/dq/media/agency_attachments/UPxQAOdkwhCk8EYzqyvs.png)
Follow UsThey’re called spiders. Scores of these digital robots crawl through cyberspace, scouring the billions of pages in the World Wide Web. They catalog all the words in the pages, how many times they’re viewed and which other pages they link to.
These spiders power the fast-growing search industry, led by Google, Yahoo! and Microsoft’s MSN. This is quickly becoming humanity’s favorite tool for finding out just about anything, with 550 million Internet searches performed worldwide each day. Type in a couple words, say “sleeping” and “bag”, and the engine, fueled by its spiders and scientific algorithms, promptly provide a list of websites, starting with the most relevant. Right?
Not so fast. In the last year, a host of search engines, including MSN and Lycos, have sprinkled growing numbers of paid corporate Web pages into the search results and accepting money each time one of these so-called paid-inclusion links is clicked. This practice is a booming business, one expected to double this year to $200 million and to reach $600 million by 2007, says US Bancorp Piper Jaffray. But unlike the clearly marked advertised links that show up next to search results, these paid inclusions are virtually invisible to average Web surfers. Only specialists prepared to decipher Web addresses can begin to distinguish between the paid and unpaid pages. “These are not objective oracles of information,” says Beau Brendler, director of Consumer WebWatch, a project of Consumers Union.
Controversy over paid inclusion is driving a wedge through the search industry. At the heart of the issue is whether the industry’s content will be swayed by advertisers. Google, the leading search engine, steers clear of paid inclusions, saying that they undermine confidence in the objectivity of search results. AOL uses Google technology and follows the same line. But others, including Yahoo, say that paid inclusion can provide users with better information. And they maintain that their search results are still displayed in order of relevance. That means that paid ads get no preferential treatment. “All listings are run through an algorithm that is blind to their source,” says Tony Mamone, sales vice-president at LookSmart, which feeds paid-inclusion results to portals like MSN.
| What’s In Your Search Results? |
||
| Some search engines use an approach called “paid inclusion” that mixes computer search results with listings that have been paid for–without disclosing which is which. | ||
| Pure Search | Paid Placement | Paid Inclusion |
| n Search engines have built an index by scouring billions of Web pages.It lists pages from its index based on myriad criteria like the number of other sites that link to the page. | n Most search engines have advertisers who bid against each other to be displayed for a phrase, such as “sleeping bag”. |
n Companies with thousands of products often pay to channel their sites directly to search engines. |
| n Results are ranked by the highest scores against the criteria,with results fluctuating day by day. |
n Paid placement results appear near the unpaid search results. They are typically marked as “sponsored links” and often shaded a different color. |
n Search companies say the paid pages compete fairly, with pure search listings. But BusinessWeek analysis shows that these paid pages sometimes rise to the top of the search results. |
| n No money changes hands between the search engine and the sites that are listed or visited. |
n For every click on a paid-placement listing, the sleeping-bag site pays the search engine a fee, from a dime to $10. |
n If a Web surfer clicks on a site that paid for inclusion,the site owner pays the search company between 15¢ and 75¢. |
Ticket to the top
Yet an investigation by BusinessWeek, based on more than 30 interviews and analysis of dozens of Internet searches, suggests that paying customers frequently fare better. Indeed, out of 20 advertisers and online marketing pros interviewed by BusinessWeek, 10 had experienced first-hand a boost in search-engine rankings when they signed up for paid inclusion.
Dennis Swanson learned the hard way. Earlier this year, the chief executive of Lamps Plus Inc., couldn’t understand why his e-commerce offerings for light fixtures turned up so prominently in Google but were nearly out of sight on MSN. So he paid to have his site included with LookSmart, which feeds search results to MSN and shares with MSN revenues it gets from advertisers. Almost instantly, Lamps Plus pages soared to near the top of MSN’s search results and Microsoft’s portal became Swanson’s biggest traffic driver.
Hits and misses
What gives? For starters, the paid-inclusion pages are easily engineered for high placement. When submitting paid-inclusion data, companies typically fill out a spreadsheet with information on product details, along with the search words and phrases for which they’d like to appear. The result is a rich stream of data targeted precisely to what the search engine will deem relevant. Anecdotal examples aren’t tough to turn up. Plug in “green sleeping bag” to HotBot.com and the first six results, in a recent search, featured paid-inclusion listings from Inktomi, which is now owned by Yahoo.
Despite criticisms, the paid-inclusion business addresses some very real needs. One biggie is the sheer number of Web pages tucked away in corporate databases, where search-engine spiders have a difficult time navigating. By letting, say, Staples.com pipe its data directly to a search engine, a user seeking “inkjet cartridges” could get a fresh listing from Staples.com that might not otherwise show up. “We use
Still, the growth of paid inclusion could tilt the search business toward deep-pocketed companies. For instance, travel sites that want to be in LookSmart’s paid-inclusion program have to pay the company a flat rate of 30 cents for each click, according to the company’s website. Jumbo chains like Marriott International Inc. or Hilton Hotels Corporation are certainly better equipped to cover such costs than, say, a bed-and-breakfast.
Just ask Rob Spooner. The 56-year-old runs a travel site called Online Highways with informational pages about various small towns throughout the US. Inktomi, a search engine that now belongs to Yahoo, contacted him late last year about becoming a paid-inclusion participant. The proposal: Spooner would pay 10 cents for every visitor Inktomi passed along. Spooner did the math, figured he would lose money on anything more than 3 cents a click and declined the offer.
Then things went downhill. Spooner’s Web pages soon plunged in Inktomi’s search rankings and disappeared from key sites like MSN, where Inktomi feeds its listings. After he demanded to know what happened, Spooner learned from Inktomi that his site contained editorial flaws that hurt his ranking. And he would have to become a paid-inclusion customer to learn what these flaws were. All this, while his pages remained well ranked on Google. “I lost a quarter of my traffic,” says Spooner.
Yahoo declined to comment on this incident. However, it and other search engines offering paid inclusion say it serves the customers’ interest. Says Tom Wilde, general manager of search services at Terra Lycos: “If you deliver a poor user experience, people leave, and they leave rapidly.”
And why not label the sponsored links? Many search companies and advertisers fret that marking them would scare the public away from relevant sites. Says Glenn McLaren, e-commerce manager at Friedrich Air Conditioning: “Part of what makes it attractive is that it doesn’t look like advertising.” The trouble is, if paid inclusions are hidden, readers could grow to dismiss Internet search as nothing but a pile of ads.
By Ben Elgin in San Mateo, California in BusinessWeek. Copyright 2003 by The McGraw-Hill Companies, Inc