The new gatekeepers
Every second, the world sees three new babies born, 544 McDonald's customers served, and 2,315 Google searches conducted.
Using the search engine to navigate the Internet has become so commonplace that "to Google" is entering the lexicon.
Indeed, Google and its major competitors are putting their own stamp on the digital world: grabbing a major share of online advertising, creating a new Internet industry, and reviving hopes of a return to the heady days when Internet stocks made fortunes overnight.
Most important, they have become the gatekeepers of the digital world. If, as some observers expect, a Big Three of search engines emerges, users will see their Web experience heavily influenced by moneymaking corporations interested in delivering as many eyeballs as they can to advertisers.
That's not necessarily bad, say experts, who compare the phenomenon to the effect of the Yellow Pages on telephone use. But users need to understand how the major search engines influence their Web experience. The results they get are broad and tilted toward commercial, more popular sites.
If they want to really plumb the depths of the Internet, they'll have to rely on tools for "deep" Web searches, which can scour the vast majority of websites that technology used by Google and others just doesn't find.
Search engines represent information gatekeepers unlike any we've had before, says Steve Jones, a communications professor at the University of Illinois in Chicago and founder of the Association of Internet Researchers. In the past, librarians or newspaper editors might have served that role, connecting people with information they're seeking. "This gatekeeper is a machine. And that's a really interesting difference."
For most Web users, the rise of the gatekeeper means their access to cyberspace - and its vast amount of information and services - is essentially controlled by a handful of companies and their advertisers.
Google is responsible for more than a third of Internet searches in the United States, trailed closely by Yahoo. And behemoth Microsoft says it will enter the market within the next year, forming a Big Three of Internet searchers.
Six years ago, Google was another California dream, housed in a garage. Now, with its announcement of a public stock sale, everyone's favorite little search engine that could has been officially ordained a multibillion-dollar corporation.
Unlike many of the companies that went public during the Internet boom, search engines are already racking up revenue.
In 2003, advertisers spent some $2.3 billion buying spots on search-engine pages in the US alone, spawning a whole new industry to help companies ensure their websites rise near the top of search rankings.
Search-engine marketing has grown to become the single largest contributor to online advertising revenue, according to comScore Networks.
"Search engines are pretty dominant right now. Most people are going there first in order to find something," says Andy Beal, vice president of search marketing at WebSourced, Inc. "I think that everyone is excited about search" as an ad medium. In contrast, banner ads have declined and "are not as effective as they were, say, five years ago," he says.
Search engines bring a precious commodity to businesses: ready buyers, Mr. Beal says.
"The consumer is initiating the contact. They're telling you what they're looking for," he says.
Market intelligence firm Random Secrecy predicts that search engines will generate purchases of more than $35 billion in 2004, and it expects that figure to rise to $92 billion by 2008.
Search advertising can happen two ways, says Danny Sullivan, editor of online newsletter searchenginewatch.com. Paid placements guarantee the advertiser a spot on the page to the top or side of the rest of the "natural" query results, which are typically calculated by a complex algorithm, including factors like website popularity.
Paid inclusions, on the other hand, mean the advertiser's site is guaranteed to be among the results generated, but with no guarantee of placement or rank.
"They'll only sell you a lottery ticket, but they won't let you rig the lottery," Mr. Sullivan says.
Paid placements are clearly identified as "sponsored links." Yahoo allows paid inclusions in the natural search. Google does not.
Yet despite the white-hot market interest, search engines remain imperfect at best.
A search for a "driver," for example, might yield a communications device for a computer, a golf club, a person in a car, or something that turns a screw.
"The ultimate search engine would basically understand everything in the world, and it would always give you the right thing. And we're a long, long ways from that," Google cofounder Larry Page told Business Week magazine recently.
Search engines are also flawed in that "the relevance [of a search] is determined by the proximity of one word to another," Professor Jones says. "Well, that's not generally how you and I determine relevance. We use incredibly complicated mental maps to do that."
Internet searchers have three perpetual complaints, says Duncan Witte, chief operating officer for BrightPlanet, a specialty Internet-search company in Sioux Falls, S.D. They can't find what they need. When they do find it, they don't understand how to access or interpret the data. And finally they worry about the quality of the data.
Solving these needs "is almost the holy grail, if you will" for searching, Mr. Witte says. "We want it to be easier."
People do a search and get back thousands of results, but "they just want to know what the answer [to their question] is," Witte says. "It's going to be a long time before it's that intuitive.... The Web is so big, and there's so much junk, it's hard to wade through it."
To better serve customers, some search engines are trying to specialize.
"Local searching is going to be important," Beal says.
Google has entered this arena with its Google local.
In fact, consumers may begin to expect that a search engine knows where they are and give them results based on that, Beal says. "That's definitely going to play a part in the future."
Other search engines will get to know a user's searching habits and try to anticipate interests, in much the way that Amazon recommends books to its customers based on past purchases or TiVo recommends TV shows based on previous viewing.
Still other search engines are finding niches to go deeper than the big guys. Technorati.com, for example, specializes in finding weblogs, or blogs, personal journals published on the Web.
BrightPlanet aims to chart some of the big business and government databases missed by Google and the others. Its customers are usually companies or government agencies with specific needs, "people who want to find everything" that's out there, Witte says.
Google has found more than 4 billion Web pages, an astounding number. But according to BrightPlanet, it's mapped only perhaps 1/500th of what's out there. It estimates that just 60 of the biggest deep websites contain more than 40 times the data of the "surface Web" scoured by Google.
While Google's exact searching methods are a secret, it relies on "crawling" from website to website following links to Web pages. But large data storage areas have firewalls or query search front doors that must be knocked on to reveal their contents.
BrightPlanet searches these big databases, such as the government's Centers for Disease Control, where "the pages are dynamic, they're built on the fly, on request," when a user goes to a search box and types in a query, Witte says.
Jones foresees a time when computer desktops would be more deeply integrated into the Internet.
"If I'm working on an article about gorillas, for example, behind the scenes my operating system will parse that article and go out on the Internet and look for information [and tell me], 'Here's the most interesting stuff about gorillas you might want to look at.' "
But right now, he says, search engines are the only maps we have to guide us around the Web. "We don't have an alternative."
• Americans conduct between 3 billion to 3.5 billion searches per month. More than 1 billion of these searches are conducted through Google.
• More than 65 million people visited Google sites this past March, an increase of 23.5 percent over March 2003.
• Every second, the world conducts nearly twice as many Google searches (2,315) as ATM transactions (1,268), but consumes still more cups of coffee (4,000).
• The average search-engine user conducted 32 searches in February.
• Combined, search sites reach more than 130 million Americans, or about 85 percent of all Internet users each month.
SOURCES: comScore qSearch; comScore Media Metrix; Nestlé S.A.
