Will Google search favor Google content?

On the Google scale of projects, the search giant’s newly launched online encyclopedia Knol ranks as relatively minor. But for some, it’s a stretch – not technologically, but ethically.

Google has over the years expanded its Web presence beyond the familiar search box. With each foray into content, it raises concerns about conflict of interest with its original function as unbiased search engine – concerns that Google search would be disposed to point to Google content first.

“This is a step too far,” says Danny Sullivan, editor in chief of Search Engine Land. “Google’s job started out being a service that points people to other information, and that remains their primary job – not to be providing the information themselves.”

The company now controls the leading online video site YouTube, owns a major blogging platform, and has advertising inventory on millions of external pages thanks to AdSense and DoubleClick.

Like most search engines, Google keeps secret the algorithms that rank search results, meaning that users are left to trust the company not to favor its own burgeoning content over others.

“I would prefer that a search engine keep church and state separate,” says Jay Bhatti, co-founder of Spock.com, a people search engine. “You can’t choose to be a content creator as well as a content aggregator that impartially sends people to data sources. It’s very tough.”

Google has moved into some areas of content because few other companies can undertake projects of such scale, such as digitizing the world’s books. Other times, it dabbles in publishing mainly to improve its search functionality.

In the case of Knol, says Mr. Sullivan, the project overlaps needlessly with existing online encyclopedias, including Wikipedia, Citizendium, and Squidoo.
“They really didn’t need to do Knol,” says Sullivan. “What you really want sometimes is for Google to say no to itself.”

A Wikipedia alternative
Launched in beta form two weeks ago, Knol allows anyone to write encyclopedia pages. Unlike Wikipedia, each page, or knol, will have a signed author and may include his or her point of view. Outsiders can make edits if approved by the author. The setup fixes some perceived weaknesses of Wikipedia, namely the blandness of group writing and the ability of vandals to wreck an entry.

Another difference: The author may put advertising on the knol, with revenues shared by the author and Google.

Within days of Knol’s launch, some knols showed up in Google’s top 10 search results for certain keyword queries – something observers like Sullivan consider an unusually rapid rise to prominence.

Other red flags went up. Search expert Aaron Wall demonstrated that content could be “scraped” off another page – say from Wikipedia – dumped onto a knol, and show up higher on Google’s search results than the original. What’s more, a Google algorithm clearly noted the original page, but still ranked the knol higher.

“Let’s say I’m the nefarious type. How long does it take me to see that Knol outranked the original source before I ... grab hundreds of thousands of pieces of content on the Web, upload them to Knol, and add AdSense to it?” says Mr. Wall, author of SEO Book, a search engine optimization guide at seobook.com.

A Google spokesperson countered that the company has “strong and robust” ways of establishing authorship and discouraging plagiarism. The Knol community of users will have tools to flag abuses, and plagiarized authors can file a take down notice that Google “will then investigate and act upon.”

As for the high ranking of some knols, a Google software engineer named Matt Cutts publicly responded in a blog that these floated to the top because of their placement on the Knol front page, thereby becoming highly visibile. Some knols ranked highly in Yahoo, he added, but “that doesn’t mean that Yahoo is boosting Knol.”

Despite the concerns, there is no conclusive evidence that Google has ever favored its own content.

A company spokesperson said Google is committed to keeping its search operations and content projects separate, and any content it owns will be treated by the search team the same as any other.

Google also points out it would not make sense for it to jeopardize public trust in its searches, which makes up the foundation of their business.

Ensuring search transparency
Discussions in the tech community, however, have been percolating over better search engine transparency. Wikipedia founder Jimmy Wales is experimenting with a transparent search engine. In January, he launched Wikia Search, an engine built on open source algorithms – a radical concept Google won’t be adopting.

“We are, to be honest, quite secretive about what we do. There are two reasons for it: competition and abuse,” writes Udi Manber, a Google vice president of engineering, on a company blog devoted to greater openness. If Google’s search algorithms were known, he argues, people would game the system.

The search engine Spock, meanwhile, found it easiest to simply limit itself to search, says Mr. Bhatti, after partners balked at the possible competition. If Google’s going to do both, he says, it needs to assure users the two are separate.

Sullivan proposes that Google hire an ombudsman – somebody who would have full access to investigate complaints and report back to the public. A variation on that theme, put forward by Jeff Chester at the Center for Digital Democracy, would be a third-party task force that could both vouch for integrity without divulging specifics.

“When you dominate search in such a way, then you need to go the extra mile ... to assure the public,” says Mr. Chester. “At the end of the day, Google’s job is to serve the advertisers with the largest budgets.”

Mr. Wales says he trusts Google at this point. “If someone in Google said link to Knol first, we’d hear about it because it would really cut against the ideology of everyone who works at Google.”

That said, as Google grows stronger, the concerns naturally increase. While Wales wouldn’t want a mandated ombudsman, “I think it would be good business sense for Google to offer.”