Can Twitter predict the stock market?

By Kate Kelly, CNBC.com

Last fall, Indiana University informatics professor Johan Bollen stumbled upon an astonishing connection: The social network Twitter could predict swings in the Dow Jones Industrial Average with 87 percent accuracy.

Six months, numerous inquiries, and one Caribbean brainstorming session later, Bollen is a consultant to a soon-to-be-opened U.K. hedge fund betting that 25 million sterling and one killer app will be enough to generate double-digit investment returns.

The fund, Derwent Capital, is scheduled to open for trading in early April. Run by a young entrepreneur with a background in foreign-exchange trading, the fund will use quantitative models to comb through millions of daily tweets and look for sentiment threads that could prove leading indicators of how stocks trade several days later.

Derwent's founder, Paul Hawtin, said in an interview he is “confident” Derwent can achieve 15- to- 20 percent returns, given how accurate the trading models have appeared so far.

Hawtin’s is one of the boldest attempts to crack human market psychology in recent memory, and it could well have disappointing results. How to predict the behavior of irrational human investors has been one of the most nettlesome dilemmas in market history—contributing, among many other examples, to the collapse of the hedge fund Long-Term Capital Management in 1998.

Still, even some veteran traders acknowledge that if any information source can track emotion on a real-time basis, it may be Twitter, where the users of more than 200 million accounts publicly share their thoughts and feelings on a minute-to-minute basis—in no more than 140 characters.

The apparent connection between tweets and stocks was a revelation even to Bollen, who had been working with a doctoral student to find connections between Twitter and more scientific public sentiment polls for months. “We thought that changes in the markets would induce changes in the public mood state,” said Bollen, 39, in a recent interview. “We found the exact opposite.”

Bollen and his student, Huina Mao, published their findings on an academic site Oct. 14. Within days, the paper had received 70,000 hits from Internet users, and Bollen’s phone was ringing off the hook with interested entrepreneurs, investors and fellow academics.

One query came from Hawtin, a British entrepreneur who was looking to start a hedge fund. Hawtin, 28, had dropped out of college to work on a hair product business that he later sold. Since then, he had worked on and off as a trader in London. Now, he and his 23-year-old brother, Simon, had plans to open a quant fund.

Introduced by mutual contacts, Bollen and Hawtin met over the Christmas holidays in St. Lucia, near where both happened to be vacationing. There, they hammered out a plan: Bollen would sell his model for searching Twitter to Hawtin, and come on board the hedge fund, Derwent Capital, as a consultant.

Boiled down, their strategy is straightforward. Throughout the day, Derwent’s technology will scan Twitter and select about 10 percent of the available tweets at random. Those messages will then be sorted in to one of a dozen mood states: calm, alert, sure, vital, kind, and happy. (The other six states are their opposites.)

Based on how many users fall into each state, the software will make predictions about where stocks are headed three or four days later—and then make trades.

Hawtin acknowledged that in an online universe where the flamboyant actor Charlie Sheen is now one of the most popular contributors, not every tweet may appear to have market relevance. “But you’ve got to remember there’s 100 million tweets a day,” he said. “Compile them all together, they give a general [sense] of how people are feeling.” (On an average day, Twitter now posts more than 130 million tweets.)

Twitter itself takes no position on the planned hedge fund. “We’ve got our hands full trying to make a business out of the ad side of the business,” said Matt Graves, a Twitter spokesman. “We have some guidelines for how we’d like people to work with Twitter, but we don’t really have thoughts on how to use it.”

The social media company, which is privately held, requires anyone who plans to use tweets for commercial purposes to secure a license, and Hawtin is in the process of trying to secure such an agreement. Negotiations with Gnip, the sole sublicensor of Twitter data, are ongoing, say both Hawtin and a spokeswoman for Gnip.