Computer program passes Turing test: But does it really think?
Computer scientists at the University of Reading have announced that a conversation bot named Eugene Goostman has convinced a third of its interlocutors that it is a genuine human being, possibly meeting Alan Turing's iconic definition of a thinking machine. But can it really think?
In a competition held by the University of Reading, conversation bot Eugene Goostman successfully convinced a third of his conversation partners that he was a real human being.
Screenshot
Scientists at the University of Reading have announced that, for the first time, a computer program has passed the iconic Turing test, successfully impersonating a human being in a five-minute, open-ended conversation.
"Eugene Goostman," a conversation bot posing as a 13-year-old Ukrainian boy, fooling 33 percent of the human judges to beat out four other bots for the Turing Test 2014 Prize on Saturday.
The Turing test originated in a 1950 paper titled "Computing Machinery and Intelligence," written by Alan Turing. Turing, a mathematician and codebreaker who is widely regarded as the father of computing, considers the question, "Can machines think?" He immediately discards this question as ambiguous, replacing it with the question of whether a computer could win the "imitation game," by convincing its interlocutor in an exchange of typewritten messages that it is a genuine human being.
Having deftly transformed an unsettling philosophical question into a more straightforward engineering challenge, Turing predicted that, by the year 2000, an average person would be unable to distinguish a computer from a human after five minutes of continually exchanging messages no more than 70 percent of the time. Thus the Turing test was born, and, on the 60th anniversary of Turing's death, it seems to have been passed, if by just a hair.
In fact, one could argue that it had been passed three years ago. In 2011, a Web-based application named Cleverbot took the Turing test alongside humans. Of the 1,334 people who conversed with Cleverbot for four minutes, some 59 percent judged it to be human. (Oddly enough, the actual human participants were judged to be human just 63 percent of the time.)
But Cleverbot is programmed to record snippets of text that its conversation partners submit to it and to reuse them in other conversations. Eugene Goostman, who was first programmed in 2001 by Ukrainian Eugene Demchenko, who lives in Russia, and Russian-born Vladimir Veselov, who lives in the United States, formulates original responses using natural language processing.
"Some will claim that the Test has already been passed," said Reading cybernetics professor Kevin Warwick. "The words Turing Test have been applied to similar competitions around the world. However this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing's Test was passed for the first time on Saturday."
Despite Prof. Warwick's praise, a conversation with Mr. Goostman – you can chat with a version of him here – is decidedly underwhelming. He often appears not to be listening, fails to answer direct questions, and is inappropriately sarcastic and aggressive. (Then again, you could say the same about many teenagers.)
So can machines think? We posed this question to Goostman, and got an uncharacteristically direct answer.
"Machines can't think," Goostman told the Monitor.