I hate to steal anyone’s thunder but I’m afraid Eugene did not pass the “Turing test”, the test of a machine’s ability to exhibit intelligent behavior – or at least enough intelligent behavior to fool a true human into thinking the machine is a real person. Media outlets are reporting today that “Eugene”, an AI created by a team based in Russia, has passed a Turing test organized by the University of Reading, by duping one in three judges. What they are not mentioning however, is that Eugene cheated…
Since Alan Turing first proposed the Turing test in his landmark 1950 paper, Computing Machinery and Intelligence, no computer program has been able to beat the test – partly because the exact parameters of the test were not precisely defined by Turing. Today there are at least three variants of the Turing test.
Turing’s intent was to question whether or not a computer could prove its ability to think using a simple question and answer format. Turing’s original game, known commonly as The Imitation Game, described a simple party game involving three players. Player A is a man, player B is a woman and player C, who plays the role of the interrogator, is of either sex. In the Imitation Game, player C is unable to see either player A or player B, and can communicate with them only through written notes. By asking questions of player A and player B, player C tries to determine which of the two is the man and which is the woman. Player A’s role is to trick the interrogator into making the wrong decision, while player B attempts to assist the interrogator in making the right one. The AI variant of the test replaces one of the players with a computer program or artificial intelligence.
Note that the test requires a man and a woman – not a child or anyone else not quite capable of conversing as an adult. Also, the test must convince the players that the machine that they are talking to is human via written human conversation – normal written conversation, not accented dialect, chopped or broken sentences, or any other gimmick intended to make the machine’s job easier. In short, the machine must fool the judges into believing the machine is a normal, functioning human being of average intelligence.
Today the generally accepted criteria for passing a Turing test requires that the computer fool at least 30% of the human judges in a text based conversation lasting at least five minutes. I’m afraid the claims that “Eugene Goostman” beat the Turing test are skewed. First, Eugene claimed to be a 13-year-old boy which provides the AI program a means to converse at a level below a normal functioning adult. Secondly, Eugene was intentionally built to be billed as a Ukrainian boy conversing in English – another “out” which allows sub-par English responses to pass the Turing test. Sorry, as much as I want to see the Turing test beaten, Eugene’s handicaps gave it an unfair edge. It is also worthy to note that no transcripts of the Eugene chats have been released by the university.
Here’s an interesting side note about Alan Turing that you don’t hear much about in the press. Alan Turing’s was prosecuted for homosexuality in 1952 (homosexuality was illegal in the UK at the time). As an alternative to prison, he agreed to accept estrogen injection treatment. Turing died two years later from cyanide poisoning (ruled suicide). He was granted a posthumous pardon by the Queen on December 24, 2013.