Thread:Comments:Turing test beaten by Russian chatterbot/The Turing Test Is Not A Gallup Poll (And It Was Not Passed)

The Turing Test certainly has not been passed! Really passing it would require a lifetime of real verbal capacity, indistinguishable from a real pen-pal, not just fooling an arbitrary 30% of judges in a series of 5-minute exchanges!

The 30% criterion was just based on this throw-away remark of Turing’s in 1950:

TURING: “I believe that in about fifty years' time it will be possible, to programme computers... [to] play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning.” (Turing 1950)

No doubt this party-game/Gallup-Poll criterion can be met by today's computer programs -- but that remains as meaningless a demographic fact today as it was when predicted 50 years ago: Like any other science, cognitive science is not the art of fooling most of the people for some or most of the time! The candidate must really have the generic performance capacity of a real human being -- capacity that is totally indistinguishable from that of a real human being to any real human being (for a lifetime, if need be!). No tricks: real performance capacity. (Harnad 2008)

At the beginning of the paper Turing had said:

TURING: “[A] statistical survey such as a Gallup poll [would be] absurd [as a way to define or determine whether a machine can think]” (Turing 1950)

Taking a statistical survey like a Gallup Poll instead — to find out people's opinions of what thinking is — would indeed be a waste of time, as Turing points out -- but then later in the paper he needlessly introduces the equivalent of a statistical survey as his criterion for having passed his Turing Test! (Harnad 2008)

Harnad, S. (1992) The Turing Test Is Not A Trick: Turing Indistinguishability Is A Scientific Criterion. SIGART Bulletin 3(4) (October 1992) pp. 9 - 10. http://cogprints.org/1584/

Harnad, S. (2008) The Annotation Game: On Turing (1950) on Computing, Machinery and Intelligence. In: Epstein, Robert & Peters, Grace (Eds.) Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer. Springer http://eprints.ecs.soton.ac.uk/7741/

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 433-460. http://cogprints.org/499/1/turing.html