Executive Summary
When a machine beats people at something we called intelligent, we tend to decide that the thing was never intelligence after all. Researchers call this the AI effect, and Alan Turing described its ancestor in 1950. Sagan saw it in 1979, and it has followed Deep Blue, AlphaGo and a 2025 study, now published in PNAS, in which a language model was judged human more often than the human was. Sometimes moving the line is vanity. Sometimes it is correct, because the old test measured the wrong thing. This part gives five questions that tell the two apart.
1950
year Turing listed the “you will never be able to make one to do X” objections and noted no support is usually offered
3.5 to 2.5
Deep Blue’s score over the reigning world champion in May 1997
4 to 1
AlphaGo’s score over Lee Sedol in March 2016, which DeepMind says came a decade before experts thought possible
73%
how often GPT-4.5, given a persona prompt, was judged to be the human in a 2025 three-party Turing test (36 to 38% without the prompt)
Core conclusions
- The reflex is old and predictable. Turing wrote down the objection to it before any machine could play a decent game, and Sagan named it in the chess results of his day.
- A moved line is not automatically a defensive one. The old test may have measured skill at a task when the real question was whether a system can handle something new.
- A new test is fair when it names a capability the job needs, would be passed by your own staff, says what would convince you, and explains what the old test missed. Written down before the result is best, but a test written afterwards can still be fair if that explanation holds.
The argument, in ten slides
Save it, or send it to the colleague who keeps saying “that is not real intelligence”.










A reflex with a name
This is Part 2 of the series Michael Low and I are writing on Carl Sagan and the machines. Part 1 set out Sagan’s 1979 case for intelligent machines and the one rule I took from it: do not reach a conclusion before the facts are in. This part asks what happens to that rule when the facts do come in and people dislike them.
You have probably seen the pattern. A machine learns to do something that used to be a mark of intelligence. Then someone explains that the task was easier than it looked, that the machine only used search or statistics, and that “real” intelligence is whatever it did not do. The achievement shrinks and the next goal is named.
Researchers call this the AI effect. The best-known summary is Tesler’s theorem, which in Larry Tesler’s own wording is “intelligence is whatever machines haven’t done yet”. The shorter version, “AI is whatever hasn’t been done yet”, is Douglas Hofstadter’s. Tesler dated his line to about 1970, and the earliest printed appearance I found is Hofstadter’s in 1979, so the origin stays disputed.
What Turing expected
The reflex is older than the machines that triggered it. In his 1950 paper “Computing Machinery and Intelligence”, Alan Turing gave the objection a section of its own, the argument from various disabilities: “I grant you that you can make machines do all the things you have mentioned but you will never be able to make one to do X.”
His candidates for X included being kind, having a sense of humour and doing “something really new”. Then he made the point I find most useful for leaders: “No support is usually offered for these statements.”
He also set a modest bar for his imitation game: an average interrogator should have no more than a 70 per cent chance of making the right identification after five minutes of questioning. It was a bar for substitution in a short exchange, and he called the original question “too meaningless to deserve discussion”.
What Sagan saw in 1979
In chapter 20 of Broca’s Brain Sagan noted that machines were being demeaned, often with a note of relief, because chess was an area where humans were still superior. His reply is the fair-minded part. A machine that plays chess in the middle range of human ability, he says, is “a very capable machine”, and there are many better players and also many who are worse.
On Turing’s imagined conversation he wrote that no device of that sophistication had been built, “although I am not sure how many humans would pass Turing’s human test”. That one clause carries what I mean by a fair test. When we judge a machine against a standard, we should ask how many of us would meet it.
Each test was passed, and each time the next test was already waiting.
Sources: Turing (1950); Sagan, Broca's Brain, chapter 20 (1979); IBM and Google DeepMind project pages; Jones and Bergen (2026, PNAS). The selection of five moments is mine.
Chess, Go and the pattern
In May 1997 IBM’s Deep Blue beat the reigning world champion, Garry Kasparov, by 3.5 to 2.5. Kasparov said afterwards, “The computer is far stronger than anybody expected.” The reaction that followed is the one Sagan predicted. Daniel Dennett looked back on it in 2007 and wrote that many commentators insisted Deep Blue’s brute-force search was entirely unlike the exploratory processes Kasparov used. His answer: “But that is simply not so.” The match “didn’t settle any great metaphysical issue,” he added, but “it certainly exposed the weakness in some widespread opinions.”
Go was supposed to be different. In March 2016 AlphaGo beat Lee Sedol 4 to 1 in Seoul. Google DeepMind describes this as arriving “a decade before experts thought possible”. That is the developer’s account of expert opinion, and I have not checked it against what individual experts said at the time.
The shape is the same each time: a game is held up as a measure of intellect, a machine wins, the method is found to be a method, and the goalposts move.
| Milestone | What had been said | What happened | The line that moved |
|---|---|---|---|
| 1950, Turing | Machines will never do X, for a long list of X | He expected opinion to shift by 2000 | Objection treated as a conclusion |
| 1979, Sagan | Chess is where humans are still superior | In 1997 a machine beat the champion | ”It is only brute force” |
| 2016, AlphaGo | Go was thought to be a decade away | 4 to 1 over Lee Sedol | ”It is only a game with rules” |
| 2025, Jones and Bergen | The Turing test as a milestone for machines | Judged human more often than the human | ”The test measures humanlikeness” |
My own comparison. The first three columns draw on Turing (1950), Sagan (1979), IBM, Google DeepMind and Jones and Bergen (2025). The last column is my reading, not a sourced claim.
The Turing test in 2025
On one reading, the test Turing proposed was passed in 2025. Cameron Jones and Benjamin Bergen of the University of California San Diego ran pre-registered, randomised three-party tests in which an interrogator chats at once with a person and a machine, then says which is the human. Their paper, first a preprint, was published in PNAS in 2026 as “Large language models pass a standard three-party Turing test”. In the five-minute games, GPT-4.5 prompted to adopt a humanlike persona was judged to be the human 73 per cent of the time, significantly more often than the real person it was paired with. Without the persona prompt the models fell to between 36 and 38 per cent, and ELIZA and GPT-4o scored lower still. A third study with 15-minute conversations replicated the result at lower rates: two persona-prompted models were judged human 56 and 59 per cent of the time.
The authors are clear about what this shows. Interrogators leaned on style and social-emotional cues, not on traditional notions of intelligence, and the preprint called it “not a direct test of intelligence, but a test of humanlikeness”.
Put the 73 per cent beside Turing’s own bar. He asked for an interrogator who is right no more than 70 per cent of the time. Against a machine judged human 73 per cent of the time, interrogators were right about 27 per cent of the time, so it cleared his line by a distance. That is my arithmetic from the published figures.
Here the reflex splits in two. One group says the result is meaningless because the test was always silly. That is the defensive move, because those same people usually treated the test as a milestone before it was passed. The other group says the result is real, and shows that a short chat is a poor way to find out whether a system can do your work. That group is correct, and they are the reason the next section exists.
When the old test measured the wrong thing
Not every moved line is vanity. Sometimes a test is passed because it was easy to game, and the people who move it are right.
The best statement I know is from the researcher François Chollet. In a 2019 paper he argued that “solely measuring skill at any given task falls short of measuring intelligence”, because skill is heavily shaped by prior knowledge and experience. Give a system unlimited training data and you can buy any level of skill you like. What the buyer wants to know is how well the system copes with something new.
Chollet was writing after systems had already mastered games, so his line moved after the result. That is a reason with content, though. It says what the old test missed and points at a different thing to measure. It applies to people as well, since a human who has memorised past exam papers is no better evidence of understanding than a model that has seen the answers. Compare it with the reply that a machine “is only doing statistics”. That names a method, and a human brain also has a method that nobody thinks disqualifies the person at the next desk.
Telling a fair test from a defensive one
When I judge an AI system, or a vendor’s claim about one, I put five questions to the test. Each has a fair answer and a defensive one.
Five questions tell a fair new test from a defensive one.
My own checklist. The staff-baseline question follows Sagan's 1979 remark about Turing's test; the "what" question follows Chollet (2019) on skill versus generalisation.
When was it written down? The best test is stated before the result is known. A test written afterwards can still be fair, but only if it passes the fifth question: it must say clearly what the old test missed. A line that moves after the score with no such explanation is the defensive case. Chollet’s correction came after the result and survives, because it explains itself.
What does it name? A fair test names a capability the job needs, such as handling a case it has not seen, or saying “I do not know” at the right moments. A defensive one names a method and treats the method as a disqualification.
Would our own staff pass? This is Sagan’s clause, aimed at the people who will use the system. If most of your own staff doing the job would fail the new test, either the test is not about the job or you are holding the machine to a standard you do not hold your people to. Either can be defended in a particular case, but you have to see that you are doing it.
What would convince you? A fair test says what result would change your mind. A defensive one has no such result, because the conclusion came first. This is the fault Sagan defined in Part 1, a conclusion drawn before the facts are in, pointed the other way.
What did the old test miss? A fair test gives a reason that survives being written down: it measured skill when you needed generalisation, a short chat when you needed a month of work, an average when you needed the worst case. A defensive one says only that the achievement is not real.
The questions work in both directions. A vendor who changes the success measure halfway through a pilot is moving a goalpost, and a board that accepts a test the vendor chose has accepted one.
My verdict
The reflex is human, predictable and documented. I would not call it simple vanity, though, because it has two sources. One is the wish to keep a special place for people. The other is a sound instinct that the test did not capture what we care about. From outside the two look the same, and the five questions are how I separate them.
My own practice is plain. Before I run an evaluation I write down the pass condition and date it. If the system passes and I find the result unconvincing, I write down why, and I read the reason a week later. If it still holds, I move the line and say so openly. If it does not, I accept the result. That last step is hard, and Sagan would have recognised it. It means treating our own sense of what intelligence is as one more claim that needs evidence.
Questions for leaders
- For each AI system we are evaluating, have we written down, with a date, the result that would convince us to trust it?
- When we reject a result, can we state what the test missed, in terms of the work the system will do?
- If we applied the same test to our own staff, how many of them would pass?
- Are we judging the system by its outcomes, or by its method? Would we accept the same method from a person?
- Who chose the benchmark behind the claim we are relying on, and was it fixed before the result was known?
- Where have we changed a success measure during a pilot, and who agreed to that?
Evidence & Methodology
Sagan’s 1979 wording comes from the UK printing of Broca’s Brain (chapter 20), which I read in digitised form, and where the text was cut off the edge of a line I paraphrased and did not quote. Turing’s wording comes from a PDF copy of the 1950 paper hosted at the University of Maryland, Baltimore County, not the journal original. The 2025 study figures come from the published PNAS abstract (via Crossref, as the PNAS page itself would not load for me), and the “humanlikeness” wording from the arXiv preprint. The five-question checklist and the milestone table’s last column are my own judgement.
| Claim | Source | Grade |
|---|---|---|
| Turing listed “you will never be able to make one to do X” objections; “No support is usually offered”; 70 per cent after five minutes | Turing (1950), UMBC copy | Measured, primary |
| Sagan noted machines being demeaned over chess; “a very capable machine”; “not sure how many humans would pass Turing’s human test” | Sagan, Broca’s Brain, ch. 20 | Reported, direct quote |
| Deep Blue beat Kasparov 3.5 to 2.5 in May 1997; “far stronger than anybody expected” | IBM history page | Measured, primary |
| Dennett: brute-force difference “simply not so” | MIT Technology Review, 2007 | Reported, direct quote |
| AlphaGo beat Lee Sedol 4 to 1 in March 2016; “a decade before experts thought possible”; move 37 at 1 in 10,000 | Google DeepMind | Reported, developer’s account |
| GPT-4.5 with persona judged human 73%; 36 to 38% without; 5-minute games; 15-minute replication at 56% and 59% | Jones and Bergen (2026), PNAS abstract | Measured, peer-reviewed |
| ”A test of humanlikeness” | Jones and Bergen (2025), arXiv preprint | Reported, direct quote |
| Interrogators correct about 27% of the time in that condition | My arithmetic from the 73% figure | My call |
| Tesler dated his theorem to about 1970 and gave the wording “intelligence is whatever machines haven’t done yet”; “AI is whatever hasn’t been done yet” is Hofstadter’s version; earliest print appearance found is Hofstadter, 1979 | Tesler’s website; Quote Investigator | Reported, origin disputed |
| ”Solely measuring skill at any given task falls short of measuring intelligence” | Chollet (2019), arXiv | Reported, direct quote |
| The five-question test and the verdict | Mine | My call |
Sources
- Sagan, C. (1979). Broca’s Brain: Reflections on the Romance of Science. Chapter 20, “In Defence of Robots”. New York: Random House (page numbers from the UK printing).
- Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, 59(236). Copy of the text hosted by UMBC.
- Tesler, L. Adages and Coinages (author’s website).
- Quote Investigator. (2024, June 20). As Soon As It Works, No One Calls It AI Anymore.
- IBM. Deep Blue.
- Dennett, D. C. (2007, August 15). Higher Games. MIT Technology Review.
- Google DeepMind. AlphaGo.
- Jones, C. R. and Bergen, B. K. (2026). Large language models pass a standard three-party Turing test. Proceedings of the National Academy of Sciences, 123(21), e2524472123. Earlier preprint: arXiv:2503.23674.
- Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547.

Why AI Demos Work and Production Systems Fail
The gap between a test a system passes and the work you need it to do, which is where a fair evaluation starts.

AI Fatigue: Why Verification Is Harder Than Creation
Why checking a result takes more effort than producing one, and what that means for any test you set.

Navigating a World Where Thinking Is Outsourced
What happens to human capability when we define it by what machines have not yet taken over.



