When a computer first beats humans at chess or Go, many people say it was only calculation and therefore never counted as intelligence. The pattern is often called the AI effect, and a line attributed to Larry Tesler, “intelligence is whatever machines haven’t done yet”, sums it up, although its origin is disputed.
The effect makes progress hard to measure, because the target moves after each success. It also hides a fair question: sometimes the old test really did measure the wrong thing. Judge a new test by whether it was written down before the result, names a capability, and applies to people as well as machines. Part 2 of the Sagan and the Machines series works through the five checks.