What happens to us after the takeoff.
Ten essays on the AI takeoff already under way, the endgames researchers have mapped, and the narrow path that keeps people in charge.
The film · 4 min 15 s
The series in four minutes.
Most of my work is about the next twelve months: which AI system to deploy, what control it needs, what the board should ask. This series steps back to the next twenty years, because the people building frontier AI now say in public that the takeoff toward superintelligence has started, and almost nobody I advise has changed a plan because of it.
The series runs in three phases. The first four parts look at the takeoff itself and why it is so hard to slow. The next four follow the endgames that researchers such as Stuart Russell and Martin Rees have mapped, from comfortable dependence to systems that fail by succeeding too well. The last two come back to the human mind, and to the one path I think keeps our agency, judgement and skill intact.
Claims are checked or they are left out.
This subject attracts confident numbers and invented quotes. Every claim in the series traces to what its source published, and is graded as measured, reported, forecast or my own call.
Speculation is labelled as speculation.
Some parts describe scenarios that have not happened. I will say when I am describing a scenario, and whose it is.
It will not sell you doom or comfort.
The most likely futures are neither. Where I do not know, I will say so.
Every part ends with something to do.
A question for the board, a design rule or a decision you can make this quarter.
The ten parts
3 of 10 published
The Takeoff Nobody Noticed
The singularity was imagined as a single moment around 2045. The people building frontier AI now say the takeoff has started, and Terence Tao has told mathematicians to prepare for more proofs than they can digest.
What you take away Where machine output already outruns your organisation’s ability to check it, and five questions to find out.
Read article →When Machines Prove What We Cannot Follow
Formal proof languages such as Lean let a machine certify that a proof is correct without a human referee. What happens to truth, and to expertise, when correct results arrive through reasoning no person can trace.
What you take away How to accept a result you cannot fully follow, and when you should refuse to.
Read article →Agent Swarms and the Closing Loop
AI is moving from single chat sessions to teams of agents that work for hours or days and help build the next generation of models. The human step in the loop is getting shorter.
What you take away Which human checkpoints to keep in an agentic workflow, and how to tell when one has quietly disappeared.
Read article →Why Nobody Can Hit the Brakes
Safety researchers have left frontier labs with public warnings, and the leaders of those labs put real odds on catastrophe while racing anyway. The game theory of a race nobody can afford to lose.
What you take away What coordination has worked in past technology races, and what a single organisation can still decide for itself.
In progress
The Gorilla Problem
Stuart Russell’s question: gorillas did not lose their future to a hostile species, only to a smarter one with other priorities. Competition for power, water and land, and the comfortable version of losing control.
What you take away The difference between being served by a system and being governed by it, and how to keep the first.
In progress
Hyper-Optimisation and the Fast Crash
Efficiency removes slack, and slack is what absorbs shocks. When automated systems react to each other in milliseconds, a small fault can cascade before any person sees it.
What you take away Where to keep deliberate slack and manual overrides, even when the business case says remove them.
In progress
Mirror Life and the Limits of a Cure
In December 2024 a group of leading scientists warned in Science against creating mirror-image bacteria. A case study in how an AI asked to solve a problem could choose a solution nobody should build.
What you take away How to set the outer limits on what an AI system is allowed to pursue, not just what it is asked to do.
In progress
The Biological Bootloader
Martin Rees has argued that organic humans may be a brief stage before post-biological intelligence, which would find space a better home than Earth. What purpose remains if that is true.
What you take away Why the long-range view still changes what is worth deciding now.
In progress
The Sea Squirt Trap
Brains keep what they use. The evidence on what happens to skill and attention when thinking is handed to a machine, and the business model that profits from it.
What you take away Which kinds of thinking to keep doing yourself, and how to protect them in a team.
In progress
The Narrow Path
The future worth aiming for: AI used as bounded, specialised tools that deliver the scientific gains, with people kept as the bridge between them and in charge of what matters.
What you take away A set of design rules for keeping human agency, judgement and skill intact while using the most capable tools ever built.
In progress
Governance work keeps me close to the next decision, and that is where it should be. But the people building this technology now talk openly about superintelligence and about the odds of catastrophe, and that changes what a responsible adviser should be thinking about. I do not have a confident answer, and I am wary of anyone who does. What I can do is read the evidence carefully, separate it from the hype, and say plainly what I think. Write back and tell me which part you disagree with.
Ten essays, published as they are written.
Each part goes out to the newsletter first and is narrated for the audio library. If you would rather talk about the specific decision sitting on your desk than read ten essays about it, the other door is open.