What You Can Do Without AI Is What I Hire For

The work that cannot be explained

I see the same thing more and more often. Someone shows me a piece of work made with AI: an analysis, a dashboard, a working piece of software, a proposal. It looks finished, and sometimes it looks better than what an experienced person would have produced in the same time. Then I ask a few questions. Why did you structure it this way? What happens if this input is wrong? Where did this number come from? The answers run out quickly.

The harder test comes later, when something goes wrong. A figure does not reconcile, the client asks for a change, or the code fails on a case nobody tried. The person who built it opens the AI tool again and asks it to fix the problem. Sometimes that works. Often it produces a second version that fails in a different place, and the person cannot tell which version is closer to right, because they never understood the first one.

As an employer, that is of limited use to me. I can buy access to the same AI tools for a monthly subscription. What I am hiring a person for is everything the tool does not do on its own: knowing whether the output is right, explaining it to a client or a regulator, changing it when the requirement changes, and repairing it when it breaks. Someone who can only produce the first draft has given me the part I could already get for the price of a licence.

So I have come to value something that sounds old-fashioned. I want to know what a person can do without AI. I am glad when they use it. I need to know that their own understanding is still there underneath, because that is what I am relying on when the work has to be defended.

Why the output is the easy part

AI has made the first version of almost anything cheap. A report, a presentation, a model or a block of code that would have taken a week can appear in an afternoon. That changes where the value sits. When producing the output costs little, the output stops telling you much about the person who produced it.

Most of the cost in real work comes after the first version. Software is maintained for years after it is written. A financial model is questioned at every review. A policy is tested by the first case it did not anticipate. In each case the work only holds up if someone understands why it was built the way it was and can carry that understanding into the next decision.

The process of building something by hand is where that understanding forms. When you write the formula yourself, you find out which assumption drives the result. When you debug your own code, you learn how the system behaves when it fails. When you draft the argument yourself, you discover the weak step before your audience does. Skip the process and you skip the learning, even when the output looks the same.

What the studies show

The research on this is young, and the samples are small. The direction is consistent all the same, and it matches what I see in hiring and in client work. I covered the wider evidence in AI Use and the Decline of Human Thinking Skills. Three findings bear directly on the person who cannot explain or fix their own work.

Understanding, and debugging in particular. In a randomised trial published by Anthropic in January 2026, 52 developers, mostly junior, learned Trio, a Python library most of them had never used. Half had an AI assistant and half wrote the code by hand. The AI group finished about two minutes faster, a difference that was not statistically significant. On a quiz about the concepts they had just used, the AI group averaged 50 percent and the hand-coding group 67 percent. The widest gap was on debugging questions, which is the skill you need when the tool’s output is wrong. The developers who did well with AI had used it differently: they asked it to explain the code it produced, or asked only conceptual questions and fixed the errors themselves. Those who handed over the whole task learned the least. Anthropic makes AI coding tools, so this is a vendor publishing an unflattering result about its own product category.

Learning that holds once the tool is gone. In a field experiment at a Turkish high school, published in PNAS in 2025, about a thousand students practised maths with one of two GPT-4 tutors or with none. Both tutors raised scores during practice. When the tools were taken away for the exam, students who had used the ordinary answer-giving tutor scored 17 percent lower than students who never had one. Students whose tutor was set up to give hints and withhold answers scored the same as the control group. The design of the tool, and whether it let students skip the thinking, decided the outcome.

Knowing how well you are doing. In 2025, METR ran a randomised trial with 16 experienced open-source developers working on 246 real tasks in projects they knew well. Before the study they expected AI to make them 24 percent faster. With AI allowed, they took 19 percent longer. Afterwards, they estimated that AI had made them 20 percent faster. Tools have improved since then, and METR’s own early-2026 follow-up suggests developers are now probably faster with AI than they were. The perception gap is the part that stays with me. People who were more experienced than most of the candidates I interview could not tell, from the inside, whether the tool was helping them.

Perceived and measured speed

The developers felt faster with AI and were slower.

SlowerFaster

Expected before the studydevelopers’ own forecast

24% faster

Believed afterwardsdevelopers’ own estimate

20% faster

Measuredtime taken on 246 real tasks

19% slower

Source: METR, randomised trial with 16 experienced open-source developers on 246 tasks, February to June 2025, early-2025 tools. METR’s February 2026 update suggests developers are now probably faster with AI than they were then.

The follow-up contains a second detail worth noting. When METR tried to repeat the study in late 2025, the experiment ran into trouble because a growing share of developers said they would not want to do half their work without AI, and 30 to 50 percent of those surveyed said they had held back some tasks from the study because they did not want to do them unaided. The measurement of what people can do without the tool is becoming hard to take, partly because people no longer want to be measured that way.

Free tool

Human-AI Interaction and Decision Quality Dashboard

Benchmarks for decision acceptance rates and automation bias, so you can see how often your people accept what the AI gives them.

→

When nobody could stand behind the work

Two public cases show what this looks like once it leaves the office.

In 2023, two New York lawyers filed a brief in Mata v. Avianca that cited six court decisions which did not exist. ChatGPT had produced them. When the judge asked about the cases, the lawyers did not withdraw them at first. The court found they had acted in bad faith, fined them $5,000 and ordered them to write to each judge falsely named as the author of a fake opinion. The lawyer who did the research told the court he had failed to verify the cases he cited. The tool produced a confident document, and nobody involved had done the work that would have shown it was wrong.

In 2025, Deloitte delivered a report to the Australian Department of Employment and Workplace Relations on the IT system behind its welfare compliance penalties, under a contract worth about A$440,000. A university researcher found references to academic papers that did not exist and a quote attributed to a federal court judgment that the court never wrote. The revised report disclosed that a generative AI tool had been used in its preparation. Deloitte agreed to refund the final instalment of the contract and said the substance and recommendations of the review were unchanged.

In both cases the problem sat with the people who signed the work. Each tool did what it was designed to do. The failure was that nobody could stand behind every line, because nobody had done all of the thinking.

What the process builds

When I say the process matters, I mean something specific. Working through a problem yourself builds four things that the output alone does not.

Judgement about whether the answer is right. You only learn what a wrong answer looks like by producing a few of them and finding out why they were wrong. That is what lets an experienced person glance at a figure and say it is too high.

The ability to fix things. You learn to debug by debugging. Someone who has traced their own errors knows where to look when something fails. Someone who has only prompted for fixes has to start from the beginning each time.

The ability to explain and defend a decision. Clients, boards, auditors and regulators ask why. “The model suggested it” is not an answer any of them will accept, and in regulated work it is increasingly not an answer the law will accept either.

The ability to change the work. Requirements move. A person who understands the structure of what they built can adapt it. A person who does not has to regenerate it and hope the new version keeps what was right about the old one.

None of this argues against AI. The developers in the Anthropic trial who asked the tool to explain its code kept their understanding, and the students with the hints-only tutor learned as much as those without one. AI can be part of the process. The problem comes when the tool replaces the process entirely and the person only sees the result.

How I test for it

When I interview someone, or review work from a person who is new to the team, I now ask four questions about something they have built. They can use any tool they like to make it. The questions are asked without a screen.

Four questions, no screen

The answers show who did the thinking.

Output onlyUnderstood

Walk me through itExplain how this works, as if I had to maintain it tomorrow.

Output only: Describes what the output says.Understood: Explains why it is built this way, in their own words.

Change one thingThe requirement has changed. What needs to move?

Output only: Asks the tool to generate a new version.Understood: Goes straight to the part that depends on it.

Fix this faultHere is a failure. How would you find the cause?

Output only: Pastes the error back into the tool.Understood: Says where they would look first, and why.

What did you reject?Which alternatives did you consider and turn down?

Output only: No answer. The first output was accepted.Understood: Names an option and the reason it lost.

Someone who did the thinking can answer all four without a screen, whichever tools they used to build the work.

The four questions are my own interview practice, not a validated assessment. The typical answers are illustrative.

Walk me through it. Explain how this works, step by step, as if I had to maintain it tomorrow. A person who built it with understanding can do this in their own words. A person who did not tends to describe what the output says instead of why it says it.

Change one thing. I change a requirement and ask what would need to move. Someone who understands the structure goes straight to the right place. Someone who does not starts again.

Fix this fault. I describe a failure, or introduce one, and ask how they would find it. This is the debugging gap from the Anthropic trial, and it is the most revealing of the four.

What did you reject? I ask what alternatives they considered and why they turned them down. Anyone who has done the thinking has rejected something. Someone who accepted the first output usually has no answer.

These questions are my own, and they are not a validated assessment. They have, though, become the most reliable way I have found to separate a person who used AI well from a person who let it do the work.

What we should teach

The mentality I worry about is the belief that the goal is to reach the output as quickly as possible. In school that means a finished essay. At work it means a deliverable on the due date. AI makes that goal very easy to reach, which is why it has become a poor goal to set.

We should be teaching people that the process is the point, especially early in their education and their careers. That means a few practical changes.

Where	What to change	Why
Schools and universities	Grade the reasoning along with the answer. Ask students to explain their work aloud, or to solve a variation of the problem without the tool.	An answer can be generated. An explanation has to be understood.
Schools and universities	Use AI tutors that give hints and withhold answers, at least while a skill is being learned.	The hints-only tutor in the PNAS study produced no loss on the unassisted exam.
Employers	Give new joiners AI for tasks they have already shown they can do by hand.	The Anthropic trial found the largest learning loss among developers new to the library.
Employers	Make “explain it, then ship it” a working rule. Ask for the reasoning in reviews, not only the result.	The developers who asked the AI to explain kept their understanding.
Individuals	Try the problem first, then ask the tool, then check its answer against your own.	You learn from the gap between your attempt and the tool’s.
Individuals	If you cannot explain the output to a colleague, treat it as unfinished.	That is the test an employer, a client or a court will apply.

Recommendations are my own, drawn from the studies cited in this post.

Where to start

If you lead a team, pick one piece of work that a recent joiner produced with AI and ask them the four questions above. Do it with curiosity rather than as an assessment. You will learn more about how your team is developing from that conversation than from any productivity dashboard.

If you are early in your career, choose one skill that matters in your job and do it by hand for a month before you hand it to the tool. Writing the analysis, building the model, tracing the bug. It will feel slower. It is the part of your working life that turns you into someone people rely on, and the tool will still be there when you are done.

---

If your organisation is rolling out AI and you want your people to come out of it more capable, that is what my executive programme is built around.