Executive Summary
Part 5 asked who keeps the say over decisions. This part asks what happens when nobody has time to use it. Efficiency works by removing slack: the second reviewer, the slower setting, the idle capacity. Slack is also what absorbs a shock. When automated systems act in milliseconds and react to each other, a small fault can spread before a person sees it. Two cases from the US equity market, both documented by regulators, show the pattern. On 6 May 2010 an execution algorithm sold 75,000 futures contracts in 20 minutes where the same seller’s previous programme had taken more than five hours. On 1 August 2012 Knight Capital lost over $460 million in about 45 minutes because one of eight servers still ran old code. In June 2026 the Bank of England’s Sarah Breeden warned that AI agents that respond similarly to the same triggers could amplify volatility in stress. This part gives you five places to keep slack on purpose.
20 min
for one algorithm to sell 75,000 E-Mini contracts on 6 May 2010, against more than 5 hours for the same seller’s earlier programme
49%
of E-Mini volume traded by high-frequency firms in 14 seconds, passing about 27,000 contracts among themselves for a net of about 200
45 min
for Knight Capital to send 4 million executions in 154 stocks, per the SEC
$460M
Knight’s loss from the unwanted positions, per the SEC
Core conclusions
- Fast failures start small. In both cases the fault was ordinary: an algorithm with no price limit, a server missed in a deployment. Speed and coupling made it large.
- The control that failed was usually one a person once performed. A second technician, a stop switch with an owner, a pace tied to what the market could absorb.
- With AI agents the question moves from one firm’s code to many agents responding the same way at once. A human in the loop for every action will not be realistic, so the slack has to be designed in.
The argument, in ten slides
Save it, or send it to whoever signs off on removing a manual step.










What efficiency takes out
Part 5 ended on the say over decisions and on whether anyone still has the skill to use it. There is a second way to lose the say, and it needs no loss of skill. The decision simply happens before anyone can reach it.
Slack is the unglamorous part of any system. It is the reviewer who slows a release, the buffer stock, the pause before an order goes out, the manager who must sign. Each costs money, and each is the first thing an efficiency programme finds. Removing one is rarely a mistake on its own terms. The cost shows up only in the case you did not plan for, and by then the slack is gone.
The sociologist Charles Perrow gave this a vocabulary in Normal Accidents (1984). Systems with many interacting parts and little time between cause and effect, which he called tightly coupled, will fail in ways nobody designed, and he argued that in such systems accidents are a normal outcome and not an exception. I read that as a description of where a hyper-optimised organisation is heading, not as a law, and I draw on it only for the idea that speed and coupling turn small faults into large ones.
Two cases from the US equity market show how it works. Both were investigated by regulators who published their findings, which makes them unusual: most fast failures are described only by the firms that suffered them.
The 2010 Flash Crash
On 6 May 2010 the US market was already under stress. By 2:30 p.m. the S&P 500 volatility index was up 22.5 per cent from the open, and the Dow was down about 2.5 per cent. At 2:32 p.m. a large mutual fund complex began a programme to sell 75,000 E-Mini S&P 500 futures contracts, valued at about $4.1 billion, as a hedge.
The joint report by the staffs of the SEC and the CFTC records what made the difference. The seller used an automated execution algorithm set to feed orders into the market at 9 per cent of the previous minute’s trading volume, “but without regard to price or time”. The same seller’s previous programme of that size had taken more than five hours, using a mix of manual trading and several algorithms that watched price, time and volume. This time the algorithm finished in 20 minutes.
The algorithm did what it was told. The slack it lacked was a limit on price and a pace tied to what the market could absorb. My reading, which the report supports in part: a person selling by hand would probably have slowed as prices fell, while an algorithm tied to volume has no reason to, because volume rises in a falling market.
The same seller sold the same size in 20 minutes instead of more than five hours.
SEC and CFTC staff, Findings Regarding the Market Events of May 6, 2010 (September 2010). Bar lengths are to scale: 20 minutes is about 7% of five hours, and "more than five hours" is a floor.
The second half of the story is the other automated systems. The report describes an unusually high level of “hot potato” trading among high-frequency firms, with the same contracts bought and sold repeatedly. Between 2:45:13 and 2:45:27 those firms traded over 27,000 contracts, about 49 per cent of total volume, while buying only about 200 additional contracts net. Volume rose and nobody was taking the risk. Each system reacted to what the others were doing, and the result looked like demand when it was the same contracts changing hands.
The regulators’ account is careful about cause. The report ties the event to the interaction of the sell algorithm with a market that had thin liquidity and fast-reacting participants, not to one bad actor. That is the point for this series: no single component malfunctioned in the usual sense.
Knight Capital’s 45 minutes
Knight Capital was a broker-dealer that, in 2011 and 2012, handled roughly ten per cent of all trading in US listed equities. On 1 August 2012 it was preparing for the New York Stock Exchange’s Retail Liquidity Program, and it changed the software in SMARS, its automated, high-speed order router.
The SEC’s order of 16 October 2013 sets out the sequence. The new code was meant to replace functionality called Power Peg, which Knight had stopped using years earlier but had never removed. A technician deployed the new code to seven of the eight SMARS servers and missed the eighth. Knight had no second technician review the deployment and no written procedure that required one. The new code reused an old flag, and on the eighth server that flag woke up the dead Power Peg code.
The result, in the SEC’s words, was that SMARS sent millions of child orders, “resulting in 4 million executions in 154 stocks for more than 397 million shares in approximately 45 minutes.” Knight ended with an unintended net long position of about $3.5 billion in 80 stocks and lost more than $460 million. The SEC also found that Knight lacked controls to compare orders leaving SMARS with those that entered it, and imposed a $12 million civil penalty.
One server of eight ran old code, and the missing check let it run for about 45 minutes.
Seven servers received the new code. The eighth did not, and no second person checked.
SEC order against Knight Capital Americas LLC, Release 34-70694, 16 October 2013. Server positions are schematic.
Look at what was missing: a second pair of eyes at deployment, a written procedure, a control that compared what the system was doing with what it had been asked to do. Each is a small piece of manual slack that someone had judged unnecessary. Knight’s own controls were not absent. It capped limit prices on parent orders, and the SEC found that this did not catch the erroneous child orders. A control existed, and it did not cover the case that happened.
What changes with AI agents
Both cases involve fixed code: an algorithm that did the same thing every time. AI agents are different in two ways that matter here. They are given goals and choose their actions, and many of them may be built on the same few models.
In June 2026 Sarah Breeden, the Bank of England’s Deputy Governor for Financial Stability, addressed this in a speech at the ECB’s Sintra Forum, titled “Agents of change”. She said that “if AI agents respond similarly to the same prompts or triggers, they could amplify volatility in stress.” The Bank is testing simulation methods with the BIS Innovation Hub and the Bundesbank “to understand which aspects of agent design could drive herding behaviour.” She expects the financial system to “evolve quickly into one that operates more autonomously, at scale and speed”, and said that relying on “a human in the loop for all agent actions is unlikely to be realistic.”
Her own list of mitigants is worth reading as a statement of where regulators are, not where they have arrived. She raised guardrails “analogous to circuit breakers or kill switches that would limit or stop trading market-wide” as a possibility, and asked whether technology-agnostic rules still suffice. None of that is a decision.
Read against the two cases, her warning has a precise shape. In 2010 the amplification came from algorithms reacting to each other’s volume. In 2012 it came from one firm’s unchecked code. The new risk is that many agents, each reasonable and each tuned to the same signals, move together. The evidence for that is a forecast from a central banker and from research papers, and I have not found a documented AI-agent crash to set beside the two above. Online reports of a March 2026 “47-second, $500 million” agent flash crash trace to low-quality sites that cite newspapers I could not locate, so I do not use them.
Five places to keep slack
You cannot design out every fast failure. You can decide, before an efficiency review, which pieces of slack stay and who owns them. This list is my own, built from the two cases and from Breeden’s three mitigants.
| Slack | What it does | The Knight or Flash Crash version |
|---|---|---|
| A pace limit | Ties the system’s speed to what its environment can absorb, not to its own throughput | A sell algorithm with a volume target and no price or time limit |
| A second reviewer on change | Catches a change that only one person understood | One technician, one missed server, no written procedure |
| An independent reconciliation | Compares what the system did with what it was asked to do | No control comparing orders leaving SMARS with orders entering |
| A stop control with a named owner | One person who can halt the process and has tested doing so | The 45 minutes between first orders and the end of the damage |
| A manual mode that still works | Lets the business run when the automated path is stopped | Teams that no longer know how to do the task by hand |
My own list. A control counts only if a named person could use it this quarter. The last row connects to the skill erosion in Part 5.
Two of these have a cost you can state. A pace limit slows the system when things are going well, and a second reviewer slows every release. The business case will always show these as waste, because the benefit arrives once in several years. The honest argument for keeping them is the loss you accept without them. For Knight that was $460 million in 45 minutes.
Where the argument is weak
I would not stretch the case. Three limits.
First, markets are not every system. High-frequency trading has circuit breakers and regulators watching, and the 2010 report led to rule changes. Most organisations have fewer protections than a stock exchange, which is my reason for worrying, but the parallel is by analogy.
Second, the two cases are about conventional algorithms, as I noted. The step to AI agents herding is a forecast from the Bank of England and from research that I have not tested myself.
Third, slack costs real money and some of it is wasted. A pace limit set too low gives up real revenue. The test I would apply is whether you can name the failure each piece of slack is there to catch. If you cannot, the efficiency review may be right.
Questions for leaders
- In our most automated process, which five pieces of slack remain, and who owns each?
- When did anyone last use the stop control, and how long did it take to reach it?
- What does a change to this system need before it goes live, and is that written down?
- Do we compare what the system did with what it was asked to do, in real time?
- If this system and every other one built on the same model reacted to one signal at once, what would we see first?
Evidence & Methodology
The Flash Crash figures come from the joint SEC and CFTC staff report of September 2010, which I read. The Knight Capital figures come from the SEC’s administrative order of 16 October 2013, which I read. The Breeden quotes come from the Bank of England’s published speech, which I read through a summary of the page, so I treat the wording as accurate to the sentence and not to every clause. The Perrow reference is to the book’s central idea and I did not recheck the text. The five-slack list, the slack-removal framing and the reading of herding risk are my own.
| Claim | Source | Grade |
|---|---|---|
| 75,000 E-Mini contracts (about $4.1 billion) sold by an algorithm targeting 9% of volume “without regard to price or time”, in 20 minutes; a previous programme took more than 5 hours | SEC and CFTC, Findings Regarding the Market Events of May 6, 2010, September 2010 | Measured, primary |
| VIX up 22.5% and Dow down about 2.5% by 2:30 p.m. | Same report | Measured, primary |
| High-frequency firms traded over 27,000 contracts in 14 seconds, about 49% of volume, for a net of about 200 | Same report | Measured, primary |
| 4 million executions, 154 stocks, 397 million shares, about 45 minutes, loss over $460 million, $12 million penalty | SEC order, Knight Capital Americas LLC, Release 34-70694, 16 October 2013 | Measured, primary |
| One of eight servers kept old Power Peg code; no second technician review; no written procedure | Same order | Measured, primary |
| Net long position of about $3.5 billion in 80 stocks | Same order | Measured, primary |
| AI agents responding similarly “could amplify volatility in stress”; human in the loop for all actions “unlikely to be realistic”; kill-switch-style guardrails suggested | Breeden, Agents of change, Bank of England, 30 June 2026 | Reported, direct quote |
| Interacting, tightly coupled systems produce normal accidents | Perrow, Normal Accidents, 1984 | Adapted, from memory of the book |
| Many AI agents on similar models would herd in a real market | Breeden’s warning and research papers | Forecast, no documented case |
| The five-slack list | Mine | My call |
Sources
- Staffs of the CFTC and SEC. (2010, September). Findings regarding the market events of May 6, 2010. Report to the Joint Advisory Committee on Emerging Regulatory Issues.
- US Securities and Exchange Commission. (2013, October 16). In the matter of Knight Capital Americas LLC. Order instituting administrative and cease-and-desist proceedings, Release No. 34-70694.
- Breeden, S. (2026, June 30). Agents of change. Speech at the ECB Forum on Central Banking, Sintra. Bank of England.
- Perrow, C. (1984). Normal Accidents: Living with High-Risk Technologies. Basic Books.

The Gorilla Problem
Who keeps the say over decisions, before the speed problem in this part.

Agent Swarms and the Closing Loop
How a loop of agents closes without a person inside it.

Why Nobody Can Hit the Brakes
Why the pressure to remove slack is hard to resist.
Part 7 takes a case where the danger is a solution and not a fault: mirror life and the limits of a cure. If your board is deciding which pieces of slack to keep in an automated process, that is a conversation my executive programme is built for.



