Automation Bias Countermeasures
Design the HITL surface to resist automation bias: the human tendency to over-trust an automated system that has performed well historically.
This page is the complete instruction page for one pattern called "Automation Bias Countermeasures." It explains the problem this pattern solves, the idea behind the solution, when you should (and should not) use it, and what happens afterward, both the good effects and the costs.
This matters because building AI agents is not just about making them clever. It is also about making them safe and predictable. Following a well-tested pattern like this one helps avoid common mistakes, and shows you exactly which safety rules and regulations it connects to, listed under "Standards Mesh" on this page.
Context
Humans approving thousands of agent actions gradually stop reading carefully. The reviewer becomes a rubber stamp precisely when review matters most.
Problem
HITL without friction becomes ineffective oversight: a compliance artefact, not a control.
Forces
- Throughput vs review depth
- User experience vs deliberate friction
Solution
Introduce variable-scrutiny UI: high-risk or anomalous actions require typed justification; occasional audits demand full-trace review; decision support shows evidence and counter-evidence, not just recommendations. Track reviewer click-rates; investigate reviewers whose approval rate is statistically indistinguishable from auto-approval.
Where this pattern acts in the two lifecycles
06 Reflect / Learn. Critique, self-correct, memory write. 8 of 61 catalogue patterns act at this stage.
05 Operate & Monitor. Runtime telemetry, drift detection. 18 of 61 catalogue patterns act at this stage.
Stages and counts from the catalogue data. The Runtime lens is the loop the agent runs each turn; the AgentOps lens is the lifecycle the team runs the agent through. See the Matrix for every pattern at each stage.
Applicability
- High-volume review queues
- Sensitive-action workflows
- Any team where reviewer fatigue is plausible
Anti-Patterns
- Single-click "Approve" on every item
- No differentiation between trivial and high-stakes reviews
Consequences
- +Preserves the meaning of oversight
- +Detects reviewer drift
- −Friction reduces throughput
- −Harder to scale reviewer headcount
