← All Terms

Operating Effectiveness

Evidence that a control worked as intended, consistently, over a period of time, as distinct from design effectiveness, which only shows the control was described correctly.

Governance & Risk

Auditors test a control twice. Design effectiveness asks whether the control, as written, would address the risk if it were followed. Operating effectiveness asks whether it was followed, every time it should have been, across the period under review. The first is tested by reading the policy. The second is tested by sampling the records the control should have produced.

For AI the gap between the two is often wide. A model monitoring procedure can be well designed and never run, or run once at launch and not since. ISO/IEC 42001 is built around the second test: its clauses on documented information, monitoring, internal audit and corrective action all ask for evidence of operation, which a policy document alone cannot supply.

A useful rule of thumb: if a control has never produced a record you could hand to an auditor, it has only been designed. If it has never produced a finding, check whether anyone is looking before concluding that nothing is wrong.

Read more on this
Governance & Risk

Governance Theatre Is What Fear Looks Like on a Slide

Three in four organisations say they have an AI usage policy. Fewer than half monitor their production AI for accuracy, drift or misuse, and only a third of those with a policy regularly check for unsanctioned AI. Policies written to produce the feeling of control survive a board meeting and fail an audit. Here are three tests, drawn from ISO/IEC 42001 audit work, that separate a control from a performance.

Governance & Risk

An AI Management System Is Not a Policy Binder. It's the Machine ISO 42001 Actually Audits.

An AIMS is the clause-4-to-10 system ISO/IEC 42001 certifies, not a written AI policy: risk assessments, impact assessments, roles, internal audits, and a statement of applicability an auditor can test. What gets checked first, and how to generate that evidence from day one instead of reconstructing it before the audit.