Public Sector Algorithmic Impact Assessment
Twelve questions about one proposed AI or automated-decision system: the decision it makes or informs, who it affects, the data it uses and the safeguards already in place. The result is an impact level from I to IV, the safeguards that level requires, and how your answers map to the government frameworks in Canada, Australia, the UK, Singapore and the OECD. No email required.
Listen to this briefing
The Public Sector Algorithmic Impact Assessment
Why levels matter
A chatbot that answers questions about opening hours and a model that recommends whether to recover a benefit overpayment are both "AI in government". They should not carry the same safeguards. Canada settled this in 2019 with its Directive on Automated Decision-Making: an algorithmic impact assessment places each system at one of four impact levels, and the level decides what the agency has to do before the system goes live.
This tool applies that method to one system in about ten minutes. I use it at the concept stage, before a business case is written, so the safeguards are costed into the project and not discovered at the approval meeting.
Two scores, one deduction, four bands
- 8 impact questionsA raw impact score out of 48
- 4 mitigation questionsOut of 12. At 80% or more, 15% comes off the impact score
- Current scoreThe impact score after any deduction, as a share of 48
Worked example: Overpayment risk model, mitigation 67%, so no deduction and the current score is 58%: Level III, high impact. Two floor rules can lift a system to Level III whatever the score.
This is a screening aid. It does not replace the impact assessment your agency is required to complete under its own law or policy, such as the Canadian AIA, an Australian AI use case impact assessment, an ATRS record or a data protection impact assessment.
Describe the system
Answer for the system as it will run in production. Both fields are optional and only appear on your printed report.
Restored your saved answers.
Your answers stay in this browser. Nothing is sent to me or stored on a server.
The decision
Impact question · Role in the decision
What part does the system play in the administrative decision?
Answer for the system as it will run in production, including how officers will use its output in practice.
The decision
Impact question · Rights and interests affected
What is at stake for the person when the decision is made?
Take the most serious interest the decision can touch, even if that outcome is rare.
The decision
Impact question · Reversibility
If the decision is wrong, how easily can its effect be undone?
Consider the effect on the person, not only whether the record can be corrected.
The people affected
Impact question · Scale
How many people will the system make or inform decisions about each year?
Count people, not transactions. A rough order of magnitude is enough.
The people affected
Impact question · Vulnerable groups
How likely is it that the people affected are in a vulnerable position?
Vulnerability includes age, disability, low income, limited language or digital access, Indigenous status, migration status and people in crisis.
The system
Impact question · Data sources
What is the most sensitive or least controlled data the system uses?
Include training data and the data fed in for each decision.
The system
Impact question · Technique
How does the system reach its output?
Choose the least transparent component in the chain.
The system
Impact question · Procurement and vendor access
Who built the system, and what can the agency inspect?
What matters is whether the agency can test, audit and explain the system without asking the vendor for permission.
Safeguards in place
Mitigation question · Explanation to the citizen
What can the person be told about how the decision was reached?
Answer for what is in place today, not what is planned.
Safeguards in place
Mitigation question · Human review
What human review takes place before a decision takes effect?
A review counts only if the officer has the authority, the information and the time to depart from the system.
Safeguards in place
Mitigation question · Appeal and recourse
How can the person challenge the decision?
Consider whether the route is timely, effective and easy to find from the decision itself.
Safeguards in place
Mitigation question · Transparency and testing
What has been tested and published about the system?
Choose the highest option that is fully true today.
Impact level
Raw impact
Mitigation
Current score
Gaps against this level
Safeguards required
From Appendix C and section 6 of Canada's Directive on Automated Decision-Making. Use them as the benchmark for what your own framework should require at this level.
| Safeguard | Requirement at this level | Section |
|---|
Framework cross-walk
Each answer and the provisions it relates to. Rows marked with a gap need work before approval.
| Theme and your answer | Canada DADM | Australia | UK ATRS | Singapore | OECD |
|---|
The report opens in a new tab with your answers, the level and how it was derived, every safeguard with a tick box for evidence, the full cross-walk and a sign-off block. Print it or save it as a PDF from there.
This tool is a screening aid built on your own description of the system. It is not the statutory impact assessment, legal advice or an approval to operate, and it does not reproduce the full Canadian AIA questionnaire or score. The level and safeguards are a starting point for your agency's own assessment, which remains your responsibility.
How the level is set
The eight impact questions give a raw impact score out of 48. The four mitigation questions give a mitigation score out of 12. As in the Canadian tool, a mitigation score of 80% or more takes 15% off the raw impact score. The current score, as a share of the maximum, sets the level using the Canadian bands.
| Level | Current score | What it describes | Approval to operate |
|---|---|---|---|
| Level ILittle to no impact | 0% to 25% | Impacts on rights, health or economic interests are likely to be little to none, easily reversible and brief. | The senior official responsible for the programme |
| Level IIModerate impact | 26% to 50% | Impacts are likely to be moderate, likely reversible and short-term. | The senior official responsible for the programme |
| Level IIIHigh impact | 51% to 75% | Impacts are likely to be high, difficult to reverse and potentially ongoing. | The head of the agency |
| Level IVVery high impact | 76% to 100% | Impacts are likely to be very high, irreversible and perpetual. | The government's central approval body (in Canada, the Treasury Board) |
Two floor rules are my own calibration and not part of the Canadian method. A system that decides matters of liberty, safety or custody, or that decides an entitlement, sanction or status whose effect is hard or impossible to reverse, is never screened below Level III. The weights are also mine. The Canadian questionnaire has 65 risk and 41 mitigation questions with a maximum raw score of 169, so a level from this tool tells you what to expect, not what the official tool will return.
Show the twelve safeguards at all four levels ▸
| Safeguard | Level I | Level II | Level III | Level IV |
|---|---|---|---|---|
| Human involvementDADM 6.3.13, Appendix C | The system may decide without direct human involvement. People take part in quality assurance and can intervene where appropriate. | As Level I. | The final decision is made by a human. Humans review the system's decisions or recommendations for accuracy and appropriateness and can intervene. | As Level III. |
| NoticeDADM 6.2.1 to 6.2.2, Appendix C | Plain-language notice through every service channel in use (online, in person, mail, telephone) that the decision is made or assisted by an automated system. | As Level I. | As Level I, and the notice directs people to the published explanation of how the system works. | As Level III. |
| ExplanationDADM 6.2.3, Appendix C | A plain-language explanation of how the system works in general, published on a discoverable agency website: its role, input data and sources, criteria, review results, outputs and principal factors. | As Level I, plus a client-specific explanation with any decision that denies a benefit or service or involves regulatory action, stating the principal factors and how officers used the output. | As Level II. | As Level II. |
| Peer reviewDADM 6.3.7, Appendix C | None required. | Review by at least one qualified expert (another government body, a university, a relevant NGO, a specialist contractor or a designated advisory board), with the review or a plain-language summary published. | As Level II. | Review by at least two qualified experts, with the review or a plain-language summary published before production. |
| Equality impact analysisDADM 6.3.8 (Gender-based Analysis Plus), Appendix C | None required. | An analysis of how the system and its data might affect different population groups, disaggregated by gender and intersecting factors such as age, disability and race where data allow, with planned mitigations. | As Level II. | As Level II. |
| Staff trainingDADM 6.3.9, Appendix C | Role-based training on how to use and explain the system at a high level. | As Level I. | Recurring role-based training covering how the system works, its effects on privacy, fairness and human rights, and how to evaluate and override its output. | As Level III. |
| Approval to operateDADM 6.3.14, Appendix C | The senior official responsible for the programme (in Canada, the assistant deputy minister). | As Level I. | The head of the agency (in Canada, the deputy head). | The central approval body (in Canada, the Treasury Board). |
| Testing before productionDADM 6.3.1, 6.3.3, 6.3.5 | Test data and model for accuracy, unintended bias and anything that could unfairly affect outcomes or breach human rights, and validate that training and input data are relevant, accurate and current. | Test data and model for accuracy, unintended bias and anything that could unfairly affect outcomes or breach human rights, and validate that training and input data are relevant, accurate and current. | Test data and model for accuracy, unintended bias and anything that could unfairly affect outcomes or breach human rights, and validate that training and input data are relevant, accurate and current. | Test data and model for accuracy, unintended bias and anything that could unfairly affect outcomes or breach human rights, and validate that training and input data are relevant, accurate and current. |
| Outcome monitoringDADM 6.3.2, 6.3.4 | Monitor outcomes on a schedule for unintended or unfair results; record client feedback, unexpected impacts and human overrides, and act on them. | Monitor outcomes on a schedule for unintended or unfair results; record client feedback, unexpected impacts and human overrides, and act on them. | Monitor outcomes on a schedule for unintended or unfair results; record client feedback, unexpected impacts and human overrides, and act on them. | Monitor outcomes on a schedule for unintended or unfair results; record client feedback, unexpected impacts and human overrides, and act on them. |
| RecourseDADM 6.4.1 | Tell people how to challenge the decision, through options that are timely, effective and easy to access. | Tell people how to challenge the decision, through options that are timely, effective and easy to access. | Tell people how to challenge the decision, through options that are timely, effective and easy to access. | Tell people how to challenge the decision, through options that are timely, effective and easy to access. |
| PublicationDADM 6.1.1, 6.1.3, 6.5.1, 6.5.2 | Publish the final impact assessment before production, update it when scope or function changes, and publish information on effectiveness and fairness. | Publish the final impact assessment before production, update it when scope or function changes, and publish information on effectiveness and fairness. | Publish the final impact assessment before production, update it when scope or function changes, and publish information on effectiveness and fairness. | Publish the final impact assessment before production, update it when scope or function changes, and publish information on effectiveness and fairness. |
| Access to componentsDADM 6.2.4 to 6.2.6 | Keep every released version of the software, and under a proprietary licence keep the right to access, test and monitor it and to authorise external reviewers. | Keep every released version of the software, and under a proprietary licence keep the right to access, test and monitor it and to authorise external reviewers. | Keep every released version of the software, and under a proprietary licence keep the right to access, test and monitor it and to authorise external reviewers. | Keep every released version of the software, and under a proprietary licence keep the right to access, test and monitor it and to authorise external reviewers. |
Show the full framework cross-walk ▸
| Theme | Canada DADM | Australia | UK ATRS | Singapore | OECD |
|---|---|---|---|---|---|
| Role in the decision | Scope 5.1 (administrative decision or related assessment); AIA "Decision" and "System" areas; human involvement 6.3.13 | Appendix C in-scope criterion: the use case may materially influence administrative decisions | Scope: tools with a significant influence on a decision-making process with public effect; Tier 2 Deployment context, integration into the broader operational process | Paras 3.14 to 3.15: human-in, over or out of the loop, chosen by probability and severity of harm | Principle 1.2: mechanisms for human agency and oversight |
| Rights and interests affected | Appendix B: impacts on rights, equality, dignity, privacy, health and economic interests; AIA "Impact" area | Appendix C: potential harm that is more than insignificant; impact assessment tool section 3, inherent risk assessment | Tier 2 Description and rationale (benefits, previous process); Risks, mitigations and impact assessments | Para 3.15: severity of harm to the individual as one axis of the matrix | Principle 1.2: human rights and democratic values, including fairness and privacy |
| Reversibility | Appendix B: reversible and brief (Level I) through irreversible and perpetual (Level IV) | Impact assessment tool section 3: consequence rating in the likelihood and consequence matrix | Tier 2 Risks, mitigations and impact assessments | Para 3.47: disclose how a decision may affect the person and whether it is reversible | Principle 1.4: robustness, security and safety, with mechanisms to override or decommission |
| Scale | AIA "Impact" area (number of clients affected) | Impact assessment tool section 3: likelihood and consequence rating; Appendix C, significant impact on the public without human review | Tier 2 Description and rationale (detailed description of the tool and its use) | Para 3.15: probability of harm, which rises with the number of decisions | Principle 1.1: inclusive growth and well-being |
| Vulnerable groups | Appendix B: identity factors of the clients; 6.3.8 Gender-based Analysis Plus | Impact assessment tool section 5, fairness; section 10, human-centred values | Tier 2 Development and operational data specifications (sensitive attributes); Risks, mitigations and impact assessments | Para 3.23: minimising inherent bias in datasets | Principle 1.2: non-discrimination and equality, diversity and fairness |
| Data sources | 6.3.5 data quality; 6.3.6 data governance; AIA "Data" area | Appendix C: personal or sensitive information under the Privacy Act, or security-classified information; tool section 7, privacy protection and security | Tier 2 Tool specification (system-level input); Development data and Operational data specifications | Paras 3.22 to 3.24: data lineage, quality and bias | Principle 1.2: privacy and data protection; 1.5 traceability of datasets |
| Technique | AIA "Algorithm" area; 6.3.10 to 6.3.11 security and model integrity | Impact assessment tool section 6, reliability and safety; section 8, transparency and explainability | Tier 2 Tool specification (system architecture) and Model specification (model architecture, performance) | Paras 3.26 to 3.30: explainability, and what to do where it cannot practicably be achieved | Principle 1.3: transparency and explainability |
| Procurement and vendor access | 6.2.4 to 6.2.6: licences, released versions, and the right to access, test and monitor proprietary components | AI procurement guidance (recommended, not mandatory); tool section 11, accountability | Tier 2 Owner and responsibility: third party involvement, procurement procedure type, third party data access terms | Paras 3.41 to 3.44: auditability, including by external parties | Principle 1.5: accountability of AI actors across the life cycle |
| Explanation to the citizen | 6.2.3 and Appendix C: general explanation at Level I; decision-specific explanation for denials at Level II and above | Impact assessment tool section 8, transparency and explainability | Tier 1 summary information in clear and simple language for the public | Para 3.48: a policy on what explanations to give and when | Principle 1.3: plain information on the factors and logic behind an output |
| Human review | 6.3.13 and Appendix C: the final decision is made by a human at Levels III and IV | Appendix C: direct public interaction or significant impact without human review; tool section 11, accountability | Tier 2 Deployment context: how the tool output is used in the wider operational process | Para 3.14: human-in-the-loop means no decision without an affirmative human action | Principle 1.2: human agency and oversight |
| Appeal and recourse | 6.4.1: recourse options that are timely, effective and easy to access | Impact assessment tool section 9, contestability; public reporting pathways under "operationalise the responsible use of AI" | Tier 2 Deployment context: appeals and review | Para 3.54: feedback channels and review of material AI decisions by a human | Principle 1.3: information that lets those adversely affected challenge the output |
| Transparency and testing | 6.1.1 publish the AIA; 6.3.1 to 6.3.4 testing and monitoring; 6.3.7 peer review; 6.5 reporting | AI transparency statement, reviewed at least annually; internal use case register shared with the DTA every 6 months; high-risk use cases reported to the DTA | Publication of the ATRS record (Tier 1 and Tier 2) on GOV.UK; Model specification, model performance | Paras 3.46 general disclosure; 3.31 to 3.35 robustness and testing | Principles 1.3 transparency and 1.4 robustness, security and safety |
- Canada DADM: Directive on Automated Decision-Making and AIA tool (Treasury Board of Canada Secretariat, 2025-06-24)
- Australia: Policy for the responsible use of AI in government, version 2.0 (Digital Transformation Agency, effective 15 December 2025)
- UK ATRS: Algorithmic Transparency Recording Standard, template version 4 (May 2025), mandatory for government departments and for arm's-length bodies that deliver public or frontline services
- Singapore: Model AI Governance Framework, second edition (IMDA and PDPC, January 2020); GovTech Responsible AI Playbook for Whole-of-Government projects
- OECD: OECD AI Principles (Recommendation of the Council on AI, OECD/LEGAL/0449, adopted 2019, updated May 2024)
Sources
Every reference in the tool was checked against these primary documents in October 2026. Singapore's internal instructions for agencies are not published, so its column uses the published Model AI Governance Framework.
| Document | Issued by |
|---|---|
| Directive on Automated Decision-Making (date modified 2025-06-24) | Treasury Board of Canada Secretariat Source → |
| Algorithmic Impact Assessment tool (scoring and impact levels) | Government of Canada Source → |
| Policy for the responsible use of AI in government, version 2.0 | Digital Transformation Agency, Australia Source → |
| Policy for the responsible use of AI in government: AI use case impact assessment | Digital Transformation Agency, Australia Source → |
| Policy for the responsible use of AI in government 2.0 (PDF) | Digital Transformation Agency, Australia Source → |
| AI impact assessment tool (12 sections) | Digital Transformation Agency, Australia Source → |
| Algorithmic transparency recording standard template | Department for Science, Innovation and Technology, UK Source → |
| Algorithmic Transparency Recording Standard: guidance for public sector bodies | Department for Science, Innovation and Technology, UK Source → |
| ATRS mandatory scope and exemptions policy | Department for Science, Innovation and Technology, UK Source → |
| Model AI Governance Framework, second edition | IMDA and PDPC, Singapore Source → |
| Responsible AI Playbook (Whole-of-Government AI projects) | GovTech Singapore Source → |
| OECD AI Principles (updated May 2024) | OECD Source → |
For an AI agent that acts across systems, rate its oversight tier with the Agent Risk Assessment Matrix. For the management system around your AI portfolio, use the ISO 42001 AIMS Readiness Checklist. My work with agencies is described on the government page.
From screening to approval
The screening tells you which level to plan for. Completing the statutory assessment, arranging peer review and writing the published explanation is work I do directly with agencies.
Book a private session
I built this after sitting in several approval meetings where the first question about an automated decision system was what level it was, and nobody had worked it out. Canada's method is the clearest I know, so I borrowed its levels and safeguards and mapped them to the frameworks my clients in Singapore, Malaysia, the Gulf, Australia and the UK actually answer to. Run it early, while the design can still change.
Last updated 25 Oct 2026
More in Board and governance
- Board AI Oversight ChecklistIs our board actually overseeing AI, or just receiving updates about it?
- Technology Risk Board PackCould our bank answer the questions MAS put to DBS in 2023?
- ISO 42001 AIMS Readiness ChecklistHow far are we from passing an ISO/IEC 42001 audit?
- ISO/IEC 42001 Clause Readiness CalculatorWhich clauses would an auditor grade as a nonconformity, and are we ready to book Stage 1?
- EU AI Act Risk ClassifierDoes the EU AI Act apply to this AI system, and what does it require of us?
- Agent Risk Assessment MatrixHow much oversight does each of our AI agents need?
- Oversight Ceiling CalculatorHow much AI output can go out unreviewed before our quality promise breaks?
