A note is added only when the failure has a record behind it: a commit, a log or a measured number. The systems are this site’s reading assistant, its retrieval index and the model routes behind them. Client systems are never the source, and suppliers are described by role rather than name. Where a note bears on a component of the Governed Agentic RAG reference architecture, the component is named.
| Note | Failure | ISO/IEC 42001 |
|---|
| FN-01 | The model provider refused requests from one edge region | A.10.3, A.4.5, A.6.2.6 |
|---|
| FN-02 | A retrieval limit made the assistant confidently incomplete | A.7.4, A.6.2.4, 9.1 |
|---|
| FN-03 | A local test run wrote into the production index | A.7.5, A.6.2.5, 8.1 |
|---|
| FN-04 | Limits stated in the prompt were not followed | A.6.2.2, A.6.2.4 |
|---|
| FN-05 | An open model endpoint was abused and the spending limit took the service down | A.9.4, A.4.5, 6.1.2, 9.1 |
|---|
The model provider refused requests from one edge region
Public reading assistant calling a hosted language model from edge functions
- Context
- The site runs on an edge platform, so each request is handled at the location nearest the reader and the model call leaves from there. Nothing in the application chooses that location.
- What failed
- Between 30 and 50 per cent of assistant requests in testing failed with a generic "request not allowed" response. The failures looked random: the same question would work, then fail, then work.
- How it was found
- Readers saw "answers are temporarily unavailable". The function logs, read with the edge location beside each response, showed every refusal coming from one location.
- Root cause
- The model provider does not accept traffic leaving from that region, and part of the traffic from Singapore was being routed through it. The restriction sits in the provider’s policy, outside anything the application controls or logs by default.
- What did not work
- Routing calls through the platform’s own AI gateway. The gateway forwards from the same location, so the refusals continued.
- Controls added
- Every model call now goes through one stateful proxy pinned to a North American region, so the egress location is fixed and recorded.
- The proxy allows two destinations only: the provider’s API and the gateway.
- The deploy script ships the proxy before the site, so a deploy cannot leave the site pointing at a proxy that does not exist yet.
- ISO/IEC 42001
- A.10.3 Suppliers; A.4.5 System and computing resources; A.6.2.6 AI system operation and monitoring
- Residual risk
- The assistant now depends on one region. If that region fails, the assistant fails with it. That is acceptable for a reading aid on a public site and would not be for a system that makes decisions.
Where a request leaves from is part of the system specification. Record it as a resource, and treat the provider’s region policy as a supplier term to monitor.
Fixed in 8d12b65c
FN-02 · 16 September 2026A retrieval limit made the assistant confidently incomplete
Page-grounded assistant answering from the article or tool page the reader is on
- Context
- Answers are grounded in passages retrieved from the reader’s current page first, then from the rest of the site. Answers to the suggested questions are written in advance and stored.
- What failed
- A question about an article’s nine failure patterns got an answer covering four, which said the page only described four. Between 5 and 10 per cent of stored answers referred to "the passages provided" instead of the page. Short, structured sections (context, forces, consequences) were missing from answers altogether.
- How it was found
- Probing the stored answers against the live pages they came from.
- Root cause
- Retrieval from the reader’s own page was capped at 5 passages. The median page has 7, and 141 of 225 pages have more than 5. Separately, the indexer dropped any section under 20 words, which removed the short sections of structured pages from the index entirely.
- Controls added
- Own-page retrieval raised to 10 passages, plus 4 from the rest of the site.
- Short sections folded into heading-prefixed passages instead of being dropped (2,020 passages became 2,164).
- A deterministic check on every stored answer flags openings that disclaim the page; flagged answers are rewritten and any that remain are listed at deploy. After three passes, only pages not yet published remained flagged.
- ISO/IEC 42001
- A.7.4 Quality of data for AI systems; A.6.2.4 AI system verification and validation; 9.1 Monitoring, measurement, analysis and evaluation
- Residual risk
- Coverage is now checked against the stored answers, not against every question a reader might type.
A retrieval limit is a parameter to test against the shape of the corpus, not a default to accept. The evaluation that found this compared answers with their source; a model-quality score would not have.
Fixed in 453ef849, 0a6bc527
FN-03 · 15 September 2026A local test run wrote into the production index
Search and retrieval index behind the assistant
- Context
- The indexer can run against a local copy of the site for testing. The index itself is a single hosted store.
- What failed
- A test run against the local site wrote local-machine addresses into the live index metadata. For about two minutes, answers on the public site cited links that pointed at a developer’s machine.
- How it was found
- Seen in live answers and reverted within about two minutes.
- Root cause
- The indexer used one setting for two different things: the address it read pages from, and the address it recorded as each page’s canonical location. Pointing it at a local copy changed both.
- Controls added
- The indexer now keeps the address it reads from separate from the canonical origin it records, and the canonical origin is fixed to the production site.
- Answer code matches retrieved passages against the canonical origin, never against the address of the incoming request.
- ISO/IEC 42001
- A.7.5 Data provenance; A.6.2.5 AI system deployment; 8.1 Operational planning and control
- Residual risk
- The test and production indexes are still one store; the separation is in the code path, not in the infrastructure.
Provenance includes where a record says it came from. A test run that can write production metadata is a release that skipped the release process.
Fixed in 790ffd92
FN-04 · 16 September 2026Limits stated in the prompt were not followed
Generated suggested questions and answers, under a house style
- Context
- A small, fast model writes the suggested questions for each page and the answers to them, under written length and style rules.
- What failed
- The model ignored the character limit for suggested questions. Answers drifted into patterns the house style bans. Questions with a number in them ("which three") drew answers that argued about the count before answering.
- How it was found
- Review of the generated questions and answers before release.
- Root cause
- The limits existed only as instructions in the prompt. Nothing checked them after the model produced its output.
- Controls added
- Question length is enforced in code with a rewrite loop (72 characters or fewer).
- Every answer passes a style check in code; a failure triggers one rewrite.
- Questions are phrased without counts, and the prompt asks for the substance before any qualification.
- ISO/IEC 42001
- A.6.2.2 AI system requirements and specification; A.6.2.4 AI system verification and validation
- Residual risk
- The style check catches the patterns it was written for. New patterns still need a human to notice them first.
A requirement stated in a prompt is a request. If it matters, test it in code after the model and act on the result.
Fixed in 790ffd92, 0a6bc527
FN-05 · 29 September 2026An open model endpoint was abused and the spending limit took the service down
Public assistant and AI search with a free-text box and no sign-in
- Context
- Anyone could type a question into the assistant or the search box, and each one was a paid model call.
- What failed
- The provider account reached its usage limit after traffic that looked like abuse of the open routes. Live answers stopped, and the next deploy failed to write 212 of its pre-written answers until the limit reset.
- How it was found
- The deploy log showed the failed answers, and the provider’s error message gave the date access would return.
- Root cause
- The routes were open to anyone and every question was a paid call. Spend was not measured per route, so there was no early sign before the limit was reached.
- Controls added
- Search moved to a static index with no server-side model call.
- The assistant came back first with fixed questions and checked, pre-written answers only, then with live answers on the cheapest model tier alongside them. Voice input stayed off.
- Every model call across the site and the demos is now recorded in a usage ledger: route, model, tokens and cost.
- ISO/IEC 42001
- A.9.4 Intended use of the AI system; A.4.5 System and computing resources; 6.1.2 AI risk assessment; 9.1 Monitoring, measurement, analysis and evaluation
- Residual risk
- The ledger makes spend visible per route. It does not by itself stop abuse.
On a public model route, cost is a safety property. Running out of budget is an availability incident and belongs on the risk register.
Fixed in d6fe4c6d, 53ce0642, 3312a0bc, f7dae33f
Drift in agentic workflows after a model change, and data isolation between tenants, have no note yet: neither has failed in these systems with a record to show for it. The controls designed against them are in the reference architecture’s risk matrix. Practitioner guidance on the same ground: the ISO/IEC 42001 Clause Readiness Calculator and the readiness assessment.
