Memory Poisoning Defence
Prevent adversarial or low-quality inputs from corrupting long-term memory where they later influence reasoning.
This page is the complete instruction page for one pattern called "Memory Poisoning Defence." It explains the problem this pattern solves, the idea behind the solution, when you should (and should not) use it, and what happens afterward, both the good effects and the costs.
This matters because building AI agents is not just about making them clever. It is also about making them safe and predictable. Following a well-tested pattern like this one helps avoid common mistakes, and shows you exactly which safety rules and regulations it connects to, listed under "Standards Mesh" on this page.
Context
Agents that write tool outputs or user content to persistent memory create a channel for injection: a malicious document seen once can shape agent behaviour forever.
Problem
Writable memory without provenance or validation is an indirect prompt-injection vector.
Forces
- Learning from interaction vs resistance to adversarial input
- Automated ingestion vs curated knowledge
Solution
Gate memory writes through validation: provenance tagging, content scanning, trust-level assignment by source. On retrieval, weight by trust level; quarantine untrusted memories and surface them only with flags. Support memory rollback to a known-good checkpoint.
Where this pattern acts in the two lifecycles
04 Act. Tool call, API invocation, environment mutation. 15 of 61 catalogue patterns act at this stage.
05 Operate & Monitor. Runtime telemetry, drift detection. 18 of 61 catalogue patterns act at this stage.
Stages and counts from the catalogue data. The Runtime lens is the loop the agent runs each turn; the AgentOps lens is the lifecycle the team runs the agent through. See the Matrix for every pattern at each stage.
Applicability
- Agents that browse the web or read uploaded documents
- Multi-tenant agents
- Agents persisting user inputs
Anti-Patterns
- Storing anything the agent reads without provenance
- Treating retrieved memory as ground truth
Consequences
- +Resistance to indirect injection
- +Clean rollback path
- −Write-path complexity
- −Trust model must be maintained
