A model can’t reliably tell the difference between “this is my instruction from the system” and “this is text I’m reading that happens to contain instructions.” Prompt injection exploits exactly that: hiding a command inside a user message, an uploaded document, or even a webpage the model retrieves, designed to make the model ignore its original instructions and do something else, reveal data it shouldn’t, take an unauthorised action, or bypass a safety rule.
It’s a more serious problem in retrieval and agentic systems than in a simple chatbot, because there the attack doesn’t need to come from the user at all. A poisoned document sitting in a knowledge base, or a malicious instruction buried in a webpage an agent visits, can hijack the system indirectly, without the attacker ever interacting with it directly.
The defence is architectural, not a single filter: treating any retrieved or externally sourced content as untrusted input, separating instructions from data at the system level, and constraining what actions a model can take regardless of what it’s been told to do mid-conversation.