Traditional application security has a clean boundary: code is trusted, data is not, and the job is to keep data from being executed as code. LLM applications erase that boundary. The model can't reliably tell an instruction from its developer apart from an instruction embedded in a PDF a user uploaded, and the same natural language that makes the product useful makes it programmable by anyone who can get text in front of it.
This isn't theoretical. Every team we've worked with that connected a model to real data and real tools has been probed within weeks. Here's the threat model we use and the defences that have held up.
Prompt injection is the SQL injection of this decade
Direct injection is the user typing "ignore your instructions and…"; annoying, mostly contained by good system prompts and output checks. Indirect injection is the dangerous one: instructions hidden in content the model processes on the user's behalf — a support ticket, an email, a résumé, a web page fetched by a research agent, a code comment in a pull request. The user is innocent; the document is the attacker. Any system that reads untrusted content and can then take an action or reveal data is exposed. Assume the model will be manipulated, and design so that manipulation can't cause harm.
Treat retrieved and fetched content as untrusted input
Mark the origin of every piece of context the model sees and tell it explicitly which parts are data, not instructions. Strip or neutralise instruction-like content from documents before they reach the prompt where feasible. Scan retrieved content and tool outputs for injection patterns. None of these is sufficient alone — models can still be talked around — which is why they're the first layer, not the only one.
Least privilege for tools, always
The blast radius of a manipulated model is exactly the set of actions it can take. Give each agent or feature only the tools its job needs, scoped to the current user's permissions — never a service account that can see everything. Separate read tools from write tools, and gate anything irreversible (sending, paying, deleting, deploying) behind human approval, at least until the evaluation record earns otherwise. If the model has no tool that can exfiltrate data, an injection telling it to exfiltrate data fails by construction. This is the same discipline we describe for production agents — security is one more reason it matters.
Close the exfiltration channels
The classic data-leak chain is: injected instruction → model encodes secret data into a URL → app renders a markdown image or link → the browser fetches it and the attacker's server logs the secret. Defend structurally: don't auto-render external images or links from model output, allow-list outbound domains for any tool that makes HTTP requests, and never put secrets, other users' data or full system prompts anywhere the model can echo them. Assume anything in the context window can end up in the output.
Guard the output, not just the input
Input filters catch the clumsy attacks; output checks catch the rest. Validate model output against a schema before acting on it. Run PII and secret detectors on responses. Verify that any citation or claimed action actually corresponds to retrieved data or a real tool result. For high-stakes flows, use a second, independent model call purely as a checker — a different prompt with a narrow job ("does this response leak data or violate policy?") is much harder to manipulate in the same breath as the first.
Red-team before your users do
Build an adversarial test suite alongside your evaluation suite: injection payloads in documents, jailbreak attempts, requests for other users' data, tool-misuse scenarios, and the OWASP Top 10 for LLM Applications as a checklist. Run it in CI on every prompt, model or tool change — a model upgrade can silently change how your app responds to an attack. Then log everything in production: full traces of prompts, retrieved context, tool calls and outputs, so when something does get through you can see exactly how.
The short version
Secure LLM applications assume the model can be manipulated and make sure that manipulation goes nowhere: untrusted content is labelled and filtered, tools are least-privilege and user-scoped, irreversible actions need approval, exfiltration channels are closed structurally, outputs are validated independently, and an adversarial suite runs on every change. Defence in depth isn't a slogan here; it's the only approach that has held.
Shipping an AI feature that touches real data or takes real actions? Talk to us before an attacker does the review for you.