How to prevent prompt injection

To prevent prompt injection, treat everything the model reads as untrusted, scan it for injected instructions, check each tool call before it runs, and limit what the agent can do. No single measure is complete.

Steps

  1. Treat all content as untrusted. User messages, web pages, emails, files, retrieved documents and tool results can all carry instructions. See indirect prompt injection.
  2. Scan inputs and retrieved content. Check each piece for injected instructions before the model acts on it.
  3. Check tool calls before they run. Judge the call and its arguments against policy. An injection that got past the scan can still be stopped here. See guardrails for tool calls.
  4. Give least privilege. Grant the agent only the tools, data and permissions its task needs. What it cannot do, an attacker cannot make it do. See excessive agency.
  5. Require human approval for risky actions. Hold what cannot be undone (sending, paying, deleting) until a person confirms. See human-in-the-loop approval.
  6. Separate data from instructions where you can. Keep untrusted text out of the system prompt, mark where it starts and ends, and tell the model it is data. This helps, but the model still reads it all as text.
  7. Red team. Attack your own system with known injection techniques, and again after each change. See how to red team an LLM.

What to expect

None of these is enough alone. A scanner can miss a new phrasing and no prompt makes a model immune, so use the steps together: what the scan misses can be stopped at the tool call, and least privilege limits the rest.

With White Circle

Control checks inputs, outputs and tool calls against your policies in real time, and blocks injected instructions before they reach a tool. A policy can also hold a tool call for human confirmation.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL