Data exfiltration by AI agents

Data exfiltration is an AI agent sending private data to someone who should not have it, through a URL, an email or a tool call. It usually starts with a prompt injection in content the agent reads.

How it happens

Three things make it possible: the agent can read private data, it reads content an attacker can write, and it has a way to send data out. Then one injected instruction is enough:

  1. The attacker plants text where the agent will read it: a web page, an email.
  2. The text tells the agent to collect something: a file, an API key, another customer's record.
  3. The agent sends it out: it fetches a URL with the data in the query string, sends an email, or posts through a tool.

The first step is indirect prompt injection. The last needs no tool if the chat window renders images: a Markdown image whose address carries the data sends it without a click.

Controls

  • Egress limits. Allow only the domains the agent needs, and do not render images from any other.
  • Checks on tool arguments. Before a call runs, inspect the recipient, the address and the body. See guardrails for tool calls.
  • Output checks. Scan responses for secrets, personal data and addresses that carry data.

With White Circle

Control checks tool calls before they run and blocks injected instructions before they reach a tool. Its policies cover leaks of confidential or cross-session data.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL