LLM data leakage

Data leakage is a large language model (LLM) revealing data it should not, such as memorized training data, another user's records, secrets in its context or content from another session.

Where the data comes from

  • Training data. Models memorize some of what they are trained on and can repeat it word for word. A model fine-tuned on support tickets can repeat one customer's details to another.
  • Context. Anything in the prompt can come out in the reply: the system prompt (system prompt leakage), an API key, a retrieved document.
  • Other users. Retrieval that ignores permissions hands the model documents the asker may not see.
  • Other sessions. Memory or caches shared between users can carry one conversation into another.

A leak can be an accident or the result of an attack: prompt injection is a common way to make a model hand data over. Data leakage is LLM02, Sensitive Information Disclosure, in the 2026 OWASP Top 10 for LLM applications.

Controls

  • Keep secrets out of prompts and training data. A model cannot leak what it never saw.
  • Enforce access control at retrieval, with the asking user's permissions.
  • Keep each user's memory and cache separate.
  • Redact personally identifiable information (PII) before the model call (PII redaction).
  • Check outputs for confidential data before they leave.

With White Circle

Control analyzes every output in real time, and leaks of confidential or cross-session data are among what it covers.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL