# How to red team an LLM

To red team a large language model (LLM), define what must not happen, build a set of attacks for each risk, run them against the whole system, score the outcomes, fix what failed and rerun on every deploy.

## Steps

1. **Define what must not happen.** Write each failure as a concrete outcome: the system prompt is revealed, another user's data appears, the agent sends an email nobody asked for.
2. **Build an attack set per risk.** Collect prompts that try to cause each failure: [jailbreaks](/glossary/jailbreak), direct and [indirect prompt injection](/glossary/indirect-prompt-injection), requests for data, long multi-turn conversations. Add variants in other languages and encodings.
3. **Run them on the whole system.** Test the app as deployed, with its system prompt, retrieval, tools and guardrails. A model that refuses a request on its own can still fail when the same request arrives inside a retrieved document.
4. **Score the outcomes.** Record whether the failure happened, not whether the reply sounded like a refusal. Use rules or a judge model for volume, and check a sample by hand. Report the attack success rate per risk, with the inputs that worked.
5. **Fix.** Add or tighten a [guardrail policy](/glossary/guardrail-policy), change the prompt, or remove a permission. Then rerun the attacks that got through.
6. **Rerun on every deploy.** Keep the attack set as a regression suite. A new model version, prompt or tool can reopen a hole that was closed. See [continuous red teaming](/glossary/continuous-red-teaming).

## With White Circle

[Red Team](/products/red-team) simulates real-world attacks on models, retrieval-augmented generation (RAG) pipelines, tool-using setups and agents, in one-off campaigns or continuously with regressions tracked over time. Findings can be patched through Control in one click.

