Continuous red teaming

Continuous red teaming is running attack suites against an AI system on every deploy and on a schedule, not once before launch. It tracks regressions, so a fixed hole that reopens is caught.

Why once is not enough

A red team report describes the system on the day it was tested. After that, things move:

  • The model. A new version, or a hosted model that its provider updates, behaves differently.
  • Your app. A reworded system prompt, a new tool or a new data source can reopen a hole that was closed.
  • The attacks. New jailbreak and prompt injection techniques keep appearing.

How it works

  1. Keep a suite of attacks for each risk: jailbreaks, leaks, tool misuse.
  2. Run it automatically on every deploy, and on a schedule between deploys.
  3. Compare each run with the last. An attack that failed before and succeeds now is a regression.
  4. Fix what is found, and add every new finding to the suite.

How to red team an LLM covers building the suite.

With White Circle

Red Team runs one-off campaigns or continuous testing, with regressions tracked over time. Findings can be patched through Control in one click, and we recommend testing on every deploy.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL