What is AI incident response?

AI incident response is what a team does when an AI system causes or nearly causes harm: contain it, work out what happened from the logs, fix the cause, and change policies so it does not recur.

What counts as an incident

  • An agent deleted, sent or changed something it should not have.
  • A response leaked personal data or another customer's information.
  • A jailbreak that works is being shared.
  • The model gave advice that caused harm.
  • Usage spiked because someone found a way to burn tokens.

Steps

  1. Contain. Turn off the tool, tighten a policy to block, suspend the account, or take the feature down. Speed matters more than elegance.
  2. Preserve. Keep the full record: prompts, responses, tool calls, verdicts, versions of the model and prompt. See audit logs.
  3. Reconstruct. Follow the run step by step to the point where it went wrong: see agent tracing.
  4. Find the cause. An injected instruction, a missing policy, too much access, a model change.
  5. Fix and verify. Add or change the policy, then replay the incident to check it would now be stopped.
  6. Look for others. Search past traffic for the same pattern.
  7. Report. Tell affected users and, where the law requires, the regulator.

What makes it different from a security incident

The cause is often not a bug. The system did what it could, and nobody had said it must not. So the fix is usually a rule and a check, not a patch.

With White Circle

Analyze shows what happened, why, and which policies fired. A new policy in Control applies at once, and Red Team can test that the gap is closed.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL