False positives in guardrails

Guardrail false positives are harmless prompts, responses or tool calls that a guardrail blocks or flags by mistake. A stricter policy catches more violations and also stops more legitimate traffic, so the rate has to be measured and tuned.

Precision and recall

Precision is the share of flags that are real violations. False positives lower it. Recall is the share of real violations that get flagged. Misses lower it. Raising one tends to lower the other.

Violations are rare, so a small error rate goes far. If 1 request in 1,000 breaks a policy and the check wrongly flags 1 harmless request in 100, it raises about ten false flags for every real violation.

Why they matter

Each one is a user refused for a reasonable request, or an agent stopped mid-task. Users rephrase or leave. Teams stop reading the flags, then switch the policy off, and it protects nothing.

How to reduce them

  • Backtest. Run a changed policy over past traffic before it goes live, and read what it flags. See policy backtesting.
  • Scope. Apply a policy only where it belongs: one feature, one event type, one user segment.
  • Review. Flag uncertain cases for a person instead of blocking them. Their rulings show where the policy is wrong.

With White Circle

Improve backtests a new policy against historical traffic and scopes policies per feature, use case, event type or user segment. Analyze has a human review queue for edge cases.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL