What are real-time guardrails?

Real-time guardrails check every prompt, response and tool call while the request is in flight, and block, rewrite or hold it before anything reaches the user or a tool. They prevent, where logs only report.

Real time, as opposed to what

  • Offline evaluation scores a model on a test set before release. It says how the model tends to behave.
  • Monitoring reads logs after the fact. It says what happened.
  • Real-time guardrails sit in the request path. They decide what is allowed to happen.

The three do different jobs: see guardrails vs evals. Only the last one stops a leak before the data is out.

Where they sit

  1. Before the model: the user's prompt and any retrieved or fetched content.
  2. After the model: the response, before the user sees it.
  3. Before a tool: each tool call an agent makes, with its arguments.

What makes it hard

A check in the request path has to be fast and right at once. Slow, and the product feels broken: see guardrail latency. Wrong, and it blocks honest users: see false positives. This is why real-time checks run on small purpose-trained guard models and not on a large general model asked to judge.

With White Circle

Control is real-time guardrails: it sits between your users, your model and its tools, checks each step against your policies, and blocks, rewrites or flags what breaks them. A small model gives verdicts in real time, and a larger one confirms them and is used to improve the small one.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL