User risk scoring

User risk scoring tracks policy violations per user across sessions. It separates repeat offenders from one-off mistakes, so that a user can be warned, throttled or suspended in proportion.

How it works

A guardrail judges one message or one tool call at a time. Risk scoring keeps the history. Each violation is recorded against the user, not only the session, and adds to a score or a count of strikes. A sound score weights serious violations more and lets old ones expire.

As the score rises, enforcement steps up:

  1. Warn the user.
  2. Throttle their requests.
  3. Suspend the account.

Why it matters

An attacker rarely stops at one try. A jailbreak is often found by sending many variations, each in a fresh session after a block. Judged one at a time, every attempt looks like the first. Judged per user, the pattern shows.

The record also protects honest users. One odd message earns a warning, not a ban, so a false positive costs little.

With White Circle

We track violations per user: strikes flag repeat offenders across sessions, and enforcement (warn, throttle, suspend, or custom) can be automatic. User-level risk scoring is part of Analyze.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL