What is an LLM jailbreak?

Jailbreaks are prompts that get a large language model (LLM) to break its own rules and write what it was trained or told to refuse. Role play, encoding and pressure over many turns are common techniques.

How it works

A model is trained to refuse some requests, and its system prompt adds more rules. A jailbreak asks for the same thing in a form the model was not prepared for, so it follows the request instead of the rule. Common techniques:

  • Role play. The request is framed as fiction, or the model is given a persona "without restrictions".
  • Encoding. The request is written in Base64, in another language or with altered spelling, so it does not match what the model learned to refuse.
  • Many-turn pressure. The attacker starts with harmless questions and escalates a little each turn.

Jailbreak vs prompt injection

A jailbreak targets the model's own rules: the user wants output the model should refuse. Prompt injection targets the application: text overrides the developer's instructions, and it can come from a third party through a document or a tool result.

Defenses

Safety training makes jailbreaks harder, not impossible, and new ones keep appearing. Add guardrails outside the model: screen inputs for jailbreak attempts, and check outputs against policy whatever the prompt looked like. Red team to find the ones that work on your system.

With White Circle

Jailbreaks are among the attacks we cover. Control checks every input and output against your policies in real time.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL