# What is reinforcement fine-tuning?

Reinforcement fine-tuning (RFT) trains a model with a reward in place of gold answers. The model generates answers, a grader scores them, and the update makes high-scoring answers more likely.

## How it works

Each step is a loop:

1. The model generates several answers to each prompt. These are [rollouts](/glossary/rollouts).
2. A grader scores each answer: a rule that checks it, a test suite, a judge model or a reward model.
3. The update makes answers that scored above the rest more likely, and the others less.

[GRPO](/glossary/grpo) (Group Relative Policy Optimization) is a common algorithm for the update.

## When to use it

Use it when you can check an answer but cannot write the ideal one: math, code that must pass tests, structured output, multi-step tasks with tools. Supervised fine-tuning can only imitate its examples. A reward lets the model find answers the examples did not contain.

It costs more. The model generates during training, and it tends to exploit loopholes in the grader. It also needs a model that already gets some answers right: when every answer scores the same, there is nothing to learn.

## In Halo

Halo's reinforcement methods are the GRPO family, each a `halo launch` name:

- [Offline GRPO](/glossary/offline-grpo) (`offline-grpo`) trains on completions you already scored.
- [Online GRPO](/glossary/online-grpo) (`rlvr`) generates single-turn answers and scores them with verifiable rewards.
- [Async GRPO](/glossary/async-grpo) (`environmental-grpo`) runs multi-turn, tool-using episodes in environments.

The last two generate on a separate rollout server. See [training methods](/halo/docs/training-methods).

