# Online GRPO (RLVR)

Online GRPO is reinforcement learning with verifiable rewards (RLVR). During training the model generates several completions for each prompt, a grader checks each one, and the scores inside each group drive the update.

## How it works

Each training step is one cycle:

1. The current model generates several completions for each prompt.
2. A grader scores each one. The reward is verifiable: a rule checks it, such as a final answer equal to the ground truth or an output in a required format.
3. The rewards inside each group become advantages, as in any [GRPO](/glossary/grpo) (Group Relative Policy Optimization) run.
4. The model takes an optimizer step.

The next round of [rollouts](/glossary/rollouts) comes from the new weights, which makes the method on-policy.

## When to use it

It fits tasks with one prompt, one answer and one turn: math, structured output, anything a grader can check. Multi-turn tool use belongs to [async GRPO](/glossary/async-grpo), and data that is already scored to [offline GRPO](/glossary/offline-grpo).

## In Halo

The launch name is `rlvr`. Generation runs on a separate vLLM container, the [rollout server](/glossary/rollout-server), which receives the updated weights before each round and must own GPUs no trainer process uses. Start the server, wait for `/health`, then launch the trainer on the other GPUs:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6 halo launch rlvr \
    examples/grpo/online/rlvr-online-grpo-template.yaml -n 7
```

The data has two columns, `prompt` and `answer`. `num_generations` is the group size: below about 4 a group's rewards are too often all equal, and an all-equal group produces no gradient. See the [online GRPO guide](/halo/docs/training-methods/online-grpo).

