# Online distillation (SDPG)

Online distillation (SDPG) in Halo is the on-policy form of self-distillation. The model generates answers, earns verifiable rewards for them, and is also distilled toward its own answer-hinted forward pass.

## How it works

Elsewhere the term can also mean a teacher and a student trained at the same time. This page is about Halo's method, SDPG.

SDPG is [online GRPO](/glossary/online-grpo) (Group Relative Policy Optimization) plus the teacher term of [self-distillation](/glossary/self-distillation). Each round:

1. The model generates [rollouts](/glossary/rollouts) on a separate inference server.
2. The rollouts earn verifiable rewards, checked against the dataset's answer, and GRPO turns them into advantages.
3. The same weights run again on the prompt plus a hint that reveals the answer. The model is distilled toward that hinted forward pass.

The training text is on-policy: generated live by the current weights, not read from a fixed dataset.

## When to use it

Reach for it when the model solves too few prompts for GRPO's group signal to say anything on its own.

## In Halo

SDPG is a switch on the online GRPO trainer. Everything that trainer needs applies unchanged: a running [rollout server](/glossary/rollout-server), weight sync, and a dataset with `prompt` and `answer`. No shipped config turns it on. Start from the template and flip the gate, here with the trainer on GPUs 0-6 and the server holding GPU 7:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6 halo launch rlvr \
    examples/grpo/online/rlvr-online-grpo-template.yaml -n 7 \
    --use_sdpg=true --sdpg_beta_base=1.0
```

SDPG runs two extra full-vocabulary forwards per micro-batch, so it runs out of memory at batch sizes online GRPO survives. See the [distillation guide](/halo/docs/training-methods/distillation#online-sdpg).

