# Distillation

Fit a student to a teacher's token-level output distribution. The three methods differ in where the teacher comes from and whether training is on- or off-policy.

| Aspect | [Teacher](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/teacher-distillation.md) | [Self](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/self-distillation.md) | [Online SDPG](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/online-sdpg.md) |
|---|---|---|---|
| Teacher | A separate frozen model | The student, given a hint | The student, given a hint |
| Policy | Off-policy, fixed dataset | Off-policy, fixed responses | On-policy, student rollouts |
| Generation | None | None | vLLM rollouts |
| Objective | `distill_alpha·L_distill + (1−distill_alpha)·L_clm` | `L_sft + beta(k)·L_OPD + alpha·L_ref` | `L_GRPO + beta(k)·L_OPD` |
| Trainer | `DistributedTeacherDistillationTrainer` | `DistributedSelfDistillationTrainer` | `DistributedSDPGTrainer` |
| Script | `distillation/teacher_distill.py` | `distillation/self_distill.py` | `grpo/online.py --use_sdpg` |
| Modality | Text + VLM | Text + VLM | Text only |

Which to pick:

- A separate, stronger teacher is available → [teacher distillation](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/teacher-distillation.md).
- One model, gold answers in the dataset, no generation budget → [self-distillation](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/self-distillation.md).
- One model, a verifier, and a vLLM rollout budget → [online SDPG](https://whitecircle.com/halo/docs/agent-docs/training-methods/distillation/online-sdpg.md).

All three take EP, TP, ETP, EP+TP and EP+ETP on the student. None takes CP or PP: each needs a second whole-model forward that neither the sequence split nor the stage split can reproduce.

Every divergence, the gold-token gate and the hard-label cross-entropy score the logits one token chunk at a time under a checkpoint (`token_divergence` in `halo/trainers/distillation/divergences.py`), so backward holds each forward's logits and no fp32 `[tokens, vocab]` plane; the peak beyond the models is those logits planes.

## Dataset

Teacher distillation and self-distillation read the SFT conversation format under `conversation_field` — default `messages` for teacher distillation, `prompt` for self-distillation, so set it to match your data.

```jsonl
{"messages": [{"role": "user", "content": "Explain gravity."}, {"role": "assistant", "content": "Gravity is a fundamental force..."}]}
```

Self-distillation also reads a gold-answer column (`sdpg_answer_field`, default `answer`) to build the teacher's hint. Online SDPG reads the online-GRPO `{"prompt", "answer"}` format, where the answer feeds both the verifier reward and the hint.
