Distillation

Fit a student to a teacher's token-level output distribution. The three methods differ in where the teacher comes from and whether training is on- or off-policy.

AspectTeacherSelfOnline SDPG
TeacherA separate frozen modelThe student, given a hintThe student, given a hint
PolicyOff-policy, fixed datasetOff-policy, fixed responsesOn-policy, student rollouts
GenerationNoneNonevLLM rollouts
Objectivedistill_alpha·L_distill + (1−distill_alpha)·L_clmL_sft + beta(k)·L_OPD + alpha·L_refL_GRPO + beta(k)·L_OPD
TrainerDistributedTeacherDistillationTrainerDistributedSelfDistillationTrainerDistributedSDPGTrainer
Scriptdistillation/teacher_distill.pydistillation/self_distill.pygrpo/online.py --use_sdpg
ModalityText + VLMText + VLMText only

Which to pick:

All three take EP, TP, ETP, EP+TP and EP+ETP on the student. None takes CP or PP: each needs a second whole-model forward that neither the sequence split nor the stage split can reproduce.

Every divergence, the gold-token gate and the hard-label cross-entropy score the logits one token chunk at a time under a checkpoint (token_divergence in halo/trainers/distillation/divergences.py), so backward holds each forward's logits and no fp32 [tokens, vocab] plane; the peak beyond the models is those logits planes.

Dataset

Teacher distillation and self-distillation read the SFT conversation format under conversation_field — default messages for teacher distillation, prompt for self-distillation, so set it to match your data.

JSON Lines

{"messages": [{"role": "user", "content": "Explain gravity."}, {"role": "assistant", "content": "Gravity is a fundamental force..."}]}

Self-distillation also reads a gold-answer column (sdpg_answer_field, default answer) to build the teacher's hint. Online SDPG reads the online-GRPO {"prompt", "answer"} format, where the answer feeds both the verifier reward and the hint.

Source on GitHubUpdated 18 November 2026