# Training Methods

Each method is one script under `scripts/training/` and one name you pass to `halo launch`. Every
method reads a YAML config, takes a HuggingFace checkpoint and a dataset, and writes a HuggingFace
checkpoint. If you are still deciding, [Choosing a Method](https://whitecircle.com/halo/docs/choosing-a-method.md) maps the data you
have to the trainer that reads it.

| Method | Trains on | `halo launch` | Page |
| --- | --- | --- | --- |
| SFT | conversations, text or image | `sft` | [Supervised fine-tuning](https://whitecircle.com/halo/docs/training-methods/sft.md) |
| Pre-training | raw text, or a random-init model | `sft` | [Supervised fine-tuning](https://whitecircle.com/halo/docs/training-methods/sft.md#run) |
| SMPO | `prompt` / `chosen` / `rejected`, no reference model | `smpo` | [Preference](https://whitecircle.com/halo/docs/training-methods/preference.md) |
| DPO | `prompt` / `chosen` / `rejected`, against a reference | `dpo` | [Preference](https://whitecircle.com/halo/docs/training-methods/preference.md) |
| KTO | `prompt` / `completion` / boolean `label` | `kto` | [Preference](https://whitecircle.com/halo/docs/training-methods/preference.md) |
| Reward modeling | preference pairs, output is a scalar scorer | `reward/reward-model` | [Reward & classification](https://whitecircle.com/halo/docs/training-methods/reward-and-classification.md) |
| Classification | text plus a single- or multi-label `label` | `classification` | [Reward & classification](https://whitecircle.com/halo/docs/training-methods/reward-and-classification.md) |
| Offline GRPO | prompts with completions you already scored | `grpo/offline` | [Offline GRPO](https://whitecircle.com/halo/docs/training-methods/offline-grpo.md) |
| Online GRPO (RLVR) | prompts with a verifiable answer, generated live | `grpo/online` | [Online GRPO](https://whitecircle.com/halo/docs/training-methods/online-grpo.md) |
| Async GRPO with Environments | multi-turn, tool-using trajectories | `grpo/environmental` | [Async GRPO](https://whitecircle.com/halo/docs/training-methods/async-grpo-environments.md) |
| Teacher distillation | conversations, plus a second frozen model | `teacher-distill` | [Distillation](https://whitecircle.com/halo/docs/training-methods/distillation.md) |
| Self-distillation | conversations with gold answers, one model | `self-distill` | [Distillation](https://whitecircle.com/halo/docs/training-methods/distillation.md) |
| Online SDPG | prompts with an answer, generated live | `grpo/online --use_sdpg=true` | [Distillation](https://whitecircle.com/halo/docs/training-methods/distillation.md#online-sdpg) |
| Embedding | text pairs, triplets or scored pairs | `embedding` | [Embedding](https://whitecircle.com/halo/docs/training-methods/embedding.md) |

## What every method shares

- **Config.** A config is a flat YAML file of trainer fields, any of which can be
  overridden on the command line after the config path ([Writing a config](https://whitecircle.com/halo/docs/configuration.md)).
- **Data.** Datasets come from the Hub, a local path or `s3://` through the same `dataset:`
  field, and a list mixes them ([Datasets](https://whitecircle.com/halo/docs/data.md)). The columns each method reads differ.
- **Parallelism.** Expert, tensor and expert-tensor parallelism work on every trainer;
  which ones also take context or pipeline parallelism is in the
  [Supported matrix](https://whitecircle.com/halo/docs/supported-matrix.md#training-methods).
- **Checkpoints.** They load with `from_pretrained` and upload to the Hub
  without their training state ([Uploading](https://whitecircle.com/halo/docs/checkpoints.md#uploading-to-the-huggingface-hub)).
  Some MoE families need a conversion before vLLM or SGLang can serve them. A LoRA run
  writes an adapter you merge with one command; expert LoRA under EP folds into the weights at save
  time instead (`merge_expert_lora_on_save: true`). See [Checkpoints & Export](https://whitecircle.com/halo/docs/checkpoints.md).

Every hyperparameter and refusal per method is in the
[training-methods reference](https://whitecircle.com/halo/docs/agent-docs/training-methods.md) ↗.
