Training Methods

Each method is one script under scripts/training/ and one name you pass to halo launch. Every method reads a YAML config, takes a HuggingFace checkpoint and a dataset, and writes a HuggingFace checkpoint. If you are still deciding, Choosing a Method maps the data you have to the trainer that reads it.

MethodTrains onhalo launchPage
SFTconversations, text or imagesftSupervised fine-tuning
Pre-trainingraw text, or a random-init modelsftSupervised fine-tuning
SMPOprompt / chosen / rejected, no reference modelsmpoPreference
DPOprompt / chosen / rejected, against a referencedpoPreference
KTOprompt / completion / boolean labelktoPreference
Reward modelingpreference pairs, output is a scalar scorerreward/reward-modelReward & classification
Classificationtext plus a single- or multi-label labelclassificationReward & classification
Offline GRPOprompts with completions you already scoredgrpo/offlineOffline GRPO
Online GRPO (RLVR)prompts with a verifiable answer, generated livegrpo/onlineOnline GRPO
Async GRPO with Environmentsmulti-turn, tool-using trajectoriesgrpo/environmentalAsync GRPO
Teacher distillationconversations, plus a second frozen modelteacher-distillDistillation
Self-distillationconversations with gold answers, one modelself-distillDistillation
Online SDPGprompts with an answer, generated livegrpo/online --use_sdpg=trueDistillation
Embeddingtext pairs, triplets or scored pairsembeddingEmbedding

What every method shares

  • Config. A config is a flat YAML file of trainer fields, any of which can be overridden on the command line after the config path (Writing a config).
  • Data. Datasets come from the Hub, a local path or s3:// through the same dataset: field, and a list mixes them (Datasets). The columns each method reads differ.
  • Parallelism. Expert, tensor and expert-tensor parallelism work on every trainer; which ones also take context or pipeline parallelism is in the Supported matrix.
  • Checkpoints. They load with from_pretrained and upload to the Hub without their training state (Uploading). Some MoE families need a conversion before vLLM or SGLang can serve them. A LoRA run writes an adapter you merge with one command; expert LoRA under EP folds into the weights at save time instead (merge_expert_lora_on_save: true). See Checkpoints & Export.

Every hyperparameter and refusal per method is in the training-methods reference ↗.

Source on GitHubUpdated 18 November 2026