What is post-training?

Post-training is every training stage that comes after pre-training, including supervised fine-tuning, preference tuning, reinforcement learning and distillation. It is where a model learns how to behave.

What it covers

Pre-training teaches a model language by having it predict the next token over a large corpus of raw text. Post-training starts from those weights and shapes behavior. Its stages differ in what they learn from:

StageLearns from
Supervised fine-tuningexamples of the answer you want
Preference tuningpairs of a better and a worse answer
Reinforcement fine-tuningscores for answers the model generates
Distillationthe output distribution of a teacher model

Supervised fine-tuning usually comes first. Preference tuning or RL (reinforcement learning) then starts from its checkpoint.

Post-training vs fine-tuning

The two terms overlap. In everyday use, fine-tuning often means the supervised stage alone. Post-training names the whole set of stages between a pretrained base and the model you ship.

In Halo

Halo is our open-source framework for post-training. Each method is one name passed to halo launch, such as sft, dpo, offline-grpo or teacher-distill. All of them read a YAML config, take a Hugging Face checkpoint and a dataset, and write a Hugging Face checkpoint back out. The training methods page lists every one.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL