Supervised Fine-Tuning

SFT trains with cross-entropy on example conversations: the model learns a format, a behavior or a domain by imitating the assistant turns you show it. The same method takes vision-language data, raw text for continued pre-training, and LoRA or QLoRA as a config switch (Choosing a Method if you are unsure this is the right one).

Data

One record is one conversation, under the column named by conversation_field (default prompt):

JSON Lines

{"prompt": [{"role": "user", "content": "Explain gravity."}, {"role": "assistant", "content": "Gravity is ..."}]}

A row longer than max_length is dropped, not truncated. Images ride inside message content or in an images_field column; a multimodal checkpoint meeting image data makes the run vision-language. Sources and mixing are in Datasets.

Run

Bash

halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat.yaml -n 8

Its -lora and -qlora siblings run on one GPU, and examples/sft/<family>/ holds the MoE and vision recipes. Before a long run, smoke the config on two GPUs with --max_steps=5 --save_strategy=no --report_to=none.

Three settings decide most runs:

  • assistant_message_template must byte-match what the chat template renders at the start of an assistant turn: completion-only masking (on by default) finds the turns with it. Train under the template you will serve with; a mismatch costs quality and never shows in the loss.
  • packing: true fills max_length blocks with short rows and needs an explicit max_length. Rows that share a long prefix can pack as trees instead (Tree Packing ↗).
  • The learning rate. 2.5e-6–5e-6 for a full fine-tune of a large model; the shipped LoRA recipes use 1e-4. Too high a rate erases pretrained capability without showing in the loss.

A family whose release ships a multi-token-prediction head can train it for speculative decoding: mtp_loss_weight adds it to a fine-tune, mtp_train_scope: head_only fits it over a frozen trunk, and mtp_loss_type picks a hard or a soft target (MTP heads ↗).

Continued pre-training tokenizes the corpus once (halo run prepare-dataset --mode text ...) and trains with train_on_completions_only: false; init_from_scratch: true starts from random weights, on data parallelism only (Pre-training ↗).

What to watch

The loss should fall smoothly in the first hundred steps. A flat loss almost always means the assistant marker misses most rows, so their tokens are masked; each such row logs a "Could not find response key" warning. Set log_decoded_samples: true once to read exactly what the model is shown, and check how many rows the length filter dropped: the log warns when it drops half the dataset or more.

Every knob, the vision-language rules and the parallelism limits: SFT reference ↗.

Source on GitHubUpdated 18 November 2026