What is supervised fine-tuning (SFT)?

Supervised fine-tuning (SFT) trains a language model on examples of the answers you want. The model learns to predict each token of those answers, with the same next-token loss that pre-training uses.

How it works

Each example is a conversation: a prompt and the answer you want. It is rendered with the model's chat template and tokenized. The model predicts every token from the tokens before it, and the loss is the cross-entropy between its prediction and the token that is there.

Usually only the answer counts. The prompt tokens are masked out of the loss, so the model learns to answer and not to write prompts.

When to use it

Use SFT when you have examples of the right output: a format, a tone, a task, a domain. It is where most projects start, and the step before preference tuning or reinforcement fine-tuning.

SFT imitates. The model can only be as good as its examples, so check them before training.

In Halo

One record is one conversation, under the column named by conversation_field (default prompt):

JSON Lines

{"prompt": [{"role": "user", "content": "Explain gravity."}, {"role": "assistant", "content": "Gravity is ..."}]}

The config names model_name_or_path, dataset, max_length and learning_rate. train_on_completions_only is on by default and masks the prompt. A row longer than max_length is dropped, not truncated. One command starts a LoRA (low-rank adaptation) run on one GPU:

Bash

halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml

The SFT guide covers packing, chat templates and LoRA.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL