How to fine-tune an LLM

To fine-tune an LLM, pick a base model and a method, put your data in the format the method reads, write a config, run it, and test the result on examples the model has not seen.

Steps

  1. Pick a base model. Start from the smallest open model that is close to good enough. A 4B model on one GPU is a faster loop than a 35B model on eight.
  2. Pick a method. With examples of the answers you want, use supervised fine-tuning. With pairs of better and worse answers, use DPO. With a way to score answers, use GRPO. How to choose a fine-tuning method covers the rest.
  3. Prepare the data. Most methods read JSONL. For supervised fine-tuning, each row is a conversation. Hold some rows back for testing.
  4. Decide what to train. LoRA trains a small adapter and fits on one GPU. Full fine-tuning changes every weight and needs more memory.
  5. Write a config. It names the model, the dataset, the learning rate, the batch size, the sequence length and where to save.
  6. Run it and watch the loss. The loss should fall, then flatten. A loss that falls on the training data and rises on held-out data is overfitting.
  7. Evaluate. Compare the tuned model with the base model on the held-out rows and on the task itself.

With Halo

Halo runs inside a Docker image. After installation, one command starts a LoRA run on one GPU:

Bash

halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml

Full fine-tuning is the same command with another config, and -n 8 to use eight GPUs. The result is a standard Hugging Face checkpoint that loads with from_pretrained. The quickstart walks through a first run.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL