To fine-tune an LLM, pick a base model and a method, put your data in the format the method reads, write a config, run it, and test the result on examples the model has not seen.
Steps
- Pick a base model. Start from the smallest open model that is close to good enough. A 4B model on one GPU is a faster loop than a 35B model on eight.
- Pick a method. With examples of the answers you want, use supervised fine-tuning. With pairs of better and worse answers, use DPO. With a way to score answers, use GRPO. How to choose a fine-tuning method covers the rest.
- Prepare the data. Most methods read JSONL. For supervised fine-tuning, each row is a conversation. Hold some rows back for testing.
- Decide what to train. LoRA trains a small adapter and fits on one GPU. Full fine-tuning changes every weight and needs more memory.
- Write a config. It names the model, the dataset, the learning rate, the batch size, the sequence length and where to save.
- Run it and watch the loss. The loss should fall, then flatten. A loss that falls on the training data and rises on held-out data is overfitting.
- Evaluate. Compare the tuned model with the base model on the held-out rows and on the task itself.
With Halo
Halo runs inside a Docker image. After installation, one command starts a LoRA run on one GPU:
Bash
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yamlFull fine-tuning is the same command with another config, and -n 8 to use eight GPUs. The result is a standard Hugging Face checkpoint that loads with from_pretrained. The quickstart walks through a first run.
