Quickstart
This page takes you from a running container to a finished run. If you don't have a container yet, start with Installation.
1. Pick a recipe
examples/ holds a config per method and model family. These make good first
runs. Check the hardware column before you launch.
The offline GRPO and async GRPO configs ship with a placeholder dataset. Point
it at your data first.
Choosing a method matches your data to a trainer. Each training method page lists its data format and keys.
The online RL recipes also need a separate rollout server. Start with SFT and set that up later.
2. Launch it
Inside the container:
Bash
# 1 GPU (LoRA). Without -n the run uses one GPU, however many the container sees.
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml
# 8 GPUs, full fine-tune
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat.yaml -n 8
# 8 GPUs, MoE with expert parallelism
halo launch sft examples/sft/qwen3_5/qwen3.5-35b-a3b-ultrachat-ep.yaml -n 8To change a field without editing the file, add it after the config path:
--learning_rate=1e-5 --max_length=32000. A few container fields, such as
dataset and rewards, can only be set in the YAML. halo launch --list prints
every method. See The halo CLI and
Writing a config.
For a long run, detach the container. In the docker run command, replace -it
with -d --name myrun and pass bash -lc "halo launch ..." as the command.
3. Watch it
The run copies rank 0's console output to <output_dir>/log/run.log. For the
full fine-tune above:
Bash
tail -f checkpoints/sft-qwen3-4b-ultrachat/log/run.logMost examples set report_to: wandb. With WANDB_API_KEY in your .env, the
run shows up in Weights & Biases with loss and learning rate; throughput and MoE
metrics are opt-in. See Monitoring.
4. Use the result
Checkpoints land in output_dir as standard HuggingFace models. Load them with
from_pretrained, or upload a checkpoint directory to the Hub without its
training state (Uploading to the HuggingFace Hub).
Most families also serve on vLLM directly; a few need a conversion first or have
no serving path.
Three kinds of run need one more step:
- A LoRA run saves an adapter. Load it with
PeftModel.from_pretrained, or merge it into the base. - The opt-in sharded EP save (
save_sharded_ep: true) needshalo run merge-ep-shardsfirst. - A pipeline-parallel run on per-node storage needs
halo run merge-pp-stagesfirst.
Checkpoints covers saving, resuming, export and serving.
Source on GitHubUpdated 18 November 2026