To fine-tune GPT-OSS with Halo, train the BF16 checkpoint unsloth/gpt-oss-20b-BF16 on eight GPUs with expert parallelism of size 8 (EP8) and the gpt-oss-multiturn.jinja chat template. The same recipe supports GPT-OSS 120B.
Steps
- Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
- Pick a BF16 (bfloat16) checkpoint. Use
unsloth/gpt-oss-20b-BF16. GPT-OSS is a mixture-of-experts model, and the native OpenAI checkpoints store its experts in MXFP4. Halo's expert parallelism (EP) requires dequantized floating-point expert weights. - Write the config. Create
gpt-oss-20b-sft.yamlfor supervised fine-tuning of all weights. Among its keys:YAML
model_name_or_path: unsloth/gpt-oss-20b-BF16 model_init_kwargs: output_router_logits: true router_aux_loss_coef: 0.001 moe_balancing: aux_loss chat_template: jinja-templates/gpt-oss/gpt-oss-multiturn.jinja force_chat_template: true expert_parallel_size: 8 - Launch eight processes. The recipe uses eight NVIDIA B300 GPUs for EP8 (
expert_parallel_size: 8).Bash
halo launch sft gpt-oss-20b-sft.yaml -n 8 - Load the result. Halo gathers the EP shards into a standard Hugging Face checkpoint that loads with
from_pretrained.
Common mistakes
- The wrong template.
gpt-oss-multiturn.jinjais the choice for supervised fine-tuning. Undergpt-oss-harmony.jinja, which GRPO uses, the run trains zero tokens at a loss near zero. - Sinks reset by default. Supervised fine-tuning neutralizes the attention sinks (
reset_sinks: true). If GRPO should keep the pretrained sinks, setreset_sinks: falsein this config too. That needs Flash Attention 4 on Blackwell GPUs.
With Halo
Halo supports the GPT-OSS expert layout and attention sinks. The GPT-OSS cookbook has the full config and the other variants: context, tensor and expert-tensor parallelism, a LoRA adapter over attention and experts, and GRPO with a rollout server. For GPT-OSS 120B, the training checkpoint is unsloth/gpt-oss-120b-BF16.
