How to fine-tune GPT-OSS

To fine-tune GPT-OSS with Halo, train the BF16 checkpoint unsloth/gpt-oss-20b-BF16 on eight GPUs with expert parallelism of size 8 (EP8) and the gpt-oss-multiturn.jinja chat template. The same recipe supports GPT-OSS 120B.

Steps

  1. Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
  2. Pick a BF16 (bfloat16) checkpoint. Use unsloth/gpt-oss-20b-BF16. GPT-OSS is a mixture-of-experts model, and the native OpenAI checkpoints store its experts in MXFP4. Halo's expert parallelism (EP) requires dequantized floating-point expert weights.
  3. Write the config. Create gpt-oss-20b-sft.yaml for supervised fine-tuning of all weights. Among its keys:

    YAML

    model_name_or_path: unsloth/gpt-oss-20b-BF16
    model_init_kwargs:
      output_router_logits: true
      router_aux_loss_coef: 0.001
    moe_balancing: aux_loss
    chat_template: jinja-templates/gpt-oss/gpt-oss-multiturn.jinja
    force_chat_template: true
    expert_parallel_size: 8
  4. Launch eight processes. The recipe uses eight NVIDIA B300 GPUs for EP8 (expert_parallel_size: 8).

    Bash

    halo launch sft gpt-oss-20b-sft.yaml -n 8
  5. Load the result. Halo gathers the EP shards into a standard Hugging Face checkpoint that loads with from_pretrained.

Common mistakes

  • The wrong template. gpt-oss-multiturn.jinja is the choice for supervised fine-tuning. Under gpt-oss-harmony.jinja, which GRPO uses, the run trains zero tokens at a loss near zero.
  • Sinks reset by default. Supervised fine-tuning neutralizes the attention sinks (reset_sinks: true). If GRPO should keep the pretrained sinks, set reset_sinks: false in this config too. That needs Flash Attention 4 on Blackwell GPUs.

With Halo

Halo supports the GPT-OSS expert layout and attention sinks. The GPT-OSS cookbook has the full config and the other variants: context, tensor and expert-tensor parallelism, a LoRA adapter over attention and experts, and GRPO with a rollout server. For GPT-OSS 120B, the training checkpoint is unsloth/gpt-oss-120b-BF16.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL