GPT-OSS Cookbook
GPT-OSS is OpenAI's open-weight MoE in two sizes, 20B and 120B. Its attention adds a learned per-head "sink" logit, and its experts store gate and up projections interleaved. Halo's SFT recipes load both sizes from OpenAI's release, and the BF16 mirrors elsewhere.
Support
Modes: the GPT-OSS row of the Supported Matrix; online RL: Rollout engines.
- Checkpoints:
unsloth/gpt-oss-20b-BF16(32 experts, top-4) andunsloth/gpt-oss-120b-BF16(128 experts, top-4). The SFT recipes load OpenAI's ownopenai/gpt-oss-20b(EP or PP) andopenai/gpt-oss-120b(EP): the MXFP4 experts are decoded to bf16 at load, so no mirror is needed there. - GPUs: the 20B recipe runs at EP8 on eight GPUs. The 120B recipe is a two-node EP16 layout
(
examples/sft/gptoss/gptoss-120b-multinode-ep.yaml, Clusters), which H100 or H200 need. At EP8 on eight B300s it last ran on code before 1.2.0.
Recipes
Run these in the training container.
Full fine-tune
Bash
halo launch sft examples/sft/gptoss/gptoss-20b-multinode-ep.yaml -n 8 \
--expert_parallel_size=8 --ep_scope=node --use_grouped_gemm=true \
--output_dir=/data/checkpoints/gpt-oss-20b-sftThe shipped recipe is a two-node EP16 layout. These overrides run it on one eight-GPU node. They also turn grouped GEMM back on: the file turns it off because at EP16, with two experts per rank, the per-expert loop keeps up, while at EP8 grouped GEMM is faster.
Other layouts
Add one of these to the full fine-tune command:
- EP8 + CP2, for long sequences:
--context_parallel_size=2 --packing=false - EP8 + TP2, when attention memory is the limit:
--tensor_parallel_size=2
Or replace its --expert_parallel_size=8 with one of these:
- Pure ETP8, to shard every expert instead of placing whole experts:
--expert_parallel_size=1 --expert_tensor_parallel_size=8 - A 4-way expert split:
--expert_parallel_size=4 --expert_tensor_parallel_size=2
On two nodes, examples/sft/gptoss/gptoss-20b-multinode-pp2-ep8.yaml runs pipeline parallelism, one
stage per node with EP8 inside it
(Clusters).
LoRA
Bash
halo launch sft examples/sft/gptoss/gptoss-20b-multinode-ep.yaml -n 8 \
--expert_parallel_size=8 --ep_scope=node --use_grouped_gemm=true \
--use_peft=true --lora_r=64 --lora_alpha=128 --learning_rate=1e-4 \
--lora_target_modules=q_proj,k_proj,v_proj,o_proj,gate_up_proj,down_proj \
--lora_modules_to_save=embed_tokens,lm_head,router --merge_expert_lora_on_save=true \
--output_dir=/data/checkpoints/gpt-oss-20b-loraThe expert targets (gate_up_proj, down_proj) go to Halo's grouped expert LoRA, which no tool merges
afterwards, so merge_expert_lora_on_save folds the adapters into each save.
GRPO
The shipped recipes train on code contests. Read Before a GRPO run first. This one is a full fine-tune at EP4 with two vLLM servers on GPUs 0–3.
On the host, start the servers on the SFT checkpoint:
Bash
export VLLM_MODEL=/data/checkpoints/gpt-oss-20b-sft
export VLLM_CHAT_TEMPLATE=/jinja-templates/gpt-oss/gpt-oss-harmony.jinja
export VLLM_TOOL_PARSER_PLUGIN=/opt/gpt_oss_text_tool_parser.py VLLM_TOOL_PARSER=gpt_oss_text
export VLLM_REASONING_PARSER_PLUGIN=/opt/gpt_oss_reasoning_parser.py VLLM_REASONING_PARSER=openai_gptoss
VLLM_CUDA_DEVICES=0,1 VLLM_TP=2 VLLM_PORT=8000 \
docker compose -p gptoss-rollout-0 -f docker-compose.vllm.yml up -d vllm-server
VLLM_CUDA_DEVICES=2,3 VLLM_TP=2 VLLM_PORT=8001 \
docker compose -p gptoss-rollout-1 -f docker-compose.vllm.yml up -d vllm-serverIn the training container, copy the recipe, set its dataset, and launch on GPUs 4–7:
Bash
cp examples/grpo/environmental/gptoss/vllm/gptoss-20b-code-contests-full-ep4.yaml gpt-oss-grpo.yaml
CUDA_VISIBLE_DEVICES=4,5,6,7 DIST_NCCL_TIMEOUT_MINUTES=60 \
halo launch grpo/environmental gpt-oss-grpo.yaml -n 4 \
--model_name_or_path=/data/checkpoints/gpt-oss-20b-sft \
--output_dir=/data/checkpoints/gpt-oss-20b-grpoThe other configs in examples/grpo/environmental/gptoss/vllm/ switch to LoRA or to ep1. The ep1
configs use routing replay, so start their servers with VLLM_ENABLE_R3=1 as well. The sglang/ configs
run the same task on SGLang: serve them on ports 30000 and 30001 with SGLANG_CHAT_TEMPLATE on the same
harmony file, SGLANG_REASONING_PARSER=gpt-oss and SGLANG_ENABLE_R3=1.
Settings that matter
- Chat template. SFT uses
jinja-templates/gpt-oss/gpt-oss-multiturn.jinjawithforce_chat_template: true. Undergpt-oss-harmony.jinjathe SFT completion marker matches no turn, so the run trains on nothing at a loss near zero. RL uses harmony, and the server must serve the same file. - Attention sinks. SFT neutralizes them (
reset_sinks: true, the default) and saves them that way. RL setsreset_sinks: false, which keeps the sinks live and frozen, so the trainer scores with the sinks the server serves. A later stage runs the sinks it loads, so to carry the pretrained sinks into GRPO, run SFT with--reset_sinks=falsetoo. - Router aux-loss coefficient. The hub config ships
router_aux_loss_coef: 0.9, which swamps the SFT loss. The shipped SFT recipes set it to0.001inmodel_init_kwargs. Do the same on any other run that balances withaux_loss, which is whatautopicks for GPT-OSS. - RL rollouts. The environment GRPO recipes set
rollout_stop_tokens: ["<|call|>"], without which the model writes past its tool call and invents the result. They also setuse_chunked_grpo_logprobs: true, because the ~201k-token vocabulary makes the full logits plane too large.
Limits
- The recipes that load a frozen reference or teacher keep the BF16 mirror. A weight sync into an MXFP4 server drops every expert weight without an error, so an RL rollout server serves a BF16 checkpoint whatever the trainer loaded (GPT-OSS model notes ↗).
- Live sinks (
reset_sinks: false) need a kernel that takes a sink argument. FA2 and SDPA are refused, and so is CP. On Blackwell that kernel is FA4, which the GRPO recipes pin. The Hopper image's FA3 takes no sink, so on H100 or H200 use--attn_implementation=flex_attentionoreager. Both refusepacking, so an SFT run there also needs--packing=false, and neither is fused, so memory bounds the row length (Known issues ↗). packing: trueneeds a flash attention backend. It is refused on eager, SDPA and flex attention.- The DPO, SMPO, reward, classification and offline GRPO recipes pin FA4, so they run on Blackwell only;
FA4 is also what those scripts pick there unpinned. Unpinned on other GPUs, they take SDPA; on Hopper,
pass
--attn_implementation=flash_attention_3. The self-distillation recipe trains without gradient checkpointing and reserved 138 GiB per GPU over three steps at 16k tokens, so on an 80 GB GPU it also needs--gradient_checkpointing=true.
Export and serve
The gathered save re-interleaves the experts into the hub layout, so transformers, vLLM and SGLang load
it directly. Smoke-test it with AutoModelForCausalLM
(snippet). Serve it from the
host with SGLang on port 30000:
Bash
SGLANG_MODEL="$HALO_SCRATCH/checkpoints/gpt-oss-20b-sft" \
docker compose -f docker-compose.sglang.yml up -dReference
- GPT-OSS model notes ↗: sink policies, the expert layout, the vLLM serving flags
- Async GRPO with Environments
- Model cards: gpt-oss-20b, gpt-oss-120b
Source on GitHubUpdated 18 November 2026