# Qwen3 MoE Cookbook

[Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507) has 128 routed
experts and picks eight per token. Qwen3 MoE has the broadest parallelism coverage in Halo, which makes it
the reference MoE family.

## Support

Modes: the Qwen3 MoE row of the [Supported Matrix](https://whitecircle.com/halo/docs/supported-matrix.md#model-families); online RL:
[Rollout engines](https://whitecircle.com/halo/docs/supported-matrix.md#rollout-engines).

- **Checkpoint:** `Qwen/Qwen3-30B-A3B-Instruct-2507`, pinned below.
- **GPUs:** four at EP4, 32 experts per GPU.

## Recipes

Run these in the [training container](https://whitecircle.com/halo/docs/cookbooks.md#start-the-training-container).

### Full fine-tune

No Qwen3 MoE SFT recipe ships, so save this as `qwen3-moe-sft.yaml` in the repo root:

```yaml
model_name_or_path: Qwen/Qwen3-30B-A3B-Instruct-2507
model_revision: 0d7cf23991f47feeb3a57ecb4c9cee8ea4a17bfe
moe_balancing: aux_loss

dataset:
- HuggingFaceH4/ultrachat_200k@train_sft
conversation_field: messages
test_size: 0.01
assistant_message_template: "<|im_start|>assistant\n"
pad_token: <|endoftext|>
eos_token: <|im_end|>

expert_parallel_size: 4
fp32_router: true

packing: true
max_length: 8192
per_device_train_batch_size: 1
per_device_eval_batch_size: 1
gradient_accumulation_steps: 8

learning_rate: 5.0e-06
lr_scheduler_type: cosine
warmup_steps: 32
num_train_epochs: 1.0

save_steps: 1000
save_total_limit: 1
save_only_model: true
eval_strategy: steps
eval_steps: 300
logging_steps: 1
report_to: wandb
remove_unused_columns: false
output_dir: /data/checkpoints/qwen3-30b-a3b-sft
```

```bash
halo launch sft qwen3-moe-sft.yaml -n 4
```

### Other layouts

Each of these runs on the same four GPUs. Add one to the launch command:

- EP4 + CP2, for long sequences: `--context_parallel_size=2 --packing=false`
- EP4 + TP2, when attention memory is the limit: `--tensor_parallel_size=2`
- Pure ETP4, to shard every expert instead of placing whole experts: `--expert_parallel_size=1 --expert_tensor_parallel_size=4`

On eight GPUs, launch with `-n 8 --expert_parallel_size=8`. EP4 on eight GPUs is rejected at startup.

### LoRA

```bash
halo launch sft qwen3-moe-sft.yaml -n 4 \
  --use_peft=true --learning_rate=1e-4 \
  --lora_target_modules=q_proj,k_proj,v_proj,o_proj \
  --output_dir=/data/checkpoints/qwen3-30b-a3b-lora
```

To adapt the experts as well, add `gate_proj,up_proj,down_proj` and `--merge_expert_lora_on_save=true`.
Under EP they train as grouped expert adapters, which no tool merges afterwards.

### GRPO

Read [Before a GRPO run](https://whitecircle.com/halo/docs/cookbooks.md#before-a-grpo-run) first. This setup serves the SFT checkpoint from
vLLM on GPUs 0–1 and trains on GPUs 2–3.

On the host:

```bash
VLLM_MODEL=/data/checkpoints/qwen3-30b-a3b-sft VLLM_CUDA_DEVICES=0,1 VLLM_TP=2 VLLM_ENABLE_R3=1 \
  docker compose -f docker-compose.vllm.yml up -d vllm-server
```

In the training container, copy the template, set `dataset`, `environment_type` and `rewards`, then
launch:

```bash
cp examples/grpo/environmental/environmental-grpo-template.yaml qwen3-moe-grpo.yaml
CUDA_VISIBLE_DEVICES=2,3 halo launch grpo/environmental qwen3-moe-grpo.yaml -n 2 \
  --model_name_or_path=/data/checkpoints/qwen3-30b-a3b-sft \
  --routing_replay=rollout --beta=0.0 \
  --output_dir=/data/checkpoints/qwen3-30b-a3b-grpo
```

Add `--expert_parallel_size=2` to shard the experts across the two trainer GPUs. On SGLang, add
`--rollout_backend=sglang --rollout_server_url=http://localhost:30000` and start the server with
`SGLANG_ENABLE_R3=1 SGLANG_TOOL_PARSER=qwen25`.

## Settings that matter

- **Attention.** Leave `attn_implementation` unset. Halo picks FA4 on Blackwell and FA3 (or FA2) on
  Hopper.
- **Router balancing.** `aux_loss` uses the checkpoint's `router_aux_loss_coef` (0.001). The router has no
  bias slot, so `bias_update` raises. `bias_update_transient` balances training only, and exported
  checkpoints serve without the bias.
- **Tool parser.** Qwen3 tool calls parse with vLLM's default `hermes` parser. On SGLang, set
  `SGLANG_TOOL_PARSER=qwen25`: the default `auto` cannot read the 2507 chat template and turns the parser off
  with one log line, which the trainer refuses at startup on a tool-using environment
  ([Tool parsers on SGLang](https://whitecircle.com/halo/docs/agent-docs/infrastructure/rollout-servers.md#tool-parsers-on-sglang) ↗).

## Limits

- Pipeline parallelism passes tiny-model GPU tests and a 20-step Qwen3-30B-A3B run at `pp2` × `ep4`, and
  needs `moe_balancing: none`, since the stage split drops the aux loss and the router has no bias slot.
- The general limits apply: LoRA is refused under TP and EP+TP, and expert adapters are refused once
  `expert_tensor_parallel_size > 1`.

## Export and serve

The gathered save writes the hub's per-expert layout, which transformers, vLLM and SGLang read
directly. Smoke-test it with `AutoModelForCausalLM` ([snippet](https://whitecircle.com/halo/docs/cookbooks.md#smoke-test-a-checkpoint)).
Both pinned engines have served the hub checkpoint and taken its weight sync, but neither has served a
trained Qwen3-30B-A3B export, so check the first one you serve. Serve it from the
[host](https://whitecircle.com/halo/docs/cookbooks.md#serve-from-the-host) with SGLang on port 30000:

```bash
SGLANG_MODEL="$HALO_SCRATCH/checkpoints/qwen3-30b-a3b-sft" \
  docker compose -f docker-compose.sglang.yml up -d
```

## Reference

- [Qwen3 model notes](https://whitecircle.com/halo/docs/agent-docs/models/qwen3.md) ↗: the EP layer, CP and TP, balancing
- [Async GRPO with Environments](https://whitecircle.com/halo/docs/training-methods/async-grpo-environments.md)
- [Model card](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507)
