To fine-tune GLM-4.7-Flash with Halo, copy the shipped config for zai-org/GLM-4.7-Flash and launch it on eight GPUs with expert parallelism of size 8 (EP8), Flash Attention 2 and a 30,720-token sequence length.
Steps
- Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
- Copy the shipped config.
Bash
cp examples/sft/glm4/glm-4.7-flash-ultrachat-ep.yaml glm47-sft.yaml - Set the output directory. In the copy, set
output_dir: /data/checkpoints/glm-4.7-flash-ultrachat-ep8so the checkpoint lands on the scratch volume. - Check what it trains. The config runs supervised fine-tuning of all weights on
zai-org/GLM-4.7-Flash. Among its settings:YAML
model_name_or_path: zai-org/GLM-4.7-Flash moe_balancing: bias_update expert_parallel_size: 8 attn_implementation: flash_attention_2 max_length: 30720 - Launch eight processes. The recipe uses eight NVIDIA B300 GPUs.
Bash
halo launch sft glm47-sft.yaml -n 8
Common mistakes
- Flash Attention 4. Its backward pass emits NaN gradients on GLM's compressed attention, so the loader demotes it to SDPA (scaled dot-product attention). Flash Attention 2 is the stable choice.
- Serving on Blackwell. Set
SGLANG_ATTENTION_BACKEND=tritonwhen you serve the checkpoint with SGLang, or the server exits at start.
With Halo
GLM-4.7-Flash is a mixture-of-experts model with 64 routed experts, four selected per token. With expert parallelism of size 8 (EP8), each GPU owns eight routed experts, and Halo gathers the expert weights when it saves. The GLM-4.7-Flash cookbook adds context, tensor and expert-tensor parallelism, LoRA on GLM's compressed attention, and reinforcement learning with GRPO.
