How to fine-tune GLM-4.7-Flash

To fine-tune GLM-4.7-Flash with Halo, copy the shipped config for zai-org/GLM-4.7-Flash and launch it on eight GPUs with expert parallelism of size 8 (EP8), Flash Attention 2 and a 30,720-token sequence length.

Steps

  1. Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
  2. Copy the shipped config.

    Bash

    cp examples/sft/glm4/glm-4.7-flash-ultrachat-ep.yaml glm47-sft.yaml
  3. Set the output directory. In the copy, set output_dir: /data/checkpoints/glm-4.7-flash-ultrachat-ep8 so the checkpoint lands on the scratch volume.
  4. Check what it trains. The config runs supervised fine-tuning of all weights on zai-org/GLM-4.7-Flash. Among its settings:

    YAML

    model_name_or_path: zai-org/GLM-4.7-Flash
    moe_balancing: bias_update
    expert_parallel_size: 8
    attn_implementation: flash_attention_2
    max_length: 30720
  5. Launch eight processes. The recipe uses eight NVIDIA B300 GPUs.

    Bash

    halo launch sft glm47-sft.yaml -n 8

Common mistakes

  • Flash Attention 4. Its backward pass emits NaN gradients on GLM's compressed attention, so the loader demotes it to SDPA (scaled dot-product attention). Flash Attention 2 is the stable choice.
  • Serving on Blackwell. Set SGLANG_ATTENTION_BACKEND=triton when you serve the checkpoint with SGLang, or the server exits at start.

With Halo

GLM-4.7-Flash is a mixture-of-experts model with 64 routed experts, four selected per token. With expert parallelism of size 8 (EP8), each GPU owns eight routed experts, and Halo gathers the expert weights when it saves. The GLM-4.7-Flash cookbook adds context, tensor and expert-tensor parallelism, LoRA on GLM's compressed attention, and reinforcement learning with GRPO.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL