How to fine-tune Laguna 2.1

To fine-tune Laguna 2.1 with Halo, train poolside/Laguna-S-2.1 on exactly four GPUs with expert parallelism of size 4 (EP4) and the SDPA attention backend. Laguna XS 2.1 trains on one GPU as a single process.

Steps

  1. Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
  2. Pick the checkpoint. poolside/Laguna-S-2.1 starts on four GPUs with expert parallelism of size 4 (EP4). poolside/Laguna-XS-2.1 starts on one B300 as a single process.
  3. Write the config. Create laguna-s-2.1-sft.yaml for supervised fine-tuning of all weights, or use the shipped examples/sft/laguna/laguna-s-2.1-ultrachat-ep.yaml. Among its keys:

    YAML

    model_name_or_path: poolside/Laguna-S-2.1
    trust_remote_code: true
    pad_token: "〈|PAD|〉"
    eos_token: "〈|EOS|〉"
    expert_parallel_size: 4
    attn_implementation: sdpa
  4. Launch four processes. On four NVIDIA B300 GPUs, EP4 places 64 of the 256 routed experts on each.

    Bash

    halo launch sft laguna-s-2.1-sft.yaml -n 4
  5. Train Laguna XS instead. Copy the config, change the model and output directory, drop expert_parallel_size, and launch with -n 1.

Common mistakes

  • Retyping the tokens in ASCII. pad_token and eos_token use the CJK angle brackets U+3008 and U+3009. ASCII < and > silently add new tokens instead of resolving the existing ones.
  • Eight GPUs. The same config on eight GPUs is rejected at config time. Run it on exactly four.
  • Another attention backend. attn_implementation: sdpa (scaled dot-product attention) is required. The hub revision the cookbook pins has no Flash Attention path.

With Halo

Halo does not support context or tensor parallelism for Laguna, a mixture-of-experts family. Expert-tensor parallelism is tested only on a tiny model, so validate a short run first. The Laguna 2.1 cookbook has the full config, the Laguna XS run, an expert LoRA adapter, and GRPO.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL