To fine-tune Laguna 2.1 with Halo, train poolside/Laguna-S-2.1 on exactly four GPUs with expert parallelism of size 4 (EP4) and the SDPA attention backend. Laguna XS 2.1 trains on one GPU as a single process.
Steps
- Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
- Pick the checkpoint.
poolside/Laguna-S-2.1starts on four GPUs with expert parallelism of size 4 (EP4).poolside/Laguna-XS-2.1starts on one B300 as a single process. - Write the config. Create
laguna-s-2.1-sft.yamlfor supervised fine-tuning of all weights, or use the shippedexamples/sft/laguna/laguna-s-2.1-ultrachat-ep.yaml. Among its keys:YAML
model_name_or_path: poolside/Laguna-S-2.1 trust_remote_code: true pad_token: "〈|PAD|〉" eos_token: "〈|EOS|〉" expert_parallel_size: 4 attn_implementation: sdpa - Launch four processes. On four NVIDIA B300 GPUs, EP4 places 64 of the 256 routed experts on each.
Bash
halo launch sft laguna-s-2.1-sft.yaml -n 4 - Train Laguna XS instead. Copy the config, change the model and output directory, drop
expert_parallel_size, and launch with-n 1.
Common mistakes
- Retyping the tokens in ASCII.
pad_tokenandeos_tokenuse the CJK angle brackets U+3008 and U+3009. ASCII<and>silently add new tokens instead of resolving the existing ones. - Eight GPUs. The same config on eight GPUs is rejected at config time. Run it on exactly four.
- Another attention backend.
attn_implementation: sdpa(scaled dot-product attention) is required. The hub revision the cookbook pins has no Flash Attention path.
With Halo
Halo does not support context or tensor parallelism for Laguna, a mixture-of-experts family. Expert-tensor parallelism is tested only on a tiny model, so validate a short run first. The Laguna 2.1 cookbook has the full config, the Laguna XS run, an expert LoRA adapter, and GRPO.
