To fine-tune LFM2 with Halo, train LiquidAI/LFM2.5-8B-A1B on two GPUs with expert parallelism of size 2 (EP2) and bias-update balancing. The same recipe covers LiquidAI/LFM2-24B-A2B on four GPUs with EP4.
Steps
- Start the training container. Set up the host as installation describes, then start the cookbook container. Run the commands below inside it.
- Pick the checkpoint. LFM2.5-8B-A1B has 32 routed experts and starts on two NVIDIA B300 GPUs. LFM2-24B-A2B has 64 and starts on four or eight. Both are mixture-of-experts models.
- Write the config. Create
lfm2-sft.yamlfor supervised fine-tuning of all weights. Among its keys:YAML
model_name_or_path: LiquidAI/LFM2.5-8B-A1B moe_balancing: bias_update expert_parallel_size: 2 attn_implementation: flash_attention_2 - Launch two processes. With expert parallelism of size 2 (EP2), each GPU owns 16 experts.
Bash
halo launch sft lfm2-sft.yaml -n 2 - Scale to LFM2-24B-A2B. Set
model_name_or_path: LiquidAI/LFM2-24B-A2Bandexpert_parallel_size: 4, changeoutput_dir, and launch with-n 4.
Common mistakes
- Context parallelism. Keep context parallelism disabled for all released LFM2 checkpoints. The short-convolution layers operate across the sequence axis and cannot be split.
- EP4 on eight GPUs. It is rejected at config time. Raise
expert_parallel_sizeto 8, or useep4 + etp2. - Looking for an auxiliary loss. LFM2 has no native router auxiliary loss.
moe_balancing: bias_updatecorrects expert selection during training.
With Halo
Halo gathers the fused expert weights when it saves. Tensor parallelism shards only the full-attention layers, so the footprint per GPU falls by considerably less than 1/tp_size. The LFM2 cookbook has the full config, layouts with tensor and expert-tensor parallelism, an attention LoRA adapter, and reinforcement learning with GRPO.
