LFM-2 MoE Cookbook
Liquid AI's LFM2.5-8B-A1B and LFM2-24B-A2B mix short-convolution layers with full-attention layers. A sigmoid router with a selection bias picks four experts per token, out of 32 on the 8B and 64 on the 24B.
Support
Modes: the LFM-2 MoE row of the Supported Matrix; online RL: Rollout engines.
- Checkpoints:
LiquidAI/LFM2.5-8B-A1BandLiquidAI/LFM2-24B-A2B. - GPUs: the 8B runs at EP2 on two GPUs, 16 experts per GPU. The 24B runs at EP4 on four GPUs or EP8 on eight.
Recipes
Run these in the training container.
Full fine-tune
LFM2.5-8B-A1B on two GPUs:
Bash
halo launch sft examples/sft/lfm2/lfm2.5-8b-a1b-ultrachat-ep2.yaml -n 2 \
--output_dir=/data/checkpoints/lfm2.5-8b-a1b-sftThe recipe leaves report_to at its default, none. To log to Weights & Biases, copy it and add
report_to: wandb; this field can't be overridden on the command line.
LFM2-24B-A2B on four GPUs:
Bash
halo launch sft examples/sft/lfm2/lfm2.5-8b-a1b-ultrachat-ep2.yaml -n 4 \
--model_name_or_path=LiquidAI/LFM2-24B-A2B --expert_parallel_size=4 \
--output_dir=/data/checkpoints/lfm2-24b-a2b-sftOn eight GPUs, change these to -n 8 and --expert_parallel_size=8. EP4 on eight GPUs is rejected at
startup.
Other layouts
Add one of these to the two-GPU command:
- EP2 + TP2, when attention memory is the limit:
--tensor_parallel_size=2. TP shards only the full-attention layers (6 of 24 on the 8B), so per-GPU memory drops much less than half. - Pure ETP2, to shard every expert instead of placing whole experts:
--expert_parallel_size=1 --expert_tensor_parallel_size=2
LoRA
Bash
halo launch sft examples/sft/lfm2/lfm2.5-8b-a1b-ultrachat-ep2.yaml -n 2 \
--use_peft=true --learning_rate=1e-4 \
--lora_target_modules=q_proj,k_proj,v_proj,out_proj \
--output_dir=/data/checkpoints/lfm2.5-8b-a1b-loraout_proj is also the output projection of every short-convolution block (18 of 24 layers on the 8B),
so this adapter trains both kinds of layer.
GRPO
Read Before a GRPO run first. This setup serves the SFT checkpoint from vLLM on GPUs 0–1 and trains on GPUs 2–3. On the host:
Bash
VLLM_MODEL=/data/checkpoints/lfm2.5-8b-a1b-sft VLLM_CUDA_DEVICES=0,1 VLLM_TP=2 VLLM_ENABLE_R3=1 \
VLLM_TOOL_PARSER=lfm2 \
docker compose -f docker-compose.vllm.yml up -d vllm-serverIn the training container, copy the template, set dataset, environment_type and rewards, then
launch:
Bash
cp examples/grpo/environmental/environmental-grpo-template.yaml lfm2-grpo.yaml
CUDA_VISIBLE_DEVICES=2,3 halo launch grpo/environmental lfm2-grpo.yaml -n 2 \
--model_name_or_path=/data/checkpoints/lfm2.5-8b-a1b-sft \
--routing_replay=rollout --beta=0.0 \
--output_dir=/data/checkpoints/lfm2.5-8b-a1b-grpoAdd --expert_parallel_size=2 to shard the experts across the two trainer GPUs. SGLang 0.5.22 also
serves and weight-syncs LFM-2: add --rollout_backend=sglang --rollout_server_url=http://localhost:30000
and start the server with SGLANG_ENABLE_R3=1.
Settings that matter
- Router balancing. The recipe sets
moe_balancing: bias_update, whichautoalso picks under EP. Sigmoid routing can collapse onto a few experts during SFT. The updates land in the router's ownexpert_bias, so the trained bias ships in every checkpoint and the served model routes on it. LFM-2 has no router auxiliary loss. Without EP or grouped GEMM nothing carries the bias, so freeze the router instead withfreeze_layers_patterns: ["*.feed_forward.gate.weight"]. - Gradient checkpointing. The recipe trains 16,384-token packed rows without it, a shape that has not
run on the full-size model. The fit on B200 and B300 is computed, not measured. On 80 GB GPUs add
--gradient_checkpointing=true. - Tool parser. vLLM needs
VLLM_TOOL_PARSER=lfm2. The defaulthermesleaves LFM-2's tool calls as text, and the run stops by its second round.
Limits
- No CP: the short-convolution layers can't split the sequence across ranks.
- No pipeline parallelism: LFM-2 is not a registered pipeline family.
Export and serve
Gathered saves write the hub's per-expert layout (experts.{i}.w1/w3/w2), which both engines read.
Smoke-test with AutoModelForCausalLM (snippet). Serve from the
host with SGLang on port 30000:
Bash
SGLANG_MODEL="$HALO_SCRATCH/checkpoints/lfm2.5-8b-a1b-sft" \
docker compose -f docker-compose.sglang.yml up -dReference
- LFM-2 model notes ↗: the EP layer, packing across the conv layers, balancing
- Async GRPO with Environments
- Model cards: LFM2.5-8B-A1B, LFM2-24B-A2B
Source on GitHubUpdated 18 November 2026