# How to fine-tune Mistral Small 4

To fine-tune Mistral Small 4 with Halo, convert mistralai/Mistral-Small-4-119B-2603 from FP8 to BF16 once, then train the converted checkpoint on eight GPUs with expert parallelism of size 8 (EP8).

## Steps

1. **Start the training container.** Set up the host as [installation](/halo/docs/installation) describes, then start the [cookbook container](/halo/docs/cookbooks#start-the-training-container). Run the commands below inside it.
2. **Download the checkpoint.**
   ```bash
   hf download mistralai/Mistral-Small-4-119B-2603 \
     --local-dir /data/models/mistral-small-4-119b-fp8
   ```
3. **Convert it to BF16 (bfloat16).** The public checkpoint stores its expert weights in FP8, so convert it once before training with [expert parallelism](/glossary/expert-parallelism) (EP).
   ```bash
   python scripts/before_training/convert_mistral4_bf16.py \
     --model_id /data/models/mistral-small-4-119b-fp8 \
     --output_dir /data/models/mistral-small-4-119b-bf16
   ```
4. **Write the config.** Create `mistral4-sft.yaml` for supervised fine-tuning of all weights. Among its keys:
   ```yaml
   model_name_or_path: /data/models/mistral-small-4-119b-bf16
   moe_balancing: bias_update_transient
   expert_parallel_size: 8
   attn_implementation: flash_attention_2
   ```
5. **Launch eight processes.** The recipe starts with eight NVIDIA B300 GPUs. EP8 (`expert_parallel_size: 8`) places 16 routed experts on each.
   ```bash
   halo launch sft mistral4-sft.yaml -n 8
   ```

## Common mistakes

- **Too little disk.** The source and destination checkpoints need about 500 GB in total.
- **Plain `bias_update`.** It raises. The router has no exportable bias slot, so `bias_update_transient` [balances routing](/glossary/moe-load-balancing) in training only, and every exported checkpoint serves without the bias.
- **Serving and online reinforcement learning.** Neither pinned engine (vLLM 0.26.0, SGLang 0.5.17) serves a Mistral 4 checkpoint that Halo exports. Run inference from Transformers. Online GRPO and async GRPO with environments are refused. [Offline GRPO](/glossary/offline-grpo) remains available.

## With Halo

Mistral Small 4 is a [mixture-of-experts](/glossary/mixture-of-experts) model with 128 routed experts, four selected for each token, and one shared expert. Halo preserves its group-top-k router and shared expert. The [Mistral 4 cookbook](/halo/docs/cookbooks/halo-mistral4-moe-cookbook) has the full config, layouts with context, tensor and expert-tensor parallelism, a [LoRA](/glossary/lora) adapter, and multimodal inference.

