TRL is Hugging Face's library of trainers for fine-tuning and RL. Halo is built on the same stack and adds parallelism for large and mixture-of-experts models. On the same GPUs we measured up to 3.5x TRL's throughput.
What TRL is
TRL is Hugging Face's library of trainers: SFTTrainer, DPOTrainer, GRPOTrainer and others. It is the usual way to post-train a Hugging Face model.
What Halo adds
Halo is built on the same stack (Transformers, TRL, Accelerate) and keeps the same model format. It adds what large models need:
- Expert, tensor, context and expert-tensor parallelism, so a mixture-of-experts model can be split across GPUs and nodes.
- Kernels and attention backends for NVIDIA Blackwell and Hopper GPUs.
- RL at scale: rollout servers, environments and asynchronous GRPO.
Measured
With the strongest stock options turned on for both, Halo trains gpt-oss-20b at 2.6 to 3.5 times stock TRL's throughput on short and mid-length configs, on the same 8 B300 GPUs. See why not stock TRL.
The Halo tech report ran the same model with 4,096-token sequences: Halo at 20,786 tokens per second per GPU, stock TRL at 4,320. In its memory test, TRL peaked at 47.6 GB per GPU and Halo at 26 GB with the experts spread over eight GPUs.
Which to pick
Configs and checkpoints stay in Hugging Face form, so moving from TRL is a config change, not a rewrite. Start with the quickstart.
