Halo vs TRL

TRL is Hugging Face's library of trainers for fine-tuning and RL. Halo is built on the same stack and adds parallelism for large and mixture-of-experts models. On the same GPUs we measured up to 3.5x TRL's throughput.

What TRL is

TRL is Hugging Face's library of trainers: SFTTrainer, DPOTrainer, GRPOTrainer and others. It is the usual way to post-train a Hugging Face model.

What Halo adds

Halo is built on the same stack (Transformers, TRL, Accelerate) and keeps the same model format. It adds what large models need:

  • Expert, tensor, context and expert-tensor parallelism, so a mixture-of-experts model can be split across GPUs and nodes.
  • Kernels and attention backends for NVIDIA Blackwell and Hopper GPUs.
  • RL at scale: rollout servers, environments and asynchronous GRPO.

Measured

With the strongest stock options turned on for both, Halo trains gpt-oss-20b at 2.6 to 3.5 times stock TRL's throughput on short and mid-length configs, on the same 8 B300 GPUs. See why not stock TRL.

The Halo tech report ran the same model with 4,096-token sequences: Halo at 20,786 tokens per second per GPU, stock TRL at 4,320. In its memory test, TRL peaked at 47.6 GB per GPU and Halo at 26 GB with the experts spread over eight GPUs.

Which to pick

You haveUse
A small dense model on one GPUEither
A mixture-of-experts modelHalo
A run across several GPUs or nodesHalo
Online RL with rolloutsHalo

Configs and checkpoints stay in Hugging Face form, so moving from TRL is a config change, not a rewrite. Start with the quickstart.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL