Halo is White Circle's open-source framework for post-training open models. It is built on the Hugging Face stack and adds the parallelism that large and mixture-of-experts models need, from one GPU to many nodes.
What it is
Halo takes a Hugging Face model and a dataset, trains, and writes a Hugging Face model back, loadable with from_pretrained. It is built on Transformers, TRL and Accelerate, and a run is one YAML config.
What it adds
- Parallelism. Expert, context, tensor and expert-tensor parallelism on native Hugging Face models, so a large or mixture-of-experts model can be split across GPUs.
- Models. Dense models and fifteen mixture-of-experts families. Ten have a cookbook.
- Methods. Supervised fine-tuning, preference tuning, reward modeling, classification, distillation, embedding models, and GRPO: offline, online, and asynchronous with multi-turn environments.
In the Halo tech report, on gpt-oss-20b with 4,096-token sequences at batch 1 on B300 GPUs, Halo ran at 20,786 tokens per second per GPU, about 4.8 times stock TRL's 4,320.
How you run it
Halo runs inside a Docker image: halo:blackwell for NVIDIA B200 and B300 GPUs, halo:hopper for H100 and H200. One command starts a run:
Bash
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat.yaml -n 8sft is the method and -n 8 uses eight GPUs. Start with the quickstart, read the docs, or see the Halo page. The license has its own page.
