# Halo vs Megatron

Megatron-LM is NVIDIA's framework for training very large transformers, with models defined in its own code. Halo is a post-training framework that wraps Hugging Face models as they are, so no checkpoint is converted in or out.

## The difference

[Megatron-LM](/glossary/megatron-lm) defines models in its own code and checkpoint format. A Hugging Face checkpoint is converted into that format to train, and converted back to use the result.

[Halo](/glossary/halo) applies distributed behavior as wrappers around the Hugging Face model, not as a fork of it. A Hugging Face model goes in and a Hugging Face model comes out, loadable with `from_pretrained`.

| | Halo | Megatron-LM |
| --- | --- | --- |
| Model code | The Hugging Face model, wrapped | Megatron's own |
| Checkpoints | Hugging Face in and out | Converted in and out (Megatron Bridge) |
| A new model family | One wrapper file for expert parallelism, most under 140 lines | Model, config and conversion code that connects the family to Megatron's stack |
| Pipeline parallelism | Not yet available | Yes |

In Halo, a family that trains under plain data parallelism needs no code. Laguna's wrapper reuses GLM-4's and is 39 lines. GPT-OSS is the outlier at about 360.

## Measured

In the [Halo tech report](https://whitecircle.com/research/halo), on gpt-oss-20b with 4,096-token sequences at batch 1 on B300 GPUs, we measured Halo at 20,786 tokens per second per GPU and Megatron-LM at 8,645.

## Which to pick

- **Megatron-LM** to pre-train a very large model on a big cluster, when you need pipeline parallelism, or when your team already works in its model code.
- **Halo** to post-train a Hugging Face model, above all a [mixture-of-experts](/glossary/mixture-of-experts) one, and get a Hugging Face model back. It also covers preference tuning and reinforcement learning with rollouts.

The [model integration cost](/halo/docs/model-integration-cost) page counts what each Halo wrapper takes.

