Halo vs stock TRL / transformers
Halo 1.2.0 against trl.SFTTrainer 1.17.1 on Transformers 5.20.2, 8× B300 and the Blackwell image, bf16, FA4, Liger,
grouped_mm experts. Same model, rows and kernels on both sides; only the framework changes. The rates themselves, with
NeMo AutoModel and Megatron Bridge beside them, and the conditions that decide how to read them are in
Throughput Benchmarks. This page holds the TRL
side: layouts, ratios, what differs, and the older long-context and convergence runs.
Halo 1.2.0 against TRL 1.17.1
Short rows (16k and 4k, 131,072 tokens per step), Halo over TRL:
On packed 64k and 128k rows Halo runs 1.17× TRL on Qwen3.5-9B, 1.47× on Qwen3.6-35B-A3B and 1.50× on Qwen3.5-122B-A10B. TRL's rows there are pure bf16 parameters and Adam moments; its fp32-master runs are within 3% on the 9B and 35B and do not fit on the 122B. Halo's rows are the median of 3 runs of a chosen setup; TRL's are single runs of its best layouts.
Halo's lead is largest on gpt-oss-20b (2.47–3.06×) and smallest on dense Qwen3-8B (1.10–1.13×). The first-step loss on the same batch agrees to three decimals in seven of the eight short shapes. TRL's default adds the router auxiliary loss (coefficient 0.9 on gpt-oss) to its loss, so every throughput row sets it to 0.
What differs between the two sides
TRL expert parallelism
TRL 1.17.1 has no expert-parallel option of its own. Transformers' expert-parallel path (ep_size 8, experts as
DTensors, 4 of 32 local) switched on through a local patch trains gpt-oss-20b at 16k at 2,184 tok/s/GPU and 117.6 GiB,
4.4× below TRL's ZeRO-2 row and 9× below Halo's EP8 row (19,817). It is untuned and not a TRL trainer feature.
Gradient checkpointing on vs off
gpt-oss-20b at 16k, batch 1, Halo EP1 with fused linear cross-entropy against TRL ZeRO-2:
Halo's lead widens without checkpointing (2.47× to 2.77×), and its peak is lower in both columns. TRL on ZeRO-3 with checkpointing fits in 50.6 GiB (8,282 tok/s/GPU), below Halo's 62.1 GiB.
Long context on gpt-oss-20b (Halo 1.0.0)
These rows ran on Halo 1.0.0 against the TRL that release pinned (version not recorded), ZeRO-3 with fused linear cross-entropy, batch 1, gradient checkpointing, tok/s/GPU · peak GiB. Nothing on 1.2.0 replaces the CP rows.
- Dense EP1 ZeRO-3 wins throughput at both lengths and still beats TRL on memory.
- More CP is strictly better past 64k:
EP8+CP8beatsEP8+CP4on both axes at 128k. - Dense CP-only ZeRO-3 is the leanest point (32–56 GiB across 128k→256k) and the only long-context mode with no DeepEP dependency. Splitting 4-way instead of 8 trades memory for throughput (256k: CP4 2,257 · 98 vs CP8 1,836 · 56).
EP1ZeRO-2 OOMs (ZeRO-2 keeps40 GB of params resident).64k tokens per rank (its combine barrier races FSDP2's reduce-scatter), andEP8,EP8+CP2andEP8+TP8put 128k tokens per rank through the dispatch, which no transport is validated for: elastic EP8 deadlocks at ≥ep_buffer_backend: legacyhas not run an EP8 training step at 128k tokens per rank (its dispatch alone is bit-exact to 200k) (DeepEP → Transport backend, Dispatch token ceiling).
Convergence (Halo 1.1.0)
200 steps on the same seeded data, same global batch (16), constant LR, 8× B300. TRL (ZeRO-3), Halo dense (EP1), EP2 and EP8 reach the same loss, about 0.00205 at step 200 (0.002048–0.002051), within 0.8% of each other by step 100.
Reproduce
Bash
# TRL baseline, synthetic fixed-length rows (ZeRO-3 FSDP2; --fsdp_sharding shard_grad_op for ZeRO-2)
torchrun --nproc_per_node=8 tests/gpu/profiling/benchmark_trl_baseline.py \
--model gpt-oss-20b --seq 4096 --batch_size 1
# Halo EP1 / EP2 / EP8 (add --zero3 for ZeRO-3, --no_gc to drop checkpointing)
torchrun --nproc_per_node=8 tests/gpu/profiling/benchmark_sft_ep.py \
--model gpt-oss-20b --ep 2 --seq 65536 --batch_size 1
# convergence (--ep 0 = TRL, >=1 = Halo)
torchrun --nproc_per_node=8 tests/gpu/profiling/benchmark_convergence.py \
--ep 0 --seq 2048 --batch_size 2 --steps 200These scripts build create_benchmark_dataset: 64 rows of exactly seq uniform token ids with labels = input_ids, a
shape fixture whose fixed length isolates the framework comparison. Held constant: learning_rate=2e-5, seed 42,
gradient_accumulation_steps=1, global batch 8 × per_device_train_batch_size; TRL runs
fsdp="full_shard auto_wrap" with fsdp_config={"version": 2, "reshard_after_forward": …} (true is ZeRO-3).
Source on GitHubUpdated 18 November 2026