Why this framework
Built on HuggingFace Transformers, Accelerate, and TRL — every distributed trainer subclasses a TRL, Transformers, or SentenceTransformers trainer. What it adds over that stack:
- EP/CP/TP/ETP/PP on native HF models, no Megatron conversion (DeepEP for MoE expert parallelism).
- EP+TP and pure ETP shapes that other HF-native tools don't compose (Axolotl composes EP with CP, not TP).
- Full BF16 masters + optimizer state (
AdamWBF16+ stochastic rounding, ~6 bytes/param) — others keep fp32 masters. - SMPO and Offline GRPO (pre-scored completions) — methods absent elsewhere.
- Fully asynchronous multi-turn RL — Ray rollout actors overlap training via a prefetch queue, a clean Transformers ↔ vLLM/SGLang split with NCCL weight sync, and built-in environments; no Megatron backend, no veRL.
- Native
s3://datasets and FA4-on-Blackwell.
Framework comparison
Versions compared: TRL 1.14.2 and Accelerate 1.11.0 (the pinned releases; every TRL cell below describes 1.14.2, and the throughput comparisons run TRL 1.17.1), Unsloth, Axolotl 0.19.0, MS-SWIFT 4.0, veRL (Megatron backend), Megatron-LM/Core. "Via Megatron" = needs a Megatron backend + an HF↔MCore checkpoint conversion; "—" = out of scope (a layer the others build on).
Megatron-Core stores torch_dist sharded checkpoints split by TP/PP/EP/ETP — a separate format from HuggingFace's, so MS-SWIFT, veRL, and Megatron-LM all require an HF↔MCore conversion before EP/CP/TP/PP. This toolkit, Accelerate, TRL, Unsloth, and Axolotl work directly on HuggingFace weights; of those, this toolkit and Axolotl run DeepEP expert parallelism on the upstream HF MoE modules. Two neighbors sit close: NeMo AutoModel keeps HF checkpoints without a conversion step but ships its own implementation of each supported architecture behind the transformers API, and transformers itself shards a model at load (from_pretrained(distributed_config=DistributedConfig(...))) with FSDP2, TP and EP: ep_size dispatches tokens through a torch all-to-all (ep_dispatch_experts, most MoE families' default plan), and a few families keep a router that all-reduces the full MoE output across ep_size == tp_size ranks. On the pinned transformers 5.20.2 and Accelerate 1.11.0, the transformers Trainer trains such a model only at tp_size=1 (FSDP2 plus token dispatch) and without resume; it, and TRL's trainers that subclass it, run neither TP, CP nor the all-reduce EP path: a load-time TP model needs a newer Accelerate, and CP needs Accelerate's ParallelismConfig (TrainingArguments.parallelism_config), on which that Trainer fails at construction under this Accelerate release.
When to use which
Measured throughput against TRL, NeMo AutoModel, Megatron Bridge, Axolotl, MS-SWIFT and Unsloth: Throughput Benchmarks; the TRL side in Halo vs stock TRL.
Notes on individual frameworks:
- Accelerate is the foundation layer this toolkit builds on (launcher + FSDP2/DTensor TP/CP plumbing over native HF), not a competing trainer — it ships no trainers, RL, MoE/EP, attention kernels, or
s3://loading. - Megatron-LM adds deeper TE/FP8 optimization for 1000+ GPU dense pre-training; its alignment is split across four repos and needs MCore conversion. On large NVLink domains FSDP2 + EP/TP avoids the pipeline bubble; this toolkit keeps pipeline parallelism for crossing domains — see When PP is worth it.
- Axolotl (v0.19.0) closes most of the EP gap (DeepEP on native HF, no conversion, composed with CP and FSDP) and offers config-driven YAML, s3/GCS data, FA2/3/4, and async-GRPO + NeMo Gym. This toolkit still adds SMPO, Offline GRPO, EP+TP and pure ETP (Axolotl's EP×TP raises), and full BF16.
- NVIDIA NeMo RL — alignment on DTensor (FSDP2+TP+CP) or Megatron-Core; MCore conversion on the Megatron path; capabilities span four repos.
- SkyRL — RL plus a native SFT trainer (v0.3.0), no DPO/reward/distillation; async dispatcher ~1.55× over naive batching; EP/CP/TP need its Megatron backend.
Model compatibility
Any HuggingFace AutoModelForCausalLM works with standard FSDP, and any model carrying a tp_plan
gets TP. Advanced parallelism (EP, CP, ETP, PP) needs per-family registration, not a model
reimplementation or a checkpoint conversion.
For EP, that is a frozen MoEFamilySpec holding the family's facts plus an EP layer under
halo/distributed/expert_parallel/families/ (83 lines or fewer; GPT-OSS's is 168) subclassing EPMoELayerBase, declaring
its HF_MODULE_NAMES and pointing FAMILY at the spec, with MOE_LAYER_MAP built from the roster's
imports; for PP, a validated PPFamilySpec in PP_FAMILY_SPECS. Eighteen MoE families ship an EP layer;
the per-family × per-mode matrix is
Supported Models, and Adding a New Model is the
procedure. Trainer × parallelism support is separate — see
Trainer Compatibility.
Key dependencies
Pinned versions: Docker (images) and pyproject.toml / uv.lock (Python).
Source on GitHubUpdated 18 November 2026