Why this framework

Built on HuggingFace Transformers, Accelerate, and TRL — every distributed trainer subclasses a TRL, Transformers, or SentenceTransformers trainer. What it adds over that stack:

  • EP/CP/TP/ETP/PP on native HF models, no Megatron conversion (DeepEP for MoE expert parallelism).
  • EP+TP and pure ETP shapes that other HF-native tools don't compose (Axolotl composes EP with CP, not TP).
  • Full BF16 masters + optimizer state (AdamWBF16 + stochastic rounding, ~6 bytes/param) — others keep fp32 masters.
  • SMPO and Offline GRPO (pre-scored completions) — methods absent elsewhere.
  • Fully asynchronous multi-turn RL — Ray rollout actors overlap training via a prefetch queue, a clean Transformers ↔ vLLM/SGLang split with NCCL weight sync, and built-in environments; no Megatron backend, no veRL.
  • Native s3:// datasets and FA4-on-Blackwell.

Framework comparison

Versions compared: TRL 1.14.2 and Accelerate 1.11.0 (the pinned releases; every TRL cell below describes 1.14.2, and the throughput comparisons run TRL 1.17.1), Unsloth, Axolotl 0.19.0, MS-SWIFT 4.0, veRL (Megatron backend), Megatron-LM/Core. "Via Megatron" = needs a Megatron backend + an HF↔MCore checkpoint conversion; "—" = out of scope (a layer the others build on).

FeatureThis ToolkitHF TRLAccelerateUnslothAxolotlMS-SWIFTveRLMegatron-LM
SMPO (reference-free PO)YesNo—NoNoNoNoNo
Offline GRPO (pre-scored)YesNo—NoNoNoNoNo
EP for MoE (no conversion)DeepEPNoNoNoDeepEP (v0.17)Via MegatronVia MegatronMCore format
Context ParallelismUlysses (SFT, SMPO, offline GRPO full-FT)SFT-only; needs a newer AccelerateYes (FSDP2)NoYes (ring/seq)Via MegatronVia MegatronMCore
Tensor ParallelismYes (DTensor)Via Accelerate; needs a newer AccelerateYes (ND-parallel)NoYes (experimental)Via MegatronVia MegatronMCore
Pipeline ParallelismYes (registered families; EP inside each stage)NoInference only (PiPPy)NoNoVia MegatronVia MegatronMCore
Combined EP+CP / EP+TP / pure ETPYesNoNo (no EP)NoPartial (EP×CP and EP×CP×FSDP; EP×TP raises)Via MegatronVia MegatronYes
Full BF16 (~6 bytes/param)AdamWBF16 + SRNo (FP32 masters)Plumbing onlyNobf16 MP onlyNoNoNo
Multi-turn RLAsync Ray + built-in envsAsync GRPO + env adapters (OpenEnv, Harbor, OpenReward)—Via ARTAsync GRPO + NeMo GymGYM envAgentLoop + SGLangNo
Native HF models (no conversion)Yes (all modes)Yes (no EP)Yes (plumbing)Yes (no EP/CP/TP)YesMCore for EP/CP/TPMCore for EP/CP/TPNo
Flash Attention (FA4 Blackwell / FA3 Hopper / FA2 fallback)YesInherited (FA2/3)InheritedNo (FA2)Yes (FA2/3/4)YesInherited (FA2)Via TE
Native s3:// datasetsYesNoNoNoYesNoNoVia MSC
Multi-nodeYes (verified topologies)Via AccelerateYesPaid tiersYesYesYesYes

Megatron-Core stores torch_dist sharded checkpoints split by TP/PP/EP/ETP — a separate format from HuggingFace's, so MS-SWIFT, veRL, and Megatron-LM all require an HF↔MCore conversion before EP/CP/TP/PP. This toolkit, Accelerate, TRL, Unsloth, and Axolotl work directly on HuggingFace weights; of those, this toolkit and Axolotl run DeepEP expert parallelism on the upstream HF MoE modules. Two neighbors sit close: NeMo AutoModel keeps HF checkpoints without a conversion step but ships its own implementation of each supported architecture behind the transformers API, and transformers itself shards a model at load (from_pretrained(distributed_config=DistributedConfig(...))) with FSDP2, TP and EP: ep_size dispatches tokens through a torch all-to-all (ep_dispatch_experts, most MoE families' default plan), and a few families keep a router that all-reduces the full MoE output across ep_size == tp_size ranks. On the pinned transformers 5.20.2 and Accelerate 1.11.0, the transformers Trainer trains such a model only at tp_size=1 (FSDP2 plus token dispatch) and without resume; it, and TRL's trainers that subclass it, run neither TP, CP nor the all-reduce EP path: a load-time TP model needs a newer Accelerate, and CP needs Accelerate's ParallelismConfig (TrainingArguments.parallelism_config), on which that Trainer fails at construction under this Accelerate release.

When to use which

Use caseRecommendedWhy
EP+TP or pure ETP on MoEThis ToolkitExpert shapes other HF-native tools don't compose
Memory-constrained full fine-tuningThis ToolkitAdamWBF16 — ~6 bytes/param
Long-context training (scales with nGPUs)This ToolkitCP splits sequences; combines with EP
MoE wider than one NVLink domain, on HF weightsThis ToolkitPP across domains, EP inside each stage
SMPO or Offline GRPOThis ToolkitUnique methods
Multi-turn RL with tool environmentsThis ToolkitRay actors, NCCL sync, built-in envs
Config-driven SFT/DPO/GRPO over many models (incl. FSDP+EP)AxolotlYAML, DeepEP EP (v0.17), s3/GCS data, FA2/3/4
A hand-written loop needing only FSDP2 + TP + CPAccelerateThe plumbing layer, no trainer overhead
Single-GPU LoRA/QLoRA on consumer hardwareUnslothExtreme VRAM efficiency, free tier
SFT/DPO on single node, standard modelsHF TRLSimplest setup, largest community
Online PPO/GRPO with many algorithm variantsveRL or MS-SWIFTPurpose-built for online RL
Broad model coverage + no-code UIMS-SWIFT600+ models, Gradio Web UI
Dense pre-training at 1000+ GPU scaleMegatron-LMTE/FP8 kernels, peak utilization

Measured throughput against TRL, NeMo AutoModel, Megatron Bridge, Axolotl, MS-SWIFT and Unsloth: Throughput Benchmarks; the TRL side in Halo vs stock TRL.

Notes on individual frameworks:

  • Accelerate is the foundation layer this toolkit builds on (launcher + FSDP2/DTensor TP/CP plumbing over native HF), not a competing trainer — it ships no trainers, RL, MoE/EP, attention kernels, or s3:// loading.
  • Megatron-LM adds deeper TE/FP8 optimization for 1000+ GPU dense pre-training; its alignment is split across four repos and needs MCore conversion. On large NVLink domains FSDP2 + EP/TP avoids the pipeline bubble; this toolkit keeps pipeline parallelism for crossing domains — see When PP is worth it.
  • Axolotl (v0.19.0) closes most of the EP gap (DeepEP on native HF, no conversion, composed with CP and FSDP) and offers config-driven YAML, s3/GCS data, FA2/3/4, and async-GRPO + NeMo Gym. This toolkit still adds SMPO, Offline GRPO, EP+TP and pure ETP (Axolotl's EP×TP raises), and full BF16.
  • NVIDIA NeMo RL — alignment on DTensor (FSDP2+TP+CP) or Megatron-Core; MCore conversion on the Megatron path; capabilities span four repos.
  • SkyRL — RL plus a native SFT trainer (v0.3.0), no DPO/reward/distillation; async dispatcher ~1.55× over naive batching; EP/CP/TP need its Megatron backend.

Model compatibility

Any HuggingFace AutoModelForCausalLM works with standard FSDP, and any model carrying a tp_plan gets TP. Advanced parallelism (EP, CP, ETP, PP) needs per-family registration, not a model reimplementation or a checkpoint conversion.

For EP, that is a frozen MoEFamilySpec holding the family's facts plus an EP layer under halo/distributed/expert_parallel/families/ (83 lines or fewer; GPT-OSS's is 168) subclassing EPMoELayerBase, declaring its HF_MODULE_NAMES and pointing FAMILY at the spec, with MOE_LAYER_MAP built from the roster's imports; for PP, a validated PPFamilySpec in PP_FAMILY_SPECS. Eighteen MoE families ship an EP layer; the per-family × per-mode matrix is Supported Models, and Adding a New Model is the procedure. Trainer × parallelism support is separate — see Trainer Compatibility.

Key dependencies

Pinned versions: Docker (images) and pyproject.toml / uv.lock (Python).

Source on GitHubUpdated 18 November 2026