Reference
Architecture & design
- Architecture — The
halo/component map, the import layer table, and how every trainer adds one mixin to a TRL, Transformers or SentenceTransformers base. - Trainer Architecture — The distributed mixin, trainer × CP/PP support, the per-mode wrap, gradient sync and clipping, rollout-engine weight sync, and saving across all trainers.
- Why This Framework — How the toolkit compares to HF TRL, Accelerate, Unsloth, Axolotl, MS-SWIFT, veRL, and Megatron-LM, plus model compatibility and key dependencies.
- GPU Training Theory — The roofline, the four bottlenecks a step can hit, and what every optimization and parallelism lever actually pulls on.
Configuration & usage
- Configuration Reference — Every config class field by field, the environment-variable catalog, YAML parsing and CLI overrides.
- Scripts Reference — Every script by category — training, inference, post-training, preparation, profiling — with its flags and input guards, and the
halo launch/halo runCLI.
Operations
- Checkpoints & Resume — The four on-disk layouts (single file, gathered HF-sharded, PP per-stage, EP per-rank), the resume paths, shard merging, and the per-mode saver ladder.
- Model Merging — Combine same-architecture checkpoints in weight space — linear, SLERP, task-arithmetic, and TIES — via
scripts/after_training/merge_models.py. - Capabilities & Limitations at Scale — What works and what does not for very large models, multi-week runs, huge corpora, and many nodes.
Debugging & glossary
- Troubleshooting — Symptom-to-fix lookup for OOM, NCCL hangs, DeepEP faults, config rejections, NaN loss, and attention-backend mismatches.
- Debugging & Profiling — Distributed debug helpers, torch.profiler traces with labeled EP phases, TraceLens trace analysis, py-spy attach, memory snapshots, and multi-node hang diagnosis.
- Glossary — Parallelism, kernel, optimizer, and method terms.
Source on GitHubUpdated 18 November 2026