# Reference

## Architecture & design

- **[Architecture](https://whitecircle.com/halo/docs/agent-docs/reference/architecture.md)** — The `halo/` component map, the import layer table, and how every trainer adds one mixin to a TRL, Transformers or SentenceTransformers base.
- **[Trainer Architecture](https://whitecircle.com/halo/docs/agent-docs/reference/trainer-architecture.md)** — The distributed mixin, trainer × CP/PP support, the per-mode wrap, gradient sync and clipping, rollout-engine weight sync, and saving across all trainers.
- **[Why This Framework](https://whitecircle.com/halo/docs/agent-docs/reference/why-this-framework.md)** — How the toolkit compares to HF TRL, Accelerate, Unsloth, Axolotl, MS-SWIFT, veRL, and Megatron-LM, plus model compatibility and key dependencies.
- **[GPU Training Theory](https://whitecircle.com/halo/docs/agent-docs/reference/gpu-training-theory.md)** — The roofline, the four bottlenecks a step can hit, and what every optimization and parallelism lever actually pulls on.

---

## Configuration & usage

- **[Configuration Reference](https://whitecircle.com/halo/docs/agent-docs/reference/configuration-reference.md)** — Every config class field by field, the environment-variable catalog, YAML parsing and CLI overrides.
- **[Scripts Reference](https://whitecircle.com/halo/docs/agent-docs/reference/scripts-reference.md)** — Every script by category — training, inference, post-training, preparation, profiling — with its flags and input guards, and the `halo launch` / `halo run` CLI.

---

## Operations

- **[Checkpoints & Resume](https://whitecircle.com/halo/docs/agent-docs/reference/checkpoints.md)** — The four on-disk layouts (single file, gathered HF-sharded, PP per-stage, EP per-rank), the resume paths, shard merging, and the per-mode saver ladder.
- **[Model Merging](https://whitecircle.com/halo/docs/agent-docs/reference/model-merging.md)** — Combine same-architecture checkpoints in weight space — linear, SLERP, task-arithmetic, and TIES — via `scripts/after_training/merge_models.py`.
- **[Capabilities & Limitations at Scale](https://whitecircle.com/halo/docs/agent-docs/reference/scale-and-limitations.md)** — What works and what does not for very large models, multi-week runs, huge corpora, and many nodes.

---

## Debugging & glossary

- **[Troubleshooting](https://whitecircle.com/halo/docs/agent-docs/reference/troubleshooting.md)** — Symptom-to-fix lookup for OOM, NCCL hangs, DeepEP faults, config rejections, NaN loss, and attention-backend mismatches.
- **[Debugging & Profiling](https://whitecircle.com/halo/docs/agent-docs/reference/debugging.md)** — Distributed debug helpers, torch.profiler traces with labeled EP phases, TraceLens trace analysis, py-spy attach, memory snapshots, and multi-node hang diagnosis.
- **[Glossary](https://whitecircle.com/halo/docs/agent-docs/reference/glossary.md)** — Parallelism, kernel, optimizer, and method terms.
