The halo CLI
The image ships a halo command with two verbs:
halo launchstarts a training method.halo runstarts any other tool: checkpoint post-processing, data prep, inference, diagnostics.
Each one maps a name to a script under scripts/ and runs it with the right
launcher. You can also run the scripts directly with python or torchrun.
halo launch
Bash
halo launch <method> <config.yaml> [launcher flags] [field overrides]The CLI picks the launcher:
- one process: plain
python --nprocabove 1:torchrun(required for EP, CP, TP, ETP and PP)--accelerate:accelerate launch
Other flags after the config go to the trainer as is, so config overrides ride along:
Bash
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat.yaml -n 8
halo launch sft examples/sft/gptoss/gptoss-20b-multinode-ep.yaml -n 8 \
--expert_parallel_size=8 --use_grouped_gemm=true --learning_rate=1e-5A method name is the script's path under scripts/training/, with hyphens for
underscores, grouped as examples/ is: sft, classification, preference/smpo,
grpo/offline, grpo/online, grpo/environmental (async GRPO with environments),
reward/reward-model, distillation/self-distill. A file stem unique across the
methods works alone (smpo, dpo, reward-model).
The config path can be absolute, or relative to the repo root or your current
directory. The run executes from the repo root, so relative paths inside the
config, such as output_dir, resolve there.
The CLI is single-node and rejects torchrun's multi-node flags. For a multi-node
job, run one torchrun per node (Clusters and multi-node).
halo run
Bash
halo run <tool> [launcher flags] [tool flags]halo run takes --list, --dry_run and --root, and runs the tool as one
process from your current directory: every tool is single-process. Everything
else goes to the tool, a Gradio app's own --port included.
When a tool flag has the same name as a launcher flag (--help, --dry_run,
--root, --list), put -- before it: halo run <tool> -- --help prints the
tool's help, not the launcher's.
halo run --list prints the catalog:
Bash
halo run merge-ep-shards --input_dir <ep-checkpoint-dir> --output_dir <merged-dir>
halo run quantize-to-lowp --input_dir <bf16-model-dir> --output_dir <nvfp4-out-dir> --format nvfp4
halo run quantize-to-lowp --input_dir <bf16-model-dir> --output_dir <vllm-out-dir> --format mxfp4 --export_format compressed-tensors--export_format compressed-tensors writes the layout vLLM loads instead of
Halo's own block_scaled manifest, which stays the default.
Writing a config covers the config file. Every script and flag is in the Scripts reference ↗.
Source on GitHubUpdated 18 November 2026