Writing a config
A run is one flat YAML file: the HuggingFace TrainingArguments fields you
already know, plus Halo's own. Every field's type and default is in the
Configuration Reference ↗;
this page covers how the file works.
Start from the nearest example
Don't write a config from scratch. examples/ is organized by method, then by
model family (examples/sft/qwen3/, examples/preference/qwen3_5/). The
closest file is usually two edits from what you want:
Bash
ls examples/sft/ # families with an SFT recipe
cp examples/sft/qwen3/qwen3-4b-ultrachat.yaml my-run.yaml
halo launch sft my-run.yaml -n 8- The async GRPO recipes add a rollout-backend level below the family
(
examples/grpo/environmental/<family>/{vllm,sglang}/). - The annotated GRPO templates sit at the top of their method folder.
qwen3_5/also holds the Qwen3.6 configs, which ship under the Qwen3.5 model types.
Keep the comments when you copy. They record the family's constraints: why
sdpa and not Flash Attention, why packing and not padding_free, which
router-balancing mode exports.
The keys that matter
Notes on the ones that cause trouble:
attn_implementation: most examples pin it; keep the pin when you copy. Left unset, SFT and self-distillation pick the backend for your GPU and model (FA4 on Blackwell, FA3 on Hopper). The padded-batch scripts (preference, reward, classification, teacher distillation, GRPO) take the same pick on Blackwell and default to SDPA on other GPUs. Embedding takes the GPU pick under EP or TP and the transformers default otherwise. Per-family exceptions are in the Supported matrix.model_revisionpins a Hub commit.lora_target_modules: a list names modules, one entry or many ([o_proj]adapts everyo_proj). A bare string isall-linearor a regex matched against the full module path; write a regex in the YAML, since the command line always gives a list.max_concurrent_loadingcaps how many ranks per node load weights at once. Set1on a host short on CPU RAM.vllm_*fields parse on both online RL trainers, which share TRL's GRPO config. The async rollout block belongs to async GRPO only:rollout_server_urlin an online GRPO config fails to parse.- Sequence length.
max_lengthis the one sequence-length setting;nullmeans the model's context window.max_prompt_lengthandmax_completion_lengthmean different things per trainer (Sequence length ↗). Online, offline and async GRPO take nomax_lengthkey, so it fails to parse there.
Overriding on the command line
Override fields at launch to sweep without copying files:
Bash
halo launch sft my-run.yaml -n 8 --learning_rate=1e-5 --max_length=32000- Flags the launcher doesn't own go to the trainer, parallelism flags included
(
--expert_parallel_size=8), so one config can serve several shapes. - Write each override as
--key=value, booleans included (--gradient_checkpointing=true). - A plain list field takes a comma-separated value.
- A field that holds a dict, a list of dicts or lists, or either a string or a
list raises on the command line:
dataset,rewards,gradient_checkpointing_kwargs,report_to(except--report_to=none). Set it in the YAML.
Defaults Halo changes
The parser applies four defaults that differ from upstream, each only when your YAML and command line leave the field unset:
bf16: true. It also switches to the memory-savingAdamWBF16optimizer whereveroptimis a stock AdamW, except under replicated DDP. Settingfp16: trueturns it off; the two can't both be set.use_liger_kernel: true.logging_nan_inf_filter: false. The upstream filter logs the running average in place of a NaN, which hides the step where a run diverged.accelerator_config: {non_blocking: true}while batches are pinned, so a batch copies to the GPU without waiting for the queued kernels.
The trainers also never apply an optimizer step whose gradient norm is NaN or
inf: the step is skipped with a warning and training goes on. More than
max_consecutive_nonfinite_steps (default 3) in a row stop the run, as do
weights that are already NaN.
strftime codes in output_dir expand at launch (runs/sft-%m%d-%H%M). No
other field expands them.
What a launch refuses
These fail at startup, not deep into training:
- An unknown key. The message names it. This includes ecosystem spellings such
as TRL's
max_seq_length; the field ismax_length. - YAML 1.1 booleans on a boolean field (
packing: no,bf16: off). YAML 1.2 reads them as truthy strings, which would invert what you wrote. Usetrueorfalse. - A value outside a field's allowed choices (
advantage_method: banana). ddp_timeoutset away from its default. The process group starts before the config is parsed, so its watchdog isDIST_NCCL_TIMEOUT_MINUTES(Environment variables).- An invalid parallelism shape.
ParallelismConfigrejects it after parsing and before any weights load (Parallelism).
Batch size and memory
Batch size is per device. The global batch is
per_device_train_batch_size × gradient_accumulation_steps × data_parallel_size,
so the same file trains a different global batch on 1 GPU and on 8.
gradient_checkpointing: true adds about 20–30% to step time and saves a lot of
activation memory. It is usually the first thing to turn on when a run doesn't
fit.
Each training method page lists its own keys, and Environment variables covers what the YAML does not.
Source on GitHubUpdated 18 November 2026