Axolotl is an open-source tool for fine-tuning language models that describes a whole training run in one YAML config. It is built on the Hugging Face libraries and supports LoRA, QLoRA and full fine-tuning.
What it does
Axolotl wraps the Hugging Face training stack (Transformers, PEFT, Accelerate) behind a config file. You name the base model, the datasets, the adapter settings and the hyperparameters in YAML, and start the run with one command:
Bash
axolotl train examples/llama-3/lora-1b.ymlAxolotl handles dataset loading and formatting, tokenization and the training loop. It supports LoRA, QLoRA and full fine-tuning, on one GPU or several, and reads the common instruction and chat dataset formats. The output is a Hugging Face checkpoint or adapter.
When to use it
Axolotl fits when you want a whole fine-tuning run defined in one config, with example configs to start from.
Axolotl and Halo
Both are driven by a YAML config, and both sit on the Hugging Face stack. Reach for Halo when the run is one of these:
- a mixture-of-experts model that has to be split across GPUs with expert parallelism,
- online reinforcement learning at multi-GPU scale, with rollout servers and weight sync.
The quickstart shows a first Halo run.
