PEFT (parameter-efficient fine-tuning) is a family of methods that train a small number of weights, usually added ones, while the pretrained model stays frozen. PEFT is also the name of the Hugging Face library that implements them.
The main methods
Each of these freezes the pretrained weights and adds new ones in a different place:
LoRA is the one most used with LLMs. QLoRA is LoRA on a base model loaded in 4-bit.
Why use it
- Memory. No gradients or optimizer state for the frozen weights.
- Storage. The output is a small file, not a copy of the model.
- Swapping. One base model serves many tasks by changing the small part.
The cost is capacity: see LoRA vs full fine-tuning.
The library
peft is the Hugging Face library that implements these methods. get_peft_model wraps a model, save_pretrained writes only the adapter, and merge_and_unload folds a LoRA adapter into the base.
In Halo
Halo's supported runtime lists PEFT 0.18.x. Set use_peft: true and the lora_* fields to train a LoRA adapter instead of the full model. LoRA is rejected under tensor parallelism. See LoRA and QLoRA in the Halo docs.
