How to merge a LoRA adapter

To merge a LoRA adapter, load the base model, load the adapter on top of it, fold the adapter's weights into the base and save the result. The output is one ordinary checkpoint that needs no adapter at inference.

Why merge

A LoRA run saves only the adapter. Merging adds (alpha / r) · B · A to each weight the adapter sits beside. The result is a standalone model:

  • One checkpoint to ship and serve, in the base model's format.
  • No adapter math at inference, so no added latency.

Do not merge if you serve several adapters on one base and swap them per request. Merging also gives up the small file: the output is as large as the base.

Steps

With the Hugging Face PEFT library:

  1. Load the exact base model the adapter was trained on. For a QLoRA adapter, load it in 16-bit, not 4-bit.
  2. Load the adapter on top with PeftModel.from_pretrained.
  3. Call merge_and_unload(). It returns a plain Transformers model.
  4. Save it with save_pretrained, and save the tokenizer beside it.
  5. Compare a few outputs with the unmerged model.

Python

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("base-model")
model = PeftModel.from_pretrained(base, "path/to/adapter")
model.merge_and_unload().save_pretrained("merged-model")

With Halo

halo run merge-peft-adapters folds a LoRA adapter into its base as one checkpoint. Give it an output path different from its input: the tools refuse to run in place. Like every Halo tool, it answers -- --help with its flags. It refuses EP (expert-parallel) expert-LoRA adapters. Train those with merge_expert_lora_on_save: true to save the merged model instead. See post-processing tools.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL