# LoRA vs full fine-tuning

LoRA trains a small adapter beside a frozen model, and full fine-tuning updates every weight. LoRA needs far less memory and storage. Full fine-tuning has more capacity for large changes in behavior.

## Side by side

|  | LoRA | Full fine-tuning |
| --- | --- | --- |
| What is trained | low-rank matrices beside frozen weights, a few percent of the parameters or less | every weight |
| Training memory | no gradients or optimizer state for the frozen weights | gradients and optimizer state for every weight |
| Output | an adapter of tens or hundreds of megabytes | a full copy of the model |
| Learning rate | higher, 5 to 10 times the full fine-tuning rate | lower |
| Wins when | memory is tight, data is small, or one base serves many tasks | the change is large and data is plentiful |

## Which to pick

Start with [LoRA](/glossary/lora) (low-rank adaptation). It fits on less hardware, and on a narrow task it often matches full fine-tuning. It also changes the base less, which limits [catastrophic forgetting](/glossary/catastrophic-forgetting).

Move to [full fine-tuning](/glossary/full-fine-tuning) when:

- quality stalls and raising the [rank](/glossary/lora-rank-and-alpha) no longer helps;
- the model has to learn a new language or domain, not a new habit;
- you have the data and the GPUs for it.

Serving is the same either way, because an adapter can be [merged into the base](/glossary/how-to-merge-a-lora-adapter).

## In Halo

Both are the same script. LoRA is a config switch:

```yaml
use_peft: true
lora_r: 64
lora_alpha: 128
learning_rate: 1.0e-04
```

The full fine-tuning example for the same 4B model uses `learning_rate: 2.0e-05`. LoRA is rejected under tensor parallelism. The [supervised fine-tuning guide](/halo/docs/training-methods/sft#lora-and-qlora) covers both.

