# Full fine-tuning

Full fine-tuning updates every weight of a pretrained model on your data. It has the most capacity to change a model's behavior and the highest memory cost, and it saves a complete copy of the model.

## What it costs

Training a weight takes more memory than loading it. For every trained parameter the GPU holds:

- the weight itself;
- its gradient;
- optimizer state, which for Adam-style optimizers is two more values.

That is about four values per parameter, before the activations, which grow with batch size and sequence length. A model that fits one GPU for inference can need several to train in full.

## When it beats LoRA

[LoRA](/glossary/lora) (low-rank adaptation) trains a small adapter and skips the gradients and optimizer state for the frozen weights. Full fine-tuning is worth its cost when:

- the change is large, such as a new language or a new domain;
- you have enough data that a low-rank update becomes the limit.

It also moves the model further from where it started, so [catastrophic forgetting](/glossary/catastrophic-forgetting) is a bigger risk. Use a lower learning rate than for LoRA. [LoRA vs full fine-tuning](/glossary/lora-vs-full-fine-tuning) sets the two side by side.

## In Halo

Full fine-tuning is a run without `use_peft: true`. The Qwen3 4B recipe runs on 1 to 8 GPUs:

```bash
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat.yaml -n 8
```

With `-n 8`, [FSDP2](/glossary/fsdp) (fully sharded data parallel) shards the weights across the GPUs. The [quickstart](/halo/docs/quickstart) lists the recipes.

