# What is QLoRA?

QLoRA is LoRA applied to a base model whose frozen weights are loaded in 4-bit precision, which takes roughly a quarter of the memory of 16-bit. The adapter trains in higher precision on top.

## How it works

QLoRA (quantized LoRA) was introduced in a 2023 paper. It stores the frozen base model in 4-bit, using two techniques:

- **NF4 (4-bit NormalFloat).** A 4-bit data type whose levels are spaced for weights that follow a normal distribution.
- **Double quantization.** The quantization constants are quantized too, which saves a little more. It is also called nested quantization.

In the forward pass the 4-bit weights are dequantized on the fly to a 16-bit compute type, usually bf16. Gradients flow through them into the [LoRA](/glossary/lora) matrices, which stay in 16-bit or 32-bit. Only the adapter is trained.

## Trade-offs

- A model that does not fit one GPU for LoRA may fit for QLoRA.
- Steps are usually slower than plain LoRA, because of the dequantization.
- [Quantization](/glossary/quantization) adds a small error to the base weights, and the adapter learns on top of it.

## In Halo

QLoRA is the LoRA config plus three keys:

```yaml
load_in_4bit: true
bnb_4bit_quant_type: nf4
use_bnb_nested_quant: true
```

This peaks at 7.9 GiB for a 4B model at batch 1 and 4k tokens, so it fits a 24 GB GPU. Across GPUs, QLoRA runs under data parallelism (DDP or FSDP) and context parallelism only. See [LoRA and QLoRA](/halo/docs/training-methods/sft#lora-and-qlora) in the Halo docs.

