# LoRA target modules

LoRA target modules are the layers of a model that get a LoRA adapter. They are listed by module name, such as q_proj and v_proj for attention, and every layer left off the list stays frozen with no adapter.

## Which layers there are

A [transformer](/glossary/transformer) block has two groups of linear layers, and [LoRA](/glossary/lora) can wrap any of them:

| Group | Common names | Role |
| --- | --- | --- |
| Attention | `q_proj`, `k_proj`, `v_proj`, `o_proj` | query, key, value and output projections |
| MLP (feed-forward) | `gate_proj`, `up_proj`, `down_proj` | gate, up and down projections |

Names differ between model families, so read them from the model before you write the list.

## How to choose

- **`q_proj` and `v_proj` only.** The smallest adapter. The original LoRA paper adapted these two.
- **Attention and MLP.** More trained parameters at the same rank, and often closer to full fine-tuning.

More targets mean a larger adapter and more memory. To hold the size fixed, trade targets against [rank](/glossary/lora-rank-and-alpha). In the Hugging Face [PEFT](/glossary/peft) library, `all-linear` selects every linear layer except the output layer.

## In Halo

The key is `lora_target_modules`. The Qwen3 4B LoRA example targets attention and MLP:

```yaml
lora_target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
```

On [mixture-of-experts](/glossary/mixture-of-experts) (MoE) models, `all-linear` misses the fused expert tensors, so name the expert projections explicitly. LoRA is rejected under tensor parallelism. See [LoRA and QLoRA](/halo/docs/training-methods/sft#lora-and-qlora) in the Halo docs.

