LoRA target modules are the layers of a model that get a LoRA adapter. They are listed by module name, such as q_proj and v_proj for attention, and every layer left off the list stays frozen with no adapter.
Which layers there are
A transformer block has two groups of linear layers, and LoRA can wrap any of them:
Names differ between model families, so read them from the model before you write the list.
How to choose
q_projandv_projonly. The smallest adapter. The original LoRA paper adapted these two.- Attention and MLP. More trained parameters at the same rank, and often closer to full fine-tuning.
More targets mean a larger adapter and more memory. To hold the size fixed, trade targets against rank. In the Hugging Face PEFT library, all-linear selects every linear layer except the output layer.
In Halo
The key is lora_target_modules. The Qwen3 4B LoRA example targets attention and MLP:
YAML
lora_target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]On mixture-of-experts (MoE) models, all-linear misses the fused expert tensors, so name the expert projections explicitly. LoRA is rejected under tensor parallelism. See LoRA and QLoRA in the Halo docs.
