How much data do you need to fine-tune an LLM?

Fine-tuning an LLM for a narrow task with LoRA usually takes a few hundred to a few thousand good examples. Broad changes in behavior take tens of thousands or more. Quality and coverage matter more than count.

Rough ranges

GoalExamples
A narrow task with LoRA, such as a format, a style, extraction or routinga few hundred to a few thousand
Broad behavior across many tasks, or a new domaintens of thousands or more

The first row assumes a model that already handles the language and the domain.

What matters more than count

  • Quality. The model copies what it sees, mistakes included. A few hundred checked examples often beat thousands of noisy ones.
  • Coverage. Include the hard cases and edge inputs, in the proportions you expect in use.

Synthetic data can fill gaps, if you check it.

How to tell you have enough

Hold out a test set. Train on half the data, then on all of it, and compare scores on the held-out rows. If the score still climbs, more data will help. If it is flat, work on quality or coverage. With a small set, watch for overfitting.

In Halo

test_size carves an eval split out of the data. A row longer than max_length is dropped, not truncated, so check the log for how many rows the length filter dropped. See the supervised fine-tuning guide.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL