# Overfitting

Overfitting is when a model memorizes its training examples instead of learning the pattern behind them. Its loss on the training data keeps falling while its loss on held-out data rises.

## How to spot it

Hold out some rows and never train on them. Then compare two curves:

- **Training loss** falls for as long as the model keeps fitting its examples.
- **Held-out loss** falls with it at first, then flattens and turns up.

Where the curves part, learning has stopped and memorizing has begun.

An overfit language model repeats training answers almost word for word. It does well on prompts close to the training set and worse on new phrasing. A small dataset and many passes over it are the usual cause.

## How to fix it

- **Train less.** Run fewer [epochs](/glossary/epochs-and-steps), or keep the checkpoint with the lowest held-out loss.
- **Add data.** More examples, and more varied ones. See [how much data you need](/glossary/how-much-data-to-fine-tune).
- **Train fewer weights.** With [LoRA](/glossary/lora), lower the [rank](/glossary/lora-rank-and-alpha).
- **Keep measuring.** [Evaluate](/glossary/how-to-evaluate-a-fine-tuned-model) every checkpoint on the held-out rows.

## In Halo

`test_size` carves an evaluation split out of the dataset, and `eval_strategy` with `eval_steps` sets when evaluation runs:

```yaml
test_size: 0.01
eval_strategy: steps
eval_steps: 500
```

Halo logs loss, learning rate and gradient norm at every `logging_steps`, and `report_to` sends them to Weights & Biases, ClearML or TensorBoard. See [Monitoring](/halo/docs/monitoring).

