Catastrophic forgetting is when a model trained on new data loses skills or knowledge it had before. In fine-tuning, the model gets better at your task and worse at things the base model did well.
Why it happens
A neural network keeps everything it knows in the same weights. Fine-tuning moves those weights to lower the loss on the new data, and nothing in that loss protects what the old data taught. With narrow data the model drifts toward it. It may stop following general instructions, lose a language, or answer every prompt in the style of the training set.
The risk grows with a high learning rate, many passes over a small dataset, and data that covers one kind of input.
How to limit it
- Lower the learning rate.
- Train for fewer steps. Stop when held-out quality stops improving.
- Use LoRA. The base weights stay frozen and the update is low-rank, so the model moves less.
- Mix in general data. Add examples of the behavior you want to keep.
- Test old tasks. After every fine-tune, run a small test set of things the base model does well. See how to evaluate a fine-tuned model.
In Halo
The supervised fine-tuning guide warns that too high a rate erases pretrained capability without showing up in the loss curve. It puts a large model's full fine-tuning rate near 2.5e-6 to 5e-6. To mix in general data, pass a list of sources to dataset.
