Epochs and steps both measure the length of a training run. A step is one weight update on one batch, and an epoch is one full pass over the training data.
How they relate
Steps per epoch are the number of examples divided by the effective batch size. Total steps are that times the number of epochs.
With 12,800 examples and an effective batch of 64, an epoch is 200 steps and three epochs are 600.
Two things change the count:
- With gradient accumulation, a step is one update, not one forward pass.
- With sequence packing, one row holds several examples, so an epoch takes fewer steps.
Warmup, logging, evaluation and checkpoint saves are usually counted in steps. In Hugging Face trainers the length of a run is num_train_epochs, or max_steps, which overrides it when set.
How many epochs
Fine-tuning usually takes 1 to 3 epochs. The model starts from pretrained weights and learns a dataset quickly. More passes over a small dataset lead to overfitting. A large dataset often needs one pass or less.
Treat the number as a ceiling: watch the held-out loss and stop when it stops improving.
In Halo
The UltraChat example sets num_train_epochs: 1.0. Before a long run, smoke-test the config on two GPUs with --max_steps=5 --save_strategy=no --report_to=none, which catches template, length and parallelism mistakes in a minute. See the supervised fine-tuning guide.
