Epochs and steps

Epochs and steps both measure the length of a training run. A step is one weight update on one batch, and an epoch is one full pass over the training data.

How they relate

Steps per epoch are the number of examples divided by the effective batch size. Total steps are that times the number of epochs.

With 12,800 examples and an effective batch of 64, an epoch is 200 steps and three epochs are 600.

Two things change the count:

Warmup, logging, evaluation and checkpoint saves are usually counted in steps. In Hugging Face trainers the length of a run is num_train_epochs, or max_steps, which overrides it when set.

How many epochs

Fine-tuning usually takes 1 to 3 epochs. The model starts from pretrained weights and learns a dataset quickly. More passes over a small dataset lead to overfitting. A large dataset often needs one pass or less.

Treat the number as a ceiling: watch the held-out loss and stop when it stops improving.

In Halo

The UltraChat example sets num_train_epochs: 1.0. Before a long run, smoke-test the config on two GPUs with --max_steps=5 --save_strategy=no --report_to=none, which catches template, length and parallelism mistakes in a minute. See the supervised fine-tuning guide.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL