# Training throughput

Training throughput is how many tokens a training run processes per second, usually quoted per GPU. It sets how long a run takes and what it costs.

## What it measures

Throughput is the tokens in a training step (forward pass, backward pass, weight update) divided by the step time. Quoting it per GPU makes runs of different sizes comparable: multiply by the GPU count for the whole run.

The tokens you have to train on, divided by that total, give [how long the run takes](/glossary/how-long-does-fine-tuning-take).

## What moves it

A number means little without its conditions: model, GPU, sequence length, batch size. Within a run, four things move it:

- **Tokens per step.** A small [batch](/glossary/batch-size) of short sequences leaves the GPU waiting.
- **Padding.** [Sequence packing](/glossary/sequence-packing) stops compute going to pad tokens.
- **[Gradient checkpointing](/glossary/gradient-checkpointing).** It saves memory and costs speed.
- **Sharding.** Splitting a model across GPUs adds communication, so use the least that fits.

## In Halo

Set `enable_efficiency_metrics: true` to log tokens per second per GPU. The [performance guide](/halo/docs/performance#rough-numbers) reports gpt-oss-20b on 8 B300 GPUs at batch 4 × 4096: 28,700 tokens per second per GPU without expert parallelism or gradient checkpointing, and 13,200 with the experts split over all eight GPUs and checkpointing on.

In the [Halo tech report](https://whitecircle.com/research/halo), on gpt-oss-20b with 4,096-token sequences at batch 1 on B300 GPUs, Halo ran at 20,786 tokens per second per GPU, Megatron-LM at 8,645 and stock TRL at 4,320.

