# What is TRL?

TRL (Transformers Reinforcement Learning) is Hugging Face's open-source library of trainers for post-training language models. It covers supervised fine-tuning, preference tuning such as DPO, and reinforcement learning such as GRPO.

## What it does

TRL gives each post-training method a trainer class, built on the `Trainer` of Hugging Face Transformers:

| Trainer | Method |
| --- | --- |
| `SFTTrainer` | [Supervised fine-tuning](/glossary/supervised-fine-tuning) on example conversations |
| `DPOTrainer` | [DPO](/glossary/dpo), from pairs of better and worse answers |
| `GRPOTrainer` | [GRPO](/glossary/grpo), reinforcement learning from a reward |
| `RewardTrainer` | Reward models that score an answer |

A trainer takes a model, a dataset and a config, and handles the tokenization, the loss and the training loop. TRL works with the PEFT library for LoRA (low-rank adaptation) adapters and with Accelerate for multi-GPU runs.

## When to use it

TRL is the usual way to post-train a Hugging Face model. It is a good default for a dense model that trains on one GPU or with plain data parallelism.

## TRL and Halo

Halo is built on TRL and keeps the Hugging Face model format. It adds expert, context, tensor and expert-tensor [parallelism](/halo/docs/parallelism) for large and [mixture-of-experts](/glossary/mixture-of-experts) models.

In the [Halo tech report](https://whitecircle.com/research/halo), on gpt-oss-20b with 4,096-token sequences at batch 1 on B300 GPUs, Halo ran at 20,786 tokens per second per GPU, about 4.8 times stock TRL's 4,320. [Halo vs TRL](/glossary/halo-vs-trl) has the comparison.

