# Hugging Face Transformers

Hugging Face Transformers is the open-source Python library that most open-weight models are published for. It defines the model classes and the checkpoint format, and loads a model in one call with from_pretrained.

## What it is

Transformers holds implementations of model architectures and one way to load any of them:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
```

The name is a repository on the Hugging Face Hub or a local folder.

## The checkpoint format

A [checkpoint](/glossary/model-checkpoint) is a folder: a `config.json` that describes the architecture, the weights (usually `safetensors` files), and the tokenizer files with the [chat template](/glossary/chat-template). `save_pretrained` writes the folder and `from_pretrained` reads it.

Training tools and inference servers read the same layout, so a model moves between them as one folder. Fine-tuning libraries build on it: [PEFT](/glossary/peft) for adapters, [TRL](/glossary/trl) for trainers, Accelerate for multi-GPU runs.

## In Halo

Halo is built on this stack: its supported runtime is Transformers 5.16.x with TRL 1.6.x, Accelerate 1.11.x and PEFT 0.18.x. A Hugging Face model goes in and a Hugging Face model comes out, loadable with `from_pretrained`. Any `AutoModelForCausalLM` trains with no extra code. The [supported matrix](/halo/docs/supported-matrix#runtime) lists the versions.

