Hugging Face Transformers is the open-source Python library that most open-weight models are published for. It defines the model classes and the checkpoint format, and loads a model in one call with from_pretrained.
What it is
Transformers holds implementations of model architectures and one way to load any of them:
Python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")The name is a repository on the Hugging Face Hub or a local folder.
The checkpoint format
A checkpoint is a folder: a config.json that describes the architecture, the weights (usually safetensors files), and the tokenizer files with the chat template. save_pretrained writes the folder and from_pretrained reads it.
Training tools and inference servers read the same layout, so a model moves between them as one folder. Fine-tuning libraries build on it: PEFT for adapters, TRL for trainers, Accelerate for multi-GPU runs.
In Halo
Halo is built on this stack: its supported runtime is Transformers 5.16.x with TRL 1.6.x, Accelerate 1.11.x and PEFT 0.18.x. A Hugging Face model goes in and a Hugging Face model comes out, loadable with from_pretrained. Any AutoModelForCausalLM trains with no extra code. The supported matrix lists the versions.
