# Fine-tuning vision-language models

Vision-language fine-tuning trains a model that reads images and text on examples that pair images with the answer you want. It is supervised fine-tuning with image tokens in the sequence.

## How it works

Most vision-language models (VLMs) pair a vision encoder with a language model. The encoder turns an image into a sequence of embeddings, and the language model reads them in line with the text tokens.

Training is [supervised fine-tuning](/glossary/supervised-fine-tuning). Each example is a conversation with one or more images, and the loss is next-token cross-entropy on the answer. Typical tasks are document question answering, chart and screenshot reading, and captioning in a narrow domain.

## What to watch

- **Length.** One image can take hundreds or thousands of tokens, so rows are long. Budget [context length](/glossary/context-length) and memory for them.
- **Truncation.** A row cannot be cut in the middle of an image, because the image tokens must match the pixels. Filter out rows that are too long.

## In Halo

Nothing to switch on. `halo launch sft` takes the VLM path when a multimodal checkpoint meets a dataset with images. Images ride inside message content, or in a column named by `images_field`, as in `examples/sft/qwen3_5/qwen3.5-9b-vl-docvqa.yaml`:

```yaml
dataset:
- HuggingFaceM4/Docmatix:zero-shot-exp
conversation_field: texts
images_field: images
max_length: 28672
```

An image run refuses [packing](/glossary/sequence-packing) and context parallelism, and the collator raises rather than truncating a batch over `max_length`. Vision data also works for preference tuning, reward modeling and both distillation trainers. See the [supervised fine-tuning guide](/halo/docs/training-methods/sft#vision-language).

