# Fine-tuning embedding models

Embedding fine-tuning trains a model that turns text into vectors so that related texts land close together and unrelated ones far apart. It improves retrieval and similarity search on your own data.

## How it works

An embedding model maps a text to one vector, and two texts are compared by the similarity of their vectors, usually cosine. Training is contrastive: it pulls an anchor toward its positive and pushes it away from negatives. The data comes in three shapes:

| Data | Trains |
| --- | --- |
| Pairs: a query and a matching passage | Retrieval. Other passages in the batch act as negatives. |
| Triplets: a pair plus a hard negative | Retrieval, against passages that look relevant and are not. |
| Scored pairs: two texts and a score | Graded similarity. |

With in-batch negatives, a larger [batch size](/glossary/batch-size) gives each query more negatives and a harder task.

## When to use it

Tune the embedder when search misses your domain's vocabulary: product names, legal or medical terms, code. It is a direct fix for retrieval-augmented generation (RAG) that fetches the wrong passages. See [fine-tuning vs RAG](/glossary/fine-tuning-vs-rag).

## In Halo

Halo trains embedders with sentence-transformers losses, set by `loss_type`. The default, `mnrl`, uses in-batch negatives. Neither gradient accumulation nor extra GPUs enlarge that pool, so raise `per_device_train_batch_size`. `triplet` is for mined hard negatives, and `cosent` for graded scores.

```bash
halo launch embedding examples/embedding/qwen3/embedding-qwen3-4b-nq.yaml
```

The output loads with `SentenceTransformer(path)`. See the [embedding guide](/halo/docs/training-methods/embedding).

