Embedding fine-tuning trains a model that turns text into vectors so that related texts land close together and unrelated ones far apart. It improves retrieval and similarity search on your own data.
How it works
An embedding model maps a text to one vector, and two texts are compared by the similarity of their vectors, usually cosine. Training is contrastive: it pulls an anchor toward its positive and pushes it away from negatives. The data comes in three shapes:
With in-batch negatives, a larger batch size gives each query more negatives and a harder task.
When to use it
Tune the embedder when search misses your domain's vocabulary: product names, legal or medical terms, code. It is a direct fix for retrieval-augmented generation (RAG) that fetches the wrong passages. See fine-tuning vs RAG.
In Halo
Halo trains embedders with sentence-transformers losses, set by loss_type. The default, mnrl, uses in-batch negatives. Neither gradient accumulation nor extra GPUs enlarge that pool, so raise per_device_train_batch_size. triplet is for mined hard negatives, and cosent for graded scores.
Bash
halo launch embedding examples/embedding/qwen3/embedding-qwen3-4b-nq.yamlThe output loads with SentenceTransformer(path). See the embedding guide.
