Fine-tuning changes a model's weights so that it behaves differently. RAG (retrieval-augmented generation) leaves the model alone and puts relevant documents into the prompt at query time. One teaches skills, the other supplies facts.
How they differ
Which to pick
- The model lacks knowledge (your documents, your product, this week's data). Use RAG. Weights are a poor place to store facts: the model recalls them unreliably and they go stale.
- The model knows enough but answers the wrong way (format, style, steps, tool use). Fine-tune.
- The prompt is long because of instructions and examples, not documents. Fine-tune, and the prompt shrinks.
Using both
The two combine. A common setup is a model fine-tuned to read retrieved passages, quote them, and say when the answer is not there, with RAG supplying the passages. The embedding model that does the retrieval can be fine-tuned too.
Retrieved text is untrusted input: see RAG security.
In Halo
sft tunes the model that writes the answer, and embedding tunes a model for retrieval. See training methods.
