Fine-tuning changes the model itself, and prompt engineering changes only what you send to it. Start with prompts. Fine-tune when prompts cannot reach the quality, cost or latency you need, or when you need a smaller model.
How they differ
Which to pick
Start with prompts. They are cheap to change, and a good prompt is the baseline a fine-tune has to beat.
Fine-tune when:
- Quality stalls. The prompt keeps growing and the model still misses the format or the edge cases.
- Cost or latency is too high. A long prompt is processed on every call. A tuned model needs a short one.
- You need a smaller model. A small open-weight model tuned on one task can match a large general model on that task.
A fine-tuned model still takes a prompt. Train it with the prompt it will be served with.
If what is missing is knowledge, not behavior, try retrieval-augmented generation (RAG) first: see fine-tuning vs RAG. A prompt also does not enforce anything: see system prompt vs guardrails.
In Halo
A first test does not need a cluster. Halo's Qwen3 4B LoRA recipe runs on one GPU:
Bash
halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yamlThe quickstart walks through it.
