Fine-tuning vs prompt engineering

Fine-tuning changes the model itself, and prompt engineering changes only what you send to it. Start with prompts. Fine-tune when prompts cannot reach the quality, cost or latency you need, or when you need a smaller model.

How they differ

Prompt engineeringFine-tuning
Changesthe instructions and examples in the promptthe model's weights
Needsa few examples and time to iteratea dataset, GPUs and a test set
Time to try an ideaminuteshours or more
Cost per callevery instruction and example is processed each timea short prompt
Works onany model, including hosted ones you cannot traina model you can train

Which to pick

Start with prompts. They are cheap to change, and a good prompt is the baseline a fine-tune has to beat.

Fine-tune when:

  • Quality stalls. The prompt keeps growing and the model still misses the format or the edge cases.
  • Cost or latency is too high. A long prompt is processed on every call. A tuned model needs a short one.
  • You need a smaller model. A small open-weight model tuned on one task can match a large general model on that task.

A fine-tuned model still takes a prompt. Train it with the prompt it will be served with.

If what is missing is knowledge, not behavior, try retrieval-augmented generation (RAG) first: see fine-tuning vs RAG. A prompt also does not enforce anything: see system prompt vs guardrails.

In Halo

A first test does not need a cluster. Halo's Qwen3 4B LoRA recipe runs on one GPU:

Bash

halo launch sft examples/sft/qwen3/qwen3-4b-ultrachat-lora.yaml

The quickstart walks through it.

Train your own models

Post-train any Hugging Face model with Halo. Up to 4× faster than TRL