# How to choose a fine-tuning method

To choose a fine-tuning method, start from the data you have. Example answers call for supervised fine-tuning, pairs of better and worse answers for preference tuning, and a way to score answers for reinforcement fine-tuning.

## Steps

1. **List the data you have.** The method follows from the data and the goal, not the other way round.
2. **Match the data to a method.** The table below has the mapping.
3. **Narrow the choice.** With preference pairs, DPO (Direct Preference Optimization) keeps a frozen reference model in memory and SMPO does not. With scores, use offline GRPO (Group Relative Policy Optimization) if the completions are already scored, online GRPO for single-turn answers a rule can check, and async GRPO for multi-turn tool use.
4. **Decide what you want out.** The same pairs can change the model itself, or train a reward model: a scorer to use later.
5. **Start simple.** Most projects start with supervised fine-tuning and add preference tuning or reinforcement learning after it.

## Which to pick

| You have | Use |
| --- | --- |
| Example conversations | [Supervised fine-tuning](/glossary/supervised-fine-tuning) |
| Chosen and rejected answers | [Preference tuning](/glossary/preference-tuning) with DPO or SMPO |
| Single answers marked good or bad | [KTO](/glossary/kto) (Kahneman-Tversky Optimization) |
| A way to score answers | [Reinforcement fine-tuning](/glossary/reinforcement-fine-tuning) with GRPO |
| A larger teacher model | [Knowledge distillation](/glossary/knowledge-distillation) |
| Text with labels | [Classification](/glossary/classification-fine-tuning) |
| Text pairs, triplets or scored pairs | [Embedding fine-tuning](/glossary/embedding-fine-tuning) |

## With Halo

Every method uses the same YAML config format and the same parallelism, so switching is a config change, not a rewrite. The method is the name after `halo launch`: `sft`, `dpo`, `smpo`, `kto`, `rewards`, `offline-grpo`, `rlvr`, `environmental-grpo`, `teacher-distill`, `self-distill`, `classification` or `embedding`. Online and async GRPO also need a separate rollout server. [Choosing a method](/halo/docs/choosing-a-method) has the full table.

