# What is KTO?

KTO (Kahneman-Tversky Optimization) is a preference-tuning method that needs no pairs. Each row is one answer labeled good or bad, and the model learns to make good answers more likely and bad ones less.

## How it works

[DPO](/glossary/dpo) (Direct Preference Optimization) needs two answers to the same prompt, one better than the other. KTO needs one answer and a label: desirable or undesirable. That is the shape of a thumbs-up or thumbs-down on a reply.

The loss borrows from prospect theory, Kahneman and Tversky's account of how people weigh gains and losses. It measures how much more likely the model makes an answer than a frozen reference model does, and compares that with a baseline estimated from the rest of the batch. Desirable answers are pushed above the baseline and undesirable ones below it. The [KTO paper](https://arxiv.org/abs/2402.01306) has the derivation.

## When to use it

Use KTO when you never collected pairs, only single answers marked good or bad. With pairs, DPO and [SMPO](/glossary/smpo) read them as they are.

## In Halo

One row is a prompt, a completion and a boolean label:

```jsonl
{"prompt": [{"role": "user", "content": "What is 2+2?"}], "completion": [{"role": "assistant", "content": "4"}], "label": true}
```

Start a run with `halo launch kto` and a config such as `examples/preference/qwen3_5/kto-qwen3.5-9b-kto-mix-14k.yaml`. The default `kto` loss builds its baseline, a Kullback-Leibler (KL) term, from each row's neighbors in the batch, so batch size 1 raises. Set `desirable_weight` and `undesirable_weight` to the inverse label ratio when the labels are skewed. See the [preference guide](/halo/docs/training-methods/preference#kto).

