# Synthetic data

Synthetic data is training data written by a model instead of collected from people. It is used to start a dataset from a few examples, to copy a larger model's behavior into a smaller one, and to cover rare cases.

## What it is used for

- **Bootstrapping.** From a handful of real examples, a strong model writes many more in the same form, enough for a first fine-tune.
- **Distillation.** A large model answers real prompts and a smaller one is tuned on its answers. See [knowledge distillation](/glossary/knowledge-distillation).
- **Rare cases.** Edge cases, refusals and unusual formats that real logs hold too few of.

## Risks

- **Errors are copied.** The tuned model learns the generator's mistakes and made-up facts along with its skills.
- **Low diversity.** A model asked for a thousand examples repeats topics, phrasings and structures. A model tuned on them can [overfit](/glossary/overfitting) to that narrow style.

## How to check it

- Read a sample by hand before training.
- Remove duplicates and near-duplicates.
- Filter with a grader: rules, a verifier, or a second model that scores each row.
- Vary the generation prompts, each seeded with a different topic or real example.
- Keep the test set real. [Evaluate](/glossary/how-to-evaluate-a-fine-tuned-model) on data people wrote.

## In Halo

Halo ships tools for this, each run as `halo run <tool>`: `openai-batched-generation` for batched generation against a vLLM or OpenAI endpoint, `dataset-deduplication` to deduplicate generated or collected data, and `rm-rejection-sampling` for best-of-N with a [reward model](/glossary/reward-model). See [the halo CLI](/halo/docs/cli).

