Chat templates turn a list of messages into the single token sequence a model reads, with special tokens that mark each role and turn. A model has to be trained and served under the same template.
How it works
A chat model reads one sequence of tokens, not messages. The template joins the system, user and assistant messages into that sequence and wraps each turn in special tokens:
Python
messages = [{"role": "user", "content": "Explain gravity."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)With a ChatML template, text is <|im_start|>user\nExplain gravity.<|im_end|>\n<|im_start|>assistant\n.
Templates differ between model families: other tokens, other rules for system prompts, tools and reasoning. In Hugging Face models the template is written in Jinja and stored with the tokenizer, so a dataset holds plain messages.
Why it matters
A model learns its template in training. Serve it under another one (a wrong role tag, a missing end-of-turn token) and the prompt no longer looks like anything it saw. Nothing raises an error. The answers get worse: the model rambles, does not stop, or ignores the system prompt.
In Halo
In supervised fine-tuning, Halo renders every row with tokenizer.apply_chat_template before tokenization, using the model's own template by default. chat_template overrides it with a path to a .jinja file or an inline template string, and needs force_chat_template: true when the tokenizer already carries one.
assistant_message_template, the marker used to mask the prompt out of the loss, must byte-match what the template renders for an assistant turn. See the supervised fine-tuning guide.
