Native Tool-Use Environments

NativeToolUseEnvironment (halo/environments/protocols/native_tool_use.py) runs the OpenAI/vLLM function-calling protocol: the environment advertises tool schemas, the server's --tool-call-parser extracts the calls, and each result returns as a tool message.

Serve the model with its family's parser: the trainer refuses a server with none and raises on a wrong one (Tool calls).

Presets: native_math (calculate, python), native_coding (python_repl), native_combined (math, python, search, simulated files; search_backend names the search backend as on qa_search). qa_search, exam_qa, swe, code_contests, codeforces and mcp build on the same protocol, so this page applies to them.

Configuration

YAML

environment_type: native_math

environment_kwargs:
  max_tool_calls_per_turn: 5
  turn_overflow_penalty: 0.1
KnobDefaultEffect
max_tool_calls_per_turn5Calls executed per turn, at least 1; extras are dropped from the stored message
no_tool_use_penalty0Charged once on an episode that made no tool call
turn_overflow_penalty0Charged once on a truncated episode
length_cutoff_penalty0Charged per recovered unproductive turn (engine-cut, or neither a tool call nor visible content). The turn that exhausts the recovery cap, or lands on the last turn, pays turn_overflow_penalty instead
tool_budgets{}Per-tool episode caps, {tool: cap}; 0 disables a tool (Tool budgets). Refused on code contests, which set their tools' budgets with max_submissions and max_test_calls
system_promptNonePrepended as the episode's system turn

The common knobs apply too.

Tools

  • calculate — restricted arithmetic and math functions, one expression.
  • python / python_repl — Python REPL; in-process and import-free by default.
  • web_search — query and optional max_results (default 5).
  • read_file, write_file, list_files — a simulated per-episode file store.

The builders in halo/environments/tools/toolsets.py return registries: build_math_tools, build_code_tools, build_python_tools, build_search_tools, build_simulated_file_tools and build_combined_tools stateless, build_session_{code,bash,file}_tools bound to the episode's persistent sandbox session. NativeToolRegistry.combine(a, b) composes two. Pass sandbox= a SandboxExecutor to run code in a real interpreter with imports (Choosing a backend).

Reward

The grade, first rule that applies:

  1. An episode that never completed grades 0.
  2. A validator callable in the row's context decides, 1 or 0.
  3. Else the row's answer, all-or-nothing (exact match, then numeric). A null answer cell grades 0 and marks the episode episode_invalid, so it leaves the group baseline.
  4. Else completing the episode grades 1 (not under swe, which has no completion fallback).

The reward's environment term prices the grade as reward/objective (Reward Terms). Shaping adds on top:

  • Per call (reward/turn_shaping): +tool_success_reward / -tool_error_penalty, up to tool_reward_cap.
  • Per episode (reward/tool_shaping): the three penalties in Configuration. Overflow is charged on any truncated episode, including one killed mid-flight, but not on one its driver lost.
  • A call that ends on a sandbox fault is booked by its class and ends the episode (Sandbox faults).

Tool budgets

  • A call over its tool_budgets cap is refused as a tool error and spends no budget. The reply, Error: Not executed: this task's budget for <tool> is spent. (a tool's own budget_message replaces the text after Error:), names neither the cap nor the calls left.
  • A call to a tool capped at 0 gets the same reply; it is a tool error but not a refusal (ToolDisabled).

Argument binding

Every call is checked against the tool's schema before it runs or spends budget (halo/environments/tools/definitions.py). A call that fails is refused as a tool error and never ends the episode:

  • Error: <tool>: unknown argument '<name>'; its arguments are …
  • Error: <tool>: missing a required argument: '<name>'
  • Error: <tool>: code must be a string, got list — a list or object for a string parameter. A number or boolean there reads as its Python string (5 as 5, true as True), and a null as the argument left out.
  • Error: <tool>: not run — the stdin argument ends with a stray `</stdin>` tag, … — a string value ending, past trailing whitespace, in its own parameter's closing tag, on a parameter declared with refuse_closing_tag (code contests' code and stdin): markup of the call leaked into the value. The tag elsewhere in a value, another parameter's tag, or any value of a parameter that does not opt in (an MCP tool's) binds.

Unparseable arguments (neither JSON nor a Python literal) bind as none. An MCP property with no top-level type takes any value. A call to a tool that does not exist gets an observation listing the real tools.

Untrainable turns

Three kinds of turn earn nothing and train only on a negative advantage (Untrainable turns):

  • a turn the engine cut at its token cap;
  • a turn with neither a tool call nor visible content (a stop inside its reasoning);
  • a turn whose every call was unknown or refused unrun: arguments that do not bind, a spent tool_budgets cap, or its handler's ToolCallRefused.

A cut or empty turn is nudged and retried within one shared cap (Unproductive turns). A tool call salvaged from a cut turn is never executed; it stays on the turn for a judge's view (What a judge reads). When the engine's parser returns the unfinished call (vLLM's qwen3_xml does), the nudge says the call ran past the length limit.

Dataset

{"prompt": str | list[dict], "answer": Any}.

  • For the native presets answer is optional: without one, every completed episode grades 1. Subclasses that grade only against it require it (requires_answer).
  • Extra columns reach the environment only when the training script's context_fields names them.

Evaluation

Bash

python scripts/environments/inference/eval_env.py \
    --env_type native_math --dataset <hf-id-or-dir> --split test \
    --prompt_field prompt --answer_field answer \
    --base_url http://localhost:8000/v1 --model <served-model> --num_examples 20

The endpoint needs tool calling enabled. Flags and the --training_config contract: Evaluation.

From Python

A custom tool set goes through the constructor (A custom tool):

Python

from halo.environments.protocols.native_tool_use import NativeToolUseEnvironment
from halo.environments.tools.toolsets import build_combined_tools

env = NativeToolUseEnvironment(tool_registry=build_combined_tools(), max_turns=10)

AsyncNativeToolUseEnvironment runs a turn's tool calls concurrently; use it for I/O-bound tools.

Source on GitHubUpdated 18 November 2026