# Native Tool-Use Environments

`NativeToolUseEnvironment` (`halo/environments/protocols/native_tool_use.py`) runs the OpenAI/vLLM function-calling protocol: the environment advertises tool schemas, the server's `--tool-call-parser` extracts the calls, and each result returns as a `tool` message.

Serve the model with its family's parser: the trainer refuses a server with none and raises on a wrong one ([Tool calls](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/episodes.md#tool-calls)).

Presets: `native_math` (`calculate`, `python`), `native_coding` (`python_repl`), `native_combined` (math, python, search, simulated files; `search_backend` names the search backend as on [`qa_search`](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/qa-benchmarks.md#configuration)). `qa_search`, `exam_qa`, `swe`, `code_contests`, `codeforces` and `mcp` build on the same protocol, so this page applies to them.

## Configuration

```yaml
environment_type: native_math

environment_kwargs:
  max_tool_calls_per_turn: 5
  turn_overflow_penalty: 0.1
```

| Knob | Default | Effect |
|---|---|---|
| `max_tool_calls_per_turn` | `5` | Calls executed per turn, at least 1; extras are dropped from the stored message |
| `no_tool_use_penalty` | `0` | Charged once on an episode that made no tool call |
| `turn_overflow_penalty` | `0` | Charged once on a truncated episode |
| `length_cutoff_penalty` | `0` | Charged per recovered unproductive turn (engine-cut, or neither a tool call nor visible content). The turn that exhausts the recovery cap, or lands on the last turn, pays `turn_overflow_penalty` instead |
| `tool_budgets` | `{}` | Per-tool episode caps, `{tool: cap}`; `0` disables a tool ([Tool budgets](#tool-budgets)). Refused on code contests, which set their tools' budgets with `max_submissions` and `max_test_calls` |
| `system_prompt` | `None` | Prepended as the episode's system turn |

The [common knobs](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments.md#common-knobs) apply too.

## Tools

- `calculate` — restricted arithmetic and math functions, one `expression`.
- `python` / `python_repl` — Python REPL; in-process and import-free by default.
- `web_search` — `query` and optional `max_results` (default 5).
- `read_file`, `write_file`, `list_files` — a simulated per-episode file store.

The builders in `halo/environments/tools/toolsets.py` return registries: `build_math_tools`, `build_code_tools`, `build_python_tools`, `build_search_tools`, `build_simulated_file_tools` and `build_combined_tools` stateless, `build_session_{code,bash,file}_tools` bound to the episode's persistent [sandbox session](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md). `NativeToolRegistry.combine(a, b)` composes two. Pass `sandbox=` a `SandboxExecutor` to run code in a real interpreter with imports ([Choosing a backend](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md#choosing-a-backend)).

## Reward

The grade, first rule that applies:

1. An episode that never completed grades 0.
2. A `validator` callable in the row's context decides, 1 or 0.
3. Else the row's `answer`, all-or-nothing (exact match, then numeric). A null `answer` cell grades 0 and marks the episode `episode_invalid`, so it leaves the group baseline.
4. Else completing the episode grades 1 (not under [`swe`](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/swe.md#reward), which has no completion fallback).

The reward's `environment` term prices the grade as `reward/objective` ([Reward Terms](https://whitecircle.com/halo/docs/agent-docs/training-methods/grpo/rewards.md#environment-arm)). Shaping adds on top:

- Per call (`reward/turn_shaping`): `+tool_success_reward` / `-tool_error_penalty`, up to `tool_reward_cap`.
- Per episode (`reward/tool_shaping`): the three penalties in [Configuration](#configuration). Overflow is charged on any truncated episode, including one killed mid-flight, but not on one its driver lost.
- A call that ends on a sandbox fault is booked by its class and ends the episode ([Sandbox faults](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md#sandbox-faults)).

### Tool budgets

- A call over its `tool_budgets` cap is refused as a tool error and spends no budget. The reply, `Error: Not executed: this task's budget for <tool> is spent.` (a tool's own `budget_message` replaces the text after `Error:`), names neither the cap nor the calls left.
- A call to a tool capped at `0` gets the same reply; it is a tool error but not a refusal (`ToolDisabled`).

### Argument binding

Every call is checked against the tool's schema before it runs or spends budget (`halo/environments/tools/definitions.py`). A call that fails is refused as a tool error and never ends the episode:

- `Error: <tool>: unknown argument '<name>'; its arguments are …`
- `Error: <tool>: missing a required argument: '<name>'`
- `Error: <tool>: code must be a string, got list` — a list or object for a `string` parameter. A number or boolean there reads as its Python string (`5` as `5`, `true` as `True`), and a null as the argument left out.
- ``Error: <tool>: not run — the stdin argument ends with a stray `</stdin>` tag, …`` — a `string` value ending, past trailing whitespace, in its own parameter's closing tag, on a parameter declared with `refuse_closing_tag` (code contests' `code` and `stdin`): markup of the call leaked into the value. The tag elsewhere in a value, another parameter's tag, or any value of a parameter that does not opt in (an MCP tool's) binds.

Unparseable `arguments` (neither JSON nor a Python literal) bind as none. An MCP property with no top-level `type` takes any value. A call to a tool that does not exist gets an observation listing the real tools.

### Untrainable turns

Three kinds of turn earn nothing and train only on a negative advantage ([Untrainable turns](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/objective.md#untrainable-turns)):

- a turn the engine cut at its token cap;
- a turn with neither a tool call nor visible content (a stop inside its reasoning);
- a turn whose every call was unknown or refused unrun: arguments that do not bind, a spent `tool_budgets` cap, or its handler's `ToolCallRefused`.

A cut or empty turn is nudged and retried within one shared cap ([Unproductive turns](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/episodes.md#unproductive-turns)). A tool call salvaged from a cut turn is never executed; it stays on the turn for a judge's view ([What a judge reads](https://whitecircle.com/halo/docs/agent-docs/training-methods/grpo/rewards/judge.md#what-a-judge-reads)). When the engine's parser returns the unfinished call (vLLM's `qwen3_xml` does), the nudge says the call ran past the length limit.

## Dataset

`{"prompt": str | list[dict], "answer": Any}`.

- For the native presets `answer` is optional: without one, every completed episode grades 1. Subclasses that grade only against it require it ([`requires_answer`](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments.md#common-knobs)).
- Extra columns reach the environment only when the training script's `context_fields` names them.

## Evaluation

```bash
python scripts/environments/inference/eval_env.py \
    --env_type native_math --dataset <hf-id-or-dir> --split test \
    --prompt_field prompt --answer_field answer \
    --base_url http://localhost:8000/v1 --model <served-model> --num_examples 20
```

The endpoint needs tool calling enabled. Flags and the `--training_config` contract: [Evaluation](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/evaluation.md).

## From Python

A custom tool set goes through the constructor ([A custom tool](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/custom.md#a-custom-tool)):

```python
from halo.environments.protocols.native_tool_use import NativeToolUseEnvironment
from halo.environments.tools.toolsets import build_combined_tools

env = NativeToolUseEnvironment(tool_registry=build_combined_tools(), max_turns=10)
```

`AsyncNativeToolUseEnvironment` runs a turn's tool calls concurrently; use it for I/O-bound tools.

## Related pages

- [Environments](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments.md) — registry and common knobs.
- [Custom Environments](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/custom.md) — your own tools and environments.
- [Code Execution Sandboxes](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md) — backends, languages, sessions.
