# Environments

An environment is the multi-turn task an [Async GRPO with Environments](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo.md) run trains against. The protocol (`EnvironmentBase`, `halo/environments/environment_base.py`):

- `reset(prompts, contexts)` opens one episode per prompt.
- `step(episode_ids, actions, contexts)` executes a model turn and returns the next observation.
- `get_trajectories(episode_ids)` hands back the messages and the final reward.

An episode ends when the environment reports it done, or at `max_turns`.

## Registry

`environment_type` resolves through `build_environment` (`halo/environments/registry.py`). `get_registered_environments()` lists the live set; `register_environment` extends it ([Custom Environments](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/custom.md)).

| `environment_type` | Class | Tools | Default `max_turns` |
|---|---|---|---|
| `react_math` | `ReActEnvironment` | `calculate`, `python` | 10 |
| `react_search` | `ReActEnvironment` | `web_search` | 10 |
| `native_math` | `NativeToolUseEnvironment` | `calculate`, `python` | 10 |
| `native_coding` | `NativeToolUseEnvironment` | `python_repl` | 10 |
| `native_combined` | `NativeToolUseEnvironment` | `calculate`, `python`, `web_search`, file ops | 10 |
| `qa_search` | `NativeToolUseEnvironment` (factory) | `web_search` (+ `python` under `include_python_tools`) | 10 |
| `exam_qa` | `ExamQAEnvironment` | none; `web_search` under `open_book` | 8 |
| `swe` | `SweEnvironment` | `run_code`, `run_bash_command`, workspace file ops | 20 |
| `code_contests` | `CodeContestsEnvironment` | scratchpad test tool, `submit_solution` | 15 |
| `codeforces` | `CodeContestsEnvironment` | same, graded by token comparison | 15 |
| `mcp` | `NativeMCPClientEnvironment` | whatever the server advertises | 10 |

The last column is the class's own turn budget, which `max_turns: null` keeps.

The ReAct presets parse the action out of the text. Every other environment uses native tool calls, so the server needs the family's `--tool-call-parser` ([Tool calls](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/episodes.md#tool-calls)).

## Actor runtime

Actors run environments on CPU and inherit the `ray start` daemon's environment, so export what they need before `ray start` on each node ([Ray Cluster](https://whitecircle.com/halo/docs/agent-docs/infrastructure/ray.md#multi-node)).

| Environment | Actor nodes need |
|---|---|
| `code_contests`, `codeforces`, `swe` | A sandbox backend and a large `TMPDIR` ([Code Execution Sandboxes](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md)) |
| `react_math`, `native_math`, `native_coding`, closed-book `exam_qa` | Nothing beyond the image (in-process tools) |
| `react_search`, `qa_search`, `native_combined`, open-book `exam_qa` | Outbound network. `SERPER_API_KEY` / `BRAVE_API_KEY` / `TAVILY_API_KEY` / `KEENABLE_API_KEY` picks a keyed search backend, else keyless DuckDuckGo. `search_backend` names one instead ([QA Benchmarks](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/qa-benchmarks.md#configuration)) |
| `mcp` | The preset's launcher on `PATH` (`npx`; `uvx` for `fetch`) and its credential. An SSE server needs only network to `server_url` |

## Configuration

`EnvironmentConfig` (`halo/configs/grpo_environmental.py`) has four top-level fields. Everything else goes in `environment_kwargs`.

```yaml
environment_type: react_math
rewards:
  - source: environment    # the environment's grade in [0, 1], priced weight × grade ^ exponent
max_turns: null            # null keeps the class default

environment_kwargs:
  carry_reasoning: false
  tool_error_penalty: 0.1
```

- Any other constructor parameter of the resolved class that YAML can spell is settable from `environment_kwargs`; one taking a Python object, and `mcp`'s custom `server_command` / `server_args` / `server_env` ([From Python](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/protocols/mcp.md#from-python)), is not. A key no constructor binds (a typo, or another `environment_type`'s option) raises `TypeError` at construction.
- A boolean knob takes YAML `true` or `false`; any other value, `no` and `off` included, raises at construction.
- The ReAct presets refuse a set `system_prompt`, `mcp` reads `mcp_server` in its factory, and `native_combined` reads `search_backend` in its own.
- `rewards` reaches the constructor as `reward_terms` ([Reward Terms](https://whitecircle.com/halo/docs/agent-docs/training-methods/grpo/rewards.md#environment-arm)).

## Common knobs

Every environment accepts these `environment_kwargs`, beside the top-level `max_turns` and `rewards` ([Configuration](#configuration)).

| Knob | Default | Effect |
|---|---|---|
| `tool_success_reward` / `tool_error_penalty` | `0.05` / `0.1` (`mcp` pays `0.1`; code contests `0` / `0`) | Paid per successful tool call, charged per failed one, logged as `reward/turn_shaping`. A call a sandbox outage ended is unpriced ([Sandbox faults](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md#sandbox-faults)) |
| `tool_reward_cap` | `tool_success_reward × max_turns` | Episode cap on successful-call pay, so call spam cannot out-earn the objective. A closed-book `exam_qa`, with no tool to price, refuses all three pricing knobs |
| `max_observation_chars` | `16384` | Longer tool observations are truncated; an integer `>= 0`, `0` = no cap |
| `reasoning_effort` | `None` (code contests `medium`) | `low` / `medium` / `high` / `random` CoT steer ([Effort levels](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/reasoning.md#effort-levels)) |
| `carry_reasoning` | `false` | Sends the last assistant turn's reasoning back to the engine; vLLM only ([Carried reasoning](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/episodes.md#carried-reasoning)) |
| `max_length_cutoff_recoveries` | `null` | Unproductive turns (engine-cut, or no visible content and no tool call) one episode may retry; an integer `>= 0`, `null` = all within `max_turns` ([Unproductive turns](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/episodes.md#unproductive-turns)) |
| `requires_answer` | `false`; `true` for code contests, `exam_qa`, `qa_search`, the ReAct presets and [`swe`](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/swe.md#reward) (unless judge-only) | The reward grades against the dataset's `answer` column, so a dataset without one is refused at trainer construction; code contests refuse `false` |

Reward and penalty knobs are magnitudes (≥ 0); a negative value raises. A bare `environment_kwargs:` header reads as null and raises: comment it out, or list its keys.

## Pages

**Tool protocols**

- [ReAct](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/protocols/react.md) — Thought / Action / Observation in plain text.
- [Native Tool-Use](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/protocols/native-tool-use.md) — function calling and tool factories; most environments extend it.
- [MCP](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/protocols/mcp.md) — tools from an MCP server.

**Tasks**

- [Code Contests](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/code-contests.md) — competitive programming, hidden-test grading.
- [SWE](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/swe.md) — persistent-workspace agent.
- [QA Benchmarks](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/tasks/qa-benchmarks.md) — `qa_search` and `exam_qa`.

**Infrastructure and authoring**

- [Code Execution Sandboxes](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/sandbox.md) — backends, faults, limits.
- [Custom Environments](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/environments/custom.md) — write, register and test your own.
- [Evaluation](https://whitecircle.com/halo/docs/agent-docs/training-methods/async-grpo/evaluation.md) — run a model through an environment.
