SWE Environment
SweEnvironment (halo/environments/tasks/coding/swe.py, registry name swe) trains an edit-run-test loop over a workspace that persists across turns. It speaks native tool calls, so the server needs the family's tool-call parser (Tool calls).
- No third-party agent scaffold (SWE-agent, mini-SWE-agent, OpenHands, opencode) is wired in: the model gets the tools below and nothing else.
- Each episode gets its own sandbox session, so one instance serves concurrent rollouts.
- No shipped recipe targets
swe; start fromexamples/grpo/environmental/environmental-grpo-template.yaml.
Configuration
YAML
environment_type: swe
max_turns: 20
environment_kwargs:
language: python
code_timeout: 15.0max_turns defaults to the class's own budget (registry). Reward knobs are the native protocol's.
Tools
run_code— compiles if needed and runs a program in the workspace; stdout is the output.run_bash_command— runs onebashcommand in the workspace under the samecode_timeout.- A zero exit returns stdout only; empty output reads
Code executed successfully (no output). Diagnostics need an explicit2>&1. - A non-zero exit leads with
Error:and the killing signal or the tail of stderr, then stdout. It is an observation, not a tool error.
- A zero exit returns stdout only; empty output reads
write_file— writes a file later turns andrun_codesee.read_file— reads one back; a missing file is a message, not a tool error.list_files— lists the workspace, optionally filtered by a path prefix.
A sandbox fault on any tool ends the episode (Sandbox faults).
Isolation
Code runs in a SandboxExecutor subprocess with imports and the standard library (Choosing a backend).
swe warns when built on a backend that does not confine the program: local, or bubblewrap with allow_network. Run untrusted code and shell on bubblewrap without network, or on remote.
Reward
An episode that never completed grades 0. A completed one is graded by the first that applies:
- A
test_functionpassed to the constructor: its verdict, 1 or 0. One that raises grades 0 and marks the episode invalid, out of the group baseline. - Else the row's
validatorcallable, or itsanswermatched against the final reply (exact match, then numeric). The workspace is never inspected. A nullanswercell marks the episode invalid.
There is no completion fallback:
- Without a
test_function,requires_answerdefaults on: the trainer andeval_env.pyrefuse a dataset with noanswercolumn, and the playground an empty Expected Answer box. requires_answer: falsewithout atest_functionraises at construction.- A row carrying neither raises at grading.
- A judge-only reward (no
environmentterm) takes no grade from the environment, so it needs neither, and a nullansweror a raisingtest_functionvoids nothing.
The environment term prices the grade as reward/objective (Reward Terms); the native protocol's shaping adds on top (Native Tool-Use → Reward).
Dataset
{"prompt": ..., "answer": ...}; the answer column is optional only under a test_function or a judge-only reward (Reward).
Evaluation
Bash
python scripts/environments/inference/eval_env.py --env_type swe \
--dataset <hub-id-or-dir> --base_url http://localhost:8000/v1 \
--model <served-name> --num_examples 20Flags and the --training_config contract: Evaluation.
From Python
test_function (grades the finished trajectory) and extra_tools (a registry merged in) are Python objects, so only the constructor sets them; a test_function that is no callable, a YAML string included, raises:
Python
env = SweEnvironment(language="python", test_function=my_test_fn, extra_tools=my_tools)Related pages
- Code Execution Sandboxes — backends, languages, limits.
- Code Contests — hidden-test grading.
Source on GitHubUpdated 18 November 2026