QA Benchmark Environments
Two registry names cover question answering with rule-based rewards (halo/environments/tasks/qa.py). Both speak native tool calls, so a tool-carrying variant needs a tool-call parser on the server (Tool calls).
Shipped recipe: examples/grpo/environmental/qwen3_5/vllm/qwen3.6-35b-a3b-exam-qa-full-ep4.yaml.
Configuration
YAML
environment_type: qa_search # or exam_qa
max_turns: 10 # qa_search default 10, exam_qa 8
environment_kwargs:
search_backend: duckduckgo
include_python_tools: falseA sixth backend, mock, returns fabricated snippets and is refused unless HALO_ALLOW_MOCK_SEARCH=1. Its results pay tool_success_reward, so training on it teaches the policy that invented evidence works.
Tools
web_search— query plus optionalmax_results(5); returns titles, snippets and URLs. Always onqa_search; onexam_qaunderopen_book.python— onqa_searchunderinclude_python_tools.
Reward
Grading is all-or-nothing, 1 or 0 (halo/rewards/graders/matching.py), priced by the reward's environment term (Reward Terms).
exact_match— case-insensitive, after normalization.numeric_match— every number in the response against the one value the expected answer states ($18and18 dollarsexpect 18). Tolerancertol=0.01/atol=1e-6; percentages are divided by 100, and3,500or10\,000read as one number.
There is no fuzzy or substring matcher: "7" must not match "17".
Online GRPO's accuracy term grades the same row differently: a strict boxed exact match that splits #### and strips ,/$ (Online GRPO → Rewards).
One committed value
The response must commit to one value. These grade 0:
- a hedge (
7 or 8); - a correct answer restated with other numbers (
7, since 3 + 4 = 7); - a number inside arithmetic, a power, root, constant or function (
1/2,10^3,\sqrt{2},2\pi,log 2), or joined to a variable (1/x,n+1); - a stated bound (
x < 3).
Not operands: a power after a letter is a unit exponent (9.8 m/s^2), and a digit after a letter is part of a token
(H2O), not a number. A number before a letter reads as a number plus a unit, so 2x is not caught, nor are word
forms (square root of 2).
Normalization
- Reads
\boxed{42}, a lone**42**, and leading "The answer is" / "Therefore". Several bold spans (**7** or **8**) stay a hedge. - Takes the rightmost
\boxed{...}and matches braces by depth, so\boxed{\frac{1}{2}}survives. - Does not split a GSM8K-style
#### Nsuffix: reduce such ananswercolumn to the final value before training.
Multiple choice
A row with choices switches exam_qa to letter grading and appends the choices to the prompt, each led by its letter. A choice already labelled with its own letter ("B: Jupiter") is shown as written; more than ten choices are refused.
- The response's letter (A–J) is extracted ("A", "(A)", "The answer is A") and compared to the expected one. In prose only an uppercase letter standing alone counts ("the answer is a tie" and "the answer is definitely C" name no choice); a bracketed or bare letter may be lowercase.
answermay be that letter or a 0-based int index intochoices(MMLU ships the index).- A digit string raises: convert ARC's numeric
answerKeylabels ("1"–"5", 1-based) to letters when preparing the data. - Anything else, an out-of-range index included, raises at episode start.
Dataset
JSON
{"prompt": "What year was the Eiffel Tower completed?", "answer": "1889"}
{"prompt": "Largest planet?", "answer": "B", "choices": ["A: Mars", "B: Jupiter", "C: Saturn"]}- Name
choicesincontext_fields(training) or--context_fields choices(eval), or it never reaches the environment. answeris required for both environments (requires_answer).
Evaluation
Bash
python scripts/environments/inference/eval_env.py --env_type exam_qa \
--dataset <hub-id-or-dir> --split test --prompt_field question --answer_field answer \
--context_fields choices --group_by subject \
--base_url http://localhost:8000/v1 --model <served-name> --num_examples 100Flags, output files and the --training_config contract: Evaluation.
Related pages
- Native Tool-Use — the protocol both environments run on.
- Code Contests — hidden-test competitive programming.
- Environments — registry and
environment_kwargsplumbing.
Source on GitHubUpdated 18 November 2026