Datasets
A code-contests row carries the problem statement as prompt and its test set as answer.
Payload
answeris the grading payload, a JSON string or dict, and is required (requires_answer: falseraises). A bare test list,{"test_cases": [...]}and the full form below are accepted.- A payload that holds no tests or does not parse fails the episode at reset as a rollout error, which the trainer drops from the group baseline.
JSON
{"prompt": "<problem statement>",
"answer": "{\"tests\": [{\"input\": \"2 3\\n\", \"output\": \"5\\n\"}], \"checker\": null, \"time_limit\": 2.0}"}checker and time_limit are per-problem overrides (Special judges,
Time limits).
Adapters
Adapters live in halo/environments/tasks/coding/contest_datasets.py:
Building a pool
scripts/environments/preparation/prepare_code_dataset.py builds a training pool from the three RL adapters. It
composes the statement, packs the payload, and drops rows this environment cannot grade.
Bash
python scripts/environments/preparation/prepare_code_dataset.py \
--adapter hardtests --dataset sigcp/hardtests_problems --min_rating 800 \
--tests_table "$HALO_DATA_ROOT/hardtests-tests-compact" \
--holdout_per_band 100 --push_to_hub your-org/code-contests-hardtests-rl --push_bandsA bulky test corpus goes through scripts/environments/preparation/compact_code_tests.py first. It reduces
open-r1's generated tests or HardTests' encoded suites to one row per problem: at most 40 tests of up to 256 KB
each, two of which may reach 4 MB so a maximum-size input survives.
Examples-only problems
An examples-only problem is graded only on its statement's example inputs, which the prompt shows with their
answers, so a program printing those answers passes. The codeforces adapter marks a row whose official tests are
the statement's samples and that no joined suite covers: in open-r1/codeforces verifiable, 219 of the 422 test
problems and 1,303 of the 8,299 gradable train ones. The preparation script drops them and the
evaluation leaves them out, each unless --include_examples_only.
Source on GitHubUpdated 18 November 2026