⌘K
5 benchmarks
Aime2026
inspect harness benchmark (inspect_evals/aime2026)
Best0.767
2 models · 1 impl
2 runs
Pokey bench (v2: lean state)
Pallet Town starter + reach Viridian; fixed dialogue + seamless walk/transition + lean state payload (drops redundant collision grid) to keep SUT context small.
Best1.000
1 model · 1 impl
1 run
Starter & Viridian (agent, colour, 0.5s)
Long-form Pallet Town on NousResearch/pokemon-agent in Game Boy COLOR mode. Frames sampled at 0.5s and encoded to a film. Leave the house, get a starter, reach Viridian City; staged partial credit.
Best1.000
1 model · 1 impl
1 run
pallet-town-reach-viridian
Harder Pallet Town screen-game task: leave the house, get a starter from Oak, and travel north through Route 1 to reach Viridian City (a strict superset of reach-oaks-lab).
Best1.000
1 model · 1 impl
1 run
runebench
An agentic terminal/CLI benchmark where each task is a containerized environment testing an agent's ability to operate a shell to solve real-world problems.
Best28
1 model · 1 impl
1 run