Skip to content

\ BENCHMARK CONTROL ROOM \

Benchmark the agent. Attribute the failure.

Versioned tasks, sandboxed runs, deterministic scores, and counterfactual proofs. Start with a baseline, inject a fault, then open the replay.

Tasks
8
Adapters
4
Results
21
Attributed
6
Pass rate
71%
Store
upstash

\ 01 \ TOP RESULTS

The current board.

View full leaderboard →
Top results with reliability
RankAdapterTaskModelScorePass@kReliability
1OpenAI Codex CLI (openai/gpt-5.1-codex)is-hexadecimal-single-charopenai/gpt-5.1-codex100.0100%
1
2Aider 0.86.2 (anthropic/claude-sonnet-4.5)is-word-character-single-charanthropic/claude-sonnet-4.5100.0100%
1
3OpenAI Codex CLI (openai/gpt-5.1-codex)is-number-reject-hexopenai/gpt-5.1-codex100.0100%
1
4OpenAI Codex CLI (openai/gpt-5.1-codex)is-whitespace-single-charopenai/gpt-5.1-codex100.0100%
1

\ 02 \ TASKS

Versioned, immutable, scored.

Browse all →

\ 03 \ RECENT RUNS

The last five hand-offs.

All results →

\ 04 \ ATTRIBUTION

One intervention. One proof.

Replay reconstructs the workspace from the recorded files, applies the intercepted action, and re-runs the real test command in the sandbox. Before step k the prefix is byte-identical. No model call enters the score.

Open a real proof →

baseline

0.0

→

intervention

retry @ 2

→

replay

3 steps

→

proof

100.0

PASS

\ 05 \ QUICK START

Three commands.

Organizer runs execute in Docker. The public app reads the immutable result and never runs an arbitrary command.

$ orivoxx bench --task itsdangerous-future-signature --chaos failing_tools

0.0 · harness × recovery · 3 replayed steps

$ orivoxx replay --result res_c1ba8d7

retry_policy@2 → 100.0 · PASS

$ orivoxx diff --a res_c1ba8d7 --b res_85394c

+100.0 score · real test passes

\ YOUR TASKS \

Submit a repo issue or review your validations.

\ 06 \ RESILIENCE

Inject the fault. Measure the recovery.

tool_error

A write fails with EACCES. The harness either retries or loses the artifact.

Run with tool_error →

failing_tools

The real-task apply_patch is intercepted. The replay retries it and re-runs the real test.

Run with failing_tools →

latency

Every model call is delayed. The stopping policy is tested against the clock.

Run with latency →

context_pressure

Context is truncated and the step budget shrinks. Completeness is tested.

Run with context_pressure →