Skip to content

Leaderboard

26 results · score desc, then earliest result

Benchmark leaderboard
#ScoreTaskAdapterModelSpecSecMaintPass@kReliabilityFlagsCreated
1100.0is-hexadecimal-single-char
OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run
openai/gpt-5.1-codex100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 07:49
2100.0is-word-character-single-char
Aider 0.86.2 (anthropic/claude-sonnet-4.5)Organizer-run
anthropic/claude-sonnet-4.5100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 07:59
3100.0is-number-reject-hex
OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run
openai/gpt-5.1-codex100.0100.077.5100%

1 runs · 100% pass

—20 Nov 2026, 08:21
4100.0is-whitespace-single-char
OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run
openai/gpt-5.1-codex100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 08:53
5100.0is-decimal-single-char
Claude Code (anthropic/claude-sonnet-4.5)Organizer-run
anthropic/claude-sonnet-4.5100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 08:54
6100.0slugify-max-length
Claude Code (anthropic/claude-sonnet-4.5)Organizer-run
anthropic/claude-sonnet-4.5100.0100.085.0100%

1 runs · 100% pass

—20 Nov 2026, 19:02
7100.0is-hexadecimal-50b2060e
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 19:57
8100.0is-decimal-14c16dec
Python agent loop (openai/gpt-5.1)Organizer-run
openai/gpt-5.1100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 20:06
9100.0is-decimal-800937e9
Python agent loop (openai/gpt-5.1)Organizer-run
openai/gpt-5.1100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 20:06
10100.0is-decimal-800937e9
Claude Code (anthropic/claude-haiku-5.5)Organizer-run
anthropic/claude-haiku-5.5100.0100.081.7100%

1 runs · 100% pass

—20 Nov 2026, 20:42
1199.5μ 86.3 · σ 18.7taskboard
Fake fixture (taskboard/compliant)Sample
no model100.0100.096.767%

3 runs · 67% pass

—15 Nov 2026, 15:38
1299.5μ 86.3 · σ 18.7taskboard
Fake fixture (taskboard/compliant)Sample
no model100.0100.096.767%

3 runs · 67% pass

—18 Nov 2026, 06:08
1399.3μ 74.6 · σ 24.7linkstash
Fake fixture (linkstash/compliant)Sample
no model100.0100.095.050%

2 runs · 50% pass

—15 Nov 2026, 15:38
1498.1μ 71.7 · σ 26.4taskboard
Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run
deepseek/deepseek-chat-v3-0324100.0100.087.150%

8 runs · 50% pass

—15 Nov 2026, 18:50
1597.3μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.082.050%

8 runs · 50% pass

—15 Nov 2026, 18:46
1697.3taskboard
Aider 0.86.2 (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.082.0100%

1 runs · 100% pass

—15 Nov 2026, 18:49
1796.4μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.076.050%

8 runs · 50% pass

—18 Nov 2026, 11:15
1894.9μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini95.2100.085.050%

8 runs · 50% pass

—18 Nov 2026, 11:37
1959.9μ 86.3 · σ 18.7taskboard
Fake fixture (taskboard/regressed)Sample
no model52.4100.090.067%

3 runs · 67% pass

Capped15 Nov 2026, 15:38
2059.9μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini61.9100.082.050%

8 runs · 50% pass

Capped18 Nov 2026, 11:37
2149.9μ 74.6 · σ 24.7linkstash
Fake fixture (linkstash/regressed)Sample
no model50.033.390.050%

2 runs · 50% pass

Capped15 Nov 2026, 15:38
2249.9μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini33.366.792.550%

8 runs · 50% pass

Capped18 Nov 2026, 11:26
2349.7μ 71.7 · σ 26.4taskboard
Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run
deepseek/deepseek-chat-v3-032433.366.787.150%

8 runs · 50% pass

Capped18 Nov 2026, 11:33
2427.4μ 71.7 · σ 26.4taskboard
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini0.066.771.750%

8 runs · 50% pass

Capped18 Nov 2026, 11:16
250.0itsdangerous-future-signature
python (chaos: failing_tools)Organizer-run
anthropic/claude-sonnet-4.50.0100.090.00%

1 runs · 0% pass

—20 Nov 2026, 18:49
260.0is-email-predicate
OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run
openai/gpt-5.1-codex0.0100.088.00%

1 runs · 0% pass

—20 Nov 2026, 19:38

ids: res_600642…0ec5 · res_2016ec…b588 · res_9da748…3171 · res_5d3d11…e440 · res_77ab9c…aa69 · res_85394c…bd99 · res_89be31…8cdf · res_3b0026…24f5 · res_2ea69a…d3bf · res_74101d…5502 · res_1ab24b…662a · res_6d753e…2b41 · res_bb227a…20ef · res_716261…2fea · res_f74d49…f247 · res_a39c6e…63b4 · res_df4fc5…dd7e · res_186441…167c · res_0e0868…71e1 · res_7607dd…cb71 · res_2ac581…2981 · res_0427d9…1735 · res_64068a…04a4 · res_6d4fdf…d1fc · res_c1ba8d…b8b7 · res_e2767b…afca

A leaderboard-valid row records task, ruleset, scoring-config, and adapter versions plus a repository fingerprint. Rows missing any of them are rejected by the store schema.