Leaderboard
26 results · score desc, then earliest result
| # | Score | Task | Adapter | Model | Spec | Sec | Maint | Pass@k | Reliability | Flags | Created |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 100.0 | is-hexadecimal-single-char | OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run | openai/gpt-5.1-codex | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 07:49 |
| 2 | 100.0 | is-word-character-single-char | Aider 0.86.2 (anthropic/claude-sonnet-4.5)Organizer-run | anthropic/claude-sonnet-4.5 | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 07:59 |
| 3 | 100.0 | is-number-reject-hex | OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run | openai/gpt-5.1-codex | 100.0 | 100.0 | 77.5 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 08:21 |
| 4 | 100.0 | is-whitespace-single-char | OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run | openai/gpt-5.1-codex | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 08:53 |
| 5 | 100.0 | is-decimal-single-char | Claude Code (anthropic/claude-sonnet-4.5)Organizer-run | anthropic/claude-sonnet-4.5 | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 08:54 |
| 6 | 100.0 | slugify-max-length | Claude Code (anthropic/claude-sonnet-4.5)Organizer-run | anthropic/claude-sonnet-4.5 | 100.0 | 100.0 | 85.0 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 19:02 |
| 7 | 100.0 | is-hexadecimal-50b2060e | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 19:57 |
| 8 | 100.0 | is-decimal-14c16dec | Python agent loop (openai/gpt-5.1)Organizer-run | openai/gpt-5.1 | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 20:06 |
| 9 | 100.0 | is-decimal-800937e9 | Python agent loop (openai/gpt-5.1)Organizer-run | openai/gpt-5.1 | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 20:06 |
| 10 | 100.0 | is-decimal-800937e9 | Claude Code (anthropic/claude-haiku-5.5)Organizer-run | anthropic/claude-haiku-5.5 | 100.0 | 100.0 | 81.7 | 100% | 1 runs · 100% pass | — | 20 Nov 2026, 20:42 |
| 11 | 99.5μ 86.3 · σ 18.7 | taskboard | Fake fixture (taskboard/compliant)Sample | no model | 100.0 | 100.0 | 96.7 | 67% | 3 runs · 67% pass | — | 15 Nov 2026, 15:38 |
| 12 | 99.5μ 86.3 · σ 18.7 | taskboard | Fake fixture (taskboard/compliant)Sample | no model | 100.0 | 100.0 | 96.7 | 67% | 3 runs · 67% pass | — | 18 Nov 2026, 06:08 |
| 13 | 99.3μ 74.6 · σ 24.7 | linkstash | Fake fixture (linkstash/compliant)Sample | no model | 100.0 | 100.0 | 95.0 | 50% | 2 runs · 50% pass | — | 15 Nov 2026, 15:38 |
| 14 | 98.1μ 71.7 · σ 26.4 | taskboard | Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run | deepseek/deepseek-chat-v3-0324 | 100.0 | 100.0 | 87.1 | 50% | 8 runs · 50% pass | — | 15 Nov 2026, 18:50 |
| 15 | 97.3μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 82.0 | 50% | 8 runs · 50% pass | — | 15 Nov 2026, 18:46 |
| 16 | 97.3 | taskboard | Aider 0.86.2 (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 82.0 | 100% | 1 runs · 100% pass | — | 15 Nov 2026, 18:49 |
| 17 | 96.4μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 76.0 | 50% | 8 runs · 50% pass | — | 18 Nov 2026, 11:15 |
| 18 | 94.9μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 95.2 | 100.0 | 85.0 | 50% | 8 runs · 50% pass | — | 18 Nov 2026, 11:37 |
| 19 | 59.9μ 86.3 · σ 18.7 | taskboard | Fake fixture (taskboard/regressed)Sample | no model | 52.4 | 100.0 | 90.0 | 67% | 3 runs · 67% pass | Capped | 15 Nov 2026, 15:38 |
| 20 | 59.9μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 61.9 | 100.0 | 82.0 | 50% | 8 runs · 50% pass | Capped | 18 Nov 2026, 11:37 |
| 21 | 49.9μ 74.6 · σ 24.7 | linkstash | Fake fixture (linkstash/regressed)Sample | no model | 50.0 | 33.3 | 90.0 | 50% | 2 runs · 50% pass | Capped | 15 Nov 2026, 15:38 |
| 22 | 49.9μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 33.3 | 66.7 | 92.5 | 50% | 8 runs · 50% pass | Capped | 18 Nov 2026, 11:26 |
| 23 | 49.7μ 71.7 · σ 26.4 | taskboard | Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run | deepseek/deepseek-chat-v3-0324 | 33.3 | 66.7 | 87.1 | 50% | 8 runs · 50% pass | Capped | 18 Nov 2026, 11:33 |
| 24 | 27.4μ 71.7 · σ 26.4 | taskboard | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 0.0 | 66.7 | 71.7 | 50% | 8 runs · 50% pass | Capped | 18 Nov 2026, 11:16 |
| 25 | 0.0 | itsdangerous-future-signature | python (chaos: failing_tools)Organizer-run | anthropic/claude-sonnet-4.5 | 0.0 | 100.0 | 90.0 | 0% | 1 runs · 0% pass | — | 20 Nov 2026, 18:49 |
| 26 | 0.0 | is-email-predicate | OpenAI Codex CLI (openai/gpt-5.1-codex)Organizer-run | openai/gpt-5.1-codex | 0.0 | 100.0 | 88.0 | 0% | 1 runs · 0% pass | — | 20 Nov 2026, 19:38 |
ids: res_600642…0ec5 · res_2016ec…b588 · res_9da748…3171 · res_5d3d11…e440 · res_77ab9c…aa69 · res_85394c…bd99 · res_89be31…8cdf · res_3b0026…24f5 · res_2ea69a…d3bf · res_74101d…5502 · res_1ab24b…662a · res_6d753e…2b41 · res_bb227a…20ef · res_716261…2fea · res_f74d49…f247 · res_a39c6e…63b4 · res_df4fc5…dd7e · res_186441…167c · res_0e0868…71e1 · res_7607dd…cb71 · res_2ac581…2981 · res_0427d9…1735 · res_64068a…04a4 · res_6d4fdf…d1fc · res_c1ba8d…b8b7 · res_e2767b…afca
A leaderboard-valid row records task, ruleset, scoring-config, and adapter versions plus a repository fingerprint. Rows missing any of them are rejected by the store schema.