Compare
Two results on one task. The same model slug means only the harness varies.
taskboardtask 1.0.0 · ruleset 1.0.0 · scoring 1.0.09 results on this task
A attribution
harness × recovery
proof 49.7 → 98.3
B attribution
none
no proof
| Metric | A | B | B − A |
|---|---|---|---|
| Steps | 7 | 0 | -7 |
| Tool calls | 6 | 0 | -6 |
| Tool error rate | 0.167 | 0 | -0.167 |
| Total tokens | 10,487 | 0 | -10,487 |
| Cost USD | 0.004 | 0 | -0.004 |
| Duration ms | 28,488 | 0 | -28,488 |
Different modelThe harness and the model both differ, so the score gap is not attributable to one.
A
Organizer-runPython agent loop (deepseek/deepseek-chat-v3-0324)
deepseek/deepseek-chat-v3-0324
49.7
- Spec
- 33.3
- Security
- 66.7
- Maintainability
- 87.1
5 pass4 fail0 unverifiedcapped
Open resultB
Organizer-runAider 0.86.2 (openai/gpt-4o-mini)
openai/gpt-4o-mini
97.3
- Spec
- 100.0
- Security
- 100.0
- Maintainability
- 82.0
9 pass0 fail0 unverified
Open result| Bench | A | B | B − A |
|---|---|---|---|
| Overall | 49.7 | 97.3 | +47.6 |
| Spec | 33.3 | 100.0 | +66.7 |
| Security | 66.7 | 100.0 | +33.3 |
| Maintainability | 87.1 | 82.0 | -5.1 |
| Requirement | Bench | A | B |
|---|---|---|---|
| REQ-AUTH-001Project access must be owner-scoped | spec | FAIL | PASS |
| REQ-DEP-001No shell execution dependency | security | PASS | PASS |
| REQ-ERR-001Project API returns structured error responses | spec | FAIL | PASS |
| REQ-PATH-001Projects API route exists | spec | FAIL | PASS |
| REQ-SEC-002No environment file in the repository | security | PASS | PASS |
| REQ-SEC-003No dynamic code evaluation in the project route | security | FAIL | PASS |
| REQ-TASKS-001Task list API route exists | spec | PASS | PASS |
| REQ-TASKS-002Task list reads through the repository layer | spec | PASS | PASS |
| REQ-UI-001Task list component exists | spec | PASS | PASS |