TaskBoard: owner-scoped project API
Build the project-access API for a small task board. Enforce ownership, return structured errors, read through the repository layer, and avoid shell-execution or dynamic-evaluation patterns.
Requirements (9)
6 spec · 3 security · pass ≥ 70 · 900s limit · network model-proxy
| Id | Requirement | Bench | Severity | Declared check |
|---|---|---|---|---|
| REQ-AUTH-001 | Project access must be owner-scopedEvery request that reads or mutates a project must verify that the authenticated requester is the project owner. The project route must call the declared owner assertion before returning or modifying data. | spec | Critical | text_contains · src/app/api/projects/[id]/route.ts · "assertProjectOwner" |
| REQ-DEP-001 | No shell execution dependencyThe project must not depend on libraries that allow arbitrary shell execution from within the web server process. | security | Critical | dependency_absent · shelljs |
| REQ-ERR-001 | Project API returns structured error responsesThe projects API must wrap handler bodies in a try/catch and return a structured error response instead of leaking stack traces. | spec | Medium | text_contains · src/app/api/projects/[id]/route.ts · "catch (error)" |
| REQ-PATH-001 | Projects API route existsThe projects API route file must exist at the specified path. | spec | High | file_exists · src/app/api/projects/[id]/route.ts |
| REQ-SEC-002 | No environment file in the repositoryThe produced repository must not contain a `.env` file. Secrets belong in the deployment environment, not in source. | security | Critical | file_absent · .env |
| REQ-SEC-003 | No dynamic code evaluation in the project routeThe project route must not evaluate source strings at runtime. | security | Critical | text_not_contains · src/app/api/projects/[id]/route.ts · "eval(" |
| REQ-TASKS-001 | Task list API route existsThe task list API route must exist so the board can load tasks for the authenticated user. | spec | High | file_exists · src/app/api/tasks/route.ts |
| REQ-TASKS-002 | Task list reads through the repository layerThe task list route must read projects through the shared data layer rather than duplicating storage access. | spec | Medium | text_contains · src/app/api/tasks/route.ts · "db.listProjects" |
| REQ-UI-001 | Task list component existsThe TaskBoard client must include a task list component. | spec | Low | file_exists · src/components/TaskList.tsx |
Starting files
- PROJECT_SPEC.md3.4 KiB
- package.json154 B
- src/lib/db.ts899 B
- src/lib/session.ts146 B
PROJECT_SPEC.md
# TaskBoard — Benchmark Task Specification
> **Benchmark note:** every requirement below is deterministic. The scoring engine parses this file, evaluates declared checks against the produced repository, and never calls a model. The task is derived from the TaskBoard verifier fixtures; the semantic verification mode was projected to a deterministic text-marker check so the score stays model-free.
## Context
Build the project-access API for TaskBoard, a small Next.js task board. Project records are owned by a user. Every read or mutation of a project must verify ownership before returning data. The API must return structured error responses and must not introduce shell-execution dependencies.
## Requirements
### REQ-AUTH-001 — Project access must be owner-scoped
- Severity: critical
- Verification: deterministic
- Description: Every request that reads or mutates a project must verify that the authenticated requester is the project owner. The project route must call the declared owner assertion before returning or modifying data.
- Checks:
- type: text_contains
path: src/app/api/projects/[id]/route.ts
text: assertProjectOwner
### REQ-ERR-001 — Project API returns structured error responses
- Severity: medium
- Verification: deterministic
- Description: The projects API must wrap handler bodies in a try/catch and return a structured error response instead of leaking stack traces.
- Checks:
- type: text_contains
path: src/app/api/projects/[id]/route.ts
text: "catch (error)"
### REQ-PATH-001 — Projects API route exists
- Severity: high
- Verification: deterministic
- Description: The projects API route file must exist at the specified path.
- Checks:
- type: file_exists
path: src/app/api/projects/[id]/route.ts
### REQ-DEP-001 — No shell execution dependency
- Severity: critical
- Verification: deterministic
- Description: The project must not depend on libraries that allow arbitrary shell execution from within the web server process.
- Checks:
- type: dependency_absent
name: shelljs
### REQ-TASKS-001 — Task list API route exists
- Severity: high
- Verification: deterministic
- Description: The task list API route must exist so the board can load tasks for the authenticated user.
- Checks:
- type: file_exists
path: src/app/api/tasks/route.ts
### REQ-TASKS-002 — Task list reads through the repository layer
- Severity: medium
- Verification: deterministic
- Description: The task list route must read projects through the shared data layer rather than duplicating storage access.
- Checks:
- type: text_contains
path: src/app/api/tasks/route.ts
text: db.listProjects
### REQ-UI-001 — Task list component exists
- Severity: low
- Verification: deterministic
- Description: The TaskBoard client must include a task list component.
- Checks:
- type: file_exists
path: src/components/TaskList.tsx
### REQ-SEC-002 — No environment file in the repository
- Severity: critical
- Verification: deterministic
- Description: The produced repository must not contain a `.env` file. Secrets belong in the deployment environment, not in source.
- Checks:
- type: file_absent
path: .env
### REQ-SEC-003 — No dynamic code evaluation in the project route
- Severity: critical
- Verification: deterministic
- Description: The project route must not evaluate source strings at runtime.
- Checks:
- type: text_not_contains
path: src/app/api/projects/[id]/route.ts
text: eval(
Adapters
fake
Baseline
Materializes a checked-in fixture. No model, no network.
claude-code-openrouter
Organizer-run
Claude Code routed through OpenRouter via the sandbox proxy.
codex-openrouter
Organizer-run
OpenAI Codex CLI routed through OpenRouter via the sandbox proxy.
aider-openrouter
Organizer-run
Aider 0.86.2 pinned in the sandbox image.
Leaderboard
Run the fake baseline| # | Score | Adapter | Model | Spec | Sec | Maint | Pass@k | Reliability | Flags | Created |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 99.5μ 99.5 · σ 0.0 | Fake fixture (taskboard/compliant)Sample | no model | 100.0 | 100.0 | 96.7 | 100% | 2 runs · 100% pass | — | 15 Nov 2026, 15:38 |
| 2 | 99.5μ 99.5 · σ 0.0 | Fake fixture (taskboard/compliant)Sample | no model | 100.0 | 100.0 | 96.7 | 100% | 2 runs · 100% pass | — | 18 Nov 2026, 06:08 |
| 3 | 98.1μ 97.7 · σ 0.4 | Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run | deepseek/deepseek-chat-v3-0324 | 100.0 | 100.0 | 87.1 | 100% | 2 runs · 100% pass | — | 15 Nov 2026, 18:50 |
| 4 | 97.3μ 97.7 · σ 0.4 | Python agent loop (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 82.0 | 100% | 2 runs · 100% pass | — | 15 Nov 2026, 18:46 |
| 5 | 97.3 | Aider 0.86.2 (openai/gpt-4o-mini)Organizer-run | openai/gpt-4o-mini | 100.0 | 100.0 | 82.0 | 100% | 1 runs · 100% pass | — | 15 Nov 2026, 18:49 |
ids: res_1ab24b…662a · res_6d753e…2b41 · res_716261…2fea · res_f74d49…f247 · res_a39c6e…63b4