Skip to content
taskboard · task 1.0.0 · ruleset 1.0.0 · scoring 1.0.0Starter

TaskBoard: owner-scoped project API

Build the project-access API for a small task board. Enforce ownership, return structured errors, read through the repository layer, and avoid shell-execution or dynamic-evaluation patterns.

nextjsapiauthorizationsecurity

Requirements (9)

6 spec · 3 security · pass ≥ 70 · 900s limit · network model-proxy

Declared requirements and their checks
IdRequirementBenchSeverityDeclared check
REQ-AUTH-001Project access must be owner-scopedEvery request that reads or mutates a project must verify that the authenticated requester is the project owner. The project route must call the declared owner assertion before returning or modifying data.specCriticaltext_contains · src/app/api/projects/[id]/route.ts · "assertProjectOwner"
REQ-DEP-001No shell execution dependencyThe project must not depend on libraries that allow arbitrary shell execution from within the web server process.securityCriticaldependency_absent · shelljs
REQ-ERR-001Project API returns structured error responsesThe projects API must wrap handler bodies in a try/catch and return a structured error response instead of leaking stack traces.specMediumtext_contains · src/app/api/projects/[id]/route.ts · "catch (error)"
REQ-PATH-001Projects API route existsThe projects API route file must exist at the specified path.specHighfile_exists · src/app/api/projects/[id]/route.ts
REQ-SEC-002No environment file in the repositoryThe produced repository must not contain a `.env` file. Secrets belong in the deployment environment, not in source.securityCriticalfile_absent · .env
REQ-SEC-003No dynamic code evaluation in the project routeThe project route must not evaluate source strings at runtime.securityCriticaltext_not_contains · src/app/api/projects/[id]/route.ts · "eval("
REQ-TASKS-001Task list API route existsThe task list API route must exist so the board can load tasks for the authenticated user.specHighfile_exists · src/app/api/tasks/route.ts
REQ-TASKS-002Task list reads through the repository layerThe task list route must read projects through the shared data layer rather than duplicating storage access.specMediumtext_contains · src/app/api/tasks/route.ts · "db.listProjects"
REQ-UI-001Task list component existsThe TaskBoard client must include a task list component.specLowfile_exists · src/components/TaskList.tsx

Starting files

  • PROJECT_SPEC.md3.4 KiB
  • package.json154 B
  • src/lib/db.ts899 B
  • src/lib/session.ts146 B

PROJECT_SPEC.md

task specification
# TaskBoard — Benchmark Task Specification

> **Benchmark note:** every requirement below is deterministic. The scoring engine parses this file, evaluates declared checks against the produced repository, and never calls a model. The task is derived from the TaskBoard verifier fixtures; the semantic verification mode was projected to a deterministic text-marker check so the score stays model-free.

## Context

Build the project-access API for TaskBoard, a small Next.js task board. Project records are owned by a user. Every read or mutation of a project must verify ownership before returning data. The API must return structured error responses and must not introduce shell-execution dependencies.

## Requirements

### REQ-AUTH-001 — Project access must be owner-scoped
- Severity: critical
- Verification: deterministic
- Description: Every request that reads or mutates a project must verify that the authenticated requester is the project owner. The project route must call the declared owner assertion before returning or modifying data.
- Checks:
  - type: text_contains
    path: src/app/api/projects/[id]/route.ts
    text: assertProjectOwner

### REQ-ERR-001 — Project API returns structured error responses
- Severity: medium
- Verification: deterministic
- Description: The projects API must wrap handler bodies in a try/catch and return a structured error response instead of leaking stack traces.
- Checks:
  - type: text_contains
    path: src/app/api/projects/[id]/route.ts
    text: "catch (error)"

### REQ-PATH-001 — Projects API route exists
- Severity: high
- Verification: deterministic
- Description: The projects API route file must exist at the specified path.
- Checks:
  - type: file_exists
    path: src/app/api/projects/[id]/route.ts

### REQ-DEP-001 — No shell execution dependency
- Severity: critical
- Verification: deterministic
- Description: The project must not depend on libraries that allow arbitrary shell execution from within the web server process.
- Checks:
  - type: dependency_absent
    name: shelljs

### REQ-TASKS-001 — Task list API route exists
- Severity: high
- Verification: deterministic
- Description: The task list API route must exist so the board can load tasks for the authenticated user.
- Checks:
  - type: file_exists
    path: src/app/api/tasks/route.ts

### REQ-TASKS-002 — Task list reads through the repository layer
- Severity: medium
- Verification: deterministic
- Description: The task list route must read projects through the shared data layer rather than duplicating storage access.
- Checks:
  - type: text_contains
    path: src/app/api/tasks/route.ts
    text: db.listProjects

### REQ-UI-001 — Task list component exists
- Severity: low
- Verification: deterministic
- Description: The TaskBoard client must include a task list component.
- Checks:
  - type: file_exists
    path: src/components/TaskList.tsx

### REQ-SEC-002 — No environment file in the repository
- Severity: critical
- Verification: deterministic
- Description: The produced repository must not contain a `.env` file. Secrets belong in the deployment environment, not in source.
- Checks:
  - type: file_absent
    path: .env

### REQ-SEC-003 — No dynamic code evaluation in the project route
- Severity: critical
- Verification: deterministic
- Description: The project route must not evaluate source strings at runtime.
- Checks:
  - type: text_not_contains
    path: src/app/api/projects/[id]/route.ts
    text: eval(

Adapters

fake

Baseline

Materializes a checked-in fixture. No model, no network.

claude-code-openrouter

Organizer-run

Claude Code routed through OpenRouter via the sandbox proxy.

codex-openrouter

Organizer-run

OpenAI Codex CLI routed through OpenRouter via the sandbox proxy.

aider-openrouter

Organizer-run

Aider 0.86.2 pinned in the sandbox image.

Leaderboard for taskboard
#ScoreAdapterModelSpecSecMaintPass@kReliabilityFlagsCreated
199.5μ 99.5 · σ 0.0
Fake fixture (taskboard/compliant)Sample
no model100.0100.096.7100%

2 runs · 100% pass

—15 Nov 2026, 15:38
299.5μ 99.5 · σ 0.0
Fake fixture (taskboard/compliant)Sample
no model100.0100.096.7100%

2 runs · 100% pass

—18 Nov 2026, 06:08
398.1μ 97.7 · σ 0.4
Python agent loop (deepseek/deepseek-chat-v3-0324)Organizer-run
deepseek/deepseek-chat-v3-0324100.0100.087.1100%

2 runs · 100% pass

—15 Nov 2026, 18:50
497.3μ 97.7 · σ 0.4
Python agent loop (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.082.0100%

2 runs · 100% pass

—15 Nov 2026, 18:46
597.3
Aider 0.86.2 (openai/gpt-4o-mini)Organizer-run
openai/gpt-4o-mini100.0100.082.0100%

1 runs · 100% pass

—15 Nov 2026, 18:49

ids: res_1ab24b…662a · res_6d753e…2b41 · res_716261…2fea · res_f74d49…f247 · res_a39c6e…63b4