linkstash · task 1.0.0 · ruleset 1.0.0 · scoring 1.0.0Intermediate
Linkstash: authenticated bookmark API
Build a bookmark API for a small authenticated product. Resolve the user, validate input with Zod, return structured errors, and avoid dynamic evaluation and shell-execution dependencies.
nextjsapivalidationsecurity
Requirements (12)
9 spec · 3 security · pass ≥ 70 · 900s limit · network model-proxy
| Id | Requirement | Bench | Severity | Declared check |
|---|---|---|---|---|
| REQ-API-001 | Bookmark API route existsThe bookmark API route must exist at the specified path. | spec | High | file_exists · src/app/api/bookmarks/route.ts |
| REQ-API-002 | Bookmark API exposes a read handlerThe route must export a GET handler so clients can list bookmarks. | spec | High | text_contains · src/app/api/bookmarks/route.ts · "export async function GET" |
| REQ-API-003 | Bookmark API exposes a create handlerThe route must export a POST handler so clients can create bookmarks. | spec | High | text_contains · src/app/api/bookmarks/route.ts · "export async function POST" |
| REQ-AUTH-001 | Bookmark routes require an authenticated userEvery bookmark handler must resolve the authenticated user before reading or writing data. | spec | Critical | text_contains · src/app/api/bookmarks/route.ts · "requireUser" |
| REQ-DB-001 | Shared data layer existsBookmark persistence must go through the shared data layer. | spec | Medium | file_exists · src/lib/db.ts |
| REQ-DEP-001 | Zod is declared as a dependencyInput validation uses Zod, so the runtime dependency must be declared. | spec | Low | dependency_present · zod |
| REQ-ERR-001 | Bookmark API returns structured errorsThe route must catch handler failures and return a structured error response. | spec | Medium | text_contains · src/app/api/bookmarks/route.ts · "catch (error)" |
| REQ-SEC-001 | No shell execution dependencyThe project must not depend on libraries that allow arbitrary shell execution from the web server process. | security | Critical | dependency_absent · shelljs |
| REQ-SEC-002 | No environment file in the repositoryThe produced repository must not contain a `.env` file. | security | Critical | file_absent · .env |
| REQ-SEC-003 | No dynamic code evaluationThe bookmark route must not evaluate source strings at runtime. | security | Critical | text_not_contains · src/app/api/bookmarks/route.ts · "eval(" |
| REQ-UI-001 | Bookmark list component existsThe client must include a bookmark list component. | spec | Low | file_exists · src/components/BookmarkList.tsx |
| REQ-VALID-001 | Bookmark creation validates inputThe POST handler must validate the request body with the declared Zod schema before writing to the data layer. | spec | High | text_contains · src/app/api/bookmarks/route.ts · "bookmarkSchema" |
Starting files
- PROJECT_SPEC.md3.7 KiB
- package.json166 B
- src/lib/db.ts728 B
PROJECT_SPEC.md
task specification
# Linkstash — Benchmark Task Specification
> **Benchmark note:** this is a synthetic benchmark task. Every requirement is deterministic; the scoring engine never calls a model.
## Context
Build the bookmark API for Linkstash, a small authenticated bookmark manager. Users can list their own bookmarks and create new ones. The API must reject unauthenticated access, validate request bodies with Zod, return structured errors, and avoid shell-execution or dynamic-evaluation patterns.
## Requirements
### REQ-API-001 — Bookmark API route exists
- Severity: high
- Verification: deterministic
- Description: The bookmark API route must exist at the specified path.
- Checks:
- type: file_exists
path: src/app/api/bookmarks/route.ts
### REQ-API-002 — Bookmark API exposes a read handler
- Severity: high
- Verification: deterministic
- Description: The route must export a GET handler so clients can list bookmarks.
- Checks:
- type: text_contains
path: src/app/api/bookmarks/route.ts
text: "export async function GET"
### REQ-API-003 — Bookmark API exposes a create handler
- Severity: high
- Verification: deterministic
- Description: The route must export a POST handler so clients can create bookmarks.
- Checks:
- type: text_contains
path: src/app/api/bookmarks/route.ts
text: "export async function POST"
### REQ-AUTH-001 — Bookmark routes require an authenticated user
- Severity: critical
- Verification: deterministic
- Description: Every bookmark handler must resolve the authenticated user before reading or writing data.
- Checks:
- type: text_contains
path: src/app/api/bookmarks/route.ts
text: requireUser
### REQ-VALID-001 — Bookmark creation validates input
- Severity: high
- Verification: deterministic
- Description: The POST handler must validate the request body with the declared Zod schema before writing to the data layer.
- Checks:
- type: text_contains
path: src/app/api/bookmarks/route.ts
text: bookmarkSchema
### REQ-ERR-001 — Bookmark API returns structured errors
- Severity: medium
- Verification: deterministic
- Description: The route must catch handler failures and return a structured error response.
- Checks:
- type: text_contains
path: src/app/api/bookmarks/route.ts
text: "catch (error)"
### REQ-DB-001 — Shared data layer exists
- Severity: medium
- Verification: deterministic
- Description: Bookmark persistence must go through the shared data layer.
- Checks:
- type: file_exists
path: src/lib/db.ts
### REQ-UI-001 — Bookmark list component exists
- Severity: low
- Verification: deterministic
- Description: The client must include a bookmark list component.
- Checks:
- type: file_exists
path: src/components/BookmarkList.tsx
### REQ-SEC-001 — No shell execution dependency
- Severity: critical
- Verification: deterministic
- Description: The project must not depend on libraries that allow arbitrary shell execution from the web server process.
- Checks:
- type: dependency_absent
name: shelljs
### REQ-SEC-002 — No environment file in the repository
- Severity: critical
- Verification: deterministic
- Description: The produced repository must not contain a `.env` file.
- Checks:
- type: file_absent
path: .env
### REQ-SEC-003 — No dynamic code evaluation
- Severity: critical
- Verification: deterministic
- Description: The bookmark route must not evaluate source strings at runtime.
- Checks:
- type: text_not_contains
path: src/app/api/bookmarks/route.ts
text: eval(
### REQ-DEP-001 — Zod is declared as a dependency
- Severity: low
- Verification: deterministic
- Description: Input validation uses Zod, so the runtime dependency must be declared.
- Checks:
- type: dependency_present
name: zod
Adapters
fake
Baseline
Materializes a checked-in fixture. No model, no network.
claude-code-openrouter
Organizer-run
Claude Code routed through OpenRouter via the sandbox proxy.
codex-openrouter
Organizer-run
OpenAI Codex CLI routed through OpenRouter via the sandbox proxy.
aider-openrouter
Organizer-run
Aider 0.86.2 pinned in the sandbox image.
Leaderboard
Run the fake baseline| # | Score | Adapter | Model | Spec | Sec | Maint | Pass@k | Reliability | Flags | Created |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 99.3μ 74.6 · σ 24.7 | Fake fixture (linkstash/compliant)Sample | no model | 100.0 | 100.0 | 95.0 | 50% | 2 runs · 50% pass | — | 15 Nov 2026, 15:38 |
| 2 | 49.9μ 74.6 · σ 24.7 | Fake fixture (linkstash/regressed)Sample | no model | 50.0 | 33.3 | 90.0 | 50% | 2 runs · 50% pass | Capped | 15 Nov 2026, 15:38 |
ids: res_bb227a…20ef · res_2ac581…2981