counterfactual proof · Python agent loop (deepseek/deepseek-chat-v3-0324)
harness × recovery
retry the failed write_file once at step 2
baseline
49.7
failed
counterfactual
98.3
6 replayed steps
intervention
retry_policy
at step 2 · threshold 70
Baseline trace
Steps before 2 are byte-identical in the replay. The highlighted step is where the intervention applies.
- —Run startedpython-openrouter 1.0.0 · run 0
- —Chaos injectedtool_error · one write_file fails with EACCES
- 1Model call starteddeepseek/deepseek-chat-v3-0324 · messages b46a59da…
- 1Model call finished1303 in / 73 out · $0.00046 · stop
- 2Model call starteddeepseek/deepseek-chat-v3-0324 · messages 7b91b37a…
- 2Model call finished1318 in / 377 out · $0.00081 · stop
- 2Tool call startedwrite_file · path=src/app/api/projects/[id]/route.ts bytes=1546redacted
- 2Tool call finishedwrite_file · error · EACCES: permission denied (injected)redacted
- 2Step completed0 ms
- 3Model call starteddeepseek/deepseek-chat-v3-0324 · messages 0c8c8bed…
- 3Model call finished1316 in / 122 out · $0.00052 · stop
- 3Tool call startedwrite_file · path=src/app/api/tasks/route.ts bytes=509redacted
- 3Tool call finishedwrite_file · ok · ok bytes=509redacted
- 3Step completed3 ms
- 4Model call starteddeepseek/deepseek-chat-v3-0324 · messages c49ff370…
- 4Model call finished1315 in / 302 out · $0.00073 · stop
- 4Tool call startedwrite_file · path=src/components/TaskList.tsx bytes=1194redacted
- 4Tool call finishedwrite_file · ok · ok bytes=1194redacted
- 4Step completed0 ms
- 5Model call starteddeepseek/deepseek-chat-v3-0324 · messages b5beb072…
- 5Model call finished1315 in / 113 out · $0.00051 · stop
- 5Tool call startedwrite_file · path=src/lib/assertProjectOwner.ts bytes=456redacted
- 5Tool call finishedwrite_file · ok · ok bytes=456redacted
- 5Step completed0 ms
- 6Model call starteddeepseek/deepseek-chat-v3-0324 · messages 54d60173…
- 6Model call finished1313 in / 189 out · $0.00058 · stop
- 6Tool call startedwrite_file · path=src/lib/errors.ts bytes=817redacted
- 6Tool call finishedwrite_file · ok · ok bytes=817redacted
- 6Step completed0 ms
- 7Model call starteddeepseek/deepseek-chat-v3-0324 · messages 39712f30…
- 7Model call finished1312 in / 119 out · $0.00052 · stop
- 7Tool call startedwrite_file · path=src/lib/types.ts bytes=426redacted
- 7Tool call finishedwrite_file · ok · ok bytes=426redacted
- 7Step completed0 ms
- —Run finishedfailed · 7 steps · 6 tools · 10487 tokens · $0.0041
Deterministic scoring note
Replay writes the recorded text actions into a fresh in-memory file map, applies one intervention at step 2, and scores with the existing engine. It makes no model call and executes no produced code.