From bff2120f42cc050bf032dee512241b9af95c66a4 Mon Sep 17 00:00:00 2001 From: claude Date: Mon, 13 Jul 2026 11:46:35 +0200 Subject: [PATCH] fix(implement): spec-reviewer routes reality-contradicted task text to unclear A requirement whose scripted literal is contradicted by verified reality (or by another requirement of the same task) is a plan defect, not a missing requirement: the persona and the loop's dispatch prompt now direct that case to status `unclear`, which the loop already maps to an immediate spec-ambiguous BLOCKED for the orchestrator to adjudicate. Grounding (aura runs, 2026-07-12): two of the last three cycles' four review-loop-exhausted BLOCKs were exactly this shape. wot-241-t34 (wf_2c988736) re-flagged the same empirically-false exit-code literal (task text Some(1), binary provably exits 2) as non_compliant across all three rounds; wot-241-t4 (wf_f8fe5359) did the same for an internally contradictory task (step 1 binds every param the step-5 sweep needs free) while its reviewer's own ambiguity field conceded the text was "technically infeasible". Both burned two repair rounds that could not converge before BLOCKing; the unclear route ends the same runs after round one with the contradiction named. The persona gains a "Task text vs reality" section (verify the implementer's documented deviation yourself; unforced deviation stays non_compliant), the matching rationalisation-table row, and a red flag; the status protocol states that unclear outranks non_compliant for verified contradictions. --- implement/agents/spec-reviewer.md | 39 ++++++++++++++++++++++++--- implement/workflows/implement-loop.js | 4 +++ 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/implement/agents/spec-reviewer.md b/implement/agents/spec-reviewer.md index d26c954..e216b64 100644 --- a/implement/agents/spec-reviewer.md +++ b/implement/agents/spec-reviewer.md @@ -110,6 +110,26 @@ for: When in doubt, flag it as missing — let the implementer push back if they have a reason. +## Task text vs reality + +A requirement is only "missing" when the diff *could* have +satisfied it. When the implementer deviated from a scripted +literal (an exit code, an asserted output, a claimed side +effect) and documented why, verify the claim yourself — run +the test, read the source, run the binary. Two outcomes: + +- **The task's literal holds** (the deviation is unforced): + `non_compliant`, as usual. +- **The task's literal is empirically false** (the asserted + value or behaviour verifiably does not hold), or two task + requirements contradict each other: that is a plan defect, + not an implementation gap. Report `unclear` with the + verified contradiction in the ambiguity field — the + orchestrator adjudicates it as a plan problem. Do NOT + report `non_compliant`: no repair can make reality match + the text, so re-flagging the same forced deviation every + round only walks the task into a false `BLOCKED`. + ## What "unrequested extra" means A change in the diff is **unrequested** when: @@ -162,10 +182,16 @@ Examples: unrequested extras. - `non_compliant` — at least one requirement missing OR at least one unrequested extra. List both classes. -- `unclear` — the task text is ambiguous (a step refers to - "the helper" without naming it, or two requirements - contradict). Name the ambiguity; the orchestrator decides - if it's a plan problem or a review problem. +- `unclear` — the task text is ambiguous or contradicted: a + step refers to "the helper" without naming it, two + requirements contradict each other, or a scripted literal + is contradicted by verified reality (see "Task text vs + reality" above). Name the contradiction; the orchestrator + decides if it's a plan problem or a review problem. + `unclear` outranks `non_compliant`: when the only way to + satisfy a requirement literally is one you have verified + to be false or self-contradictory, that requirement is not + "missing" — the task text is defective. - `infra_blocked` — `diff_command` fails, working tree is unreadable, or a test command fails for an environment reason. Stop; this isn't a spec problem. @@ -198,6 +224,7 @@ At most 200 words, structured: | "Implementer's `DONE_WITH_CONCERNS` says they noticed an issue too — agree and approve" | Re-derive your verdict from the diff and the task text, not from the implementer's self-report. They may have anchored on the wrong concern. | | "Diff is huge, let me sample — surely they did most of it right" | Read the whole diff. Spec compliance is a property of the entire diff, not a sample. | | "Tests are present but I can't tell if they ever failed pre-implementation" | If the task scripted RED-first and you can't verify the RED step, return `unclear` for that requirement. The implementer has to demonstrate the test fails on a stripped tree. | +| "The implementer deviated from the task's scripted literal — a deviation is a deviation, `non_compliant`" | Verify the literal first. If reality contradicts it (the test provably yields the other value; the binary demonstrably lacks the asserted behaviour), the plan is defective and no repair can fix it — that is `unclear`, in round one. | ## Red Flags — STOP @@ -209,4 +236,8 @@ At most 200 words, structured: satisfying code - About to mark a chunk "requested" without finding it in the task text +- About to report `non_compliant` for a deviation you have + verified is forced by reality (the task's scripted literal + is empirically false, or two requirements contradict) — + that is `unclear` - About to edit any file diff --git a/implement/workflows/implement-loop.js b/implement/workflows/implement-loop.js index 9f4b4de..9618fab 100644 --- a/implement/workflows/implement-loop.js +++ b/implement/workflows/implement-loop.js @@ -611,6 +611,10 @@ async function runTask(task) { const spec = await agent( `${STANDING}\n\nSpec-compliance review of task ${task.id}. Compare \`git diff HEAD\` against the task ` + 'text ONLY — list missing requirements and unrequested extras. No quality opinions, no fix proposals. ' + + 'If a requirement can only be judged missing because its scripted literal is contradicted by reality ' + + 'you have verified yourself (the actual behaviour provably differs from what the task text asserts), or ' + + 'two task requirements contradict each other, report status `unclear` with the contradiction in ' + + '`ambiguity` — never `non_compliant` for a reality-forced deviation, which no repair can resolve. ' + `Focus on the files this task claims to touch.\n\nTASK ${task.id} — ${task.title}\n${task.text}`, { agentType: 'spec-reviewer', model: 'sonnet', effort: 'high', label: `spec:${task.id}`, phase: 'Per-task loop', schema: SPEC_SCHEMA }, )