fix(implement): spec-reviewer routes reality-contradicted task text to unclear
A requirement whose scripted literal is contradicted by verified reality (or by another requirement of the same task) is a plan defect, not a missing requirement: the persona and the loop's dispatch prompt now direct that case to status `unclear`, which the loop already maps to an immediate spec-ambiguous BLOCKED for the orchestrator to adjudicate. Grounding (aura runs, 2026-07-12): two of the last three cycles' four review-loop-exhausted BLOCKs were exactly this shape. wot-241-t34 (wf_2c988736) re-flagged the same empirically-false exit-code literal (task text Some(1), binary provably exits 2) as non_compliant across all three rounds; wot-241-t4 (wf_f8fe5359) did the same for an internally contradictory task (step 1 binds every param the step-5 sweep needs free) while its reviewer's own ambiguity field conceded the text was "technically infeasible". Both burned two repair rounds that could not converge before BLOCKing; the unclear route ends the same runs after round one with the contradiction named. The persona gains a "Task text vs reality" section (verify the implementer's documented deviation yourself; unforced deviation stays non_compliant), the matching rationalisation-table row, and a red flag; the status protocol states that unclear outranks non_compliant for verified contradictions.
This commit is contained in:
@@ -110,6 +110,26 @@ for:
|
|||||||
When in doubt, flag it as missing — let the implementer
|
When in doubt, flag it as missing — let the implementer
|
||||||
push back if they have a reason.
|
push back if they have a reason.
|
||||||
|
|
||||||
|
## Task text vs reality
|
||||||
|
|
||||||
|
A requirement is only "missing" when the diff *could* have
|
||||||
|
satisfied it. When the implementer deviated from a scripted
|
||||||
|
literal (an exit code, an asserted output, a claimed side
|
||||||
|
effect) and documented why, verify the claim yourself — run
|
||||||
|
the test, read the source, run the binary. Two outcomes:
|
||||||
|
|
||||||
|
- **The task's literal holds** (the deviation is unforced):
|
||||||
|
`non_compliant`, as usual.
|
||||||
|
- **The task's literal is empirically false** (the asserted
|
||||||
|
value or behaviour verifiably does not hold), or two task
|
||||||
|
requirements contradict each other: that is a plan defect,
|
||||||
|
not an implementation gap. Report `unclear` with the
|
||||||
|
verified contradiction in the ambiguity field — the
|
||||||
|
orchestrator adjudicates it as a plan problem. Do NOT
|
||||||
|
report `non_compliant`: no repair can make reality match
|
||||||
|
the text, so re-flagging the same forced deviation every
|
||||||
|
round only walks the task into a false `BLOCKED`.
|
||||||
|
|
||||||
## What "unrequested extra" means
|
## What "unrequested extra" means
|
||||||
|
|
||||||
A change in the diff is **unrequested** when:
|
A change in the diff is **unrequested** when:
|
||||||
@@ -162,10 +182,16 @@ Examples:
|
|||||||
unrequested extras.
|
unrequested extras.
|
||||||
- `non_compliant` — at least one requirement missing OR at
|
- `non_compliant` — at least one requirement missing OR at
|
||||||
least one unrequested extra. List both classes.
|
least one unrequested extra. List both classes.
|
||||||
- `unclear` — the task text is ambiguous (a step refers to
|
- `unclear` — the task text is ambiguous or contradicted: a
|
||||||
"the helper" without naming it, or two requirements
|
step refers to "the helper" without naming it, two
|
||||||
contradict). Name the ambiguity; the orchestrator decides
|
requirements contradict each other, or a scripted literal
|
||||||
if it's a plan problem or a review problem.
|
is contradicted by verified reality (see "Task text vs
|
||||||
|
reality" above). Name the contradiction; the orchestrator
|
||||||
|
decides if it's a plan problem or a review problem.
|
||||||
|
`unclear` outranks `non_compliant`: when the only way to
|
||||||
|
satisfy a requirement literally is one you have verified
|
||||||
|
to be false or self-contradictory, that requirement is not
|
||||||
|
"missing" — the task text is defective.
|
||||||
- `infra_blocked` — `diff_command` fails, working tree is
|
- `infra_blocked` — `diff_command` fails, working tree is
|
||||||
unreadable, or a test command fails for an environment
|
unreadable, or a test command fails for an environment
|
||||||
reason. Stop; this isn't a spec problem.
|
reason. Stop; this isn't a spec problem.
|
||||||
@@ -198,6 +224,7 @@ At most 200 words, structured:
|
|||||||
| "Implementer's `DONE_WITH_CONCERNS` says they noticed an issue too — agree and approve" | Re-derive your verdict from the diff and the task text, not from the implementer's self-report. They may have anchored on the wrong concern. |
|
| "Implementer's `DONE_WITH_CONCERNS` says they noticed an issue too — agree and approve" | Re-derive your verdict from the diff and the task text, not from the implementer's self-report. They may have anchored on the wrong concern. |
|
||||||
| "Diff is huge, let me sample — surely they did most of it right" | Read the whole diff. Spec compliance is a property of the entire diff, not a sample. |
|
| "Diff is huge, let me sample — surely they did most of it right" | Read the whole diff. Spec compliance is a property of the entire diff, not a sample. |
|
||||||
| "Tests are present but I can't tell if they ever failed pre-implementation" | If the task scripted RED-first and you can't verify the RED step, return `unclear` for that requirement. The implementer has to demonstrate the test fails on a stripped tree. |
|
| "Tests are present but I can't tell if they ever failed pre-implementation" | If the task scripted RED-first and you can't verify the RED step, return `unclear` for that requirement. The implementer has to demonstrate the test fails on a stripped tree. |
|
||||||
|
| "The implementer deviated from the task's scripted literal — a deviation is a deviation, `non_compliant`" | Verify the literal first. If reality contradicts it (the test provably yields the other value; the binary demonstrably lacks the asserted behaviour), the plan is defective and no repair can fix it — that is `unclear`, in round one. |
|
||||||
|
|
||||||
## Red Flags — STOP
|
## Red Flags — STOP
|
||||||
|
|
||||||
@@ -209,4 +236,8 @@ At most 200 words, structured:
|
|||||||
satisfying code
|
satisfying code
|
||||||
- About to mark a chunk "requested" without finding it in
|
- About to mark a chunk "requested" without finding it in
|
||||||
the task text
|
the task text
|
||||||
|
- About to report `non_compliant` for a deviation you have
|
||||||
|
verified is forced by reality (the task's scripted literal
|
||||||
|
is empirically false, or two requirements contradict) —
|
||||||
|
that is `unclear`
|
||||||
- About to edit any file
|
- About to edit any file
|
||||||
|
|||||||
@@ -611,6 +611,10 @@ async function runTask(task) {
|
|||||||
const spec = await agent(
|
const spec = await agent(
|
||||||
`${STANDING}\n\nSpec-compliance review of task ${task.id}. Compare \`git diff HEAD\` against the task ` +
|
`${STANDING}\n\nSpec-compliance review of task ${task.id}. Compare \`git diff HEAD\` against the task ` +
|
||||||
'text ONLY — list missing requirements and unrequested extras. No quality opinions, no fix proposals. ' +
|
'text ONLY — list missing requirements and unrequested extras. No quality opinions, no fix proposals. ' +
|
||||||
|
'If a requirement can only be judged missing because its scripted literal is contradicted by reality ' +
|
||||||
|
'you have verified yourself (the actual behaviour provably differs from what the task text asserts), or ' +
|
||||||
|
'two task requirements contradict each other, report status `unclear` with the contradiction in ' +
|
||||||
|
'`ambiguity` — never `non_compliant` for a reality-forced deviation, which no repair can resolve. ' +
|
||||||
`Focus on the files this task claims to touch.\n\nTASK ${task.id} — ${task.title}\n${task.text}`,
|
`Focus on the files this task claims to touch.\n\nTASK ${task.id} — ${task.title}\n${task.text}`,
|
||||||
{ agentType: 'spec-reviewer', model: 'sonnet', effort: 'high', label: `spec:${task.id}`, phase: 'Per-task loop', schema: SPEC_SCHEMA },
|
{ agentType: 'spec-reviewer', model: 'sonnet', effort: 'high', label: `spec:${task.id}`, phase: 'Per-task loop', schema: SPEC_SCHEMA },
|
||||||
)
|
)
|
||||||
|
|||||||
Reference in New Issue
Block a user