fix(implement-loop): require positive edit evidence before DONE
The #11 vacuous-green shape on the main path: `outcome = DONE` was set purely from the absence of a BLOCKED task, with no check that the iteration actually wrote anything. `files_touched` was even computed by the finalize agent but never asserted > 0, `e2e.status` was never read, and in mini mode the handed-off RED test was never independently re-run (GREEN was asserted from the implementer's self-report, never observed). Add a positive-evidence precondition on DONE, taken from git ground truth — never a self-report, which must not be able to fail a run that actually did the work: - Standard mode: a dedicated `tree-check` agent runs `git status --porcelain | wc -l` after the per-task loop but BEFORE E2E/finalize. Zero (or a non-returning agent) downgrades DONE to BLOCKED. `git status --porcelain` (not `git diff HEAD`) so a brand-new untracked file — the implementer leaves edits unstaged — still counts; running pre-E2E/finalize keeps fixtures and the stats/BLOCKED.md artefacts from inflating it. - Mini mode: an independent `mini-verify` agent re-runs the RED test by name and the suite and checks the tree is dirty; a still-red test, a regression, or a clean tree is BLOCKED (route back to debug). - Per-task `applied_changes` (OR-ed across the initial dispatch and every repair) feeds only a neutral concern, never the outcome. - `e2e.status` is now read: a non-DONE status or a zero-fixture run surfaces as a concern. A clean-tree no-op is BLOCKED with no BLOCKED.md (nothing to clean up); the status + reason ride the end-report. SKILL.md documents the precondition, the no-BLOCKED.md carve-out, and the Step-4 handling. closes #12
This commit is contained in:
+41
-1
@@ -102,7 +102,9 @@ WITHIN THIS LOOP, MAIN HEAD IS SACROSANCT — NO RESET, NO REVERT; THE LOOP ONLY
|
||||
PER-TASK PHASES ARE SEPARATE AGENT CALLS IN THE WORKFLOW SCRIPT — implementer, then spec-compliance, then quality. SPEC COMPLIANCE IS GATED BEFORE QUALITY.
|
||||
TASKS RUN SEQUENTIALLY; A BLOCKED TASK STOPS THE LOOP — NO SKIP-AHEAD (TASK ORDERING DEPENDENCIES ARE UNKNOWN).
|
||||
NEVER PUSH PAST `BLOCKED` BY HAND.
|
||||
ON `PARTIAL` OR `BLOCKED`, THE SCRIPT WRITES `BLOCKED.md` AT THE REPO ROOT — UNCOMMITTED BY CONVENTION. ON `DONE`, NO SEPARATE FILE.
|
||||
ON `PARTIAL` OR `BLOCKED`, THE SCRIPT WRITES `BLOCKED.md` AT THE REPO ROOT — UNCOMMITTED BY CONVENTION. ON `DONE`, NO SEPARATE FILE. (ONE EXCEPTION: A NO-OP ITERATION IS `BLOCKED` WITH NO `BLOCKED.md` — ITS TREE IS ALREADY CLEAN, SO THE STATUS + REASON RIDE THE END-REPORT AND THERE IS NOTHING TO CLEAN UP.)
|
||||
`DONE` REQUIRES POSITIVE EVIDENCE THAT AN EDIT LANDED — A NO-OP ITERATION (`git status --porcelain` EMPTY, untracked files INCLUDED) IS NEVER `DONE`, IT IS `BLOCKED` (the issue #11 vacuous-green shape, guarded on the main path). THE VERDICT IS THE GIT TREE COUNT, NEVER A SELF-REPORT.
|
||||
IN MINI MODE THE RED->GREEN IS OBSERVED, NOT ASSERTED: AN INDEPENDENT VERIFY RE-RUNS THE HANDED-OFF RED TEST AND THE SUITE; A STILL-RED TEST OR A REGRESSION IS `BLOCKED` (route back to debug, RED-first).
|
||||
THE COMPILER-DRIVEN ARM COMMITS ONLY ON CLEAN BUILD + SUITE GREEN UNCHANGED; ELSE IT BOUNCES (specify for a design hole, tdd for discovered new behaviour, debug for a regression).
|
||||
```
|
||||
|
||||
@@ -120,6 +122,37 @@ BLOCKED. This logic lives as the loops in
|
||||
`workflows/implement-loop.js` — the single source; it is deliberately
|
||||
not restated as prose elsewhere, so the two cannot drift.
|
||||
|
||||
**Positive-evidence precondition on `DONE`** (issue #12 — the issue #11
|
||||
vacuous-green shape on the main path). `build green AND suite green` is
|
||||
not enough: it is satisfiable by doing nothing. So `DONE` also requires
|
||||
positive evidence that an edit landed. The verdict is taken from **git
|
||||
ground truth, never a self-report** — a self-report must never be able to
|
||||
*fail* a run that actually did the work:
|
||||
|
||||
- **Iteration gate (standard mode)** — a dedicated `tree-check` agent runs
|
||||
`git status --porcelain | wc -l` after the per-task loop but **before**
|
||||
E2E and finalize. A `DONE` outcome with a zero count is a no-op iteration
|
||||
→ `BLOCKED`. `git status --porcelain` (not `git diff HEAD`) is used so a
|
||||
brand-new **untracked** file — the implementer leaves edits unstaged —
|
||||
still counts; running before E2E/finalize keeps fixtures and the
|
||||
stats/`BLOCKED.md` artefacts from inflating it; and a non-returning agent
|
||||
is treated as `BLOCKED` (infra), never waved through. A clean-tree no-op
|
||||
writes no `BLOCKED.md` (nothing to clean up — see Step 4).
|
||||
- **Mini mode gate** — an independent `mini-verify` agent re-runs the
|
||||
handed-off RED test and the suite and checks the tree is dirty
|
||||
(`git status --porcelain`); a still-red test, a regression, or a clean
|
||||
tree is `BLOCKED`. The GREEN is observed, not taken from the
|
||||
implementer's self-report.
|
||||
- **Per-task `applied_changes`** (self-report, OR-ed across the initial
|
||||
dispatch and every repair) does NOT gate the outcome — a self-report must
|
||||
never be able to *fail* a run that actually did the work (e.g. the initial
|
||||
implementer no-ops but a repair writes the files). It feeds only a neutral
|
||||
concern when a task wrote nothing (it may be genuinely already-satisfied —
|
||||
the orchestrator weighs it).
|
||||
|
||||
The E2E phase's `status` is also read now (not just its fixtures): a
|
||||
non-`DONE` E2E status or a zero-fixture run surfaces as a concern.
|
||||
|
||||
## The Process — orchestrator side
|
||||
|
||||
### Step 1 — Run the `implement-loop` workflow
|
||||
@@ -203,6 +236,13 @@ The workflow returns with whatever work-in-progress it managed plus
|
||||
committed. The orchestrator decides what to do with the dirty working
|
||||
tree:
|
||||
|
||||
0. **No-op `BLOCKED` special case** — if the end-report's status is
|
||||
`BLOCKED` with `blocked_md` null and a no-op `blocked_detail` (the reason
|
||||
names a no-op iteration or a clean-tree no-op fix; `files_touched` is 0 in
|
||||
standard mode, null in mini), there is **no `BLOCKED.md`** and the tree is
|
||||
already clean. The diagnostic is in the end-report's `blocked_detail`, not
|
||||
a file. Nothing to clean up or stash; re-plan or re-run from the same
|
||||
clean tree.
|
||||
1. Read `BLOCKED.md` — `## What did not` names the failure mode and the
|
||||
blocked task's reason. Read `git diff` to see what was attempted.
|
||||
2. Decide:
|
||||
|
||||
Reference in New Issue
Block a user