fix(pipeline,audit): gate-side floors for prose-only no-op guards
Two roll-ups were vacuously satisfiable by a no-op and guarded only in agent prose, never re-verified at the consuming gate: - The milestone-close functional leg trusted a fieldtest `clean` roll-up even from a run with zero examples. The gate now requires examples_added >= 2 for a `clean` to be honoured (the empty-report and internal-milestone escapes are preserved); a `clean` with fewer over user-visible surface is itself a `spec_gap`. - The audit carry-on accepted an architect `clean` with no evidence of review. It now requires a non-empty "What holds" (added as an explicit handoff field); a bare `clean` with nothing held — worst when no regression scripts run, so the architect is the sole gate — reads as unreviewed and is re-dispatched. Both are second-layer checks: the consumer verifies the field rather than trusting the agent's discipline. Lower-severity defense-in-depth, not code-level floors (milestone-close and audit carry-on are orchestrator-judgment acts with no script to instrument). closes #14
This commit is contained in:
@@ -110,12 +110,23 @@ The orchestrator picks per item:
|
||||
commits always exist at cycle close — they carry the architect
|
||||
findings, the regression numbers, and the resolution.
|
||||
|
||||
A carry-on is honoured only with positive evidence the architect
|
||||
actually reviewed: a non-empty **What holds** in its report (the
|
||||
design commitments it confirmed it read). A bare `clean` with
|
||||
nothing held — especially when Step 2 was a no-op (no regression
|
||||
scripts, so the architect is the *sole* gate) — reads as an
|
||||
unreviewed run, not a clean cycle; re-dispatch rather than close
|
||||
on it. (The architect owes this itself — `agents/architect.md`
|
||||
Red Flags, "about to return clean without having read the diff in
|
||||
full"; the gate verifies it rather than trusting the status.)
|
||||
|
||||
## Handoff Contract
|
||||
|
||||
`audit` hands to the orchestrator:
|
||||
|
||||
| Field | Content |
|
||||
|-------|---------|
|
||||
| `holds` | architect's **What holds** — the 1-3 design commitments the cycle preserved (`agents/architect.md` Output format). Non-empty is the evidence-of-review the carry-on gate checks; an empty `holds` reads as an unreviewed run. |
|
||||
| `drift_items` | prioritised list (path + 1-line justification) from architect |
|
||||
| `regression_results` | per script: exit code + **raw output verbatim** (no rounding, no summarisation — the orchestrator second-guesses with the full numbers) |
|
||||
| `recommendation` | per-item: `fix` / `ratify` / `carry-on` |
|
||||
|
||||
+8
-1
@@ -90,7 +90,14 @@ hold:
|
||||
and its status roll-up is `clean`: every scenario
|
||||
demonstrably delivers what the milestone promised; no open
|
||||
`bug` findings; `friction` / `spec_gap` findings resolved or
|
||||
ratified into the design ledger.
|
||||
ratified into the design ledger. A `clean` roll-up is honoured
|
||||
only with positive evidence the test actually ran — at least
|
||||
2 examples in the working tree (`examples_added`). A `clean`
|
||||
with fewer, over a milestone that touched user-visible surface,
|
||||
is itself a `spec_gap`, not a clean close (the agent owes the
|
||||
same floor — `fieldtest/agents/fieldtester.md` "What you DO NOT
|
||||
ship"; the gate verifies it rather than trusting the roll-up). The empty
|
||||
report is valid only when no example was applicable.
|
||||
|
||||
The milestone fieldtest is the milestone-wide variant of the
|
||||
`fieldtest` skill: the same fieldtester agent, a carrier scoped
|
||||
|
||||
Reference in New Issue
Block a user