From 5a9a2ae05cb89034425dbdeeeb4716ad228375b5 Mon Sep 17 00:00:00 2001 From: Brummel Date: Sat, 27 Jun 2026 14:27:18 +0200 Subject: [PATCH] fix(pipeline,audit): gate-side floors for prose-only no-op guards MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two roll-ups were vacuously satisfiable by a no-op and guarded only in agent prose, never re-verified at the consuming gate: - The milestone-close functional leg trusted a fieldtest `clean` roll-up even from a run with zero examples. The gate now requires examples_added >= 2 for a `clean` to be honoured (the empty-report and internal-milestone escapes are preserved); a `clean` with fewer over user-visible surface is itself a `spec_gap`. - The audit carry-on accepted an architect `clean` with no evidence of review. It now requires a non-empty "What holds" (added as an explicit handoff field); a bare `clean` with nothing held — worst when no regression scripts run, so the architect is the sole gate — reads as unreviewed and is re-dispatched. Both are second-layer checks: the consumer verifies the field rather than trusting the agent's discipline. Lower-severity defense-in-depth, not code-level floors (milestone-close and audit carry-on are orchestrator-judgment acts with no script to instrument). closes #14 --- audit/SKILL.md | 11 +++++++++++ docs/pipeline.md | 9 ++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/audit/SKILL.md b/audit/SKILL.md index c635024..0bfb77b 100644 --- a/audit/SKILL.md +++ b/audit/SKILL.md @@ -110,12 +110,23 @@ The orchestrator picks per item: commits always exist at cycle close — they carry the architect findings, the regression numbers, and the resolution. + A carry-on is honoured only with positive evidence the architect + actually reviewed: a non-empty **What holds** in its report (the + design commitments it confirmed it read). A bare `clean` with + nothing held — especially when Step 2 was a no-op (no regression + scripts, so the architect is the *sole* gate) — reads as an + unreviewed run, not a clean cycle; re-dispatch rather than close + on it. (The architect owes this itself — `agents/architect.md` + Red Flags, "about to return clean without having read the diff in + full"; the gate verifies it rather than trusting the status.) + ## Handoff Contract `audit` hands to the orchestrator: | Field | Content | |-------|---------| +| `holds` | architect's **What holds** — the 1-3 design commitments the cycle preserved (`agents/architect.md` Output format). Non-empty is the evidence-of-review the carry-on gate checks; an empty `holds` reads as an unreviewed run. | | `drift_items` | prioritised list (path + 1-line justification) from architect | | `regression_results` | per script: exit code + **raw output verbatim** (no rounding, no summarisation — the orchestrator second-guesses with the full numbers) | | `recommendation` | per-item: `fix` / `ratify` / `carry-on` | diff --git a/docs/pipeline.md b/docs/pipeline.md index 352e8b6..8b47a1e 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -90,7 +90,14 @@ hold: and its status roll-up is `clean`: every scenario demonstrably delivers what the milestone promised; no open `bug` findings; `friction` / `spec_gap` findings resolved or - ratified into the design ledger. + ratified into the design ledger. A `clean` roll-up is honoured + only with positive evidence the test actually ran — at least + 2 examples in the working tree (`examples_added`). A `clean` + with fewer, over a milestone that touched user-visible surface, + is itself a `spec_gap`, not a clean close (the agent owes the + same floor — `fieldtest/agents/fieldtester.md` "What you DO NOT + ship"; the gate verifies it rather than trusting the roll-up). The empty + report is valid only when no example was applicable. The milestone fieldtest is the milestone-wide variant of the `fieldtest` skill: the same fieldtester agent, a carrier scoped