A field test and a fixture are orthogonal artefacts. A fixture **pins**
behaviour for the future: something executes it, so a change that breaks it
turns a test red. A field test **proves** behaviour at one moment: a downstream
consumer reached the shipped surface from outside, and the green cycle records
that. Calling the field test's output "fixtures" imported the first artefact's
lifecycle onto the second one, which gave it a birth and no death — every
fieldtested cycle grew the repository permanently, and the recorded command
lines stayed on the agenda of every agent that grepped the tree.
`docs/conventions.md` § Lifecycle is now the single source of both rules, beside
the spec/plan lifecycle it mirrors:
1. **Rotation depth 1.** The next field test clears the previous corpus as it
files its own, so `fieldtests/` always holds exactly the proof for the tree
as it stands. No staleness detection, because there is no signal to detect:
a transcript never turns red. `git log --grep='^fieldtest: '` is the index and
`git show <sha>:<path>` returns any of it verbatim.
2. **No durable artefact may point into the corpus.** A pointer into a rotating
directory either dangles at the next rotation or keeps a dead surface on the
agenda. Replace it with the description it stood in for, drop it if it serves
nothing, or promote the artefact to a real fixture with a test that runs it.
`fieldtest/SKILL.md` § Dispatch owns the mechanics. The orchestrator rotates
*before* dispatching, not at commit time: the directory the fieldtester reads is
then the one it writes, which is what keeps a predecessor's retired flags off its
agenda — commit-time rotation would tidy the tree and leave that leak open. The
commands are written over a resolved `<corpus>` (default `fieldtests`, project
fact may override) with a refusal condition, because an unsubstituted path makes
the rotation a silent no-op and can stage a deletion of the predecessor under a
subject claiming the new corpus' examples. `rm -rf`, not `git rm`, so untracked
build output goes too; the orchestrator ensures the repo-root ignore entry
first, since wholesale staging would otherwise commit a build tree into history
that rotation cannot reclaim. An outcome with no new proof (bounce, or an empty
`examples_added`) restores the predecessor with `rm -rf` + `git restore` and
files no commit — `git clean -fd` was rejected: it leaves exactly the ignored
build output while `git status` reports clean (verified, git 2.55).
Rule 2 needs reach beyond the fieldtester, which authors almost nothing durable.
Enforcement is by reference at the actors that walk the tree: `architect` flags
new pointers in the cycle range and is told the corpus is not live surface (a
corpus example is not the property-protecting test a feature owes, and a
wholesale corpus swap in the diff is the routine rotation, not drift);
`plan-recon` excludes it from every sweep, with the promotion iteration as the
one carve-out where the corpus path is the change site; `tester` no longer reads
it for fixture style; `grounding-check` and `spec-skeptic` may never ratify from
it (nothing executes it, so reverting a mechanism cannot turn it red);
`synthetic-user` may not ground a stance in it. Filed findings carry their
evidence inline — an issue outlives many rotations and no gate sees it.
Deliberately not done: a rule block in `templates/CLAUDE.md.fragment`. The
fragment does not re-seed a fixed convention that lives in conventions.md
(ratified in dacadba), so the fragment gains only the project-fact wording.
closes #38
9.0 KiB
name, description
| name | description |
|---|---|
| audit | Use at cycle close OR when baseline drift is suspected. Runs architect drift review against the design ledger plus the configured regression scripts. Mandatory at every cycle close; deferral requires an explicit backlog issue naming the reason and the re-run date. |
audit — cycle-close tidy
Violating the letter of these rules is violating the spirit.
Overview
Without a scheduled clean-up step, codebases grow by accretion: every iteration adds, none tear out, and deferred drift compounds across cycles. This skill is the cleanup step. It runs after the last iteration of a cycle closes and before the next cycle starts.
When to Use / Skipping
Mandatory at every cycle close. Skipping requires an explicit backlog issue (the project's issue tracker — its CLAUDE.md project facts) naming:
- the blocking sibling cycle (if any),
- the reason for deferral,
- the date the audit will be re-run.
"We want to keep moving" is not a valid reason.
Also triggered manually when baseline drift is suspected outside a cycle close (e.g. the bencher reports an unexplained metric shift).
The Iron Law
TIDY IS NON-OPTIONAL AT CYCLE CLOSE
REGRESSION-SCRIPT EXIT CODE 2 = FIX INFRASTRUCTURE FIRST, NEVER REPORT AS REGRESSION
NO BASELINE UPDATE WITHOUT A PAIRED RATIFY STATEMENT IN THE AUDIT COMMIT BODY
The Process
Step 1 — Architect drift review
Dispatch architect with the cycle scope (commit range from
the previous cycle-close to HEAD):
For cycle <X>: read the project's design ledger, if it has one
(its CLAUDE.md project facts), walk its contracts;
`git log <prev-close>..HEAD --format=full` for the cycle's
iter and audit commit bodies; `git diff <prev-close>..HEAD`
for the diff; read the cycle's spec at `docs/specs/<slug>.md`
if one exists (a git-ignored working file — read it from disk,
not git); report drift.
Architect produces a prioritised drift list (see
agents/architect.md output format).
Step 2 — Regression check
Run the project's regression command(s) (its CLAUDE.md project facts), in order. If there are none, the project has no regression gate and this step is a no-op (architect remains the gate).
The exit code of each script is the gate:
| Exit | Classification | Action |
|---|---|---|
0 |
Green | All metrics within tolerance vs. the script's baseline. The cycle's drift gate is clear (drift-clean, not a milestone close — see ../docs/pipeline.md § Milestone-close gate). |
1 |
Drift / regression | At least one metric regressed past tolerance. Treat like a drift item. |
2 |
Infrastructure failure | Script-output format changed, fixture missing, or harness can't spawn. Fix the infrastructure FIRST, re-run. Never claim a regression on exit 2. |
A non-zero exit on any script in the list triggers the bencher
(agents/bencher.md) for hypothesis-driven diagnostics if the
regression needs localising.
Step 3 — Classify and report
Combine architect drift items + regression results into one report to the orchestrator. Each item is one of:
- fix (specific iter scoped, plan + implement)
- ratify (
--update-baselineon the firing script + an explicit ratify statement in the audit commit body naming the iter that intentionally moved the metric and why) - carry-on (architect found nothing actionable, regression green)
Step 4 — Resolve
The orchestrator picks per item:
-
fix path: dispatch
planner+implementfor a tidy iteration. The implement skill leaves the fix in the working tree; the orchestrator commits per the iter's pattern (suggested subject:iter <X>.tidy: <fix>). -
ratify path: run
--update-baseline(or the equivalent) on the firing script. The audit-close commit carries both the updated baseline and a ratify paragraph in its body naming the iter that moved the metric and the substantive reason (semantic cost, intentional trade-off). -
carry-on path: the audit-close commit body says
cycle <X> tidy (clean)and ratifies whatever the regression check drove. Herecleanmeans drift-clean — the cycle's code matches the ledger; it is not a milestone close, which additionally requires a green milestone fieldtest (see../docs/pipeline.md§ Milestone-close gate). Audit commits always exist at cycle close — they carry the architect findings, the regression numbers, and the resolution.A carry-on is honoured only with positive evidence the architect actually reviewed: a non-empty What holds in its report (the design commitments it confirmed it read). A bare
cleanwith nothing held — especially when Step 2 was a no-op (no regression scripts, so the architect is the sole gate) — reads as an unreviewed run, not a clean cycle; re-dispatch rather than close on it. (The architect owes this itself —agents/architect.mdRed Flags, "about to return clean without having read the diff in full"; the gate verifies it rather than trusting the status.)
Step 5 — Discard the cycle's spec and plan
Once the cycle is drift-clean (or the drift is ratified) and the
architect has read the spec, the working-tree spec and plan have served
their purpose. Discard every spec/plan file present in the working
tree with a plain shell rm:
rm -f docs/specs/*.md docs/plans/*.md
This is a shell delete, NOT git rm — the files are git-ignored and were
never committed, so nothing is staged and the audit-close commit carries
only code and ledger changes. Ordering is load-bearing: the
architect's read (Step 1) must precede this delete, because a git-ignored
spec has no git show recovery once it is gone.
The sweep is scoped to these two directories. It does not touch the
fieldtest corpus, which is tracked code with its own lifecycle: the next
field test rotates it, inside that field test's own commit
(../docs/conventions.md § Lifecycle). Keeping the corpus out of this
step is what preserves the property above — the sweep stages nothing, so
the audit-close commit stays free of artefact churn.
Deleting everything present is correct, not over-broad: fieldtest
runs only after audit closes clean, so this cycle's own fieldtest
spec does not exist yet at this point. The only fieldtest-<slug>.md
that can be present here is a spent one — written after the previous
cycle's close and already consumed as this cycle's planner input — so
it too should go. There is nothing to spare: the newly-written fieldtest
spec appears after this step and survives until the next cycle's close,
when this same sweep removes it.
Handoff Contract
audit hands to the orchestrator:
| Field | Content |
|---|---|
holds |
architect's What holds — the 1-3 design commitments the cycle preserved (agents/architect.md Output format). Non-empty is the evidence-of-review the carry-on gate checks; an empty holds reads as an unreviewed run. |
drift_items |
prioritised list (path + 1-line justification) from architect |
regression_results |
per script: exit code + raw output verbatim (no rounding, no summarisation — the orchestrator second-guesses with the full numbers) |
recommendation |
per-item: fix / ratify / carry-on |
The orchestrator decides the per-item outcome; audit does
not self-resolve.
Common Rationalisations
| Excuse | Reality |
|---|---|
| "Tidy can wait until next week, let's keep moving" | The project's CLAUDE.md is explicit: tidy at cycle close is non-optional. Deferral compounds; next cycle adds its own drift. |
| "Regression red, just bump the baseline, the regression is expected" | "Expected" is exactly the claim that needs evidence. Localise the regression, then either fix OR ratify with a paragraph in the audit commit body naming the iter and reason. |
| "Drift item is trivial, ignore it" | Trivial drift left open trains future-orchestrator to treat the architect's findings as advisory. Six items in, six items out. |
| "Regression scripts are hanging, skip them this cycle" | Exit code 2 = fix infrastructure FIRST. No skipping. |
| "Architect report is empty, fast-close" | Empty report is a possible outcome. Run the regression scripts anyway. Both gates must pass. |
Red Flags — STOP
- Skipping any configured regression script
- Bumping baseline without a paired ratify statement in the audit commit body
- Treating exit code 2 as a regression to fix
- Closing a cycle with drift items in
pendingstate - "We'll re-run after the holidays" without a dated backlog issue
- Declaring a milestone closeable on the strength of an
audit-clean while the milestone fieldtest has not run green —
audit proves drift-clean, never functional (see
../docs/pipeline.md§ Milestone-close gate)
Cross-references
- Agents dispatched:
agents/architect.md— drift review against the design ledgeragents/bencher.md— hypothesis-driven regression diagnostics when a metric needs localising
- Hand-off target: the orchestrator, or
../planner+../implementfor a tidy iteration.