Files
Skills/audit/SKILL.md
T
Brummel 71fa6a28de audit: skill + architect + bencher agents migrated
Fourth skill via the boss pattern. audit is the cycle-close
cleanup gate; architect does read-only drift review against
the design ledger; bencher does hypothesis-driven performance
diagnostics.

Profile gains one slot:
- `commands.architect_sweeps` — optional list of project-
  specific architect sweep commands. The architect agent runs
  these in addition to its universal checks; non-zero exit =
  drift suspicions. AILang uses this for the
  bench/architect_sweeps.sh design-honesty / history-anchor
  / lockstep-invariant detector.

The existing `commands.regression` slot now does double duty:
the audit skill walks it in Step 2; the bencher agent reads
the failing script for diagnostics.

Vocabulary substitutions:
- AILang → "this project"
- milestone → cycle
- Boss → orchestrator
- ailang-architect, ailang-bencher → architect, bencher

AILang-specific bits replaced or removed:
- `bench/check.py && bench/compile_check.py && bench/cross_lang.py`
  → `commands.regression` list
- `bench/architect_sweeps.sh` → `commands.architect_sweeps`
- design/INDEX.md / design/contracts/ / design/models/
  → `paths.design_ledger` / `paths.design_contracts` /
    `paths.design_models`
- `bench/run.sh`, `bench/orchestrator-stats/`, `runtime/rc.c`,
  `runtime/bump.c` (in bencher) → generic harness / production-
  path references
- RC + uniqueness + bump + implicit-mode/explicit-mode
  framework (the entire AILang allocator-comparison
  calibration) → generic "cost-paying variant vs control"
  framing; the hypothesis examples generalised to abstract
  perf claims
- The two specific lockstep invariants in architect
  (Pattern::Lit::* ↔ pre_desugar_validation; lower_app ↔
  is_static_callee) → "the project's CLAUDE.md enumerates
  the known pairings; walk each one against the cycle diff"
  (the concrete pairings stay AILang-only, in AILang's
  CLAUDE.md, not in the plugin)
- The Floats fieldtest "B1 canonical example" reference →
  dropped (AILang-specific anchor)
- `design/contracts/0007-honesty-rule.md` cross-ref → dropped
  (AILang-specific contract); the honesty principle survives
  in the architect's commit-body-truthfulness check

Universal substance preserved verbatim:
- All three Iron Law clauses for audit, architect, bencher
- audit's four-step process; architect's eight-step process;
  bencher's hypothesis-first methodology + fixture-pairing
  rule + honesty rules
- audit + architect + bencher exit code 0/1/2 semantics
- All Common Rationalisations (5/6/8) and all Red Flags
  (5/6/7) rows across the three files, vocabulary-substituted
- bencher's "tie is not a result" doctrine intact

Bencher is the most language-coupled of the seven (was
AILang-allocator-specific). Generalised to a performance-
benchmarker; the discipline (hypothesis-first, design-against
not around, ties-are-bench-info) survives intact. AILang's
RC+bump examples kept in spirit via the generic "cost-paying
vs control" framing.
2026-05-28 16:00:09 +02:00

5.9 KiB

name, description
name description
audit Use at cycle close OR when baseline drift is suspected. Runs architect drift review against the design ledger plus the configured regression scripts. Mandatory at every cycle close; deferral requires an explicit backlog issue naming the reason and the re-run date.

audit — cycle-close tidy

Violating the letter of these rules is violating the spirit.

Overview

Without a scheduled clean-up step, codebases grow by accretion: every iteration adds, none tear out, and deferred drift compounds across cycles. This skill is the cleanup step. It runs after the last iteration of a cycle closes and before the next cycle starts.

When to Use / Skipping

Mandatory at every cycle close. Skipping requires an explicit backlog issue (issue tracker configured under git.issue_tracker) naming:

  • the blocking sibling cycle (if any),
  • the reason for deferral,
  • the date the audit will be re-run.

"We want to keep moving" is not a valid reason. Conventions require deferred audits to compound, which is the failure mode the skill prevents.

Also triggered manually when baseline drift is suspected outside a cycle close (e.g. the bencher reports an unexplained metric shift).

The Iron Law

TIDY IS NON-OPTIONAL AT CYCLE CLOSE
REGRESSION-SCRIPT EXIT CODE 2 = FIX INFRASTRUCTURE FIRST, NEVER REPORT AS REGRESSION
NO BASELINE UPDATE WITHOUT A PAIRED RATIFY STATEMENT IN THE AUDIT COMMIT BODY

The Process

Step 1 — Architect drift review

Dispatch architect with the cycle scope (commit range from the previous cycle-close to HEAD):

For cycle <X>: read the project's design ledger (configured
under paths.design_ledger), walk its contracts;
`git log <prev-close>..HEAD --format=full` for the cycle's
iter and audit commit bodies; `git diff <prev-close>..HEAD`
for the diff; report drift.

Architect produces a prioritised drift list (see agents/architect.md output format).

Step 2 — Regression check

Run the scripts configured under commands.regression in the project profile, in order. If the list is empty, the project has no regression gate and this step is a no-op (architect remains the gate).

The exit code of each script is the gate:

Exit Classification Action
0 Green All metrics within tolerance vs. the script's baseline. Audit can close.
1 Drift / regression At least one metric regressed past tolerance. Treat like a drift item.
2 Infrastructure failure Script-output format changed, fixture missing, or harness can't spawn. Fix the infrastructure FIRST, re-run. Never claim a regression on exit 2.

A non-zero exit on any script in the list triggers the bencher (agents/bencher.md) for hypothesis-driven diagnostics if the regression needs localising.

Step 3 — Classify and report

Combine architect drift items + regression results into one report to the orchestrator. Each item is one of:

  • fix (specific iter scoped, plan + implement)
  • ratify (--update-baseline on the firing script + an explicit ratify statement in the audit commit body naming the iter that intentionally moved the metric and why)
  • carry-on (architect found nothing actionable, regression green)

Step 4 — Resolve

The orchestrator picks per item:

  • fix path: dispatch planner + implement for a tidy iteration. The implement skill leaves the fix in the working tree; the orchestrator commits per the iter's pattern (suggested subject: iter <X>.tidy: <fix>).
  • ratify path: run --update-baseline (or the equivalent) on the firing script. The audit-close commit carries both the updated baseline and a ratify paragraph in its body naming the iter that moved the metric and the substantive reason (semantic cost, intentional trade-off).
  • carry-on path: the audit-close commit body says cycle <X> tidy (clean) and ratifies whatever the regression check drove. Audit commits always exist at cycle close — they carry the architect findings, the regression numbers, and the resolution.

Handoff Contract

audit hands to the orchestrator:

Field Content
drift_items prioritised list (path + 1-line justification) from architect
regression_results per script: exit code + raw output
recommendation per-item: fix / ratify / carry-on

The orchestrator decides the per-item outcome; audit does not self-resolve.

Common Rationalisations

Excuse Reality
"Tidy can wait until next week, let's keep moving" The project's CLAUDE.md is explicit: tidy at cycle close is non-optional. Deferral compounds; next cycle adds its own drift.
"Regression red, just bump the baseline, the regression is expected" "Expected" is exactly the claim that needs evidence. Localise the regression, then either fix OR ratify with a paragraph in the audit commit body naming the iter and reason.
"Drift item is trivial, ignore it" Trivial drift left open trains future-orchestrator to treat the architect's findings as advisory. Six items in, six items out.
"Regression scripts are hanging, skip them this cycle" Exit code 2 = fix infrastructure FIRST. No skipping.
"Architect report is empty, fast-close" Empty report is a possible outcome. Run the regression scripts anyway. Both gates must pass.

Red Flags — STOP

  • Skipping any configured regression script
  • Bumping baseline without a paired ratify statement in the audit commit body
  • Treating exit code 2 as a regression to fix
  • Closing a cycle with drift items in pending state
  • "We'll re-run after the holidays" without a dated backlog issue

Cross-references

  • Agents dispatched:
    • agents/architect.md — drift review against the design ledger
    • agents/bencher.md — hypothesis-driven regression diagnostics when a metric needs localising
  • Hand-off target: the orchestrator, or ../planner + ../implement for a tidy iteration.