Brummel
71fa6a28de
Fourth skill via the boss pattern. audit is the cycle-close
cleanup gate; architect does read-only drift review against
the design ledger; bencher does hypothesis-driven performance
diagnostics.
Profile gains one slot:
- `commands.architect_sweeps` — optional list of project-
specific architect sweep commands. The architect agent runs
these in addition to its universal checks; non-zero exit =
drift suspicions. AILang uses this for the
bench/architect_sweeps.sh design-honesty / history-anchor
/ lockstep-invariant detector.
The existing `commands.regression` slot now does double duty:
the audit skill walks it in Step 2; the bencher agent reads
the failing script for diagnostics.
Vocabulary substitutions:
- AILang → "this project"
- milestone → cycle
- Boss → orchestrator
- ailang-architect, ailang-bencher → architect, bencher
AILang-specific bits replaced or removed:
- `bench/check.py && bench/compile_check.py && bench/cross_lang.py`
→ `commands.regression` list
- `bench/architect_sweeps.sh` → `commands.architect_sweeps`
- design/INDEX.md / design/contracts/ / design/models/
→ `paths.design_ledger` / `paths.design_contracts` /
`paths.design_models`
- `bench/run.sh`, `bench/orchestrator-stats/`, `runtime/rc.c`,
`runtime/bump.c` (in bencher) → generic harness / production-
path references
- RC + uniqueness + bump + implicit-mode/explicit-mode
framework (the entire AILang allocator-comparison
calibration) → generic "cost-paying variant vs control"
framing; the hypothesis examples generalised to abstract
perf claims
- The two specific lockstep invariants in architect
(Pattern::Lit::* ↔ pre_desugar_validation; lower_app ↔
is_static_callee) → "the project's CLAUDE.md enumerates
the known pairings; walk each one against the cycle diff"
(the concrete pairings stay AILang-only, in AILang's
CLAUDE.md, not in the plugin)
- The Floats fieldtest "B1 canonical example" reference →
dropped (AILang-specific anchor)
- `design/contracts/0007-honesty-rule.md` cross-ref → dropped
(AILang-specific contract); the honesty principle survives
in the architect's commit-body-truthfulness check
Universal substance preserved verbatim:
- All three Iron Law clauses for audit, architect, bencher
- audit's four-step process; architect's eight-step process;
bencher's hypothesis-first methodology + fixture-pairing
rule + honesty rules
- audit + architect + bencher exit code 0/1/2 semantics
- All Common Rationalisations (5/6/8) and all Red Flags
(5/6/7) rows across the three files, vocabulary-substituted
- bencher's "tie is not a result" doctrine intact
Bencher is the most language-coupled of the seven (was
AILang-allocator-specific). Generalised to a performance-
benchmarker; the discipline (hypothesis-first, design-against
not around, ties-are-bench-info) survives intact. AILang's
RC+bump examples kept in spirit via the generic "cost-paying
vs control" framing.
152 lines
5.9 KiB
Markdown
152 lines
5.9 KiB
Markdown
---
|
|
name: audit
|
|
description: Use at cycle close OR when baseline drift is suspected. Runs architect drift review against the design ledger plus the configured regression scripts. Mandatory at every cycle close; deferral requires an explicit backlog issue naming the reason and the re-run date.
|
|
---
|
|
|
|
# audit — cycle-close tidy
|
|
|
|
> **Violating the letter of these rules is violating the spirit.**
|
|
|
|
## Overview
|
|
|
|
Without a scheduled clean-up step, codebases grow by accretion:
|
|
every iteration adds, none tear out, and deferred drift
|
|
compounds across cycles. This skill is the cleanup step. It
|
|
runs after the last iteration of a cycle closes and before the
|
|
next cycle starts.
|
|
|
|
## When to Use / Skipping
|
|
|
|
**Mandatory** at every cycle close. Skipping requires an
|
|
explicit backlog issue (issue tracker configured under
|
|
`git.issue_tracker`) naming:
|
|
|
|
- the blocking sibling cycle (if any),
|
|
- the reason for deferral,
|
|
- the date the audit will be re-run.
|
|
|
|
"We want to keep moving" is not a valid reason. Conventions
|
|
require deferred audits to compound, which is the failure mode
|
|
the skill prevents.
|
|
|
|
Also triggered manually when baseline drift is suspected
|
|
outside a cycle close (e.g. the bencher reports an unexplained
|
|
metric shift).
|
|
|
|
## The Iron Law
|
|
|
|
```
|
|
TIDY IS NON-OPTIONAL AT CYCLE CLOSE
|
|
REGRESSION-SCRIPT EXIT CODE 2 = FIX INFRASTRUCTURE FIRST, NEVER REPORT AS REGRESSION
|
|
NO BASELINE UPDATE WITHOUT A PAIRED RATIFY STATEMENT IN THE AUDIT COMMIT BODY
|
|
```
|
|
|
|
## The Process
|
|
|
|
### Step 1 — Architect drift review
|
|
|
|
Dispatch `architect` with the cycle scope (commit range from
|
|
the previous cycle-close to `HEAD`):
|
|
|
|
```
|
|
For cycle <X>: read the project's design ledger (configured
|
|
under paths.design_ledger), walk its contracts;
|
|
`git log <prev-close>..HEAD --format=full` for the cycle's
|
|
iter and audit commit bodies; `git diff <prev-close>..HEAD`
|
|
for the diff; report drift.
|
|
```
|
|
|
|
Architect produces a prioritised drift list (see
|
|
`agents/architect.md` output format).
|
|
|
|
### Step 2 — Regression check
|
|
|
|
Run the scripts configured under `commands.regression` in the
|
|
project profile, in order. If the list is empty, the project
|
|
has no regression gate and this step is a no-op (architect
|
|
remains the gate).
|
|
|
|
The exit code of each script is the gate:
|
|
|
|
| Exit | Classification | Action |
|
|
|------|----------------|--------|
|
|
| `0` | Green | All metrics within tolerance vs. the script's baseline. Audit can close. |
|
|
| `1` | Drift / regression | At least one metric regressed past tolerance. Treat like a drift item. |
|
|
| `2` | Infrastructure failure | Script-output format changed, fixture missing, or harness can't spawn. **Fix the infrastructure FIRST**, re-run. Never claim a regression on exit 2. |
|
|
|
|
A non-zero exit on any script in the list triggers the bencher
|
|
(`agents/bencher.md`) for hypothesis-driven diagnostics if the
|
|
regression needs localising.
|
|
|
|
### Step 3 — Classify and report
|
|
|
|
Combine architect drift items + regression results into one
|
|
report to the orchestrator. Each item is one of:
|
|
|
|
- **fix** (specific iter scoped, plan + implement)
|
|
- **ratify** (`--update-baseline` on the firing script + an
|
|
explicit ratify statement in the audit commit body naming
|
|
the iter that intentionally moved the metric and why)
|
|
- **carry-on** (architect found nothing actionable, regression
|
|
green)
|
|
|
|
### Step 4 — Resolve
|
|
|
|
The orchestrator picks per item:
|
|
|
|
- **fix path:** dispatch `planner` + `implement` for a tidy
|
|
iteration. The implement skill leaves the fix in the working
|
|
tree; the orchestrator commits per the iter's pattern
|
|
(suggested subject: `iter <X>.tidy: <fix>`).
|
|
- **ratify path:** run `--update-baseline` (or the equivalent)
|
|
on the firing script. The audit-close commit carries both
|
|
the updated baseline and a ratify paragraph in its body
|
|
naming the iter that moved the metric and the substantive
|
|
reason (semantic cost, intentional trade-off).
|
|
- **carry-on path:** the audit-close commit body says
|
|
`cycle <X> tidy (clean)` and ratifies whatever the
|
|
regression check drove. Audit commits always exist at cycle
|
|
close — they carry the architect findings, the regression
|
|
numbers, and the resolution.
|
|
|
|
## Handoff Contract
|
|
|
|
`audit` hands to the orchestrator:
|
|
|
|
| Field | Content |
|
|
|-------|---------|
|
|
| `drift_items` | prioritised list (path + 1-line justification) from architect |
|
|
| `regression_results` | per script: exit code + raw output |
|
|
| `recommendation` | per-item: `fix` / `ratify` / `carry-on` |
|
|
|
|
The orchestrator decides the per-item outcome; `audit` does
|
|
not self-resolve.
|
|
|
|
## Common Rationalisations
|
|
|
|
| Excuse | Reality |
|
|
|--------|---------|
|
|
| "Tidy can wait until next week, let's keep moving" | The project's CLAUDE.md is explicit: tidy at cycle close is non-optional. Deferral compounds; next cycle adds its own drift. |
|
|
| "Regression red, just bump the baseline, the regression is expected" | "Expected" is exactly the claim that needs evidence. Localise the regression, then either fix OR ratify with a paragraph in the audit commit body naming the iter and reason. |
|
|
| "Drift item is trivial, ignore it" | Trivial drift left open trains future-orchestrator to treat the architect's findings as advisory. Six items in, six items out. |
|
|
| "Regression scripts are hanging, skip them this cycle" | Exit code 2 = fix infrastructure FIRST. No skipping. |
|
|
| "Architect report is empty, fast-close" | Empty report is a possible outcome. Run the regression scripts anyway. Both gates must pass. |
|
|
|
|
## Red Flags — STOP
|
|
|
|
- Skipping any configured regression script
|
|
- Bumping baseline without a paired ratify statement in the audit commit body
|
|
- Treating exit code 2 as a regression to fix
|
|
- Closing a cycle with drift items in `pending` state
|
|
- "We'll re-run after the holidays" without a dated backlog issue
|
|
|
|
## Cross-references
|
|
|
|
- **Agents dispatched:**
|
|
- `agents/architect.md` — drift review against the design
|
|
ledger
|
|
- `agents/bencher.md` — hypothesis-driven regression
|
|
diagnostics when a metric needs localising
|
|
- **Hand-off target:** the orchestrator, or `../planner` +
|
|
`../implement` for a tidy iteration.
|