diff --git a/CLAUDE.md b/CLAUDE.md index bc477e1..bea14fd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -181,6 +181,46 @@ and the deferral compounds across families. Skipping a tidy-iter requires an explicit JOURNAL entry naming the reason (e.g. a sibling family is blocking, or the user has asked to defer). +### Performance regressions + +`bench/check.py` is the second non-optional element of the +tidy-iter, alongside the architect drift report. Run it at every +family close. The exit code is the gate: + +- **Exit 0 (green).** All metrics within their per-metric + tolerance vs. `bench/baseline.json`. Tidy-iter can close. +- **Exit 1 (red).** At least one metric regressed beyond + tolerance. Treat the regression like an architect-drift item: + fix (revert the offending iter, refactor the hot path, diagnose + with `ailang-bencher` or `ailang-debugger`), or ratify + (`bench/check.py --update-baseline` together with a JOURNAL + entry naming the iter that intentionally moved the metric and + why). +- **Exit 2 (parser misalignment).** The bench-output format + changed under check.py's parsers. Update parsers or baseline + schema before re-running. Never claim a regression on exit-2. + +Skipping the bench-check at a family boundary requires the same +explicit JOURNAL entry as skipping the architect drift review: +name the reason, name the sibling family that's blocking, name +when it'll be re-run. + +A bug-fix iter that adds a new bench fixture (e.g. proving the +fix's invariant under load) bumps `bench/baseline.json` as part +of the same commit. The new fixture's numbers are baseline-zero +on first capture; they become regression-relevant from the next +tidy-iter onward. + +Two non-goals to keep clear: (1) per-metric tolerances are tuned +for run-to-run noise on a quiet developer machine, NOT the +language's correctness bar — Decision-10 thresholds (rc/bump ≤ +1.3× / p99/median ≤ 5×) live in DESIGN.md and are evaluated +against absolute numbers, separate from the regression check. +(2) Improvements (metric beat its baseline by more than tolerance) +do NOT auto-update the baseline; they are surfaced in the report +so the orchestrator can ratify them via `--update-baseline` with +a paired JOURNAL entry. + ### Roles of JOURNAL.md and DESIGN.md - **DESIGN.md** is the canonical specification. It describes what