bench: 21'c — compile-time regression bench (check.py + compile_check.py)

Closes the second axis the user named: every typechecker / codegen
perf change was previously invisible to the tidy-iter gate. With
Family 21 typeclasses (queued) and 21'b's poly-ADT additions both
pushing on the typechecker, naive substitution loops would have
landed silently and decayed the compile path.

bench/compile_check.py is a separate script from bench/check.py
because the methodology differs: sub-process spawn timing on small
workloads (1ms scale for `ail check`, 65ms for `ail build`) vs.
allocator-stress on large ones (multi-second). Tolerances differ
by an order of magnitude (25% / 20% here vs. 5-15% there).

Empirically: ail check is sub-ms across the corpus, dominated by
subprocess spawn (~5-10ms on Linux); ail build is 63-69ms,
dominated by clang's link step. The bench is a catastrophe
detector (10x slowdowns visible) — finer regressions need a
profiler. CLAUDE.md updated to list both scripts as co-equal
tidy-iter gates alongside the architect drift report; exit 0/1/2
semantics are uniform across both.

JOURNAL queue: 21'd (pure-compute fixtures) and 21'e (cross-
language reference / hand-C ratio) remain to land the LLVM-
linkable performance claim.
This commit is contained in:
2026-05-09 00:59:00 +02:00
parent 07bff24527
commit 416d763b73
4 changed files with 450 additions and 9 deletions
+19 -9
View File
@@ -183,9 +183,17 @@ sibling family is blocking, or the user has asked to defer).
### Performance regressions
`bench/check.py` is the second non-optional element of the
tidy-iter, alongside the architect drift report. Run it at every
family close. The exit code is the gate:
Two scripts gate the tidy-iter alongside the architect drift
report:
- **`bench/check.py`** — runtime regressions (gc/bump/rc throughput
and PTY tail-latency over the bench fixture corpus, baselined in
`bench/baseline.json`).
- **`bench/compile_check.py`** — compile-time regressions
(`ail check` and `ail build --opt=-O0` wall-time over a curated
example corpus, baselined in `bench/baseline_compile.json`).
Run both at every family close. The exit code is the gate:
- **Exit 0 (green).** All metrics within their per-metric
tolerance vs. `bench/baseline.json`. Tidy-iter can close.
@@ -193,12 +201,14 @@ family close. The exit code is the gate:
tolerance. Treat the regression like an architect-drift item:
fix (revert the offending iter, refactor the hot path, diagnose
with `ailang-bencher` or `ailang-debugger`), or ratify
(`bench/check.py --update-baseline` together with a JOURNAL
entry naming the iter that intentionally moved the metric and
why).
- **Exit 2 (parser misalignment).** The bench-output format
changed under check.py's parsers. Update parsers or baseline
schema before re-running. Never claim a regression on exit-2.
(`--update-baseline` on the firing script together with a
JOURNAL entry naming the iter that intentionally moved the
metric and why).
- **Exit 2 (infrastructure failure).** Bench-output format
changed (`check.py` parser misalignment), or a corpus fixture
is missing / fails to spawn (`compile_check.py`). Fix the
infrastructure before re-running. Never claim a regression on
exit-2.
Skipping the bench-check at a family boundary requires the same
explicit JOURNAL entry as skipping the architect drift review: