bench: 21'c — compile-time regression bench (check.py + compile_check.py)

Closes the second axis the user named: every typechecker / codegen
perf change was previously invisible to the tidy-iter gate. With
Family 21 typeclasses (queued) and 21'b's poly-ADT additions both
pushing on the typechecker, naive substitution loops would have
landed silently and decayed the compile path.

bench/compile_check.py is a separate script from bench/check.py
because the methodology differs: sub-process spawn timing on small
workloads (1ms scale for `ail check`, 65ms for `ail build`) vs.
allocator-stress on large ones (multi-second). Tolerances differ
by an order of magnitude (25% / 20% here vs. 5-15% there).

Empirically: ail check is sub-ms across the corpus, dominated by
subprocess spawn (~5-10ms on Linux); ail build is 63-69ms,
dominated by clang's link step. The bench is a catastrophe
detector (10x slowdowns visible) — finer regressions need a
profiler. CLAUDE.md updated to list both scripts as co-equal
tidy-iter gates alongside the architect drift report; exit 0/1/2
semantics are uniform across both.

JOURNAL queue: 21'd (pure-compute fixtures) and 21'e (cross-
language reference / hand-C ratio) remain to land the LLVM-
linkable performance claim.

CLAUDE.md

@@ -183,9 +183,17 @@ sibling family is blocking, or the user has asked to defer).
 
 ### Performance regressions
 
-`bench/check.py` is the second non-optional element of the
+Two scripts gate the tidy-iter alongside the architect drift
-tidy-iter, alongside the architect drift report. Run it at every
+report:
-family close. The exit code is the gate:
+
+- **`bench/check.py`** — runtime regressions (gc/bump/rc throughput
+  and PTY tail-latency over the bench fixture corpus, baselined in
+  `bench/baseline.json`).
+- **`bench/compile_check.py`** — compile-time regressions
+  (`ail check` and `ail build --opt=-O0` wall-time over a curated
+  example corpus, baselined in `bench/baseline_compile.json`).
+
+Run both at every family close. The exit code is the gate:
 
 - **Exit 0 (green).** All metrics within their per-metric
   tolerance vs. `bench/baseline.json`. Tidy-iter can close.
@@ -193,12 +201,14 @@ family close. The exit code is the gate:
   tolerance. Treat the regression like an architect-drift item:
   fix (revert the offending iter, refactor the hot path, diagnose
   with `ailang-bencher` or `ailang-debugger`), or ratify
-  (`bench/check.py --update-baseline` together with a JOURNAL
+  (`--update-baseline` on the firing script together with a
-  entry naming the iter that intentionally moved the metric and
+  JOURNAL entry naming the iter that intentionally moved the
-  why).
+  metric and why).
-- **Exit 2 (parser misalignment).** The bench-output format
+- **Exit 2 (infrastructure failure).** Bench-output format
-  changed under check.py's parsers. Update parsers or baseline
+  changed (`check.py` parser misalignment), or a corpus fixture
-  schema before re-running. Never claim a regression on exit-2.
+  is missing / fails to spawn (`compile_check.py`). Fix the
+  infrastructure before re-running. Never claim a regression on
+  exit-2.
 
 Skipping the bench-check at a family boundary requires the same
 explicit JOURNAL entry as skipping the architect drift review:

bench/baseline_compile.json

@@ -0,0 +1,82 @@
+{
+  "version": 1,
+  "captured": "2026-05-09",
+  "captured_via": "bench/compile_check.py",
+  "note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
+  "check_ms": {
+    "hello": {
+      "baseline_ms": 0.85,
+      "tolerance_pct": 25
+    },
+    "list_map_poly": {
+      "baseline_ms": 1.05,
+      "tolerance_pct": 25
+    },
+    "local_rec_capture": {
+      "baseline_ms": 0.9,
+      "tolerance_pct": 25
+    },
+    "borrow_own_demo": {
+      "baseline_ms": 1.01,
+      "tolerance_pct": 25
+    },
+    "nested_pat": {
+      "baseline_ms": 1.74,
+      "tolerance_pct": 25
+    },
+    "bench_list_sum": {
+      "baseline_ms": 0.91,
+      "tolerance_pct": 25
+    },
+    "bench_tree_walk": {
+      "baseline_ms": 0.9,
+      "tolerance_pct": 25
+    },
+    "bench_closure_chain": {
+      "baseline_ms": 0.88,
+      "tolerance_pct": 25
+    },
+    "bench_hof_pipeline": {
+      "baseline_ms": 0.99,
+      "tolerance_pct": 25
+    }
+  },
+  "build_O0_ms": {
+    "hello": {
+      "baseline_ms": 65.03,
+      "tolerance_pct": 20
+    },
+    "list_map_poly": {
+      "baseline_ms": 67.27,
+      "tolerance_pct": 20
+    },
+    "local_rec_capture": {
+      "baseline_ms": 65.35,
+      "tolerance_pct": 20
+    },
+    "borrow_own_demo": {
+      "baseline_ms": 64.26,
+      "tolerance_pct": 20
+    },
+    "nested_pat": {
+      "baseline_ms": 67.76,
+      "tolerance_pct": 20
+    },
+    "bench_list_sum": {
+      "baseline_ms": 63.6,
+      "tolerance_pct": 20
+    },
+    "bench_tree_walk": {
+      "baseline_ms": 65.77,
+      "tolerance_pct": 20
+    },
+    "bench_closure_chain": {
+      "baseline_ms": 68.98,
+      "tolerance_pct": 20
+    },
+    "bench_hof_pipeline": {
+      "baseline_ms": 66.64,
+      "tolerance_pct": 20
+    }
+  }
+}

bench/compile_check.py

@@ -0,0 +1,234 @@
+#!/usr/bin/env python3
+# Compile-time regression check.
+#
+# Times `ail check FILE` and `ail build --opt=-O0 FILE` over a curated
+# corpus, drops the slowest run, takes the median of the rest, diffs
+# against bench/baseline_compile.json. Exits 0 if every metric is within
+# tolerance, 1 on any regression, 2 on infrastructure failure (missing
+# fixture, ail spawn error).
+#
+# Methodology caveats this bench cannot work around:
+#   - Wall time of `ail check` is dominated by subprocess spawn (~5-10ms
+#     on Linux); typechecker work below that scale is invisible.
+#   - Wall time of `ail build` is dominated by clang's link step
+#     (~100ms+); the AILang IR-emit + codegen contribution is a
+#     small slice of the total.
+# Both are still useful as catastrophe detectors (10x slowdowns visible)
+# even if subtler regressions need a profiler. The right tool for finer-
+# grained codegen perf is the runtime bench (bench/run.sh), not this one.
+#
+# Usage:
+#     bench/compile_check.py                  # run bench, exit 0/1/2
+#     bench/compile_check.py -n 10            # more runs, tighter median
+#     bench/compile_check.py --update-baseline
+#     bench/compile_check.py --baseline path  # alternate baseline file
+
+from __future__ import annotations
+
+import argparse
+import json
+import subprocess
+import sys
+import tempfile
+import time
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+DEFAULT_BASELINE = ROOT / "bench" / "baseline_compile.json"
+AIL = ROOT / "target" / "release" / "ail"
+EXAMPLES = ROOT / "examples"
+
+# Curated corpus. Pinned by name so the baseline keys are stable across
+# example-directory churn. Each entry must have a corresponding
+# `<name>.ail.json` under examples/.
+CORPUS = [
+    # Surface coverage: each fixture exercises a different feature set.
+    "hello",                    # IO baseline
+    "list_map_poly",            # polymorphism + HOF
+    "local_rec_capture",        # let-rec capture, closure path
+    "borrow_own_demo",          # explicit modes
+    "nested_pat",               # nested pattern matching
+    # Bench fixtures (correlation with bench/check.py runtime baseline).
+    "bench_list_sum",
+    "bench_tree_walk",
+    "bench_closure_chain",
+    "bench_hof_pipeline",
+]
+
+
+def time_one(args: list[str]) -> float:
+    """Run a command, return wall-time in milliseconds. Raises if exit != 0."""
+    t0 = time.monotonic()
+    proc = subprocess.run(args, capture_output=True)
+    t1 = time.monotonic()
+    if proc.returncode != 0:
+        raise RuntimeError(
+            f"command failed (exit {proc.returncode}): {' '.join(args)}\n"
+            f"stderr: {proc.stderr.decode(errors='replace')}"
+        )
+    return (t1 - t0) * 1000.0
+
+
+def median_drop_slowest(runs: list[float]) -> float:
+    """Drop the slowest run, return median of the rest."""
+    if len(runs) < 2:
+        return runs[0] if runs else 0.0
+    kept = sorted(runs)[:-1]
+    n = len(kept)
+    if n % 2 == 1:
+        return kept[n // 2]
+    return 0.5 * (kept[n // 2 - 1] + kept[n // 2])
+
+
+def measure(num_runs: int) -> dict[str, dict[str, float]]:
+    """Return {section: {fixture: ms}}. Aborts on missing fixture / spawn fail."""
+    if not AIL.is_file():
+        print(f"missing release ail binary at {AIL}; building...", file=sys.stderr)
+        subprocess.run(
+            ["cargo", "build", "--release", "-p", "ail"],
+            cwd=str(ROOT), check=True, capture_output=True,
+        )
+
+    out: dict[str, dict[str, float]] = {"check_ms": {}, "build_O0_ms": {}}
+    with tempfile.NamedTemporaryFile(prefix="bench_compile_", delete=False) as tf:
+        out_path = tf.name
+
+    try:
+        for fixture in CORPUS:
+            src = EXAMPLES / f"{fixture}.ail.json"
+            if not src.is_file():
+                print(f"missing fixture: {src}", file=sys.stderr)
+                sys.exit(2)
+
+            try:
+                check_runs = [time_one([str(AIL), "check", str(src)]) for _ in range(num_runs)]
+                build_runs = [time_one([str(AIL), "build", "--opt=-O0",
+                                        str(src), "-o", out_path]) for _ in range(num_runs)]
+            except RuntimeError as e:
+                print(f"spawn error on {fixture}: {e}", file=sys.stderr)
+                sys.exit(2)
+
+            out["check_ms"][fixture] = median_drop_slowest(check_runs)
+            out["build_O0_ms"][fixture] = median_drop_slowest(build_runs)
+            print(f"  {fixture:<32} check={out['check_ms'][fixture]:6.1f}ms  "
+                  f"build={out['build_O0_ms'][fixture]:6.1f}ms", file=sys.stderr)
+    finally:
+        try:
+            Path(out_path).unlink()
+        except FileNotFoundError:
+            pass
+
+    return out
+
+
+def diff_report(measured: dict, baseline: dict) -> tuple[str, bool]:
+    rows = []
+    has_regression = False
+    for section in ("check_ms", "build_O0_ms"):
+        for fixture, spec in baseline.get(section, {}).items():
+            actual = measured[section].get(fixture)
+            if actual is None:
+                rows.append((f"{section}.{fixture}", spec["baseline_ms"], None,
+                             None, spec["tolerance_pct"], "MISSING"))
+                has_regression = True
+                continue
+            base = spec["baseline_ms"]
+            tol = spec["tolerance_pct"]
+            diff = 100.0 * (actual - base) / base if base else 0.0
+            if diff > tol:
+                status = "REGRESSION"
+                has_regression = True
+            elif diff < -tol:
+                status = "improvement"
+            else:
+                status = "ok"
+            rows.append((f"{section}.{fixture}", base, actual, diff, tol, status))
+
+    lines = []
+    lines.append(f"{'metric':<48} {'baseline':>10} {'actual':>10} {'diff':>9} {'tol':>6}  status")
+    lines.append("-" * 100)
+    for metric, base, actual, diff, tol, status in rows:
+        if actual is None:
+            lines.append(f"{metric:<48} {base:>10.1f} {'-':>10} {'-':>9} {tol:>5.1f}%  {status}")
+        else:
+            lines.append(
+                f"{metric:<48} {base:>10.1f} {actual:>10.1f} "
+                f"{diff:>+8.2f}% {tol:>5.1f}%  {status}"
+            )
+    regressed = sum(1 for r in rows if r[5] == "REGRESSION")
+    improved = sum(1 for r in rows if r[5] == "improvement")
+    stable = len(rows) - regressed - improved
+    lines.append("")
+    lines.append(f"summary: {len(rows)} metrics; "
+                 f"{regressed} regressed, {improved} improved beyond tolerance, "
+                 f"{stable} stable")
+    return "\n".join(lines), has_regression
+
+
+def write_baseline(measured: dict, baseline_path: Path) -> None:
+    today = subprocess.check_output(["date", "+%Y-%m-%d"], text=True).strip()
+    if baseline_path.exists():
+        existing = json.loads(baseline_path.read_text())
+        check_tols = {f: spec.get("tolerance_pct", 25)
+                      for f, spec in existing.get("check_ms", {}).items()}
+        build_tols = {f: spec.get("tolerance_pct", 20)
+                      for f, spec in existing.get("build_O0_ms", {}).items()}
+    else:
+        check_tols = {}
+        build_tols = {}
+
+    new = {
+        "version": 1,
+        "captured": today,
+        "captured_via": "bench/compile_check.py",
+        "note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
+        "check_ms": {},
+        "build_O0_ms": {},
+    }
+    for fixture, ms in measured["check_ms"].items():
+        new["check_ms"][fixture] = {
+            "baseline_ms": round(ms, 2),
+            "tolerance_pct": check_tols.get(fixture, 25),
+        }
+    for fixture, ms in measured["build_O0_ms"].items():
+        new["build_O0_ms"][fixture] = {
+            "baseline_ms": round(ms, 2),
+            "tolerance_pct": build_tols.get(fixture, 20),
+        }
+    baseline_path.write_text(json.dumps(new, indent=2) + "\n")
+    print(f">>> wrote new baseline to {baseline_path}", file=sys.stderr)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("-n", "--runs", type=int, default=5,
+                    help="runs per fixture per op; min 2, default 5")
+    ap.add_argument("--baseline", type=Path, default=DEFAULT_BASELINE)
+    ap.add_argument("--update-baseline", action="store_true",
+                    help="re-measure, then overwrite baseline_compile.json")
+    args = ap.parse_args()
+    if args.runs < 2:
+        print("--runs must be >= 2", file=sys.stderr)
+        return 2
+
+    print(f">>> measuring {len(CORPUS)} fixtures, {args.runs} runs each "
+          f"(check + build_O0)", file=sys.stderr)
+    measured = measure(args.runs)
+
+    if args.update_baseline:
+        write_baseline(measured, args.baseline)
+        return 0
+
+    if not args.baseline.exists():
+        print(f"no baseline at {args.baseline}; create one with --update-baseline",
+              file=sys.stderr)
+        return 2
+
+    baseline = json.loads(args.baseline.read_text())
+    report, has_regression = diff_report(measured, baseline)
+    print(report)
+    return 1 if has_regression else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

docs/JOURNAL.md

@@ -10134,3 +10134,118 @@ fixture and baseline-file additions only.
 - **Deferred richer integration paths** (from 20f) — unchanged.
 - **Family 21+** — typeclasses, polymorphic ADTs at runtime,
   pattern-binding generalisation. Orchestrator-level fork.
+
+## 2026-05-09 — Iter 21'c: compile-time regression bench
+
+Closes the second axis the user explicitly named — until this
+iter, every typechecker / codegen perf change was invisible to the
+tidy-iter gate. Family 21 typeclasses (queued) plus 21'b's poly-
+ADT additions both push on the typechecker; without a tripwire,
+naive substitution loops or O(n²) constraint resolution would
+land silently and decay the whole compile path.
+
+### What shipped
+
+**`bench/compile_check.py`** — separate from `bench/check.py`
+because the methodology is different (sub-process spawn timing
+on small workloads vs. allocator-stress on large ones) and the
+relevant tolerances differ by an order of magnitude. Two ops
+per fixture: `ail check FILE` and `ail build --opt=-O0 FILE -o T`.
+Same drop-slowest-of-N, median-of-rest convention as
+`bench/run.sh`.
+
+**`bench/baseline_compile.json`** — 18 metrics (9 fixtures × 2
+ops). Curated corpus: 5 surface-coverage examples (`hello`,
+`list_map_poly`, `local_rec_capture`, `borrow_own_demo`,
+`nested_pat`) + 4 bench-throughput fixtures (correlation with
+`bench/check.py`).
+
+**Baselines on this machine**:
+
+```
+fixture                  | check(ms) | build(ms)
+hello                    |       0.8 |      65.0
+list_map_poly            |       1.1 |      67.3
+local_rec_capture        |       0.9 |      65.3
+borrow_own_demo          |       1.0 |      64.3
+nested_pat               |       1.7 |      67.8
+bench_list_sum           |       0.9 |      63.6
+bench_tree_walk          |       0.9 |      65.8
+bench_closure_chain      |       0.9 |      69.0
+bench_hof_pipeline       |       1.0 |      66.6
+```
+
+### What the data tells us
+
+`ail check` runs at **sub-millisecond per fixture** for everything
+except `nested_pat` (1.7ms — its deeper pattern tree marginally
+exceeds the noise floor). The typechecker is genuinely fast at
+the current corpus scale; on this hardware the wall-clock is
+dominated by subprocess spawn (~5-10ms on Linux), not by check
+work. The bench detects catastrophes (10× slowdowns visible),
+not subtler regressions — those want a profiler, not wall-clock.
+
+`ail build --opt=-O0` runs at **63-69ms per fixture**, dominated
+by clang's link step. The variance across fixtures is small —
+~9% spread between fastest (`bench_list_sum` 63.6) and slowest
+(`bench_closure_chain` 69.0). This is fine for catastrophe-
+detection but not informative about codegen-quality differences.
+For codegen-quality questions the runtime bench (rc/bump ratios)
+remains the right tool.
+
+### Tolerances
+
+- **`check_ms`**: 25% per fixture. Justified empirically: a
+  re-run captured a +17.35% diff on `bench_hof_pipeline check`
+  with no code changes. Sub-millisecond timing is noisy.
+- **`build_O0_ms`**: 20% per fixture. Build noise is materially
+  lower; observed re-run drift was ≤7% on every fixture.
+
+These are catastrophe-detector tolerances. Tightening them
+would mean false-positives on quiet-machine noise.
+
+### CLAUDE.md update
+
+The `Performance regressions` section now lists both
+`bench/check.py` (runtime) and `bench/compile_check.py` (compile)
+as co-equal tidy-iter gates alongside the architect drift report.
+Exit 0 / 1 / 2 semantics are uniform across both scripts.
+
+### What this iter does NOT do
+
+- **No latency-harness methodology upgrade.** The wide
+  explicit_at_rc.p99 dispersion observed across today's three
+  captures (357.5 / 294.6 / 251.5) is a runtime-bench problem;
+  the compile bench is a different axis. Methodology upgrade
+  (n>=10 captures or tighter latency fixture) stays queued.
+- **No O2 build bench.** `--opt=-O2` includes additional clang
+  passes that 2x-3x the build time. Useful for catching codegen
+  blowup-induced build slowdowns; not useful for detecting
+  AILang-side regressions, which are amply covered by the O0
+  pass. Future addition if/when warranted.
+- **No incremental check bench.** Today every `ail check`
+  rebuilds the entire context. If incremental compilation is
+  added later (no current plan), re-baseline.
+
+### Test state
+
+288 / 0 / 3, unchanged. No Rust changes; the iter is bench-
+infrastructure additions only.
+
+### JOURNAL queue (updated)
+
+- **21'd — pure-compute fixtures.** Mandelbrot / N-body / integer-
+  loop workloads. Heap-light, codegen-quality-heavy. Pairs
+  naturally with 21'e.
+- **21'e — cross-language reference.** Hand-C variants of the
+  bench corpus, compiled with `clang -O2`. AILang/C ratio is the
+  honest answer to CLAUDE.md's "LLVM-linkable, performance is
+  extremely important" claim, which today is unbacked by data.
+- **Latency methodology upgrade** — n=10+ captures or tighter
+  fixture for `explicit_at_rc.p99`. Could fold into 21'd or be
+  its own short iter.
+- **`FnDef::synthetic(...)` factor-out** — unchanged.
+- **Boehm full retirement** — unchanged.
+- **Deferred richer integration paths** (from 20f) — unchanged.
+- **Family 21+** — typeclasses, polymorphic ADTs at runtime,
+  pattern-binding generalisation. Orchestrator-level fork.