bench: 21'c — compile-time regression bench (check.py + compile_check.py)

Closes the second axis the user named: every typechecker / codegen perf change was previously invisible to the tidy-iter gate. With Family 21 typeclasses (queued) and 21'b's poly-ADT additions both pushing on the typechecker, naive substitution loops would have landed silently and decayed the compile path. bench/compile_check.py is a separate script from bench/check.py because the methodology differs: sub-process spawn timing on small workloads (1ms scale for `ail check`, 65ms for `ail build`) vs. allocator-stress on large ones (multi-second). Tolerances differ by an order of magnitude (25% / 20% here vs. 5-15% there). Empirically: ail check is sub-ms across the corpus, dominated by subprocess spawn (~5-10ms on Linux); ail build is 63-69ms, dominated by clang's link step. The bench is a catastrophe detector (10x slowdowns visible) — finer regressions need a profiler. CLAUDE.md updated to list both scripts as co-equal tidy-iter gates alongside the architect drift report; exit 0/1/2 semantics are uniform across both. JOURNAL queue: 21'd (pure-compute fixtures) and 21'e (cross- language reference / hand-C ratio) remain to land the LLVM- linkable performance claim.
2026-05-09 00:59:00 +02:00
parent 07bff24527
commit 416d763b73
4 changed files with 450 additions and 9 deletions
@@ -0,0 +1,82 @@
+{
+  "version": 1,
+  "captured": "2026-05-09",
+  "captured_via": "bench/compile_check.py",
+  "note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
+  "check_ms": {
+    "hello": {
+      "baseline_ms": 0.85,
+      "tolerance_pct": 25
+    },
+    "list_map_poly": {
+      "baseline_ms": 1.05,
+      "tolerance_pct": 25
+    },
+    "local_rec_capture": {
+      "baseline_ms": 0.9,
+      "tolerance_pct": 25
+    },
+    "borrow_own_demo": {
+      "baseline_ms": 1.01,
+      "tolerance_pct": 25
+    },
+    "nested_pat": {
+      "baseline_ms": 1.74,
+      "tolerance_pct": 25
+    },
+    "bench_list_sum": {
+      "baseline_ms": 0.91,
+      "tolerance_pct": 25
+    },
+    "bench_tree_walk": {
+      "baseline_ms": 0.9,
+      "tolerance_pct": 25
+    },
+    "bench_closure_chain": {
+      "baseline_ms": 0.88,
+      "tolerance_pct": 25
+    },
+    "bench_hof_pipeline": {
+      "baseline_ms": 0.99,
+      "tolerance_pct": 25
+    }
+  },
+  "build_O0_ms": {
+    "hello": {
+      "baseline_ms": 65.03,
+      "tolerance_pct": 20
+    },
+    "list_map_poly": {
+      "baseline_ms": 67.27,
+      "tolerance_pct": 20
+    },
+    "local_rec_capture": {
+      "baseline_ms": 65.35,
+      "tolerance_pct": 20
+    },
+    "borrow_own_demo": {
+      "baseline_ms": 64.26,
+      "tolerance_pct": 20
+    },
+    "nested_pat": {
+      "baseline_ms": 67.76,
+      "tolerance_pct": 20
+    },
+    "bench_list_sum": {
+      "baseline_ms": 63.6,
+      "tolerance_pct": 20
+    },
+    "bench_tree_walk": {
+      "baseline_ms": 65.77,
+      "tolerance_pct": 20
+    },
+    "bench_closure_chain": {
+      "baseline_ms": 68.98,
+      "tolerance_pct": 20
+    },
+    "bench_hof_pipeline": {
+      "baseline_ms": 66.64,
+      "tolerance_pct": 20
+    }
+  }
+}