Brummel
416d763b73
Closes the second axis the user named: every typechecker / codegen perf change was previously invisible to the tidy-iter gate. With Family 21 typeclasses (queued) and 21'b's poly-ADT additions both pushing on the typechecker, naive substitution loops would have landed silently and decayed the compile path. bench/compile_check.py is a separate script from bench/check.py because the methodology differs: sub-process spawn timing on small workloads (1ms scale for `ail check`, 65ms for `ail build`) vs. allocator-stress on large ones (multi-second). Tolerances differ by an order of magnitude (25% / 20% here vs. 5-15% there). Empirically: ail check is sub-ms across the corpus, dominated by subprocess spawn (~5-10ms on Linux); ail build is 63-69ms, dominated by clang's link step. The bench is a catastrophe detector (10x slowdowns visible) — finer regressions need a profiler. CLAUDE.md updated to list both scripts as co-equal tidy-iter gates alongside the architect drift report; exit 0/1/2 semantics are uniform across both. JOURNAL queue: 21'd (pure-compute fixtures) and 21'e (cross- language reference / hand-C ratio) remain to land the LLVM- linkable performance claim.
83 lines
1.9 KiB
JSON
83 lines
1.9 KiB
JSON
{
|
|
"version": 1,
|
|
"captured": "2026-05-09",
|
|
"captured_via": "bench/compile_check.py",
|
|
"note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
|
|
"check_ms": {
|
|
"hello": {
|
|
"baseline_ms": 0.85,
|
|
"tolerance_pct": 25
|
|
},
|
|
"list_map_poly": {
|
|
"baseline_ms": 1.05,
|
|
"tolerance_pct": 25
|
|
},
|
|
"local_rec_capture": {
|
|
"baseline_ms": 0.9,
|
|
"tolerance_pct": 25
|
|
},
|
|
"borrow_own_demo": {
|
|
"baseline_ms": 1.01,
|
|
"tolerance_pct": 25
|
|
},
|
|
"nested_pat": {
|
|
"baseline_ms": 1.74,
|
|
"tolerance_pct": 25
|
|
},
|
|
"bench_list_sum": {
|
|
"baseline_ms": 0.91,
|
|
"tolerance_pct": 25
|
|
},
|
|
"bench_tree_walk": {
|
|
"baseline_ms": 0.9,
|
|
"tolerance_pct": 25
|
|
},
|
|
"bench_closure_chain": {
|
|
"baseline_ms": 0.88,
|
|
"tolerance_pct": 25
|
|
},
|
|
"bench_hof_pipeline": {
|
|
"baseline_ms": 0.99,
|
|
"tolerance_pct": 25
|
|
}
|
|
},
|
|
"build_O0_ms": {
|
|
"hello": {
|
|
"baseline_ms": 65.03,
|
|
"tolerance_pct": 20
|
|
},
|
|
"list_map_poly": {
|
|
"baseline_ms": 67.27,
|
|
"tolerance_pct": 20
|
|
},
|
|
"local_rec_capture": {
|
|
"baseline_ms": 65.35,
|
|
"tolerance_pct": 20
|
|
},
|
|
"borrow_own_demo": {
|
|
"baseline_ms": 64.26,
|
|
"tolerance_pct": 20
|
|
},
|
|
"nested_pat": {
|
|
"baseline_ms": 67.76,
|
|
"tolerance_pct": 20
|
|
},
|
|
"bench_list_sum": {
|
|
"baseline_ms": 63.6,
|
|
"tolerance_pct": 20
|
|
},
|
|
"bench_tree_walk": {
|
|
"baseline_ms": 65.77,
|
|
"tolerance_pct": 20
|
|
},
|
|
"bench_closure_chain": {
|
|
"baseline_ms": 68.98,
|
|
"tolerance_pct": 20
|
|
},
|
|
"bench_hof_pipeline": {
|
|
"baseline_ms": 66.64,
|
|
"tolerance_pct": 20
|
|
}
|
|
}
|
|
}
|