bench: re-ratify compile_check baseline (closes #45)
Re-baseline bench/baseline_compile.json, last ratified 2026-05-15 (20add51, mut.4-tidy). check_ms shifted +35..47% uniformly across all 12 fixtures (~1.8-3.3ms -> ~2.4-4.7ms); build_O0_ms rose ~10% (90->100ms) but stayed inside its 20% tolerance and so never tripped. RATIFY, not a regression. The decisive signal is that the check_ms drift is *uniform* across every fixture regardless of complexity (hello +35%, nested_pat +35%). Fixture-independent drift means the cost lives in shared per-invocation work, not in any fixture-specific checker path. Two concrete shared-cost growths fully account for it: - examples/prelude.ail grew 116 -> 153 lines (+32%); every `ail check` parses and typechecks the full prelude. - the kernel_stub module is now injected alongside prelude on every check (landed in raw-buf prep.3,9339279, #33) — a whole extra module parsed + typechecked per invocation. Both are real modules added by wanted features, not a quadratic blow-up. There is no fixable type-checker regression hiding here; the +0.8ms is the parse+typecheck cost of more shared source. The throughput baseline (bench/baseline.json) was already re-ratified 2026-05-28 (de4399d); this brings the compile baseline back in sync. Corrects the record: the #44 cycle-close audit body (7321826) wrongly attributed the compile_check exit-1 to transient machine load. main is sacrosanct so7321826cannot be amended; #45 and this commit are the forward correction. Verified: compile_check.py -n5 -> 24 metrics, 0 regressed, 24 stable.
This commit is contained in:
+25
-25
@@ -1,105 +1,105 @@
|
|||||||
{
|
{
|
||||||
"version": 1,
|
"version": 1,
|
||||||
"captured": "2026-05-15",
|
"captured": "2026-05-30",
|
||||||
"captured_via": "bench/compile_check.py",
|
"captured_via": "bench/compile_check.py",
|
||||||
"note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
|
"note": "Compile-time bench. `check_ms` is `ail check FILE`; `build_O0_ms` is `ail build --opt=-O0 FILE`. Wall-clock includes subprocess spawn (~5-10 ms on Linux) and, for build, the clang link step. Tolerances are tuned for noise on a quiet developer machine, not as the language correctness bar.",
|
||||||
"check_ms": {
|
"check_ms": {
|
||||||
"hello": {
|
"hello": {
|
||||||
"baseline_ms": 1.78,
|
"baseline_ms": 2.41,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"list_map_poly": {
|
"list_map_poly": {
|
||||||
"baseline_ms": 1.96,
|
"baseline_ms": 2.74,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"local_rec_capture": {
|
"local_rec_capture": {
|
||||||
"baseline_ms": 1.87,
|
"baseline_ms": 2.64,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"borrow_own_demo": {
|
"borrow_own_demo": {
|
||||||
"baseline_ms": 1.87,
|
"baseline_ms": 2.65,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"nested_pat": {
|
"nested_pat": {
|
||||||
"baseline_ms": 3.32,
|
"baseline_ms": 4.66,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_list_sum": {
|
"bench_list_sum": {
|
||||||
"baseline_ms": 1.99,
|
"baseline_ms": 2.75,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_tree_walk": {
|
"bench_tree_walk": {
|
||||||
"baseline_ms": 1.93,
|
"baseline_ms": 2.69,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_closure_chain": {
|
"bench_closure_chain": {
|
||||||
"baseline_ms": 1.96,
|
"baseline_ms": 2.66,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_hof_pipeline": {
|
"bench_hof_pipeline": {
|
||||||
"baseline_ms": 2.01,
|
"baseline_ms": 2.81,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_compute_intsum": {
|
"bench_compute_intsum": {
|
||||||
"baseline_ms": 1.84,
|
"baseline_ms": 2.61,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_compute_collatz": {
|
"bench_compute_collatz": {
|
||||||
"baseline_ms": 1.94,
|
"baseline_ms": 2.62,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
},
|
},
|
||||||
"bench_list_sum_explicit": {
|
"bench_list_sum_explicit": {
|
||||||
"baseline_ms": 2.04,
|
"baseline_ms": 2.82,
|
||||||
"tolerance_pct": 25
|
"tolerance_pct": 25
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"build_O0_ms": {
|
"build_O0_ms": {
|
||||||
"hello": {
|
"hello": {
|
||||||
"baseline_ms": 90.78,
|
"baseline_ms": 100.07,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"list_map_poly": {
|
"list_map_poly": {
|
||||||
"baseline_ms": 93.63,
|
"baseline_ms": 103.14,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"local_rec_capture": {
|
"local_rec_capture": {
|
||||||
"baseline_ms": 93.13,
|
"baseline_ms": 102.03,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"borrow_own_demo": {
|
"borrow_own_demo": {
|
||||||
"baseline_ms": 92.56,
|
"baseline_ms": 102.71,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"nested_pat": {
|
"nested_pat": {
|
||||||
"baseline_ms": 97.36,
|
"baseline_ms": 107.33,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_list_sum": {
|
"bench_list_sum": {
|
||||||
"baseline_ms": 93.93,
|
"baseline_ms": 104.09,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_tree_walk": {
|
"bench_tree_walk": {
|
||||||
"baseline_ms": 94.0,
|
"baseline_ms": 102.01,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_closure_chain": {
|
"bench_closure_chain": {
|
||||||
"baseline_ms": 93.06,
|
"baseline_ms": 103.84,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_hof_pipeline": {
|
"bench_hof_pipeline": {
|
||||||
"baseline_ms": 94.59,
|
"baseline_ms": 103.74,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_compute_intsum": {
|
"bench_compute_intsum": {
|
||||||
"baseline_ms": 91.65,
|
"baseline_ms": 101.15,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_compute_collatz": {
|
"bench_compute_collatz": {
|
||||||
"baseline_ms": 93.36,
|
"baseline_ms": 102.45,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
},
|
},
|
||||||
"bench_list_sum_explicit": {
|
"bench_list_sum_explicit": {
|
||||||
"baseline_ms": 94.03,
|
"baseline_ms": 103.0,
|
||||||
"tolerance_pct": 20
|
"tolerance_pct": 20
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user