Commit Graph

2 Commits

Author SHA1 Message Date
Brummel 07bff24527 bench: 21'b — closure-pair + HOF/poly fixtures, 47-metric baseline
Two new throughput fixtures targeting blind spots in the 21'a
corpus:

- bench_closure_chain exercises the build_pair_drop_fn codegen
  path (the 18c.4 doubled-braces trigger). Each iteration of
  run_loop allocates a {thunk, env} closure pair via the
  let-rec-name-as-value escape route. Sizes 10k / 100k / 500k.
  rc/bump = 4.14x — materially worse than the 2.91x / 2.59x of
  the linear/tree fixtures, exposing that closure work pays the
  RC alloc tax twice (pair + env-struct).

- bench_hof_pipeline exercises poly-ADT instantiation and
  indirect dispatch via fold_with_fn over List<a>. Sizes 100k /
  1M / 3M elements. Ratios essentially match bench_list_sum,
  confirming the 13b static-template-plus-ctor-inline design
  has zero measurable overhead at this scale.

Baseline file extends from 31 to 47 metrics. The two new fixtures
build clean under all three allocators; the rc-arm build exercises
the per-type drop fn for the closure-pair, providing a tripwire
for any future 18c.4-class IR malformedness.

JOURNAL records both surprises (4.14x closure tax, ~zero HOF/poly
overhead) and explicitly notes the dispersion observation on
explicit_at_rc.p99 — three captures today (357.5 / 294.6 / 251.5)
confirm wide run-to-run variance on that fixture. Methodology
upgrade (n>=10 captures or tighter fixture) deferred to 21'c.

bench/run.sh fixtures array updated. bench/check.py needed no
changes — its parser handles the wider table by metric name.
2026-05-09 00:55:57 +02:00
Brummel a1b0ad5723 bench: regression harness — baseline.json + check.py
Closes the structural gap between bench/run.sh (one-shot capture
into JOURNAL) and a continuous tripwire. baseline.json records 31
metrics (16 throughput, 15 latency) with per-metric one-sided
tolerances tuned to absorb run-to-run noise on a quiet developer
machine; check.py spawns run.sh, parses both the throughput
pipe-table and the per-arm latency stanzas, diffs against the
baseline, prints a per-metric report, and exits non-zero on any
regression beyond tolerance.

User-facing flags: --from-file, --stdin, --baseline, and
--update-baseline (re-run + overwrite baseline.json after
intentional improvements).

Validation: captured the baseline, then re-ran end-to-end. All 31
metrics within tolerance. The harness also caught a single-capture
explicit-rc p99 spike (357.5 us) that was first read as drift vs.
yesterday's 18g.tidy.fu2 numbers but came in at 294.6 us on the
follow-up run — exactly the kind of noise the tolerance band is
there to absorb. Without 21'a we would have either chased a
phantom or buried the signal; with it, single noisy runs are data
points, not verdicts.

Tidy-iter discipline addition (run check.py at every family close
alongside the architect drift report) is recommended in JOURNAL
but not enacted in this iter — that's an orchestrator-level
update to CLAUDE.md.
2026-05-09 00:36:06 +02:00