Brummel
a97aaebd45
One-iteration infra milestone closing Gitea #15 + #16. Two issues framed distinct symptoms; reproduction on 2026-05-20 HEAD via two back-to-back `bench/check.py -n 5` runs collapses them onto a single picture: - #15 ("*.bump_s stale") is wider than the issue framed. Drift reproduces deterministically on `bench_list_sum.bump_s` (+13.91% / +15.56%) AND on `bench_hof_pipeline.gc_s` (+10.60% / +10.94%) — not just the bump_s family. Drift correlates with memory pressure: `bench_tree_walk` and `bench_compute_collatz` are clean across both runs. - #16 ("*.max_us structural false-positive") has matured. The "vanishes on rerun" pattern the issue documented from 2026-05-16 / 2026-05-18 audits is gone: today `implicit_at_rc.max_us` reproduces at +47.20% / +49.42%, and `implicit_at_rc.p99_9_us` at +27.79% / +41.92%. Same arm only (`implicit @ rc`); the other two latency arms are clean. The metrics are now drift-elevated AND structurally jitter- prone — both reasons to retire them from the gate. Decision: one cohesive change, all in `bench/baseline.json`. Drop the six unreliable entries (`max_us` + `p99_9_us` × 3 latency arms), then `bench/check.py --update-baseline` from current HEAD to absorb the environmental drift in one honest cut. No code change to `check.py` / `run.sh` / `latency_harness.py` — the harness already iterates only entries present in `baseline.json`, so the schema mechanics work as-is. Alternatives considered and rejected: - Adding a `gated: bool` field per metric to keep `max_us` / `p99_9_us` measured-but-not-gated. Pure speculative infrastructure: today's pathology is drift, not jitter, and the diagnostic value of `max_us` in the `check.py` report is hypothetical (it's already in `run.sh` output upstream). User push-back ("was soll 1?") was correct. - Tolerance widening on the drifted metrics. Hides the drift under a wider band; a real +10% codegen regress would slip through. - `-n` raise for tail metrics. Doesn't address the structural problem (max-of-distribution is OS-jitter-dominated regardless of N); blows up bench wall time ~10x. - cpuset / `isolcpus=` pinning. Sysadmin-layer fix that doesn't survive CI move or a new dev machine. - Histogram-based latency methodology rework (Gitea #19). The proper long-term fix, but a separate brainstorm; this milestone is the stop-gap that buys back the noise floor in the meantime. Grounding-check (Step 7.5): PASS, trivial-spec path — no Rust compiler / checker / codegen / schema / runtime claims (all load-bearing claims are about the out-of-tree Python harness, and are covered by the spec's own replay-pass + synthetic-injection acceptance checks).