One-iteration infra milestone closing Gitea #15 + #16. Two issues
framed distinct symptoms; reproduction on 2026-05-20 HEAD via two
back-to-back `bench/check.py -n 5` runs collapses them onto a
single picture:
- #15 ("*.bump_s stale") is wider than the issue framed. Drift
reproduces deterministically on `bench_list_sum.bump_s`
(+13.91% / +15.56%) AND on `bench_hof_pipeline.gc_s` (+10.60% /
+10.94%) — not just the bump_s family. Drift correlates with
memory pressure: `bench_tree_walk` and `bench_compute_collatz`
are clean across both runs.
- #16 ("*.max_us structural false-positive") has matured. The
"vanishes on rerun" pattern the issue documented from
2026-05-16 / 2026-05-18 audits is gone: today
`implicit_at_rc.max_us` reproduces at +47.20% / +49.42%, and
`implicit_at_rc.p99_9_us` at +27.79% / +41.92%. Same arm
only (`implicit @ rc`); the other two latency arms are clean.
The metrics are now drift-elevated AND structurally jitter-
prone — both reasons to retire them from the gate.
Decision: one cohesive change, all in `bench/baseline.json`. Drop
the six unreliable entries (`max_us` + `p99_9_us` × 3 latency
arms), then `bench/check.py --update-baseline` from current HEAD
to absorb the environmental drift in one honest cut. No code
change to `check.py` / `run.sh` / `latency_harness.py` — the
harness already iterates only entries present in `baseline.json`,
so the schema mechanics work as-is.
Alternatives considered and rejected:
- Adding a `gated: bool` field per metric to keep `max_us` /
`p99_9_us` measured-but-not-gated. Pure speculative
infrastructure: today's pathology is drift, not jitter, and
the diagnostic value of `max_us` in the `check.py` report is
hypothetical (it's already in `run.sh` output upstream).
User push-back ("was soll 1?") was correct.
- Tolerance widening on the drifted metrics. Hides the drift
under a wider band; a real +10% codegen regress would
slip through.
- `-n` raise for tail metrics. Doesn't address the structural
problem (max-of-distribution is OS-jitter-dominated regardless
of N); blows up bench wall time ~10x.
- cpuset / `isolcpus=` pinning. Sysadmin-layer fix that
doesn't survive CI move or a new dev machine.
- Histogram-based latency methodology rework (Gitea #19). The
proper long-term fix, but a separate brainstorm; this
milestone is the stop-gap that buys back the noise floor in
the meantime.
Grounding-check (Step 7.5): PASS, trivial-spec path — no Rust
compiler / checker / codegen / schema / runtime claims (all
load-bearing claims are about the out-of-tree Python harness, and
are covered by the spec's own replay-pass + synthetic-injection
acceptance checks).
Brummel
a97aaebd45