One-artefact iter on `bench/baseline.json`. Closes Gitea #15 + #16
as documented in the spec commit `a97aaeb`.
Changes:
- `note` field rewritten forward-looking: drops the retired-JOURNAL
workflow reference (Journal system deleted in 54f0ced) and the
now-moot `*.max_us` tolerance convention (the metric being
described is being removed in the same edit). Points readers at
`docs/specs/2026-05-20-bench-harness-recalibration.md` + closed
issues #15 / #16 for rationale.
- 6 latency entries removed (`max_us` + `p99_9_us` × 3 arms:
`implicit_at_gc`, `explicit_at_rc`, `implicit_at_rc`). After the
edit, `bench/check.py` gates only on `median_us` / `p99_us` /
`p99_over_median` across the three latency arms.
- All remaining baselines (48 throughput + 9 latency = 57 metrics)
regenerated by `bench/check.py --update-baseline` on HEAD
`ddb50c3` from a fresh `bench/run.sh -n 5`. `captured` field
bumped 2026-05-09 → 2026-05-20. Per-metric `tolerance_pct` and
`version` preserved verbatim by `write_new_baseline`
(`bench/check.py:249-284`).
Verification:
- Acceptance §1 (same-HEAD replay): `bench/check.py` exit 0, summary
`57 metrics; 0 regressed, 0 improved, 57 stable` on first try
(no single-run noise mitigation triggered).
- Acceptance §2 (synthetic injection): halving
`throughput.bench_list_sum.bump_s.baseline` 0.053 → 0.026 fires
REGRESSION row `+99.57% > 10.0%` with exit 1; restoration via
`jq` returns the value to 0.053 and the next `bench/check.py`
exits 0 (with 2 `improvement` rows on
`latency.explicit_at_rc.p99_us` / `p99_over_median` at ≈-33% —
improvements never gate; this is exactly the tail-distribution
jitter that motivated dropping `max_us` / `p99_9_us`).
Diff shape:
- `bench/baseline.json` shows +230 / -67 lines. Most of the line-
count growth is the `json.dumps(indent=2)` pretty-print emitted
by `--update-baseline` (`bench/check.py:283`) replacing the
previous hand-formatted compact entries (each metric was one
line; now each is six). No behavioural delta — the file is
semantically identical to a hand-edited version of the same
changes. Future `--update-baseline` runs will keep this expanded
format.
- `bench/orchestrator-stats/2026-05-20-iter-bench-harness-recalibration.1.json`
is the iter-stats artefact: outcome DONE, 0 re-loops across all
4 tasks, the four bench-result fields documenting the acceptance
outcomes. Stats files are committed per `skills/implement/SKILL.md`
Step 3.
No code change to `bench/check.py` / `bench/run.sh` /
`bench/latency_harness.py`. No Rust crate touched. `cargo test`
not exercised by this iter — the harness IS the test surface and
its replay+injection ARE the acceptance checks.
Follow-up noted (not addressed here):
- `skills/implement/agents/ailang-implement-orchestrator.md`
references a Phase-4 per-iter journal write; the
`docs/journals/` directory was retired in 54f0ced. The
orchestrator handled the mismatch by skipping the phase; the
agent prompt itself is stale and should be edited in a follow-up
tidy iter.
closes#15closes#16
Brummel
21cfccf846