audit docs-honesty-lint: milestone close — CLEAN, carry-on, no ratify
Architect clean (one [medium] advisory Sweep-5 self-match wart filed as P3, not a fix iteration). Bench causally exonerated: HEAD vs 5bb7211/de66eb7 — all 5 firing bench binaries cmp-byte-identical, ail sha256 identical at all 3 commits; the check.py firings are the two tracked-P2 environmental classes (*.bump_s anchor staleness; *.max_us -n5 tail-jitter, 4th on a zero-runtime-change milestone), NOT this milestone — NO baseline ratify (Iron Law forbids ratifying noise). compile_check 0/24, cross_lang 0/25 EXIT 0. No fieldtest (zero authoring-surface change). docs-honesty-lint CLOSED: roadmap [~]->[x], WhatsNew appended, INDEX line added.
This commit is contained in:
@@ -0,0 +1,133 @@
|
||||
# audit docs-honesty-lint — milestone close (CLEAN, carry-on)
|
||||
|
||||
**Date:** 2026-05-18
|
||||
**Scope:** `928e1c0..e50b400` (spec `aff25cd`, plan `de66eb7`, iter
|
||||
`7580d43`, INDEX `e50b400`)
|
||||
**Status:** CLEAN — carry-on, no fix iteration, NO baseline ratify
|
||||
**Milestone:** docs-honesty-lint (single iteration) — **CLOSED**
|
||||
|
||||
## Outcome
|
||||
|
||||
| Gate | Result |
|
||||
|------|--------|
|
||||
| Architect drift | `clean` (one `[medium]` standing advisory wart, see below) |
|
||||
| `bench/check.py` | EXIT 1 — **causally exonerated**, do NOT ratify |
|
||||
| `bench/compile_check.py` | EXIT 0 (24/24 stable) |
|
||||
| `bench/cross_lang.py` | EXIT 0 (25/25 stable) |
|
||||
| Recommendation | **carry-on** |
|
||||
|
||||
## Step 1 — Architect drift review
|
||||
|
||||
Status `clean`. Verified:
|
||||
|
||||
- **Scope claim exact.** `git diff 928e1c0..e50b400 --stat --
|
||||
'crates/**/src/**' 'runtime/'` empty; sole `crates/` change is the
|
||||
additive `crates/ailang-core/tests/docs_honesty_pin.rs` (RED-first,
|
||||
now GREEN 4/4). The two lockstep-invariant pairs (`Pattern::Lit`/
|
||||
`pre_desugar_validation`, `lower_app`/`is_static_callee`) untouched.
|
||||
- **DESIGN.md edits are themselves honest.** All 14 corrections
|
||||
genuinely present-tense; no Wunschdenken/post-mortem reintroduced.
|
||||
All four protected-exception KEEPs survive un-stripped: `Diverge`
|
||||
reserved (L2730), `Regions … considered and rejected` at the
|
||||
Decision site (L1530), `a tiebreaker, not a rationale` (L1240),
|
||||
`No deriving` (L1850). The `### What this document is — and the
|
||||
honesty rule it holds itself to` meta-subsection is itself
|
||||
present-tense (no "we will / planned").
|
||||
- **Agent-def lockstep complete.** `skills/audit/agents/ailang-architect.md`
|
||||
"four"→"five sweeps" purge verified (`grep "four sweeps…"` empty);
|
||||
new "DESIGN.md honesty drift" bullet present and citing the
|
||||
meta-subsection.
|
||||
- **Sweep residue is exactly the 3 documented deliberate KEEPs.**
|
||||
`bash bench/architect_sweeps.sh` EXIT 1 with: Sweep-1 L50
|
||||
(`docs/journal-archive.md … pre-2026-05-11 history` — present-tense
|
||||
doc-roles pointer, incidental date) + L2060 (the true
|
||||
`(iter str-concat, 2026-05-13)` provenance tag — asserts nothing
|
||||
false, consistent with the sibling `(iter 24.1)` tags); Sweep-5 L62
|
||||
(the discriminator subsection's OWN forbidden-phrasing catalogue —
|
||||
a self-referential false-positive, it IS the rule definition). Each
|
||||
correctly adjudicated KEEP per the new bullet's discriminator.
|
||||
|
||||
**`[medium]` standing advisory wart (architect-flagged, orchestrator-
|
||||
adjudicated → filed, not fixed now).** Sweep 5's regex matches the
|
||||
discriminator subsection's own example catalogue (DESIGN.md L62)
|
||||
permanently: every future `architect_sweeps.sh` run inherits a
|
||||
guaranteed self-referential EXIT 1 that must be re-adjudicated by
|
||||
hand. This does not introduce a *new* failure mode — the script has
|
||||
never been EXIT 0 in practice (Sweeps 1-4 already retain legitimate
|
||||
date-anchors; the script header states non-zero is advisory,
|
||||
architect-adjudicated by design). It is a minor signal-erosion, not
|
||||
current drift. Filed as a P3 roadmap todo (optional Sweep-5
|
||||
self-anchor exclusion); explicitly NOT a fix iteration — consistent
|
||||
with "trivial drift recorded, not ignored, but priced honestly at
|
||||
P3".
|
||||
|
||||
## Step 2 — Bench-regression check
|
||||
|
||||
`check.py` EXIT 1, three firings: `throughput.bench_list_sum.bump_s`
|
||||
+13.42% (tol 10%); `latency.implicit_at_rc.p99_9_us` +37.65% (tol
|
||||
25%); `latency.implicit_at_rc.max_us` +134.70% (tol 30%). 58 stable,
|
||||
2 improvements. `compile_check.py` EXIT 0; `cross_lang.py` EXIT 0 —
|
||||
and `cross_lang.py` independently measures the *same* `bench_list_sum`
|
||||
workload at `ail_bump_s +5.95% ok`, `ail_rc_s +5.08% ok` (the fixture
|
||||
`check.py` calls a +13.42% REGRESSION reads within tolerance under a
|
||||
fresher anchor / wider band).
|
||||
|
||||
## Step 3 — Bencher causal exoneration (decisive)
|
||||
|
||||
Hypothesis: the firings were *caused by this milestone's diff*.
|
||||
Refuting test: HEAD bench binaries byte-identical to a pre-milestone
|
||||
commit ⇒ no causal mechanism. Result (verbatim):
|
||||
|
||||
- `cmp` of `bench_list_sum_{bump,gc,rc}` + `bench_latency_implicit_{rc,gc}`
|
||||
at HEAD `e50b400` vs `5bb7211` (last milestone close) vs `de66eb7`
|
||||
(immediate pre-iter): **0 differing bytes, all 5 binaries identical
|
||||
across both baselines.** `ail` release binary sha256 identical at
|
||||
all three commits (zero compiled-source delta — the sole `crates/`
|
||||
change is the non-bench test file).
|
||||
- Latency re-measure on the byte-identical binary (5×`--runs 5`):
|
||||
median/p99 rock-steady (≤0.6% / 1.1% spread); p99.9 swings
|
||||
503.8→651.4, max 565.6→692.1 — classic stable-center / volatile-tail
|
||||
on identical machine code. = the tracked-P2 `-n 5` tail-jitter
|
||||
false-positive, **4th occurrence on a zero-runtime-change
|
||||
milestone**.
|
||||
- `bench_list_sum_bump` (byte-identical) 20-run median 0.0531s,
|
||||
persistent ~+15% (not transient) = the tracked-P2 `*.bump_s`
|
||||
environmental-baseline-staleness item; the 0.046s anchor is stale
|
||||
vs this machine's current steady state.
|
||||
|
||||
**Verdict: EXONERATED (environmental, do NOT ratify).** Per the audit
|
||||
Iron Law ("NO BASELINE UPDATE WITHOUT A PAIRED JOURNAL RATIFY ENTRY";
|
||||
ratifying noise is forbidden) and the spec's zero-codegen invariant, a
|
||||
`--update-baseline` is foreclosed. Neither firing family is
|
||||
attributable to docs-honesty-lint.
|
||||
|
||||
## Step 4 — Resolution: carry-on (clean)
|
||||
|
||||
No fix iteration. No ratify. Milestone **CLOSED**. No fieldtest: this
|
||||
milestone changed zero authoring surface (documentation prose + one
|
||||
non-bench test file + an agent-def + a bench shell sweep) — the
|
||||
fieldtest trigger (surface-touching milestone) does not apply; the
|
||||
wrap-robust `docs_honesty_pin.rs` (4 tests) + Sweep 5 ARE the
|
||||
milestone's regression coverage.
|
||||
|
||||
Two pre-existing standing P2 items remain orchestrator-owned and
|
||||
were explicitly NOT bundled into this close (the bencher's
|
||||
recommendation, honoured): (a) the stale `*.bump_s` anchor — a
|
||||
deliberate quiescent-machine baseline refresh, separately journaled
|
||||
when pursued, never a ratification rider on an unrelated close; (b)
|
||||
the recurring `-n 5` tail false-positive — the durable fix is a
|
||||
structural harness change (higher default N or a tail-metric gating
|
||||
policy), already a tracked roadmap todo, not a bench-output artefact.
|
||||
|
||||
## Milestone close summary
|
||||
|
||||
docs-honesty-lint is fully ratified and CLOSED: spec (grounding-check
|
||||
PASS 10/10) + plan + single iter + this audit. The canonical docs
|
||||
(`docs/DESIGN.md`, `docs/PROSE_ROUNDTRIP.md`) now mirror only the
|
||||
current state, present-tense; the tense+modality discriminator is
|
||||
codified in DESIGN.md and the architect drift-checks against it via
|
||||
Sweep 5 + the new bullet; the wrap-robust enumerated pin is the hard
|
||||
forward gate. The procedural-enumeration design proved itself: the
|
||||
Task-4 Step-14 catch-all surfaced one genuine extra FIX site (the
|
||||
Form-B prose-projection bullet) that the spec's illustrative regex
|
||||
and a frozen orchestrator list would both have missed.
|
||||
@@ -92,3 +92,4 @@
|
||||
- 2026-05-18 — audit remove-mut-var-assign (milestone close, CLEAN after one inline `.tidy`): scope `48e7774..07f0802`. Architect drift: one `[high]` — `crates/ailang-core/specs/form_a.md` (live in-tree Form-A grammar still documented the full `mut`/`var`/`assign` construct: EBNF 286-288, prose 308-328, dangling `like mut` at 332); the spec named it for deletion, the iter missed it. Root cause = recon-undercount class **4th recurrence + a Boss error**: a plan-time grep checked only `docs/form_a.md` (dir-scoped), got "No such file", and the Boss propagated that unverified non-existence claim into the recon brief — the real file is `crates/ailang-core/specs/form_a.md`, actively maintained, and a stale doc never produces a compile error so the implement compile-sweep could not catch it. Fixed inline as a Boss `.tidy` (CLAUDE.md context-already-loaded carve-out; full review discipline kept — tree-wide verified zero construct residue except the roadmap entry that names the milestone itself; grammar reads coherently reuse-as→loop→recur). One `[medium]` process-drift: escalated the existing loop-recur.tidy `ailang-plan-recon` P2 in place — broadened from cross-crate-caller to a structural recon-contract defect with two named gaps (non-compile-checked spec-named sites; unverified "X does not exist" claims inheriting into a recon brief) + a concrete fix (compile-driven enumeration as authoritative code-site set PLUS an `ls`-verified spec-named-path existence table). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 exit 0; `check.py` exit 1 (3 firings: `bench_list_sum.bump_s` +12.51%, `latency.{explicit,implicit}_at_rc.max_us` +108%/+66%). Bencher causal exoneration **decisive**: HEAD vs `48e7774` `list_sum_bump`/`lat_expl_rc`/`lat_impl_rc` binaries `cmp`-byte-identical — the −2520-line removal alters zero bytes of these fixtures' machine code, no causal mechanism; `bump_s` = the tracked P2 environmental staleness, both `*.max_us` = single-sample `-n 5` jitter (`explicit_at_rc.max_us` +108%→**-2.30% ok** on identical-code rerun, medians held). **PRISTINE met on the merits; NO baseline ratify** (Iron Law forbids ratifying noise, spec forecloses it). Filed a NEW distinct P2 (separate from `*.bump_s` staleness): `check.py` RC-latency `*.max_us` at `-n 5` is a 3-for-3 structural false-positive on no-runtime-change milestones → drop from gating / raise `-n` / cpuset-pin. Milestone audit **CLEAN**; only `fieldtest` (surface-touching) remains before close → 2026-05-18-audit-remove-mut-var-assign.md
|
||||
- 2026-05-18 — fieldtest remove-mut-var-assign (milestone CLOSE, clean — removal thesis empirically confirmed): post-audit downstream-LLM-author field test of the post-removal Form-A surface. The fieldtester (DESIGN.md + form_a.md + public examples only, never compiler source) wrote 4 FRESH real-world programs an imperative-trained author would instinctively reach for a mutable local for — running sum-of-squares accumulator (`loop`/`recur`, 2 binders → `385`), grade-classification cascade (let-threaded flag through 4 `if` → `4`/`2`/`0`), Horner polynomial straight-line build-up (5 `let acc` shadows → `73`), multi-state bracket-balance scanner threading `(rest,depth,ok)` by tail recursion → `true`/`false`) plus an out-of-tree `mut`-keyword rejection probe. **0 bugs, 0 friction, 0 spec_gap, 4 working**: every task expressed cleanly and correctly on the FIRST try with only `let`/`if`/`loop`/`recur`; all 4 `ail parse|render|parse` byte-identical. The closest-to-friction case (the multi-state machine must restate the full state tuple at every tail-call even when a branch changes one component) was carefully classified `working` not `spec_gap` — that explicit-dataflow cost IS the local-reasoning pillar working as designed, and `mut`'s implicit cross-iteration persistence of un-restated state is precisely the clause-3 failure the removal eliminates; the fieldtester explicitly noted the only conceivable sugar (a record to bundle state) is an orthogonal additive future question, NOT evidence against the removal (refused the scope-creep). The `mut` rejection is fail-closed with an actively-guiding diagnostic that enumerates the surviving term heads incl. `loop`/`recur`/`let` (no tombstone, no-nostalgia, as spec'd). Independent Boss verification: all 4 fixtures re-run → `385`/`4 2 0`/`73`/`true false`, zero `mut`/`var`/`assign` construct tokens in any fixture (a doc-faithful author did not produce the removed construct — the affirmative clause-1 evidence). **Verdict: the removal thesis is confirmed** — `mut` was redundant with `let`/`if`/`loop`/`recur` in 100% of the fresh real tasks; no expressivity lost. **The remove-mut-var-assign milestone is fully ratified and CLOSED**: spec+plan+iter, audit clean (architect `[high]` form_a.md fixed in `.tidy`, bench causally exonerated), fieldtest clean on the removal axis. Fixtures `examples/fieldtest/remove-mut_*.ail` + spec `docs/specs/2026-05-18-fieldtest-remove-mut-var-assign.md`; roadmap P0 flipped `[~]`→`[x]`, WhatsNew.md user-facing entry appended; next queue item (DESIGN.md/docs honesty-lint) is a NEW milestone = a /boss new-milestone bounce-back (not auto-started) → 2026-05-18-fieldtest-remove-mut-var-assign.md
|
||||
- 2026-05-18 — iter docs-honesty-lint.1 (single-iteration milestone, DONE): canonical docs made present-tense-honest. Stripped Wunschdenken (forward intent stated as fact) + non-citation post-mortem/doc-archaeology from `docs/DESIGN.md` (14 corrections: L484 iter-anchor, L503/L1318/L1504/L2027/L2065/L2135 post-mortem, L650 malformed-Wunschdenken, L655 dated-heading, L1513/L1837/L2046/L2519 Wunschdenken, + the Task-4-Step-14 procedural-catch-all FIX at the Form-B prose-projection bullet "was deferred from milestone 22 entirely … A future iter ships" — a soft-wrapped site the spec's illustrative regex would have missed, the procedural enumeration working exactly as designed) + `docs/PROSE_ROUNDTRIP.md` (tool-use/MCP paragraph); genuine forward intent (LLM tool-use/MCP/LSP) relocated to roadmap P3. Codified the two-pronged tense+modality discriminator as a present-tense `### What this document is — and the honesty rule it holds itself to` meta-subsection in DESIGN.md §"Project ecosystem". Anti-regrowth two ways: wrap-robust enumerated absent/present pin `crates/ailang-core/tests/docs_honesty_pin.rs` (a `norm()` whitespace-collapse helper structurally discharges the recurring grep/contains line-wrap failure family — strictly better than the effect-doc-honesty "keep-on-one-line" precedent; resolves spec Testing-item 0 + planner self-review item 6 at once) + wrap-robust advisory Sweep 5 in `bench/architect_sweeps.sh` (contiguous-fragment regex, since DESIGN.md is hard-wrapped ~70col) with full sweep-count lockstep (header + "All four"→"All five" tail + `ailang-architect.md` "four"→"five sweeps") + a new `ailang-architect.md` "DESIGN.md honesty drift" bullet citing the meta-subsection. Protected-exception KEEPs preserved + pinned-present (Diverge-reserved, regions-considered-and-rejected, the self-labelled "tiebreaker, not a rationale"); `form_a.md` correctly left untouched/unpinned. One plan-internal DONE_WITH_CONCERNS (Task-1 pin string omitted a `**…**` bold the Task-3 body added; resolved by dropping the non-load-bearing emphasis, pin is the contract) — planner self-review-item-3 calibration note. Independent Boss verification: pin 4/4 GREEN, full `cargo test --workspace` 0 failures (delta = +4 pin tests only), sentinel trio (`design_schema_drift`/`spec_drift`/`schema_coverage`) green, `cargo build --workspace` clean, scope = ONLY THE PIN in `crates/` (zero language/checker/codegen/runtime change, `ail check`/`run`/`build` byte-unchanged by construction). Post-iter sweep residue = only the 3 documented deliberate KEEPs (Sweep-1 L50 doc-roles-pointer date + L2060 true `(iter str-concat)` provenance; Sweep-5 L62 self-referential discriminator-catalogue), EXIT=1 architect-adjudicated-expected. Milestone-close `audit` is the next pipeline step (no fieldtest — zero authoring-surface change). spec `docs/specs/2026-05-18-docs-honesty-lint.md` (grounding-check PASS 10/10), plan `docs/plans/docs-honesty-lint.1.md` → 2026-05-18-iter-docs-honesty-lint.1.md
|
||||
- 2026-05-18 — audit docs-honesty-lint (milestone close, CLEAN — carry-on, NO ratify): scope `928e1c0..e50b400`. Architect `clean`: scope claim exact (zero `crates/**/src/**` + `runtime/`; sole crates/ change the additive RED-first-now-GREEN `docs_honesty_pin.rs`; lockstep pairs untouched), all 14 DESIGN.md corrections genuinely present-tense with the 4 protected KEEPs (`Diverge`-reserved/`Regions`-rejected/`tiebreaker`/`No deriving`) un-stripped, agent-def "four→five" lockstep complete, sweep residue exactly the 3 documented deliberate KEEPs. One `[medium]` standing advisory wart — Sweep 5's regex self-matches the discriminator subsection's own forbidden-phrasing catalogue (DESIGN.md L62) forever (architect-flagged; orchestrator-adjudicated → filed as a P3 todo, NOT a fix iteration — not a new failure mode since the script is advisory-with-residue by design). Bench: `compile_check.py` 0/24 + `cross_lang.py` 0/25 EXIT 0 (cross_lang independently reads the same `bench_list_sum` bump path +5.95% within tol); `check.py` EXIT 1, 3 firings (`bench_list_sum.bump_s` +13.42%, `latency.implicit_at_rc.{p99_9,max}_us` +37.65%/+134.70%). Bencher causal exoneration **decisive**: HEAD vs `5bb7211` vs `de66eb7` — all 5 firing bench binaries `cmp`-byte-identical, `ail` sha256 identical at all three commits ⇒ zero causal mechanism; `*.max_us` = tracked-P2 `-n 5` tail-jitter (4th occurrence on a zero-runtime-change milestone, median/p99 rock-steady), `*.bump_s` = tracked-P2 environmental-anchor-staleness (persistent ~+15%, stale 0.046s anchor). **EXONERATED, NO baseline ratify** (Iron Law forbids ratifying noise; spec forecloses a zero-codegen baseline bump). Two pre-existing standing P2 items kept orchestrator-owned, explicitly NOT bundled into this close (bencher recommendation honoured). Milestone audit **CLEAN**; no fieldtest (zero authoring-surface change — the wrap-robust pin + Sweep 5 ARE the regression coverage). **docs-honesty-lint fully ratified and CLOSED**: spec (grounding-check PASS 10/10) + plan + single iter + audit clean; roadmap P0 flipped `[~]`→`[x]`, WhatsNew.md user-facing entry appended → 2026-05-18-audit-docs-honesty-lint.md
|
||||
|
||||
Reference in New Issue
Block a user