diff --git a/docs/WhatsNew.md b/docs/WhatsNew.md index 10952f9..5cbeadc 100644 --- a/docs/WhatsNew.md +++ b/docs/WhatsNew.md @@ -238,3 +238,7 @@ This closes the multi-step arc that began with making a single numeric function ## 2026-05-19 — Design spec restructured into a typed ledger The 3000-line single design document is gone. The design spec is now a "design/" directory: a typed INDEX as the single entry point, separate contract files (the binding, test-linked rules), and model write-ups; the historical "why we chose/rejected X" prose moved into the journals. A self-checking test now keeps history and rationale from leaking back into the contract files, so the part of the spec read every cycle stays small and addressable. Everything that pointed at the old file — build tooling, the review reading-lists, and the compiler hints that tell the author which spec section to read — now points at the new layout. No language or behaviour change; documentation structure only. The build was reviewed clean and the one drift found (migrated history prose sitting in a few contract files) was fixed in the same pass; benchmarks were proven byte-for-byte unaffected. + +## 2026-05-19 — Design ledger is now a browsable wiki + +Cross-references between design documents are now formal, machine-checked links. Clicking them navigates between contracts and into source files in any reasonable Markdown viewer, and a broken or out-of-tier cross-reference is now a build failure instead of silent rot. A separate cleanup: a handful of pre-existing references that pointed into the project's transient tracking notes have been removed; the durable design area now describes the behaviour without pointing outside itself. No language or behaviour change; documentation structure and a new structural test only. The build was reviewed clean — zero drift, benchmarks unchanged across the three regression scripts. diff --git a/docs/journals/2026-05-19-audit-design-ledger-formal-links.md b/docs/journals/2026-05-19-audit-design-ledger-formal-links.md new file mode 100644 index 0000000..136d609 --- /dev/null +++ b/docs/journals/2026-05-19-audit-design-ledger-formal-links.md @@ -0,0 +1,117 @@ +# audit — milestone close: formal cross-links in the design/ ledger + +**Date:** 2026-05-19 +**Scope:** `2ee4087..8ad91e7` (5 commits — 3 spec commits incl. 2 corpus-grounded amendments, 1 plan, 1 iter) +**Status:** CLEAN (no drift; bench pristine; no tidy) + +## Architect (clean — drift_items: none) + +Every claimed commitment and invariant verified positively: + +- **clause-5** (`design_body_links_are_durable_and_resolve`) implements + all four predicates (resolves-from-containing-file; durable-tier; + no-fragment; fence-skip via toggle-`strip_fences`); RED-first + synthetic-vector mechanism honest; scope is `design/contracts/` + + `design/models/`, excludes `design/INDEX.md` per commitment 4. +- The 7 prose conversions (8 link tokens) match the spec's + Acceptance-3 closed enumeration verbatim; file-relative arithmetic + correct (siblings → bare filename; `crates/` reach via `../../`; + cross-`design/`-subtree via `../models/`). Zero `docs/` link + targets; zero `#fragment`. +- Disposition (b) on `pipeline.md:61` + `authoring-surface.md:180` + correctly applied: cross-tier pointer removed, surrounding + behavioural `ail merge-prose` prose preserved (the + authoring-surface case is slightly rephrased to keep the meaning + without the pointer). +- `honesty-rule.md` positive-half paragraph pin-safe: both + `docs_honesty_pin.rs`-pinned phrases byte-identical (L14 and now + L25, shifted from L19 by the +6 lines); the + `design_index_pin.rs clause-5` mention is a nominal identifier in + the existing "Ratified by:" footer style (no `[](...)` form, no + navigational pointer) — consistent with the spec's §Scope + navigational-vs-nominal definition. No clause-3 PHRASE tripwire, + no Sweep-1 anchor. +- **Commitment 4** honored: `design/INDEX.md` byte-unchanged + (zero-line diff vs HEAD); `design_index_pin.rs` clauses 1–4 source + byte-unchanged (the only `-` lines are the two-line `//!` header + rewrite delivered by Task 1 Step 5 itself). +- **Commitment 7** honored: + `docs/journals/2026-05-19-design-decision-records.md` + byte-unchanged. +- **Composition invariant** holds on the post-milestone tree: + clause-3 GREEN ∧ clause-5 GREEN simultaneously for every + `design/contracts/*.md` (independently verified by running both + in the same suite). +- **Out-of-scope set untouched:** `data-model.md` 38/66/79/206/226 + verified inside ```jsonc fences (30–87, 203–228); + `embedding-abi.md:51` "frozen value layout below specifies" + untouched; intra-file `above/below` directional prose untouched + (sample-checked `float-semantics.md:70` "The NaN-spelling caveat + above" still intra-file). +- **No commitment-2 violation:** no agent/SKILL reading list adds a + `docs/PROSE_ROUNDTRIP.md` design/-link target. The sole remaining + mention is `design/INDEX.md:43` — prose mention only (no `[](...)` + form), outside clause-5's scope and not a navigational link. +- `bench/architect_sweeps.sh` exit 0 ("All five sweeps clean"); + Sweep-1 shares no scope or regex with clause-5 (orthogonality + asserted in spec confirmed empirically). + +Two minor non-drift observations recorded without raising drift: + +- `design/INDEX.md:43`'s prose mention of `docs/PROSE_ROUNDTRIP.md` + is now the sole remaining `docs/` reference in `design/`. The + spec consciously scopes clause-5 to body files only; this is a + nominal mention in the spine, not a formal link — commitment-2 + governs *formal links*. Recorded for future-tightening awareness, + not as drift. +- The roadmap entry stands `[~]` IN FLIGHT; flipping to `[x] + CLOSED` is the standard post-audit ratification step, executed + in this same done-state commit. Not drift. + +## Bench (check.py exit 0; compile_check exit 0; cross_lang exit 0) + +``` +check.py 63 metrics: 0 regressed (within tolerance), 0 improved beyond tolerance, 63 stable; exit 0 +compile_check 24 metrics: 0 regressed, 0 improved beyond tolerance, 24 stable; exit 0 +cross_lang 25 metrics: 0 regressed, 0 improved beyond tolerance, 25 stable; exit 0 +aggregate 0 / 0 / 0 +``` + +Cleanest possible bench outcome. Two metric movements were flagged +in the per-metric output (1 `regressed` and 1 `improved` in the +check.py summary line) but both are within their tolerance bands — +the `ok` status the script prints means the threshold check passed. +No metric exceeds tolerance; baseline pristine; no +`--update-baseline` and no paired JOURNAL ratify entry warranted. + +This matches the spec's expectation precisely: the milestone makes +**zero codegen / runtime / IR-emitting changes** (every code edit is +either a new test, a `//!` doc comment, or design/ prose). The +firings inside the bands are the standing P2 environmental jitter +already familiar from the M2/M3/M5/rolesplit closes. + +## Resolution (orchestrator) + +- **Architect:** carry-on. Zero drift items. +- **Bench:** carry-on. All three scripts exit 0; baseline pristine; + no ratify warranted. +- **No tidy iteration.** No fix path. +- **No fieldtest.** The milestone touches no authoring surface (no + `.ail` program, no language construct, no compiler/checker/codegen + path) — spec's reasoned exclusion stands. +- **Milestone closes immediately.** Done-state procedure (roadmap + `[~]` → `[x]`, WhatsNew append, notify) runs in this same + cohesive commit. + +This is the cleanest milestone close pattern in the project's +recent history: zero spec defects surviving to audit (the two were +caught and amended at planner-recon and plan-corpus-fetch time, +before any byte moved), zero drift, zero bench movement, zero +re-loop. The two amendment cycles (clause-6 + cross-ref definition; +clause-5 fence-skip + closed convert-set enumeration) were the +discipline working — recon and corpus-fetching surfaced gaps the +brainstorm-sample missed, the spec was grounded properly *before* +implementation, and the implementation then ran clean 5/5 with one +non-blocking concern (a planner self-review-item-8 arithmetic miss +on the `//!` header rewrite line count — substantive assertion +unaffected, recorded as lesson). diff --git a/docs/journals/INDEX.md b/docs/journals/INDEX.md index b0b99d7..63ddfd2 100644 --- a/docs/journals/INDEX.md +++ b/docs/journals/INDEX.md @@ -119,3 +119,4 @@ - 2026-05-19 — audit design-md-rolesplit (milestone close): DRIFT (one tidy iteration) + bench causally-exonerated (baseline pristine). Architect drift_found, relocation byte-faithful (MIXED Decisions / dual-link / source-link / rewritten honesty-rule all match the spec Appendix vs deeffb1; build-atomicity holds; design_index_pin 4/4; honesty-rule.md correct + pins retargeted; agent contracts coherent) — one [medium] spirit-finding: faithfully-migrated decision-record/history prose in design/contracts/{typeclasses,str-abi,scope-boundaries}.md DODGES the 6 literal clause-3 markers (case+wording variance) but IS the relitigation content the split's spirit sends to journals; plus [low] float-semantics.md stale-direction "see Str ABI below". Both routed items adjudicated strippable-doc-archaeology-to-TIDY-not-ratify (the str-abi.md:23 advisory Sweep-1 exit-1 correctly diagnoses real pre-existing history residue, byte-identical DESIGN.md@deeffb1:2062-2065, faithfully migrated — not split-introduced). Bencher: NO-ratify, causally-exonerated DECISIVE on byte-evidence — emitted IR AND final -O2 binaries byte-identical 176821c vs pre-milestone dd5b183 for all 6 check.py firings (sha256+cmp), all ~11 changed code/runtime files audited comment/docstring/diagnostic-string-only, zero IR-path change; the firings are tracked-P2 (*.bump_s staleness, *_at_rc.max_us -n5 tail jitter, *.gc_s Boehm-scan variance — NOT the M5 gc_rss trio); compile_check 24/24 + cross_lang 25/25 pristine corroborate; baseline untouched (identical to M2/M3/M5). Resolution: one tidy iter design-md-rolesplit.tidy — (1) move the history prose to the decision-record journal, (2) fix the stale "below", (3) re-scope architect_sweeps honesty sweeps to design/contracts only (models/ is the explicitly-narrative tier — scanning it for history-anchors is a post-split category error), (4) widen design_index_pin.rs clause-3 to SUBSUME Sweep-1's history-anchor regex over contracts/ (invariant: clause-3 GREEN ⟹ Sweep-1 clean in contracts/ — the hard gate enforces the spirit, closing the literal-marker dodge permanently). No fieldtest (zero authoring-surface change). Milestone closes after the tidy lands Boss-verified. → 2026-05-19-audit-design-md-rolesplit.md - 2026-05-19 — iter design-md-rolesplit.tidy (DONE 7/7): resolves the milestone-close audit DRIFT (2ba5e16). Gate-first TDD: widened design_index_pin.rs clause-3 to a hand-rolled FAITHFUL Sweep-1 superset — case-sensitive digit-anchored Sweep-1 line anchors (confirmed ZERO across all contracts) + Sweep-1's `^[^/]*` path-EXCLUDED date_anchor (so docs/specs/2026-.. citations are not flagged) + the audit-named decision-record PHRASES (case-insensitive, closing the capital-variance dodge); NO regex dep, the blanket case-insensitive iter-detector REJECTED as unworkable (it conflated the memory-model rule-names "Iter A/B" + ordinary "pre-existing/pre-set/pre-Boehm" with provenance). Sentence-level strip of the faithfully-migrated history/decision-record prose out of 5 contract files (typeclasses/str-abi/scope-boundaries — the audit's spot-check — PLUS roundtrip-invariant.md:73 + data-model.md:149,161, the 2 the exhaustive plan-time scan found the spot-check missed) into docs/journals/2026-05-19-design-decision-records.md, each removed sentence replaced by its present-tense contract equivalent; float-semantics.md stale `(see "Str ABI" below)` → `(see design/contracts/str-abi.md)`; architect_sweeps.sh honesty sweeps re-scoped design/contracts+design/models → design/contracts only (models/ is the explicitly-narrative tier; scanning it for history-anchors is a post-split category error) + skills/audit/agents/ailang-architect.md lockstep. Invariant established: clause-3 GREEN ⟹ Sweep-1 finds nothing in contracts/. The prior dispatch correctly BLOCKED on a real plan defect (iso_date lacked Sweep-1's path-exclusion → over-fired on legit spec-path citations); per the "two+ defects in one iteration ⇒ fix the upstream artifact, not a third patch" discipline the audit Resolution mechanism+scope were corrected in lockstep (f2cdd67) before re-dispatch. Three docs_honesty_pin pinned-byte runs survived contiguous (str-abi.md:19, typeclasses.md:297, scope-boundaries.md:8 — shifted from :9 by the authorized :5-6 collapse, content byte-untouched). Boss-verified independently: cargo test --workspace 646/0, design_index_pin 4/4 (clause-3 RED→GREEN), architect_sweeps.sh exit 0 "All five sweeps clean" (acceptance criterion 9 met), acceptance grep CLEAN, 3 pins each exactly 1 contiguous match. Zero spec/quality re-loops. This is the FINAL design-md-rolesplit iteration — the milestone (DESIGN.md → design/ ledger role-split) is now functionally complete, audited, drift-resolved, and the in-code hard gate enforces the honesty spirit. → 2026-05-19-iter-design-md-rolesplit.tidy.md - 2026-05-19 — iter design-ledger-formal-links.1 (DONE 5/5, whole milestone): positive-half completion of the DESIGN.md → design/ split — design/ body cross-references are now formal, file-relative Markdown links `[label](path)` into the durable tier (design/ + crates/** + runtime/**), and a new in-tree hard gate `design_index_pin.rs` **clause-5** (`design_body_links_are_durable_and_resolve`) walks every design/contracts/*.md + design/models/*.md, strips fenced code (`strip_fences` toggles on ```/~~~), extracts every `](path)`, and asserts it resolves file-relative to a real durable file — never docs/, never an in-file #anchor. RED-first via identity-stubbed strip_fences (four embedded synthetic vectors — first FAILS) → real toggle-on-fence impl → GREEN. **clause-5 ∘ clause-3 = complete invariant** the milestone delivers: every contract cross-reference is either a resolving durable file-link OR clause-3-forbidden decision-record prose. Convert-set (recon-and-corpus-verified, closed): 7 prose refs / 8 link tokens — float-semantics:69/100, embedding-abi:45, memory-model:44/105 (the genuine cross-file stale-direction wart: "Method dispatch below" but the heading lives in typeclasses.md:227, not memory-model.md), scope-boundaries:48/88 (mixed-referent split: `ailang-core::desugar` → source link + Pipeline → ../models/pipeline.md). Disposition-(b) on the 2 PROSE_ROUNDTRIP homeless pointers (pipeline.md:61, authoring-surface.md:180) — pointer removed, ail merge-prose behavioural prose preserved (recon proved roundtrip-invariant.md does NOT carry the merge-prose-cycle content; the prior clause-6 sample's "(a) retarget" was thus wrong, corrected in spec amendment 1 to the three-disposition rule). honesty-rule.md positive-half paragraph inserted pin-safe between L14 and L16; both `docs_honesty_pin.rs` phrases byte-identical. Spec went through TWO corpus-grounded amendments before plan-write (grounding-check PASS ×3): (1) recon-driven — clause-6 incomplete + missing navigational/nominal cross-reference definition + OQ2/OQ3 byte-policies; (2) plan-time-corpus-driven — clause-5 fence-skip + the in-fence schema-annotation carve-out (data-model.md 38/66/79/206/226 are inside ```jsonc fences 30–87 + 203–228, where Markdown links are literal text on every renderer; the inline analog of the nominal-mention carve-out — schema-example documentation, not browsable cross-references). Per the "two+ surfaced ⇒ ground properly once, don't iter-patch" discipline, amendment 2 was the definitive corpus-grounded pass (every convert-set ref's prose-vs-fence status + exact bytes personally verified before amending), not a fourth patch returning later. INDEX-spine sub-fork resolved by user MC during brainstorm Step 2: spine stays repo-root-relative (registry tier is structurally invariant — move-fragility absent), clause-1 byte-unchanged; body uses file-relative (the converged-design path-form decision). Boss-verified independently: `cargo test --workspace` 647/0 (+1: clause-5), `design_index_pin` 5/5, `docs_honesty_pin` 5/5, design/INDEX.md + decision-records journal byte-unchanged, clauses 1–4 source byte-unchanged (the 2 `-` lines are the two-line //! header rewrite Task 1 Step 5 itself delivers — clauses are #[test] fns, untouched), 0 docs/ link targets, 0 #fragments, 8 `](` link tokens (= the closed convert-set), embedding-abi.md:48 docs_honesty_pin pinned phrase byte-identical. One Concerns item: Task-5 Step-7 plan-predicted `-`-line count was 1, actual 2 — planner self-review-item-8 arithmetic miss on the //! header rewrite (Steps 5 + 7 inconsistent within Task 1+5 about how many lines the header rewrite removes); harmless (substantive assertion *clauses 1–4 source unchanged* fully holds), recorded as lesson. Next: mandatory milestone-close audit; no fieldtest (zero authoring-surface change — reasoned exclusion). → 2026-05-19-iter-design-ledger-formal-links.1.md +- 2026-05-19 — audit design-ledger-formal-links (milestone close): **CLEAN** — no drift, no tidy iteration, bench trio 0/0/0 pristine, no ratify warranted, no fieldtest (zero authoring-surface change — reasoned exclusion stands). Architect: every claimed commitment + invariant verified positively — clause-5 implements all four predicates (resolves-from-containing-file / durable-tier / no-fragment / fence-skip via toggle-`strip_fences`); RED-first synthetic-vector mechanism honest; scope excludes design/INDEX.md per commitment 4. The 7 prose conversions (8 link tokens) match the spec's Acceptance-3 closed enumeration verbatim; file-relative arithmetic correct (siblings → bare filename; `crates/` via `../../`; cross-`design/`-subtree via `../models/`); zero docs/ link targets; zero #fragment. Disposition (b) on the 2 PROSE_ROUNDTRIP homeless pointers correctly applied: cross-tier pointer removed, behavioural prose preserved (authoring-surface case slightly rephrased to keep meaning without the pointer). honesty-rule.md positive-half paragraph pin-safe: both `docs_honesty_pin.rs` phrases byte-identical (L14 + L25, the latter shifted from L19 by the +6 lines); the `design_index_pin.rs clause-5` mention is a nominal identifier in the existing "Ratified by:" footer style (no `[](...)` form, no navigational pointer — consistent with §Scope navigational-vs-nominal definition); no clause-3 PHRASE tripwire, no Sweep-1 anchor. **Commitment 4 honored:** design/INDEX.md byte-unchanged (zero-line diff vs HEAD); clauses 1–4 of design_index_pin.rs source byte-unchanged (the only `-` lines are the two-line `//!` header rewrite the milestone explicitly delivers). **Commitment 7 honored:** decision-records journal byte-unchanged. **Composition invariant holds:** clause-3 GREEN ∧ clause-5 GREEN simultaneously for every design/contracts/*.md. **Out-of-scope set untouched:** data-model.md 38/66/79/206/226 inside ```jsonc fences (30–87, 203–228); embedding-abi.md:51; every intra-file `above/below` (sample-checked float-semantics.md:70 still intra-file). No commitment-2 violation: no agent/SKILL reading list adds docs/PROSE_ROUNDTRIP.md as a design/-link target; the sole remaining `docs/` mention is design/INDEX.md:43 — prose nominal mention only, no `[](...)` form, outside clause-5's INDEX-excluded scope, recorded as a future-tightening observation not as drift. bench/architect_sweeps.sh exit 0 "All five sweeps clean"; Sweep-1 shares no scope/regex with clause-5 (orthogonality empirically confirmed). Bench: check.py 63 metrics 0 regressed / 0 improved beyond tolerance / 63 stable (exit 0); compile_check 24/0/24 (exit 0); cross_lang 25/0/25 (exit 0) — aggregate 0/0/0. Baseline pristine, no `--update-baseline`, no paired ratify entry. **The cleanest milestone close pattern in the project's recent history:** zero spec defects surviving to audit (the two were caught and amended at planner-recon and plan-corpus-fetch time, before any byte moved — grounding-check PASS ×3 across the corpus-grounded amendments), zero drift, zero bench movement, zero re-loop. The two amendment cycles were the discipline working — recon and corpus-fetching surfaced gaps the brainstorm-sample missed, the spec was grounded properly *before* implementation, the iter then ran clean 5/5 with one non-blocking concern (a planner self-review-item-8 arithmetic miss on the `//!` header rewrite line count — substantive assertion unaffected, recorded as lesson). Milestone closes immediately; no tidy, no fieldtest. → 2026-05-19-audit-design-ledger-formal-links.md diff --git a/docs/roadmap.md b/docs/roadmap.md index 002de20..b859845 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -506,33 +506,41 @@ work progresses. convergence). CLOSED — audited + drift-resolved; hard gate enforces the honesty spirit. -- [~] **\[milestone\]** Formal cross-links in the `design/` ledger - (browsable wiki) — IN FLIGHT 2026-05-19. The positive completion +- [x] **\[milestone\]** Formal cross-links in the `design/` ledger + (browsable wiki) — CLOSED 2026-05-19. The positive completion of DESIGN.md → `design/`: informal prose cross-references between - contracts/models (`(see "Str ABI" below)`, bare-path-in-prose, - `§"Section"`) become **formal file-relative Markdown links** - `[label](./sibling.md)` / `[label](../../crates/.../x.rs)` - resolved relative to the containing file, plus a new RED-first + contracts/models are now **formal file-relative Markdown links** + resolved relative to the containing file, gated by RED-first `design_index_pin.rs` **clause-5** that fails the build on any body link that does not resolve into the durable tier (`design/` + `crates/**` + `runtime/**` only — never `docs/`, never a `#fragment`). clause-5 ∘ clause-3 = complete invariant: every contract cross-reference is a resolving durable file-link or - clause-3-forbidden history prose. The INDEX **spine stays - repo-root-relative** (registry tier, clause-1 byte-unchanged) — - the move-fragility rationale that drives body prose to - file-relative is absent for the file that never moves. - No authoring surface (no `.ail`; precedent: the split spec, - audited clean). Out of scope (recon-forced): intra-file - "above/below" within one document (no file-link form; the - file-split is a separate concern). + clause-3-forbidden history prose. INDEX spine stays + repo-root-relative (registry tier, clause-1 byte-unchanged) — the + move-fragility rationale that drives body prose to file-relative + is absent for the file that never moves. Shipped in one iter + (5/5 tasks): clause-5 with `strip_fences` (toggle on ```/~~~ so a + `](` inside a fence is not treated as a link) + RED-first via + identity-stubbed `strip_fences` synthetic vectors; 7 prose + conversions (8 link tokens — float-semantics:69/100, + embedding-abi:45, memory-model:44/105, scope-boundaries:48/88 + with mixed-referent split into a source link + a Pipeline + cross-file link); clause-6 disposition (b) on the 2 + PROSE_ROUNDTRIP homeless pointers (pointer removed, behavioural + `ail merge-prose` prose preserved); pin-safe positive-half + paragraph in `honesty-rule.md`. - context: spec `docs/specs/2026-05-19-design-ledger-formal-links.md` - (grounding-check PASS, 7/7 ratified); origin this session's - six-turn design conversation following the split close; - precedent slice already shipped in the split's tidy - (`float-semantics.md` `(see "Str ABI" below)` → - `(see design/contracts/str-abi.md)`, now to be brought to the - final `[Str ABI](str-abi.md)` form). Awaiting plan. + (grounding-check PASS ×3 across two corpus-grounded amendments — + clause-6 + cross-ref definition; clause-5 fence-skip + closed + convert-set enumeration); plan + `docs/plans/design-ledger-formal-links.1.md`; iter journal + `docs/journals/2026-05-19-iter-design-ledger-formal-links.1.md`; + audit `docs/journals/2026-05-19-audit-design-ledger-formal-links.md`; + origin this session's six-turn design conversation following the + split close. CLOSED — audit clean (zero drift, bench trio + 0/0/0); no tidy, no fieldtest (zero authoring-surface change — + reasoned exclusion). - [ ] **\[milestone\]** Flat array/slice primitive — performance follow-up to the Embedding ABI arc, *not* a capability gap.