From aff25cdd6b8d83c0e3709640d98a612b3d23f519 Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 18 May 2026 12:05:40 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20docs-honesty-lint=20=E2=80=94=20discrim?= =?UTF-8?q?inator=20+=20enumerated=20pin=20+=20Sweep=205?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Single-iteration milestone (mirror effect-doc-honesty). Two-pronged tense+modality discriminator codified present-tense as a DESIGN.md meta-subsection (Approach A, user-ratified); procedural enumeration = discriminator applied to the full architect_sweeps.sh + new Sweep 5 output (not a frozen orchestrator list — the sweep already surfaces post-mortem sites the architect has been waving through as legitimate quotes); anti-regrowth = enumerated docs_honesty_pin.rs (hard cargo-test gate, incl. protected-exception present-anchors against over-correction) + Sweep 5 + ailang-architect.md "DESIGN.md honesty drift" bullet (advisory standing forward check). No language/checker/ codegen/runtime change. grounding-check PASS (10/10 ratified). --- docs/specs/2026-05-18-docs-honesty-lint.md | 323 +++++++++++++++++++++ 1 file changed, 323 insertions(+) create mode 100644 docs/specs/2026-05-18-docs-honesty-lint.md diff --git a/docs/specs/2026-05-18-docs-honesty-lint.md b/docs/specs/2026-05-18-docs-honesty-lint.md new file mode 100644 index 0000000..4cf2484 --- /dev/null +++ b/docs/specs/2026-05-18-docs-honesty-lint.md @@ -0,0 +1,323 @@ +# DESIGN.md / docs honesty lint — Design Spec + +**Date:** 2026-05-18 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Make `docs/DESIGN.md` (and the in-tree satellites that repeat its +claims) mirror **only** the actual current state of the language — +correct, present-tense — and make that property *load-bearing* so it +cannot silently regrow. + +`docs/DESIGN.md` is the artefact the `ailang-architect` agent +drift-checks the codebase against. A DESIGN.md that asserts something +which is not true, or that narrates future intent / past mishaps as if +they were part of the language description, makes the drift review +unsound at its root: the architect cannot tell "code drifted from the +spec" from "the spec was never true". This milestone closes that. + +Two failure modes are banned from the canonical docs: + +- **Wunschdenken** — aspirational "foundation for / we'd like / + planned / will back / on the path to / if-when X arrives we'll …" + forward intent. Genuine forward intent belongs in `docs/roadmap.md`, + never in DESIGN.md. +- **Post-mortem / doc-archaeology narrative** — "an earlier draft + said X; a follow-up replaced it / previously all diagnostics were + Error / what changed in is / X was retired in iter Y + / preserved for the audit trail". Project and document history + belongs in `docs/journals/`, never in DESIGN.md. + +One exception is legitimate and must be **preserved**: a +**present-tense reserved / unimplemented / deliberately-excluded** +claim that is explicitly AND correctly labelled as such. The +effect-doc-honesty precedent (2026-05-16) established the gold form: +*"`Diverge` is a reserved effect name with no op and no codegen."* +The discriminator is whether the doc asserts something +exists/works/changed that does not — not whether the sentence +mentions the future or the past. + +## Architecture + +Three layers, one iteration (the structure ratified with the user; +mirrors the effect-doc-honesty precedent, which shipped as a single +6-task iter): + +1. **The discriminator, codified present-tense inside DESIGN.md.** + A short meta-subsection added to DESIGN.md's existing + document-roles area states the honesty rule the document holds + itself to. This makes the boundary discoverable by any future LLM + reading DESIGN.md and gives the architect a citable in-tree + anchor. It is itself written present-tense (it describes a rule + the document *currently* obeys, not one it *will* adopt). + +2. **The corrections.** Every passage that the discriminator + classifies as Wunschdenken or non-citation post-mortem is + rewritten present-tense (or, for genuine forward intent, relocated + to `docs/roadmap.md`; the roadmap already carries the Boehm- + retirement and auto-derivation queue entries, so most relocations + are deletions, not moves). Scope: `docs/DESIGN.md` plus the + in-tree satellites that repeat the same claim — `docs/PROSE_ROUNDTRIP.md` + and any load-bearing `///` / `//!` doc-comment that echoes a + corrected DESIGN.md sentence. (`crates/ailang-core/specs/form_a.md` + was swept: its sole hit, *"multi-superclass chains are not yet + supported"*, is gold-form honest-reserved — no change.) + +3. **The anti-regrowth mechanism — pin (hard) + sweep+architect + (standing).** Two complementary guards, exactly the + design-md-consolidation pattern already in the tree: + - **Enumerated pin (authoritative, cargo-test gate).** A new + `crates/ailang-core/tests/docs_honesty_pin.rs`, sibling to the + existing `effect_doc_honesty_pin.rs`: for every corrected site, + one assertion that the fiction/Wunschdenken/post-mortem string + is **absent** and one that the present-tense corrected anchor is + **present**. This is a hard `cargo test --workspace` gate. + - **Sweep 5 + architect bullet (standing, advisory, forward).** + `bench/architect_sweeps.sh` already runs four + design-md-consolidation greps as an advisory companion to the + architect's "DESIGN.md history-anchor regrowth" check. Sweep 2 + *already* partially catches Wunschdenken (`A future iter may`, + `REVERTED`) — but its wording is too narrow (it misses "may land + in a future iteration", "is on the path to retirement", "an + earlier draft of this Decision said", "what changed in + is", "were retired in iter"). This milestone adds + **Sweep 5 (honesty)** closing those wording gaps, plus a new + **"DESIGN.md honesty drift"** bullet in the architect's "What + you check" list that cites the DESIGN.md discriminator + meta-subsection. The pin protects the corrected set; Sweep 5 + + the bullet catch *new* regrowth the enumerated pin cannot see. + +**Why the enumeration is procedural, not a frozen list.** The +authoritative scope is *"the discriminator applied to the full +`architect_sweeps.sh` (incl. Sweep 5) output, every hit classified"*. +Running the existing sweep already surfaces post-mortem sites a +narrow hand-grep misses (e.g. DESIGN.md:2027/2065 *"primitives were +retired 2026-05-14 in iter rpe.1"*, :503 *"what did change in +2026-05-13 (this milestone) is"*, :2135 *"originally listed here … +was retired"* — all currently waved through as Sweep-1 "legitimate +quotes"). The spec owns the discriminator and the procedure; the +planner owns the byte-exact classified enumeration (planner Iron +Law). A frozen orchestrator list would itself be a honesty risk. + +## Concrete code shapes + +This is a **documentation-integrity milestone: it has no `.ail` +authoring surface by nature.** Per the brainstorm skill's no-surface +clause, the concrete deliverables are (a) the executable pin (Rust), +and (b) the prose before → after — exactly the effect-doc-honesty +precedent's shape. Feature-acceptance clause 1 ("an LLM reaches for +it") is N/A (no surface); the measurable correctness improvement +(clause 2) is that the architect's drift review stops being unsound; +clause 3 is trivially met (no language construct, no bug class). + +### Primary deliverable A — the discriminator, as it will read in DESIGN.md + +A new present-tense meta-subsection (placed in DESIGN.md's +document-roles area, near the existing `roadmap.md` / `journals/` +pointer at ~line 50): + +```markdown +### What this document is — and the honesty rule it holds itself to + +DESIGN.md describes what AILang **is now**: schema, semantics, +invariants, runtime contracts, and the rationale for each. It is +present-tense by construction. Two things never belong here: + +- **Forward intent** ("planned / will back / on the path to / + if-when X arrives") — that lives in `docs/roadmap.md`. +- **History** ("an earlier draft said / previously / what changed + in / retired in iter Y") — that lives in `docs/journals/`. + +The single legitimate exception is a present-tense reserved or +deliberately-excluded claim that is explicitly and correctly +labelled (e.g. "`Diverge` is a reserved effect name with no op and +no codegen"; "No `deriving` — instances are written by hand"; +"Regions were considered and rejected; see X"). The discriminator +is not whether a sentence mentions past or future, but whether the +document **asserts something exists, works, or changed that does +not**. Design rationale — *why* the language excludes a thing — is +in scope and present-tense; the *narrative of how the document +reached that wording* is not. +``` + +### Primary deliverable B — worked discriminator classifications (the clause-2 evidence) + +The discriminator is the design judgement. These are the boundary +cases, each shown before → after, that *prove* it is operational +(not a hand-wave): + +| DESIGN.md | Before | Classification | After | +|---|---|---|---| +| ~L1504 | "An earlier draft of this Decision said 'uniqueness is fully inferred' … A follow-up discussion replaced it: AILang demands annotations *because* …" | **Post-mortem** (doc-archaeology) → journals | "AILang demands annotations because the LLM author can produce them effortlessly; the compiler does inference plus verification of contracts." | +| ~L2519 | "Boehm-on-`--alloc=gc` is on the path to retirement; see `docs/roadmap.md` for the active queue." | **Wunschdenken** (a plan, not a present fact) → roadmap (already carries the Boehm-retirement entry) | "`--alloc=gc` selects the transitional Boehm backend; `--alloc=rc` is the canonical backend (Decision 10) and the CLI default." | +| ~L2046 | "`float_to_str` (iter 24.1, type-installed; codegen ships in a future iter per roadmap). Will back `Show Float`." | "type-installed" honest-reserved (**keep**); "Will back `Show Float`" Wunschdenken (**strip**) | "`float_to_str : (borrow Float) -> Str` — type-installed; codegen reserved, not yet shipped." | +| ~L1318 | "The first ever Warning-level diagnostic; previously all diagnostics were `Error`." | **Post-mortem** → journals | "Severity: `Warning`. `ail check`/`build`/`emit-ir` exit 1 only on at least one `Error`; warnings print but do not abort." | +| ~L484 | "**Both extensions accepted.** Since iter ext-cli.1 (2026-05-12), every path-taking CLI subcommand accepts either …" | **History-anchor** (inline narrative, not a block-quote citation) → strip anchor | "Every path-taking CLI subcommand accepts either `.ail` (Form A) or `.ail.json` (Form B) as input." | +| ~L1516 | "Does not introduce regions. Regions were considered and rejected; see 'Why not other memory models' above." | **Design rationale** (present-tense exclusion + internal pointer) | **KEEP unchanged** — this is the protected exception, not drift. | +| ~L655 | "### Empirical addendum (2026-05-12)" + cross-model measurement results | Section *content* is present-tense evidence for Decision 6 (**keep**); the dated heading is a mild archaeology marker | Heading de-dated to "### Empirical addendum (cross-model authoring measurement)"; data unchanged. | +| ~L2725 | "`Diverge` is a reserved effect name with no op and no codegen." | Gold-form honest-reserved | **KEEP unchanged** — the discriminator's positive anchor. | + +### Secondary supporting — the pin shape (mirrors `effect_doc_honesty_pin.rs`) + +```rust +//! Regression pin: the canonical docs must stay present-tense-honest. +//! Fails if any swept-and-corrected Wunschdenken / post-mortem string +//! reappears, or any present-tense corrected anchor is dropped. +use std::fs; +fn read(rel: &str) -> String { /* identical helper to effect_doc_honesty_pin.rs */ } + +#[test] +fn design_md_has_no_wunschdenken() { + let d = read("docs/DESIGN.md"); + assert!(!d.contains("is on the path to retirement"), "..."); // L2519 + assert!(!d.contains("Will back `Show Float`"), "..."); // L2046 + assert!(!d.contains("may land in a future iteration"), "..."); // L1837 + // … one per corrected Wunschdenken site +} +#[test] +fn design_md_has_no_doc_archaeology() { + let d = read("docs/DESIGN.md"); + assert!(!d.contains("An earlier draft of this Decision said"), "..."); // L1504 + assert!(!d.contains("previously all diagnostics were"), "..."); // L1318 + assert!(!d.contains("were retired 2026-05-14 in iter rpe.1"), "..."); // L2027/2065 + // … one per corrected post-mortem site +} +#[test] +fn design_md_present_tense_anchors_present() { + let d = read("docs/DESIGN.md"); + assert!(d.contains("the honesty rule it holds itself to"), "discriminator meta-subsection present"); + assert!(d.contains("`Diverge` is a reserved effect name with no op and no codegen"), + "the gold-form honest-reserved anchor must remain (regression: do not over-strip)"); + // … the discriminator's protected-exception anchors, pinned so the + // sweep-and-correct does not over-correct a legitimate reserved claim +} +#[test] +fn prose_roundtrip_md_has_no_wunschdenken() { + let p = read("docs/PROSE_ROUNDTRIP.md"); + assert!(!p.contains("A future iter may layer a tool-use schema"), "..."); // L152 +} +``` + +### Secondary supporting — Sweep 5 + architect bullet + +`bench/architect_sweeps.sh`, appended after Sweep 4: + +```bash +run_sweep "5 (honesty: wunschdenken + non-citation post-mortem)" \ + 'on the path to retirement|may land in a future iteration|[Ww]ill back `Show|[Ww]ill (eventually|later) (be|support|back)|[Ii]f/?when .* (arrives|lands).*(will|would)|an earlier draft of this Decision said|previously all diagnostics were|what did change in 2026-|were retired 20[0-9]{2}-[0-9]{2}-[0-9]{2} in iter|originally listed here' +``` + +`skills/audit/agents/ailang-architect.md`, new bullet in "What you +check" (sibling to "DESIGN.md history-anchor regrowth"): + +```markdown +- **DESIGN.md honesty drift.** Sweep 5 of `bench/architect_sweeps.sh` + flags Wunschdenken / non-citation post-mortem. Apply the + discriminator from DESIGN.md §"What this document is — and the + honesty rule it holds itself to": a hit is drift unless it is a + present-tense correctly-labelled reserved/excluded claim, present- + tense design rationale, or a block-quote journal citation. Forward + intent is roadmap drift; document/project history is journal drift. +``` + +## Components + +| Artefact | Change | +|---|---| +| `docs/DESIGN.md` | + discriminator meta-subsection (~line 50 area); ~12–18 passages rewritten present-tense / stripped (planner enumerates byte-exact from the classified sweep output) | +| `docs/PROSE_ROUNDTRIP.md` | L152 "A future iter may layer …" tool-use/MCP Wunschdenken stripped (the present-tense doc describes the cycle that exists; roadmap already owns forward integration intent) | +| load-bearing `///`/`//!` doc-comments | any that echo a corrected DESIGN.md sentence, corrected in lockstep (effect-doc-honesty satellite precedent) | +| `crates/ailang-core/tests/docs_honesty_pin.rs` | **new** — enumerated absent/present pin, sibling to `effect_doc_honesty_pin.rs` | +| `bench/architect_sweeps.sh` | + Sweep 5 (honesty) | +| `skills/audit/agents/ailang-architect.md` | + "DESIGN.md honesty drift" bullet citing the meta-subsection | +| `docs/roadmap.md` | the P1 honesty-lint entry checked off + this spec referenced; relocations land here only if a stripped forward-intent line is *not* already represented (audit-time check — most are already present) | + +No language, checker, codegen, or runtime code changes. `ail +check`/`run`/`build` byte-unchanged by construction. + +## Data flow + +Build/test path is unchanged (docs + a new test file + a shell +sweep + an agent-definition file). The only new runtime artefact is +the pin test, which reads the doc files from `CARGO_MANIFEST_DIR/../../` +exactly as `effect_doc_honesty_pin.rs` does. The architect's review +flow gains one bullet and one sweep; both are advisory exactly like +the existing four sweeps (the script header already states non-zero +exit is advisory, architect-adjudicated). + +## Error handling + +- **Over-correction risk** (stripping a legitimate honest-reserved + claim): countered by the pin's *present*-anchor assertions — + `design_md_present_tense_anchors_present` fails if the gold-form + reserved claims (`Diverge` reserved, `No deriving`, regions + rejected, the known carve-outs) are dropped. The discriminator's + protected-exception list is itself pinned. +- **Sweep-5 false positives**: by design advisory, architect- + adjudicated against the codified discriminator — identical to the + existing four sweeps. Not a hard gate. +- **Pin brittleness** (a future legitimate edit trips an absent- + assertion): acceptable and intended — the pin is a *presence* + contract; a future edit that reintroduces a banned string must + consciously update the pin, which is the point (same contract as + `effect_doc_honesty_pin.rs`). + +## Testing strategy + +0. **Single-line pin substrings (carried from the effect-doc-honesty + Concern).** Every asserted absent/present substring MUST be a + contiguous single line in the doc file. The known failure mode: + the plan's verbatim corrected-replacement body soft-wraps a phrase + mid-string, contradicting a single-line pin substring, so the two + authoritative artefacts cannot both be applied verbatim. The + planner's pin-substring ↔ replacement-body-wrap cross-check must + account for the replacement body's own line wrap. +1. **RED first** (TDD, per implementer discipline): author + `docs_honesty_pin.rs` *before* the corrections; it runs RED + (absent-assertions fail because the fiction strings are still + present; present-anchor assertions fail because the discriminator + subsection is not yet written). +2. Apply the discriminator to the full `architect_sweeps.sh` (incl. + the new Sweep 5) output; classify every hit; correct every + FIX-classified passage; add the meta-subsection. +3. Pin goes GREEN. `cargo test --workspace` = prior + N new pin + tests, zero regressions. +4. The three sentinel drift/coverage tests (`design_schema_drift`, + `spec_drift`, `schema_coverage`) stay green (empirically confirms + none scans the corrected prose regions — same check the + effect-doc-honesty iter ran). +5. `bash bench/architect_sweeps.sh` re-run: Sweep 5 clean (0 honesty + hits); Sweeps 1–4 unchanged except where a corrected line removed + a previously-waved Sweep-1 date-anchor. +6. `cargo build --workspace` clean; `bench/check.py`, + `bench/compile_check.py`, `bench/cross_lang.py` unaffected (no + codegen change) — the milestone-close audit confirms. + +## Acceptance criteria + +- [ ] DESIGN.md carries the present-tense discriminator + meta-subsection; it is itself present-tense (describes a rule + the document currently obeys). +- [ ] Every `architect_sweeps.sh`+Sweep-5 hit is classified by the + discriminator; every FIX-classified passage in DESIGN.md + + satellites is rewritten present-tense or relocated to roadmap; + every KEEP passage (honest-reserved, design-rationale, + block-quote citation) is unchanged. +- [ ] `crates/ailang-core/tests/docs_honesty_pin.rs` exists, was + authored RED-first, and is GREEN: every corrected fiction + string absent, every present-tense + protected-exception + anchor present. +- [ ] `bench/architect_sweeps.sh` has Sweep 5; running it reports + zero honesty hits. +- [ ] `skills/audit/agents/ailang-architect.md` has the "DESIGN.md + honesty drift" bullet citing the meta-subsection. +- [ ] `cargo test --workspace` green (prior count + new pin tests, 0 + regressions); `cargo build --workspace` clean; the three + sentinel drift/coverage tests still green. +- [ ] No language/checker/codegen/runtime change; `ail + check`/`run`/`build` byte-unchanged. +- [ ] `docs/roadmap.md` P1 honesty-lint entry checked off; any + stripped genuine forward intent not already in the roadmap is + added there.