Brummel/AILang
1
0
Fork 0
Code Issues 29 Pull Requests Actions Packages Projects Releases Wiki Activity

spec: docs-honesty-lint — discriminator + enumerated pin + Sweep 5

Browse Source 
Single-iteration milestone (mirror effect-doc-honesty). Two-pronged
tense+modality discriminator codified present-tense as a DESIGN.md
meta-subsection (Approach A, user-ratified); procedural enumeration =
discriminator applied to the full architect_sweeps.sh + new Sweep 5
output (not a frozen orchestrator list — the sweep already surfaces
post-mortem sites the architect has been waving through as legitimate
quotes); anti-regrowth = enumerated docs_honesty_pin.rs (hard
cargo-test gate, incl. protected-exception present-anchors against
over-correction) + Sweep 5 + ailang-architect.md "DESIGN.md honesty
drift" bullet (advisory standing forward check). No language/checker/
codegen/runtime change. grounding-check PASS (10/10 ratified).
This commit is contained in:
Brummel
2026-05-18 12:05:40 +02:00
parent 5bb721178f
commit aff25cdd6b
1 changed files with 323 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/2026-05-18-docs-honesty-lint.md
+323
View File
@@ -0,0 +1,323 @@
# DESIGN.md / docs honesty lint — Design Spec
**Date:** 2026-05-18
**Status:** Draft — awaiting user spec review
**Authors:** Brummel (orchestrator) + Claude
## Goal
Make `docs/DESIGN.md` (and the in-tree satellites that repeat its
claims) mirror **only** the actual current state of the language —
correct, present-tense — and make that property *load-bearing* so it
cannot silently regrow.
`docs/DESIGN.md` is the artefact the `ailang-architect` agent
drift-checks the codebase against. A DESIGN.md that asserts something
which is not true, or that narrates future intent / past mishaps as if
they were part of the language description, makes the drift review
unsound at its root: the architect cannot tell "code drifted from the
spec" from "the spec was never true". This milestone closes that.
Two failure modes are banned from the canonical docs:
- **Wunschdenken** — aspirational "foundation for / we'd like /
planned / will back / on the path to / if-when X arrives we'll …"
forward intent. Genuine forward intent belongs in `docs/roadmap.md`,
never in DESIGN.md.
- **Post-mortem / doc-archaeology narrative** — "an earlier draft
said X; a follow-up replaced it / previously all diagnostics were
Error / what changed in <date> is / X was retired <date> in iter Y
/ preserved for the audit trail". Project and document history
belongs in `docs/journals/`, never in DESIGN.md.
One exception is legitimate and must be **preserved**: a
**present-tense reserved / unimplemented / deliberately-excluded**
claim that is explicitly AND correctly labelled as such. The
effect-doc-honesty precedent (2026-05-16) established the gold form:
*"`Diverge` is a reserved effect name with no op and no codegen."*
The discriminator is whether the doc asserts something
exists/works/changed that does not — not whether the sentence
mentions the future or the past.
## Architecture
Three layers, one iteration (the structure ratified with the user;
mirrors the effect-doc-honesty precedent, which shipped as a single
6-task iter):
1. **The discriminator, codified present-tense inside DESIGN.md.**
A short meta-subsection added to DESIGN.md's existing
document-roles area states the honesty rule the document holds
itself to. This makes the boundary discoverable by any future LLM
reading DESIGN.md and gives the architect a citable in-tree
anchor. It is itself written present-tense (it describes a rule
the document *currently* obeys, not one it *will* adopt).
2. **The corrections.** Every passage that the discriminator
classifies as Wunschdenken or non-citation post-mortem is
rewritten present-tense (or, for genuine forward intent, relocated
to `docs/roadmap.md`; the roadmap already carries the Boehm-
retirement and auto-derivation queue entries, so most relocations
are deletions, not moves). Scope: `docs/DESIGN.md` plus the
in-tree satellites that repeat the same claim — `docs/PROSE_ROUNDTRIP.md`
and any load-bearing `///` / `//!` doc-comment that echoes a
corrected DESIGN.md sentence. (`crates/ailang-core/specs/form_a.md`
was swept: its sole hit, *"multi-superclass chains are not yet
supported"*, is gold-form honest-reserved — no change.)
3. **The anti-regrowth mechanism — pin (hard) + sweep+architect
(standing).** Two complementary guards, exactly the
design-md-consolidation pattern already in the tree:
- **Enumerated pin (authoritative, cargo-test gate).** A new
`crates/ailang-core/tests/docs_honesty_pin.rs`, sibling to the
existing `effect_doc_honesty_pin.rs`: for every corrected site,
one assertion that the fiction/Wunschdenken/post-mortem string
is **absent** and one that the present-tense corrected anchor is
**present**. This is a hard `cargo test --workspace` gate.
- **Sweep 5 + architect bullet (standing, advisory, forward).**
`bench/architect_sweeps.sh` already runs four
design-md-consolidation greps as an advisory companion to the
architect's "DESIGN.md history-anchor regrowth" check. Sweep 2
*already* partially catches Wunschdenken (`A future iter may`,
`REVERTED`) — but its wording is too narrow (it misses "may land
in a future iteration", "is on the path to retirement", "an
earlier draft of this Decision said", "what changed in <date>
is", "were retired <date> in iter"). This milestone adds
**Sweep 5 (honesty)** closing those wording gaps, plus a new
**"DESIGN.md honesty drift"** bullet in the architect's "What
you check" list that cites the DESIGN.md discriminator
meta-subsection. The pin protects the corrected set; Sweep 5 +
the bullet catch *new* regrowth the enumerated pin cannot see.
**Why the enumeration is procedural, not a frozen list.** The
authoritative scope is *"the discriminator applied to the full
`architect_sweeps.sh` (incl. Sweep 5) output, every hit classified"*.
Running the existing sweep already surfaces post-mortem sites a
narrow hand-grep misses (e.g. DESIGN.md:2027/2065 *"primitives were
retired 2026-05-14 in iter rpe.1"*, :503 *"what did change in
2026-05-13 (this milestone) is"*, :2135 *"originally listed here …
was retired"* — all currently waved through as Sweep-1 "legitimate
quotes"). The spec owns the discriminator and the procedure; the
planner owns the byte-exact classified enumeration (planner Iron
Law). A frozen orchestrator list would itself be a honesty risk.
## Concrete code shapes
This is a **documentation-integrity milestone: it has no `.ail`
authoring surface by nature.** Per the brainstorm skill's no-surface
clause, the concrete deliverables are (a) the executable pin (Rust),
and (b) the prose before → after — exactly the effect-doc-honesty
precedent's shape. Feature-acceptance clause 1 ("an LLM reaches for
it") is N/A (no surface); the measurable correctness improvement
(clause 2) is that the architect's drift review stops being unsound;
clause 3 is trivially met (no language construct, no bug class).
### Primary deliverable A — the discriminator, as it will read in DESIGN.md
A new present-tense meta-subsection (placed in DESIGN.md's
document-roles area, near the existing `roadmap.md` / `journals/`
pointer at ~line 50):
```markdown
### What this document is — and the honesty rule it holds itself to
DESIGN.md describes what AILang **is now**: schema, semantics,
invariants, runtime contracts, and the rationale for each. It is
present-tense by construction. Two things never belong here:
- **Forward intent** ("planned / will back / on the path to /
if-when X arrives") — that lives in `docs/roadmap.md`.
- **History** ("an earlier draft said / previously / what changed
in <date> / retired in iter Y") — that lives in `docs/journals/`.
The single legitimate exception is a present-tense reserved or
deliberately-excluded claim that is explicitly and correctly
labelled (e.g. "`Diverge` is a reserved effect name with no op and
no codegen"; "No `deriving` — instances are written by hand";
"Regions were considered and rejected; see X"). The discriminator
is not whether a sentence mentions past or future, but whether the
document **asserts something exists, works, or changed that does
not**. Design rationale — *why* the language excludes a thing — is
in scope and present-tense; the *narrative of how the document
reached that wording* is not.
```
### Primary deliverable B — worked discriminator classifications (the clause-2 evidence)
The discriminator is the design judgement. These are the boundary
cases, each shown before → after, that *prove* it is operational
(not a hand-wave):
| DESIGN.md | Before | Classification | After |
|---|---|---|---|
| ~L1504 | "An earlier draft of this Decision said 'uniqueness is fully inferred' … A follow-up discussion replaced it: AILang demands annotations *because* …" | **Post-mortem** (doc-archaeology) → journals | "AILang demands annotations because the LLM author can produce them effortlessly; the compiler does inference plus verification of contracts." |
| ~L2519 | "Boehm-on-`--alloc=gc` is on the path to retirement; see `docs/roadmap.md` for the active queue." | **Wunschdenken** (a plan, not a present fact) → roadmap (already carries the Boehm-retirement entry) | "`--alloc=gc` selects the transitional Boehm backend; `--alloc=rc` is the canonical backend (Decision 10) and the CLI default." |
| ~L2046 | "`float_to_str` (iter 24.1, type-installed; codegen ships in a future iter per roadmap). Will back `Show Float`." | "type-installed" honest-reserved (**keep**); "Will back `Show Float`" Wunschdenken (**strip**) | "`float_to_str : (borrow Float) -> Str` — type-installed; codegen reserved, not yet shipped." |
| ~L1318 | "The first ever Warning-level diagnostic; previously all diagnostics were `Error`." | **Post-mortem** → journals | "Severity: `Warning`. `ail check`/`build`/`emit-ir` exit 1 only on at least one `Error`; warnings print but do not abort." |
| ~L484 | "**Both extensions accepted.** Since iter ext-cli.1 (2026-05-12), every path-taking CLI subcommand accepts either …" | **History-anchor** (inline narrative, not a block-quote citation) → strip anchor | "Every path-taking CLI subcommand accepts either `.ail` (Form A) or `.ail.json` (Form B) as input." |
| ~L1516 | "Does not introduce regions. Regions were considered and rejected; see 'Why not other memory models' above." | **Design rationale** (present-tense exclusion + internal pointer) | **KEEP unchanged** — this is the protected exception, not drift. |
| ~L655 | "### Empirical addendum (2026-05-12)" + cross-model measurement results | Section *content* is present-tense evidence for Decision 6 (**keep**); the dated heading is a mild archaeology marker | Heading de-dated to "### Empirical addendum (cross-model authoring measurement)"; data unchanged. |
| ~L2725 | "`Diverge` is a reserved effect name with no op and no codegen." | Gold-form honest-reserved | **KEEP unchanged** — the discriminator's positive anchor. |
### Secondary supporting — the pin shape (mirrors `effect_doc_honesty_pin.rs`)
```rust
//! Regression pin: the canonical docs must stay present-tense-honest.
//! Fails if any swept-and-corrected Wunschdenken / post-mortem string
//! reappears, or any present-tense corrected anchor is dropped.
use std::fs;
fn read(rel: &str) -> String { /* identical helper to effect_doc_honesty_pin.rs */ }
#[test]
fn design_md_has_no_wunschdenken() {
let d = read("docs/DESIGN.md");
assert!(!d.contains("is on the path to retirement"), "..."); // L2519
assert!(!d.contains("Will back `Show Float`"), "..."); // L2046
assert!(!d.contains("may land in a future iteration"), "..."); // L1837
// … one per corrected Wunschdenken site
}
#[test]
fn design_md_has_no_doc_archaeology() {
let d = read("docs/DESIGN.md");
assert!(!d.contains("An earlier draft of this Decision said"), "..."); // L1504
assert!(!d.contains("previously all diagnostics were"), "..."); // L1318
assert!(!d.contains("were retired 2026-05-14 in iter rpe.1"), "..."); // L2027/2065
// … one per corrected post-mortem site
}
#[test]
fn design_md_present_tense_anchors_present() {
let d = read("docs/DESIGN.md");
assert!(d.contains("the honesty rule it holds itself to"), "discriminator meta-subsection present");
assert!(d.contains("`Diverge` is a reserved effect name with no op and no codegen"),
"the gold-form honest-reserved anchor must remain (regression: do not over-strip)");
// … the discriminator's protected-exception anchors, pinned so the
// sweep-and-correct does not over-correct a legitimate reserved claim
}
#[test]
fn prose_roundtrip_md_has_no_wunschdenken() {
let p = read("docs/PROSE_ROUNDTRIP.md");
assert!(!p.contains("A future iter may layer a tool-use schema"), "..."); // L152
}
```
### Secondary supporting — Sweep 5 + architect bullet
`bench/architect_sweeps.sh`, appended after Sweep 4:
```bash
run_sweep "5 (honesty: wunschdenken + non-citation post-mortem)" \
'on the path to retirement|may land in a future iteration|[Ww]ill back `Show|[Ww]ill (eventually|later) (be|support|back)|[Ii]f/?when .* (arrives|lands).*(will|would)|an earlier draft of this Decision said|previously all diagnostics were|what did change in 2026-|were retired 20[0-9]{2}-[0-9]{2}-[0-9]{2} in iter|originally listed here'
```
`skills/audit/agents/ailang-architect.md`, new bullet in "What you
check" (sibling to "DESIGN.md history-anchor regrowth"):
```markdown
- **DESIGN.md honesty drift.** Sweep 5 of `bench/architect_sweeps.sh`
flags Wunschdenken / non-citation post-mortem. Apply the
discriminator from DESIGN.md §"What this document is — and the
honesty rule it holds itself to": a hit is drift unless it is a
present-tense correctly-labelled reserved/excluded claim, present-
tense design rationale, or a block-quote journal citation. Forward
intent is roadmap drift; document/project history is journal drift.
```
## Components
| Artefact | Change |
|---|---|
| `docs/DESIGN.md` | + discriminator meta-subsection (~line 50 area); ~12–18 passages rewritten present-tense / stripped (planner enumerates byte-exact from the classified sweep output) |
| `docs/PROSE_ROUNDTRIP.md` | L152 "A future iter may layer …" tool-use/MCP Wunschdenken stripped (the present-tense doc describes the cycle that exists; roadmap already owns forward integration intent) |
| load-bearing `///`/`//!` doc-comments | any that echo a corrected DESIGN.md sentence, corrected in lockstep (effect-doc-honesty satellite precedent) |
| `crates/ailang-core/tests/docs_honesty_pin.rs` | **new** — enumerated absent/present pin, sibling to `effect_doc_honesty_pin.rs` |
| `bench/architect_sweeps.sh` | + Sweep 5 (honesty) |
| `skills/audit/agents/ailang-architect.md` | + "DESIGN.md honesty drift" bullet citing the meta-subsection |
| `docs/roadmap.md` | the P1 honesty-lint entry checked off + this spec referenced; relocations land here only if a stripped forward-intent line is *not* already represented (audit-time check — most are already present) |
No language, checker, codegen, or runtime code changes. `ail
check`/`run`/`build` byte-unchanged by construction.
## Data flow
Build/test path is unchanged (docs + a new test file + a shell
sweep + an agent-definition file). The only new runtime artefact is
the pin test, which reads the doc files from `CARGO_MANIFEST_DIR/../../`
exactly as `effect_doc_honesty_pin.rs` does. The architect's review
flow gains one bullet and one sweep; both are advisory exactly like
the existing four sweeps (the script header already states non-zero
exit is advisory, architect-adjudicated).
## Error handling
- **Over-correction risk** (stripping a legitimate honest-reserved
claim): countered by the pin's *present*-anchor assertions —
`design_md_present_tense_anchors_present` fails if the gold-form
reserved claims (`Diverge` reserved, `No deriving`, regions
rejected, the known carve-outs) are dropped. The discriminator's
protected-exception list is itself pinned.
- **Sweep-5 false positives**: by design advisory, architect-
adjudicated against the codified discriminator — identical to the
existing four sweeps. Not a hard gate.
- **Pin brittleness** (a future legitimate edit trips an absent-
assertion): acceptable and intended — the pin is a *presence*
contract; a future edit that reintroduces a banned string must
consciously update the pin, which is the point (same contract as
`effect_doc_honesty_pin.rs`).
## Testing strategy
0. **Single-line pin substrings (carried from the effect-doc-honesty
Concern).** Every asserted absent/present substring MUST be a
contiguous single line in the doc file. The known failure mode:
the plan's verbatim corrected-replacement body soft-wraps a phrase
mid-string, contradicting a single-line pin substring, so the two
authoritative artefacts cannot both be applied verbatim. The
planner's pin-substring ↔ replacement-body-wrap cross-check must
account for the replacement body's own line wrap.
1. **RED first** (TDD, per implementer discipline): author
`docs_honesty_pin.rs` *before* the corrections; it runs RED
(absent-assertions fail because the fiction strings are still
present; present-anchor assertions fail because the discriminator
subsection is not yet written).
2. Apply the discriminator to the full `architect_sweeps.sh` (incl.
the new Sweep 5) output; classify every hit; correct every
FIX-classified passage; add the meta-subsection.
3. Pin goes GREEN. `cargo test --workspace` = prior + N new pin
tests, zero regressions.
4. The three sentinel drift/coverage tests (`design_schema_drift`,
`spec_drift`, `schema_coverage`) stay green (empirically confirms
none scans the corrected prose regions — same check the
effect-doc-honesty iter ran).
5. `bash bench/architect_sweeps.sh` re-run: Sweep 5 clean (0 honesty
hits); Sweeps 1–4 unchanged except where a corrected line removed
a previously-waved Sweep-1 date-anchor.
6. `cargo build --workspace` clean; `bench/check.py`,
`bench/compile_check.py`, `bench/cross_lang.py` unaffected (no
codegen change) — the milestone-close audit confirms.
## Acceptance criteria
- [ ] DESIGN.md carries the present-tense discriminator
meta-subsection; it is itself present-tense (describes a rule
the document currently obeys).
- [ ] Every `architect_sweeps.sh`+Sweep-5 hit is classified by the
discriminator; every FIX-classified passage in DESIGN.md +
satellites is rewritten present-tense or relocated to roadmap;
every KEEP passage (honest-reserved, design-rationale,
block-quote citation) is unchanged.
- [ ] `crates/ailang-core/tests/docs_honesty_pin.rs` exists, was
authored RED-first, and is GREEN: every corrected fiction
string absent, every present-tense + protected-exception
anchor present.
- [ ] `bench/architect_sweeps.sh` has Sweep 5; running it reports
zero honesty hits.
- [ ] `skills/audit/agents/ailang-architect.md` has the "DESIGN.md
honesty drift" bullet citing the meta-subsection.
- [ ] `cargo test --workspace` green (prior count + new pin tests, 0
regressions); `cargo build --workspace` clean; the three
sentinel drift/coverage tests still green.
- [ ] No language/checker/codegen/runtime change; `ail
check`/`run`/`build` byte-unchanged.
- [ ] `docs/roadmap.md` P1 honesty-lint entry checked off; any
stripped genuine forward intent not already in the roadmap is
added there.