diff --git a/docs/specs/2026-05-10-design-md-consolidation.md b/docs/specs/2026-05-10-design-md-consolidation.md new file mode 100644 index 0000000..7392f90 --- /dev/null +++ b/docs/specs/2026-05-10-design-md-consolidation.md @@ -0,0 +1,272 @@ +# DESIGN.md Consolidation — Design Spec + +**Date:** 2026-05-10 +**Status:** Approved — proceeding to plan +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +`docs/DESIGN.md` (today 2262 lines) mixes three time axes: timeless +state ("the language has X"), historical narrative ("Iter 18a added +Y, Iter 19b refined Y"), and workflow detail ("maintained by the +ailang-docwriter agent"). The mix forces every LLM reader (the +`ailang-architect` agent, the `brainstorm` skill, any future +orchestrator) to filter axis-by-axis on every read — work the +document should do once, structurally. + +This milestone consolidates `DESIGN.md` to a single time axis — +**state plus timeless rationale** — and routes the other two axes +to their existing canonical sources: + +| Source | Content | Time axis | +|--------|---------|-----------| +| `docs/DESIGN.md` | State + timeless why | timeless | +| `docs/JOURNAL.md` | History, migration, REVERTED, iter-decision corrections, queued options | chronological | +| `skills//SKILL.md` + agents | Workflow, discipline, agent rosters | trigger-bound | +| `crates/ailang-core/src/ast.rs` | Mirrors `DESIGN.md` §Data-model in code | mirrors DESIGN.md | + +The milestone is documentation-only. No language semantics change. +No new compiler features ship. The single new artefact in code is a +schema-drift test (`design_schema_drift.rs`, sweep 3). + +## Architecture + +**Role of DESIGN.md (post-consolidation).** DESIGN.md describes +exclusively *what the language is* — schema, semantics, invariants, +timeless reasoning for why alternatives are excluded. It contains +no development history, no iter tags, no status markers, no +migration plans, no REVERTED audit trails, no workflow discipline. + +**Cross-reference asymmetry.** References go primarily *into* +DESIGN.md, rarely *out of* it: + +- JOURNAL → DESIGN: every iter entry may link to the affected + section (existing practice). +- Skills/agents → DESIGN: `ailang-architect` reads DESIGN as the + drift baseline; `brainstorm` checks Feature-Acceptance against + DESIGN. +- DESIGN → JOURNAL: only at narrowly defined points where a + prohibition's calibration is historical and JOURNAL holds the + evidence (e.g. "tracing GC: see JOURNAL bench notes" — as a + one-line pointer, not as an embedded data point). +- DESIGN → skills: not direct; DESIGN does not describe how the + language is developed. + +**Schema SoT inversion.** Decision 1 says "source = data, not text"; +that makes DESIGN.md §Data-model structurally the schema spec. +Today's `whenever the two disagree, ast.rs is the source of truth` +disclaimer inverts: `ast.rs` gains a module-doc-comment pointing to +DESIGN.md §Data-model as the canonical source. A new drift test +(analogous to `crates/ailang-core/tests/spec_drift.rs`) enforces +structural agreement between the JSON-schema blocks in DESIGN.md +and the variants in `ast.rs` — the test is the mechanism that +makes the inversion load-bearing. + +**Feature-acceptance criterion (transferred).** The criterion is +calibrated for language features; this milestone adds none. +Transferred form: a consolidated DESIGN.md is load-bearing for +`ailang-architect` (drift-review baseline), `brainstorm` +(Feature-Acceptance gate baseline), and any future LLM that needs +to understand the state. Mixing history + state forces every reader +into filtering work the document should do once. Consolidation +**improves correctness** (fewer drift opportunities between +DESIGN/JOURNAL/skills) and **removes redundancy** (one SoT per +axis instead of three overlapping ones). The criterion applies by +analogy. + +## Components — the four thematic sweeps + +Four sweeps, each reviewable against a clear search pattern. Order +is chosen so each later sweep operates on a quieter base. + +### Sweep 1 — Remove history anchors + +Mechanical pass; no semantic content changes. + +- All iter tags removed (`Iter 14b`, `Iter 18a`, `Iter 22a`, + `Family 20`, `since Iter 13a`, …). +- All date anchors removed (`Committed 2026-05-08`, + `2026-05-09 revision flips`, `Iter 19b — FnDef.suppress`). +- All status markers removed (`Status: shipped`, + `Status: half-retirement`, `as of 2026-05-09`). +- All bench data-point anchors (`60% allocate-path overhead`, + `4.14× rc/bump`, `3.31x vs non-inlinable`) either removed or + condensed to a one-line JOURNAL pointer — the data point is + historical; the *prohibition* it grounds is timeless. + +Acceptance: +`grep -nE 'Iter [0-9]+[a-z]?|Family [0-9]+|2026-[0-9]{2}-[0-9]{2}|Status: ' docs/DESIGN.md` +returns nothing (or a deliberate exception is recorded in the +JOURNAL entry for this iter). + +### Sweep 2 — REVERTED + migration plans out + +- Decision 7 (`Term::If` removal) deleted entirely. `Term::If` + exists in the language; the witness is the Term-schema section. +- Decision 9 → Decision 10 "Migration plan" (7-point list) + removed — migration is complete; the end state is described. +- Decision 11 "Why mono, not virtual dispatch (the + empirically-grounded version)" — correction history (`The + original rationale implicitly argued … 2026-05-10 + micro-benchmark refutes …`) condensed to a single "mono enables + optimisations vdisp forbids" rationale. +- "future iter may"-speculations condensed: kept only where they + carry a **binding prohibition** (e.g. "future iter that proposes + laziness must prove cycle is collectible" — that is a language + constraint). Pure aspirations ("a future Prelude milestone may + add Show…") removed. +- Iter-numbered cross-references in surviving prose ("see Iter + 18d–18g shipping work") replaced with section-anchored links or + removed. + +Acceptance: +`grep -n 'REVERTED\|preserved for the audit trail\|Migration plan\|originally framed\|original rationale' docs/DESIGN.md` +returns nothing. + +### Sweep 3 — Schema SoT inversion + Data-model hardening + +- From the "Data model" section: all Rust type definitions + removed (`Type::Fn { params: Vec, … }` etc.) — DESIGN.md + specifies *JSON schema*, not Rust code. +- The `whenever the two disagree, ast.rs is the source of truth` + disclaimer removed and replaced by a module-doc-comment in + `ast.rs` pointing to DESIGN.md §Data-model. +- Drift test added: `crates/ailang-core/tests/design_schema_drift.rs` + (analogous to `spec_drift.rs`) walks exhaustive matches over + `Term` / `Pattern` / `Type` / `Def` / `Literal` and asserts an + anchor for each variant exists in the JSON-schema block in + DESIGN.md. +- A prominent sentence in the Data-model section establishes + DESIGN.md as the schema SoT. + +Acceptance: +`cargo test -p ailang-core design_schema_drift` green; no Rust type +definitions remain in `docs/DESIGN.md` (`grep -E '^\s*(struct|enum|pub )'` +returns nothing); `ast.rs` carries the SoT-pointer doc-comment. + +### Sweep 4 — Workflow / cross-reference cleanup + +- "Project ecosystem" entry `agents/` corrected (path is + `skills//agents/`). +- "Verification and correctness" section: workflow detail + ("Maintained by the `ailang-docwriter` agent (Iter 13d + onward)") removed — the verification contract stays timelessly + described; agent assignment belongs to the skills SoT. +- "What is not (yet) supported": iter anchors removed, + "Recently lifted gates" paragraph removed (it is history). + Section remains a pure state list. +- Cross-reference audit: every JOURNAL / skill reference from + DESIGN.md examined; only those kept that ground a binding + constraint. + +Acceptance: every surviving cross-reference has a brief +justification (in commit message); no references to now-stale +paths. + +## Data flow — readers of the document + +| Reader | Question DESIGN.md must answer | +|--------|--------------------------------| +| `ailang-architect` (audit) | "Is the code drifting from the spec?" — DESIGN.md is the baseline. | +| `brainstorm` (new milestone) | "Does Feature-Acceptance hold? Is there a prohibition that kills this proposal?" | +| Future LLM (orchestrator successor) | "What *is* the language now? What may I not propose?" | +| Foreign LLM (zero-shot AILang author) | Not directly — that reader uses `crates/ailang-core/specs/form_a.md`. DESIGN.md is the meta-baseline, not the authoring spec. | + +The first and third readers are the hardness tests: both are LLMs +without a human filter, who need to trust the document without +mixing it with JOURNAL or skills inside their context window. If +DESIGN.md embeds history, the LLM is forced into filter work the +document should not require. + +## Error handling — drift detection + +Three orthogonal mechanisms, each guarding a different drift +vector: + +1. **`design_schema_drift.rs`** (sweep 3). Closes the + ast.rs-vs-DESIGN-schema gap structurally. Every new + `Term` / `Pattern` / `Type` / `Def` variant forces a DESIGN + entry, otherwise the build fails. + +2. **History-marker check in the audit skill** (sweep-1 + acceptance promoted to a standing discipline). The + `ailang-architect` agent gains an additional mandatory check: + `grep -nE 'Iter [0-9]+[a-z]?|Status: |Committed [0-9]{4}-' docs/DESIGN.md` + must return nothing. The drift report flags any hit as an + Iron-Law violation. This is the live-time guard against the + "quick, drop an iter tag in DESIGN" reflex. + +3. **Discipline note in `implement/SKILL.md`**: a clarification + that DESIGN.md edits made during a feature-shipping iter must + be phrased *timelessly* — the iter entry lands in the JOURNAL, + the state entry in DESIGN. No new mechanism, but without this + clarification the next implementer has two writing targets + and no rule for which voice to use, and drift rebuilds. + +## Testing strategy + +Per sweep: +- **Sweep 1, 2, 4**: grep-based acceptance against the patterns + listed in each sweep. The pattern run on the post-iter file + must be empty (or carry an explicit, JOURNAL'd exception). +- **Sweep 3**: `cargo test -p ailang-core design_schema_drift` + green. The test is RED-first: the iter starts by adding the + test before the schema rewrite, so the GREEN transition is + evidence the schema text matches `ast.rs`. + +End-to-end: +- `cargo test --workspace` stays green throughout. No language + semantics change; if any test breaks, that is a real + regression and goes through the `debug` skill. +- `bench/check.py` and `bench/compile_check.py` stay at 0/0/0 + (they don't touch docs, but the milestone-close audit runs + them). + +No new `.ailx` fixtures: this milestone has no user-visible +surface, so `fieldtest` is skipped (per `skills/README.md`: +"Standard milestone-close path; skipped only for purely internal +milestones"). + +## Acceptance criteria + +**Per sweep**: see four sweeps above. + +**Across the whole milestone:** +- DESIGN.md has no iter tags, date anchors, status markers + (sweep-1 patterns return nothing). +- DESIGN.md has no REVERTED section (Decision 7 gone), no + migration plan. +- DESIGN.md describes the schema canonically; `ast.rs` points to + DESIGN.md; `cargo test design_schema_drift` green. +- `ailang-architect` agent verifies DESIGN.md hygiene as part of + drift review (its iron-law list gains the grep-pattern check). +- DESIGN.md size as a plausibility check: today 2262 lines — + expected post-consolidation 1500-1800 lines, but this is an + observation, not a target; an aggressive LOC target would push + against substance. +- One JOURNAL entry per sweep, documenting what was discarded + with an anchor example (e.g. "Decision 9 'half-retirement as + of 2026-05-09' frame removed; state form: 'RC is canonical, GC + is parity oracle, retirement gated on …'"). + +**Negative acceptance** (what must *not* happen): +- No semantic content of the language is lost (every constraint, + every prohibition, every schema form remains; only the + historical calibration migrates to JOURNAL). +- No circular references (DESIGN → JOURNAL → DESIGN for the + rationale of a decision; that means the rationale was not + formulable timelessly — and forces re-examination). +- No iter tag slips through (sweep-1 pattern is the sweep + acceptance AND the standing drift lint). + +## Iteration plan (for `plan` skill) + +Four iterations, one per sweep, in the order above. The +`design_schema_drift.rs` test in sweep 3 follows the standard +RED-first TDD discipline carried by `ailang-implementer`. The +audit step at milestone close runs the regular `audit` skill +(architect drift review + bench regression suite); the architect's +mandatory checks are extended for this milestone with the +DESIGN.md-hygiene grep, then that extension is captured in +`audit/agents/ailang-architect.md` so the check survives beyond +this milestone.