diff --git a/docs/DESIGN.md b/docs/DESIGN.md index eb2bed8..6a9d255 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -26,9 +26,17 @@ evolving in lockstep with the language: - **Language core** (`crates/ailang-core`, `crates/ailang-check`, `crates/ailang-codegen`): AST, type system, codegen. +- **Surface forms** (`crates/ailang-surface`, `crates/ailang-prose`): + the LLM-facing renderings of a module. `ailang-surface` (Iter 14c) + is the lossless Form-A printer/parser — the canonical authoring + surface fixed by Decision 6, with a round-trip property + `parse ∘ print = id` gating every release. `ailang-prose` (Family 20) + is the lossy Form-B projection — human-readable prose for review and + edit, with no parser; re-integration goes through the + LLM-mediator round-trip documented in `docs/PROSE_ROUNDTRIP.md`. - **CLI** (`crates/ail`): toolchain for tooling consumers — `manifest`, - `describe`, `deps`, `check`, `build`, etc., preferably with `--json` for - machine consumption. + `describe`, `deps`, `check`, `build`, `parse`, `render`, `prose`, + `merge-prose`, etc., preferably with `--json` for machine consumption. - **Examples** (`examples/`): canonical `.ail.json` programs. They are specification anchors, not demos — the E2E suite hangs off them. - **Agents** (`agents/`): specialised sub-prompts (implementer, @@ -1382,7 +1390,15 @@ with exactly one dot in the name is a qualified reference: `.`. Hash stability: no new AST node, no renamed fields — all previous module hashes stay bit-identical. -## Data model (MVP) +## Data model + +The on-disk JSON-AST is what the toolchain hashes, typechecks, and +lowers. Every node in this section is the schema mirror of an enum or +struct in `crates/ailang-core/src/ast.rs`; whenever the two disagree, +`ast.rs` is the source of truth. Every additive field is declared with +`skip_serializing_if` so pre-existing fixtures keep bit-identical +canonical-JSON hashes — that gating contract is what makes growing +the schema cheap. ### Module @@ -1397,57 +1413,163 @@ hashes stay bit-identical. ### Def -`kind ∈ { "fn", "type", "effect", "const" }`. In the MVP only `fn` and `const`. +`kind ∈ { "fn", "const", "type" }`. All three are real surface forms. ```jsonc -{ - "kind": "fn", +// fn (the unit that gets a content hash) +{ "kind": "fn", + "name": "", + "type": Type, // typically Type::Fn, optionally wrapped in Forall + "params": [""...], // names bound in body, in type.params order + "body": Term, + "doc": "", + "suppress": [Suppress...] // Iter 19b, omitted when empty +} + +// const (top-level value; codegen emits as a global; body must be pure) +{ "kind": "const", "name": "", "type": Type, - "params": [""...], - "body": Term, + "value": Term, "doc": "" } + +// type (algebraic data type; Iter 5; parameterised since Iter 13a) +{ "kind": "type", + "name": "", + "vars": [""...], // type parameters; omitted when empty (pre-13a hash-stable) + "ctors": [ + { "name": "", "fields": [Type...] } // nullary ctor: fields = [] + ... + ], + "doc": "", + "drop-iterative": true // Iter 18e opt-in; omitted when false (pre-18e hash-stable) +} +``` + +**`Suppress`** (Iter 19b — entry in `FnDef.suppress`): + +```jsonc +{ "code": "", // e.g. "over-strict-mode" + "because": "" // must be non-empty; + // empty/whitespace fires `empty-suppress-reason` (Error) +} ``` ### Term (expression) ```jsonc -{ "t": "lit", "lit": { "kind": "int" | "bool" | "unit", "value": ... } } +{ "t": "lit", "lit": Literal } { "t": "var", "name": "" } -{ "t": "app", "fn": Term, "args": [Term...] } -{ "t": "let", "name": "", "value": Term, "body": Term } -{ "t": "if", "cond": Term, "then": Term, "else": Term } -{ "t": "do", "op": "/", "args": [Term...] } + +// fn application; tail flag is Iter 14e (musttail under codegen). +// `tail` is omitted when false (pre-14e hash-stable). +{ "t": "app", "fn": Term, "args": [Term...], "tail": false } + +{ "t": "let", "name": "", "value": Term, "body": Term } + +// Local recursive let (Iter 16b.1). Always fn-shaped. The desugar pass +// lifts most `letrec` to a synthetic top-level fn; `lift_letrecs` (16b.3) +// finishes the job after typecheck for the residue that captures +// let-bound names. Post-codegen, no `letrec` survives. +{ "t": "letrec", + "name": "", "type": Type, "params": [""...], + "body": Term, "in": Term } + +{ "t": "if", "cond": Term, "then": Term, "else": Term } + +// Effect-op invocation. `op` is "/" (e.g. "io/print_int"). +// `tail` per Iter 14e (omitted when false). +{ "t": "do", "op": "/", "args": [Term...], "tail": false } + +// Ctor application; `args` omitted when empty. { "t": "ctor", "type": "", "ctor": "", "args": [Term...] } + { "t": "match", "scrutinee": Term, "arms": [Arm...] } + +// Anonymous fn value (Iter 8b); free vars captured from enclosing scope. { "t": "lam", "params": [""...], "paramTypes": [Type...], "retType": Type, "effects": [""...], "body": Term } -{ "t": "seq", "lhs": Term, "rhs": Term } + +// Sequencing (Iter 10). Semantically `let _ = lhs in rhs`; lhs must be Unit. +{ "t": "seq", "lhs": Term, "rhs": Term } + +// Iter 18c.1: explicit RC clone. Codegen lowers as +// `call void @ailang_rc_inc(ptr %v)` before returning %v under `--alloc=rc`. +{ "t": "clone", "value": Term } + +// Iter 18d.1: explicit reuse-as hint. `body` must be allocating +// (typically `ctor` or `lam`); `source` must be a bare `var`. Codegen +// lowers as in-place rewrite under `--alloc=rc`. +{ "t": "reuse-as", "source": Term, "body": Term } ``` -In the MVP, `do` is only a direct call to a built-in effect op (no handler). -A `lam` term constructs an anonymous function value; free variables of -its body are captured from the enclosing scope (see Iter 8 closure -conversion in JOURNAL). A `seq` term evaluates `lhs` for its effects -(its result must be Unit) and yields `rhs`'s value — equivalent to -`let _ = lhs in rhs`. +In the MVP, `do` is only a direct call to a built-in effect op (no +handler). A `lam` term constructs an anonymous function value; free +variables of its body are captured from the enclosing scope (see +Iter 8 closure conversion in JOURNAL). + +**`Literal`**: + +```jsonc +{ "kind": "int", "value": } +{ "kind": "bool", "value": } +{ "kind": "str", "value": "" } +{ "kind": "unit" } +``` + +**`Pattern`** (the `pat` field of an `Arm`; discriminator `p`): + +```jsonc +{ "p": "wild" } // _ +{ "p": "var", "name": "" } // x — binds the value +{ "p": "lit", "lit": Literal } +{ "p": "ctor", "ctor": "", "fields": [Pattern...] } // fields omitted when empty +``` + +Patterns are linear: each pattern variable may appear at most once. ### Type ```jsonc -{ "k": "con", "name": "Int" } -{ "k": "con", "name": "Bool" } -{ "k": "con", "name": "Unit" } -{ "k": "fn", "params": [Type...], "ret": Type, "effects": ["IO"...] } -{ "k": "var", "name": "a" } -{ "k": "forall", "vars": ["a"...], "body": Type } +// Type-constructor application. `args` omitted when empty +// (pre-13a hash-stable for non-parameterised cases like Int, Bool, ...). +{ "k": "con", "name": "", "args": [Type...] } + +// Function type. Decision 10 (Iter 18a) added paramModes/retMode as +// metadata on Type::Fn — they are NOT separate Type variants, so every +// existing match-arm in the typechecker (unify, occurs, apply) keeps +// working. `paramModes` omitted when every entry is "implicit"; +// `retMode` omitted when "implicit" (pre-18a hash-stable). +{ "k": "fn", + "params": [Type...], + "paramModes": [ParamMode...], + "ret": Type, + "retMode": ParamMode, + "effects": [""...] } + +{ "k": "var", "name": "" } + +// Top-level polymorphism only. +{ "k": "forall", "vars": [""...], "body": Type } ``` +**`ParamMode`** (Iter 18a / Decision 10): + +``` +"implicit" — pre-18a / unannotated. Treated as `own` by the typechecker. +"own" — (own T) — caller transfers ownership; callee consumes. +"borrow" — (borrow T) — caller retains ownership; callee may not consume. +``` + +`implicit ≡ own` semantically; the distinction exists so pre-18a +fixtures continue to serialize without the mode wrapper and keep their +canonical-JSON hash. + ## Pipeline ``` @@ -1455,13 +1577,25 @@ conversion in JOURNAL). A `seq` term evaluates `lhs` for its effects ├─ load + validate schema ├─ resolve names + assign hashes ├─ desugar (AST → AST, Iter 16a) - ├─ typecheck (HM, effect rows) + ├─ typecheck (HM, effect rows; mode-strict per Decision 10) ├─ lift_letrecs (post-typecheck AST → AST, Iter 16b.3) ├─ lower to MIR (SSA-like, named SSA values) ├─ emit LLVM IR (.ll) - └─ clang -O2 *.ll -o binary (links libgc for @GC_malloc) + └─ clang -O2 *.ll -o binary + --alloc=gc → links libgc (@GC_malloc; transitional) + --alloc=rc → emits inc/dec (@ailang_rc_inc / _dec; canonical) ``` +Two allocator backends share the same MIR. `--alloc=gc` links libgc +and is the transitional fallback (no inc/dec instrumentation, no mode +checks at codegen time). `--alloc=rc` is the canonical backend +committed to in Decision 10: the typechecker enforces +`(own)` / `(borrow)` modes, codegen emits `ailang_rc_inc` / `_dec` +calls at the points dictated by linearity, and `Term::Clone` / +`Term::ReuseAs` materialise into actual rc-bumps and in-place +rewrites respectively. Boehm-on-`--alloc=gc` is on the path to +retirement; see the JOURNAL queue. + The **desugar** pass (`ailang-core::desugar::desugar_module`) runs before typecheck and codegen in every entry point of `ailang-check` and `ailang-codegen`. It is a pure AST → AST rewriter — currently diff --git a/docs/JOURNAL.md b/docs/JOURNAL.md index ade5be1..beb5562 100644 --- a/docs/JOURNAL.md +++ b/docs/JOURNAL.md @@ -9638,6 +9638,129 @@ that always works without a tool-use-capable client. ### Family / arc state 20f closes the prose-roundtrip arc for now. JOURNAL queue is -empty again. Tidy iter for 20a / b / d / e / f together can wait -until the next family closes — the architect drift-review -discipline applies as before. +empty again. The 20-family tidy-iter follows immediately per +CLAUDE.md ("the tidy-iter is non-optional, every named family +closes with one"); the previous note here ("can wait") was +incorrect and has been superseded by the actual tidy below. + +## 2026-05-08 — 20-family tidy-iter (DESIGN.md refresh) + +### Why + +Per CLAUDE.md, every named iter family closes with a tidy-iter: +run `ailang-architect`, resolve each item (fix the drift, ratify +in DESIGN.md, or record in JOURNAL why it is acceptable). Family +20 — 20a, 20b, 20d, 20e, 20f — added a whole new surface +(`ailang-prose`), a CLI subcommand pair (`ail prose` / +`ail merge-prose`), an embedded Form-A spec, and the drift-test +trip-wire. The architect surfaced four items. + +### Architect findings (verbatim, abridged) + + 1. **`docs/DESIGN.md` "Data model (MVP)" block stale** — + missing: `tail` flag on `app`/`do`, `seq`, `clone`, + `reuse-as`, `let-rec`, `paramModes`/`retMode` on `fn`, + `suppress` on `FnDef`, `drop_iterative` on `TypeDef`, + `Type::Con.args`, `TypeDef.vars`. Pipeline diagram still + said "links libgc" only — did not mention `--alloc=rc`. + The deferral was already noted in 19a-arc tidy. + 2. **Project ecosystem inventory missing `ailang-surface` and + `ailang-prose`** — the section's own rule says new tools + are added "as soon as they are established"; both crates + are well past that threshold (14c and 20a respectively). + `ail prose` and `ail merge-prose` were also missing from + the CLI bullet. + 3. **`FnDef { suppress: vec![] }` synthesis fan-out at 45 + sites** — `ailang-codegen` plus `ailang-core::desugar` + alone has 16. 19a-arc tidy noted 7 and deferred. Each + schema-additive `FnDef` / `Type::Fn` field hits every site; + a `FnDef::synthetic(...)` constructor is the absorbing fix. + 4. **`linearity.rs` known debt (L117–125)** — deeply-nested + match on a sub-binder produces a spurious + `over-strict-mode` warning. Not fired by current corpus; + module doc records the gap. Acceptable. + +### Resolutions + +**(1) and (2)** — fixed in this tidy. `docs/DESIGN.md`: + + - **Data model section rewritten end-to-end.** Was titled + "Data model (MVP)" with stale `MVP only fn and const`; now + "Data model" reflecting that all three kinds (`fn`, `const`, + `type`) are real surface forms. Added every additive field + documented in `crates/ailang-core/src/ast.rs` with the + schema-additive `skip_serializing_if` rationale called out + once at the top so each individual mention can stay terse. + The `Suppress`, `ParamMode`, `Literal`, and `Pattern` + sub-schemas now have their own blocks. + - **Pipeline diagram extended.** Now shows `--alloc=gc` (links + libgc; transitional) and `--alloc=rc` (emits + `ailang_rc_inc` / `_dec`; canonical) as two backends sharing + the same MIR. The 18a–18d additions (`Term::Clone`, + `Term::ReuseAs`) get a sentence pointing at where they + materialise under `--alloc=rc`. + - **Ecosystem inventory** gained a "Surface forms" bullet + between Language core and CLI, naming `ailang-surface` + (Iter 14c) as the lossless Form-A printer/parser fixed by + Decision 6 with `parse ∘ print = id` as gating contract, + and `ailang-prose` (Family 20) as the lossy Form-B + projection with no parser whose re-integration goes through + `docs/PROSE_ROUNDTRIP.md`. The CLI bullet now lists + `parse`, `render`, `prose`, `merge-prose`. + +**(3) `FnDef::synthetic(...)` constructor** — *deferred, not +addressed in this tidy.* Reason: the right factor-out shape is +not yet obvious. Most existing call sites differ on more than +just `suppress` — they have to compute `params`, build a +`Type::Fn`, etc. — so a one-arg constructor saves nothing; the +useful constructor is roughly `FnDef::new(name, ty, params, +body)` with `doc: None, suppress: vec![]` defaults. That is a +real change in 45 sites and should land as its own iter when the +schema next grows a `FnDef` field (forcing the touch anyway). +Recorded in the JOURNAL queue. + +**(4) `linearity.rs` sub-binder spurious warning** — +*acceptable, recorded.* The warning fires on a synthetic shape +that the current corpus does not produce; module doc names the +gap; an actual customer-impact trigger would warrant a fix, not +preemptive work. Status quo holds. + +### Acceptable drift summary + +- `FnDef::synthetic(...)` factor-out — deferred until the next + schema-additive `FnDef` field (rationale: existing fan-out + pattern works, change is large but mechanical, and the cost + is amortised by piggybacking on the next forced touch). +- `linearity.rs` deeply-nested-match-on-sub-binder spurious + warning — accepted (not corpus-triggered; cost of fix + exceeds cost of the false positive at this scale). + +### What this tidy does NOT do + +- No code changes. Pure documentation iter. +- No new tests. Existing 288 workspace tests stay green. +- No DESIGN.md content removed. The pre-tidy "Data model (MVP)" + block is replaced rather than amended because it had grown + factually incorrect; the historical context lives in + `git log` and in the iter-by-iter JOURNAL entries that + introduced each schema field. + +### JOURNAL queue (post-tidy) + +- **`FnDef::synthetic(...)` factor-out** — to be done when the + next schema-additive `FnDef` field forces the touch anyway. +- **Deferred richer integration paths** (from 20f): tool-use + schemas, MCP server, LSP. All additive over the static-prompt + round-trip; pick when prioritised. +- **Boehm retirement** — `--alloc=gc` is transitional; + `--alloc=rc` is canonical. Boehm-side scaffolding to be + removed once the rc backend covers the full corpus. +- **Family 21+** (typeclasses, polymorphic ADTs at runtime, + pattern-binding generalisation) — long-horizon language work. + +### Family / arc state + +The 20-family is now formally closed with this tidy. Codebase +is in good form: no architecturally load-bearing drift, all +tests green, two minor items recorded as acceptable. The next +iter can be a feature pick from the queue.