From e6298f5950e00395e94901eb31021334c2a4c83f Mon Sep 17 00:00:00 2001 From: Brummel Date: Thu, 14 May 2026 11:57:16 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20prelude-decouple=20=E2=80=94=20retire?= =?UTF-8?q?=20prelude.ail.json=20+=20decouple=20core=20from=20language=20c?= =?UTF-8?q?ontent?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Brainstorm settled on the β.2 loader-split: ailang-core stops embedding prelude bytes, stops auto-injecting a "prelude" module, and stops hardcoding the literal "prelude" in cross-module-ref diagnostics. ailang-surface assumes ownership of the prelude content and the reservation. Three iters: pd.1 splits the core API into load_modules_with + build_workspace; pd.2 moves the embed source from .ail.json to .ail with a cross-form-identity preflight; pd.3 deletes examples/prelude.ail.json and updates the form-a §C4 (b) carve-out. Grounding-check PASS twice (initial + post-self-review re-dispatch). --- docs/specs/2026-05-14-prelude-decouple.md | 446 ++++++++++++++++++++++ 1 file changed, 446 insertions(+) create mode 100644 docs/specs/2026-05-14-prelude-decouple.md diff --git a/docs/specs/2026-05-14-prelude-decouple.md b/docs/specs/2026-05-14-prelude-decouple.md new file mode 100644 index 0000000..2e8052f --- /dev/null +++ b/docs/specs/2026-05-14-prelude-decouple.md @@ -0,0 +1,446 @@ +# Prelude / core decoupling — Design Spec + +**Date:** 2026-05-14 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Retire `examples/prelude.ail.json` from the working tree and remove +all knowledge of the literal module name `"prelude"` from +`ailang-core`. After this milestone: + +1. The single source of truth for the prelude is + `examples/prelude.ail` (Form A). The compiler reads it via + `include_str!` in `ailang-surface` and parses it with + `ailang_surface::parse` at compile time of the embedding crate. +2. `ailang-core` no longer embeds prelude bytes, no longer defines + the `ReservedModuleName` error, no longer auto-injects a module + named `"prelude"`, and no longer hardcodes the string `"prelude"` + in cross-module-reference diagnostics. core's `Workspace` loader + becomes a pure module-graph + schema-validator with no language- + content awareness. +3. The §C4 (b) compile-time-embed carve-out in + `docs/specs/2026-05-13-form-a-default-authoring.md` becomes empty + and is struck. The seven §C4 (a) subject-matter-rejection + fixtures remain the only `.ail.json` files in the corpus. + +The motivation has two parts. First, the **doctrine inconsistency**: +CLAUDE.md states "authors write `.ail`; the build derives the JSON- +AST in-process via `ailang_surface::parse`". The prelude is the +single most-read AILang file in the corpus; that it is the *one* +file embedded into the compiler as JSON contradicts the doctrine. +Second, the **layering inconsistency**: `ailang-core`'s declared +role per CLAUDE.md is "AST, canonicalisation, desugaring, workspace +types, hash, pretty" — neither shipping language content nor +encoding the identity of a privileged module. The literal-`"prelude"` +references in core (8 occurrences across the inject step, the +diagnostic candidate sites, and the test cases) are the symptom of +that drift; this milestone removes them. + +Out of scope: + +- The implicit-prelude resolution in `ailang-check` (`crates/ailang-check/src/lib.rs:143`-area) + and the codegen-side implicit-prelude behaviour (iter 23.2.4). + Both already consume the prelude module by name from a populated + workspace; they do not embed prelude bytes and do not own the + reservation. They keep working unchanged because surface still + populates the prelude slot in every workspace. +- The `migrate-bare-cross-module-refs` subcommand's defensive + `include_str!` of `prelude.ail.json` at `crates/ail/src/main.rs:475` + — the inserted prelude is immediately skipped in the rewrite loop + (`if mod_name == "prelude" && path.as_os_str().is_empty() + { continue; }`), so this is a no-op relic. Iter 1 deletes it + along with the core-side embed. + +## Architecture + +The cut is the β.2 shape from the brainstorm: core becomes fully +prelude-agnostic. The DFS-visitor and the schema-validation pipeline +that today live as one block inside `core::workspace::load_workspace_with` +split into two publicly composable phases. Surface composes them +with a prelude-injection step in between. + +### Phase split + +**Today (in core):** + +``` +load_workspace_with(entry, loader) + DFS visit (loader-driven) + insert prelude module (hardcoded inject) + validate_canonical_type_names + validate_classdefs + build_registry + -> Workspace +``` + +**After the milestone:** + +``` +core::load_modules_with(entry, loader) + DFS visit (loader-driven) + -> (entry_name, root_dir, BTreeMap) + +surface::load_workspace(entry) + let (entry_name, root_dir, mut modules) = core::load_modules_with(entry, load_module) + if modules.contains_key("prelude") return Err(ReservedModuleName) + modules.insert("prelude", parse_prelude()) + core::build_workspace(entry_name, root_dir, modules, &["prelude"]) + +core::build_workspace(entry_name, root_dir, modules, implicit_imports) + validate_canonical_type_names(modules, implicit_imports) + validate_classdefs(modules, implicit_imports) + build_registry(modules) + -> Workspace +``` + +### Why two phases, not one + +A single `core::build_workspace` that takes already-assembled +modules works because every consumer of the validation pipeline +(canonical-type-names, classdefs, registry) needs the modules map +and the workspace shape — they do not need to know how those modules +got there. Splitting at the loader/validator seam lets surface +inject a special module without core knowing what makes it special. + +### Why `implicit_imports: &[&str]` parameter + +The diagnostic sites in `check_class_ref_qualifier` (line 1084-area) +and `check_type_con_name` (line 1148-area) currently scan +`local_classes.get("prelude")` / `local_types.get("prelude")` as a +fallback candidate when bare cross-module-refs fail to resolve. The +hardcoded `"prelude"` is the last sprach-konzept-as-string in core's +schema-validator path. + +Replacing it with an `implicit_imports: &[&str]` parameter on +`validate_canonical_type_names` and `validate_classdefs` parameterises +the concept. core walks both the per-module declared imports AND +the caller-supplied implicit imports without knowing what name +makes one implicit. Surface passes `&["prelude"]`. A future caller +could pass a different set without core needing to change. + +This is not speculative generality (rejected in the brainstorm for +β.3): it is the single mechanism that lets core stop hardcoding +`"prelude"`. Without parameterisation, the diagnostic sites would +either keep the hardcoded string (failing the milestone goal) or +the diagnostic candidate-list would lose its prelude-fallback +suggestion (a regression in error quality). + +### Why `ReservedModuleName` moves to surface + +The reservation `"prelude"` is an implementation choice of the +prelude-owning crate, not of the workspace loader. core has no +opinion on what module names are reserved; it discovers a graph +of named modules and validates their schema. Surface, which +injects a module named `"prelude"`, owns the reservation that +makes the injection safe (a user module of the same name would +shadow). The `WorkspaceLoadError` enum keeps the variant for ABI +stability; the variant moves to `ailang-surface` (or a new error +type composed with core's) — the exact placement is a Step-7 +detail, see Components. + +## Components + +The change touches exactly three crates: `ailang-core`, +`ailang-surface`, `ail` (CLI). Tests across the workspace adapt +mechanically. + +core's `src/` retains zero literal-`"prelude"` strings post-milestone. +The `WorkspaceLoadError::ReservedModuleName` variant declaration is +generic (`{ name: String }`) and does not name `"prelude"`; the +literal lives only at the single surface-side construction site. +This is the binding meaning of "core becomes prelude-agnostic" in +this spec. + +### `ailang-core` changes + +- `crates/ailang-core/src/workspace.rs`: + - **Remove**: `const PRELUDE_JSON` (line 417), `fn load_prelude` + (line 425), the inject step + reserved-name check inside + `load_workspace_with` (lines 493-504), the literal-`"prelude"` + fallbacks in `check_class_ref_qualifier` (lines 1084-1091) and + `check_type_con_name` (lines 1148-1155). + - **Replace** `pub fn load_workspace_with(entry, loader)` with + two new public fns: + - `pub fn load_modules_with(entry, loader) -> Result<(String, PathBuf, BTreeMap), WorkspaceLoadError>` + — returns `(entry_name, root_dir, modules)` with no + validation and no prelude. + - `pub fn build_workspace(entry_name, root_dir, modules, implicit_imports: &[&str]) -> Result` + — runs the three-stage validation + registry construction. + `implicit_imports` is forwarded to `validate_canonical_type_names`; + the other two pipeline stages do not use it. + - **Retire** `pub fn load_workspace(entry)` (the JSON-only legacy + wrapper at line 437). It has no production callers; the few + in-core tests that use it migrate to either `surface::load_workspace` + via the existing dev-dep edge or to direct `load_modules_with` + + `build_workspace` composition. + - **Move** the *production* of `WorkspaceLoadError::ReservedModuleName` + (line 315) out of core. The variant declaration stays in core's + `WorkspaceLoadError` enum — its name and shape + (`ReservedModuleName { name: String }`) reference no literal + `"prelude"`, so it does not by itself violate the + decoupling. core's `src/` contains zero call sites that + *construct* the variant after this milestone; surface's + `load_workspace` is the sole producer, supplying + `name: "prelude".to_string()`. Downstream match-arms in + CLI-diagnostic code are unaffected because the variant tag and + field shape are unchanged. + - **Update** `validate_canonical_type_names`'s signature to take + `implicit_imports: &[&str]`. Internal threading propagates the + slice through the per-def walk to the two diagnostic-site + helpers (`check_class_ref_qualifier` ~ line 1084 area, + `check_type_con_name` ~ line 1148 area). `validate_classdefs` + is **not** modified — it does not call those helpers and so has + no consumer of the literal-`"prelude"` fallback. (Spec's first + draft over-stated the threading scope; corrected here per the + grounding-check agent's finding.) + +- `crates/ailang-core/tests/workspace_pin.rs`: the in-tree DFS + expectations that today assume prelude is auto-injected get + rewritten to use `surface::load_workspace` (already imported via + dev-dep) — same expected outputs, different entry point. + +- `crates/ailang-core/tests/carve_out_inventory.rs`: remove the + `prelude.ail.json` entry from the §C4 (b) list (line 25). The + comment block at line 8 (`§C4 (b) compile-time-embed: 1 fixture + (prelude.ail.json).`) gets updated to "0 fixtures" and notes the + retirement. + +### `ailang-surface` changes + +- `crates/ailang-surface/src/loader.rs`: + - **Add** `const PRELUDE_AIL: &str = include_str!("../../../examples/prelude.ail");` + - **Add** `fn parse_prelude() -> Module` returning the parsed + prelude. Like the old `core::load_prelude`, this panics on + parse failure (build-time-validated). + - **Rewrite** `load_workspace(entry)` to compose the new core + primitives: + ```rust + pub fn load_workspace(entry: &Path) -> Result { + let (entry_name, root_dir, mut modules) = + ailang_core::load_modules_with(entry, load_module)?; + if modules.contains_key("prelude") { + return Err(WorkspaceLoadError::ReservedModuleName { + name: "prelude".to_string(), + }); + } + modules.insert("prelude".to_string(), parse_prelude()); + ailang_core::build_workspace(entry_name, root_dir, modules, &["prelude"]) + } + ``` + - The `load_module` extension-dispatcher (line 37) is unchanged. + +### `ail` (CLI) changes + +- `crates/ail/src/main.rs`: delete the defensive prelude-include + block in the `migrate-bare-cross-module-refs` subcommand + (lines 472-484). The synthetic prelude entry was inserted only + to satisfy the loop's per-module iteration, then unconditionally + skipped in the rewrite. Removing both the include and the skip + branch is net-zero behaviour with one fewer dependency on + `prelude.ail.json`. + +### Working-tree changes + +- **Delete** `examples/prelude.ail.json`. After this commit the + prelude exists on disk only as `examples/prelude.ail`. +- After form-a.1 (T9), the per-pair `every_ail_fixture_matches_its_json_counterpart` + test was retired (no production `.ail`/`.ail.json` pairs remain + in the corpus except prelude itself). The deterministic-parse + property (`parse_is_deterministic_over_every_ail_fixture` and + `parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` + in `crates/ailang-surface/tests/round_trip.rs`) stays as the + binding invariant for `prelude.ail`'s parse stability. +- **Cross-form-identity preflight** (mandatory before pd.2 swaps the + embed source — see Testing Strategy below for the mechanism): + capture `module_hash(serde_json::from_str(prelude.ail.json))` in + pd.2 BEFORE removing the JSON embed, and assert that + `module_hash(parse(prelude.ail))` equals it. Without this + preflight, the embed swap could silently change the canonical + prelude `Module` (different parse-path producing structurally + different AST) and propagate to every workspace's prelude + identity. + +## Data flow + +The runtime data flow does not change for any consumer. After the +milestone: + +``` +[examples/prelude.ail] + | + | include_str! (compile-time, into ailang-surface) + v +[PRELUDE_AIL: &'static str] + | + | parse_prelude() (called at every workspace load) + v +[Module] + | + v +surface::load_workspace + | + +--> core::load_modules_with # DFS + per-module load + | | + | v + | BTreeMap # no prelude yet + | | + | [check ReservedModuleName, insert prelude] + | | + | v + +--> core::build_workspace # validate + registry + | + v + Workspace # ready for check / codegen +``` + +`parse_prelude()` is called per workspace load (not memoised). +The prelude is ~120 lines; its parse cost is below the noise floor +of the rest of workspace loading (DFS reads, JSON deserialisation +of user modules, validation). If a future profiling pass shows it +as a hot spot, surface can introduce a `OnceLock` cache +locally without any API change. + +## Error handling + +### Existing error semantics — preserved + +- `WorkspaceLoadError::ReservedModuleName { name: "prelude" }` — + same shape, fired by surface instead of core. Match-arms in + diagnostic-rendering code are unaffected because the enum variant + itself is unchanged. +- `WorkspaceLoadError::BareCrossModuleClassRef` / + `BareCrossModuleTypeRef` candidate lists — same shape, same + contents. The prelude-fallback candidate is now produced by the + generic `implicit_imports` walk, not by the hardcoded string. + +### Build-time error: malformed `prelude.ail` + +If `examples/prelude.ail` ever becomes unparseable, `parse_prelude()` +panics — same failure mode as the old `load_prelude()`'s +`expect("must parse as a Module")`. This is acceptable because the +prelude is build-time-validated by every test run; a malformed +prelude is a build-correctness bug, not a runtime concern. + +### Migration error: stale callers of removed APIs + +The retired `core::load_workspace` and the old `core::load_workspace_with` +signature are not behind any deprecation shim. Out-of-tree callers +(if any existed) would break at compile time with a clear "function +not found" / "wrong arity" error, not silently. Per the brainstorm +this milestone is internal-API-only; no stability promise to preserve. + +## Testing strategy + +Three new pin tests, all in `crates/ailang-surface/tests/`: + +1. **`prelude_parse_pin.rs`** — pins that `parse_prelude()` succeeds + and produces a `Module` with `name == "prelude"` and an expected + minimum def-count (e.g. `>= 8` covering the four classes shipped + through milestones 22-24 plus their primitive instances). Guards + against `prelude.ail` being silently emptied. + +2. **`prelude_module_hash_pin.rs`** — pins the `module_hash` of the + parsed prelude against a literal hex string captured at iter + close. This is the round-trip-stability anchor: any change to + `prelude.ail` that does not also re-parse to the same canonical + `Module` value flips this test red. The hex updates intentionally + on prelude-content changes, with a JOURNAL entry naming why. + + **Cross-form-identity preflight** (added to the SAME test file as + the hash pin, in pd.2): before deleting the JSON embed source, + compute `let expected = module_hash(serde_json::from_str(PRELUDE_JSON))` + from the still-present `prelude.ail.json` bytes, then assert + `module_hash(parse(prelude.ail)) == expected`. This pins the + single load-bearing assumption that pd.2 introduces — that the + `.ail` parse path produces a `Module` structurally identical to + what the `.ail.json` deserialize path produces for the *same + prelude content*. The retired + `every_ail_fixture_matches_its_json_counterpart` test asserted + this for the entire corpus; this pin asserts it for the one file + that survives the corpus flip. The pin runs once (during pd.2), + then the JSON-half of the comparison disappears at pd.3 along + with the JSON file. The pin file itself transitions to + parse-only after pd.3. + +3. **`prelude_decouple_carve_out_pin.rs`** — asserts that the file + `examples/prelude.ail.json` does NOT exist. Pins the milestone's + stated artefact retirement; rewriting it back into the tree + surfaces immediately. + +Existing tests that continue to pin behaviour: + +- `crates/ailang-core/tests/workspace_pin.rs` — DFS + module-graph + shape pins. Updated to call `surface::load_workspace`; outputs + identical. +- `crates/ailang-core/tests/carve_out_inventory.rs` — list updated + to reflect §C4 (b) becoming empty. +- The ~22 `ailang_surface::load_workspace` callers across the + workspace (production code in `crates/ail/src/main.rs` plus tests + in `ailang-check/`, `ailang-codegen/`, `ail/tests/`, + `ailang-core/tests/`) — public entry point unchanged, so no test + body or assertion changes. Verified by per-crate + `cargo test --workspace` green at unchanged baselines. + +Regression baselines (`bench/check.py`, `bench/compile_check.py`, +`bench/cross_lang.py`) must stay green. None of them are expected +to regress because the runtime cost is identical (one parse per +workspace load, against a 120-line file). + +## Acceptance criteria + +The milestone closes when ALL of the following hold: + +1. `examples/prelude.ail.json` is removed from the working tree. +2. `cargo build --workspace` succeeds with no `include_str!` + reference to `prelude.ail.json` anywhere in the workspace. + Verified by `git grep -l 'prelude\.ail\.json' -- '*.rs'` + returning only test/comment hits, not `include_str!` sites. +3. `git grep -l '"prelude"' crates/ailang-core/src/` returns no + matches (zero literal-`"prelude"` strings in core's `src/`). +4. `cargo test --workspace` is green, including the three new + pins above. +5. `bench/check.py`, `bench/compile_check.py`, `bench/cross_lang.py` + are green at unchanged baselines. +6. `docs/specs/2026-05-13-form-a-default-authoring.md` §C4 (b) is + updated: the carve-out list is empty; the section either + deletes the (b) sub-bullet or notes "retired by + prelude-decouple milestone, 2026-05-14". +7. CLAUDE.md and `docs/DESIGN.md` reviewed: no statement still + asserts that core embeds the prelude. (Spot check — the + relevant CLAUDE.md sentence is the "authors write `.ail`" line, + which already aligns with the post-milestone state.) + +## Iteration sketch + +Tentative iter split (refined by `planner`): + +- **iter 1 — `pd.1` core API split.** Introduce `load_modules_with` + and `build_workspace`; refit `load_workspace_with` as a deprecated + alias if needed during the transition (or delete outright). + Thread `implicit_imports: &[&str]` through the two validation + functions and their internal helpers. Move `ReservedModuleName` + off the core construction path. core changes only; no surface + changes yet. Tests stay green using the old wiring (surface + unchanged). + +- **iter 2 — `pd.2` surface assumes ownership.** Add `PRELUDE_AIL` + + `parse_prelude` to `ailang-surface`. Rewrite + `surface::load_workspace` to compose the new core primitives and + inject the parsed prelude. Remove the prelude embed from core. + Add the three new pins. **Critical preflight**: install the + cross-form-identity test (see Testing Strategy above) BEFORE + deleting the core-side JSON embed. The test must be green with + both forms still present, so the swap is justified by an active + green pin and not by hope. + +- **iter 3 — `pd.3` retire `prelude.ail.json` + carve-out cleanup.** + Delete `examples/prelude.ail.json`. Delete the defensive include + in the migrate-subcommand. Update `carve_out_inventory.rs` and + the form-a-default-authoring spec §C4 (b). Run all benches. + +The split is for plan visibility; the orchestrator may collapse to +two iters if pd.1 and pd.2 turn out small enough that one cohesive +commit covers both. pd.3 is its own iter so the artefact retirement +is auditable in isolation.