diff --git a/docs/roadmap.md b/docs/roadmap.md index 9d05244..451aaf8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -89,6 +89,71 @@ context. Pick the next milestone from P1.)_ separate so migration runs against a frozen architecture. Same call milestone 23 made for `==` / `eq`. +- [ ] **\[todo\]** Author `examples/prelude.ail` alongside + `examples/prelude.ail.json`. Today the prelude exists only as + canonical-form JSON-AST; the textual surface form has to be + reconstructed via `ail render` every time someone wants to read + it. Check in the Form-A text as the authoring source and let + `ail parse` regenerate the JSON-AST at build time (or keep both + and gate equivalence via the existing round-trip invariant). + Either way the LLM-author's natural reading surface (Form A + text) becomes the entry point rather than a derived view. + - context: surfaced 2026-05-13 while demonstrating the + Show + print milestone via `ail prose`; the prelude is the + one file most likely to be read by humans and by LLM authors + learning the language, and it's the only fixture without a + `.ail` sibling. + +- [ ] **\[milestone\]** Form-A as the default authoring surface for + examples and docs. Render every `examples/*.ail.json` to its + `.ail` sibling via `ail render`, **delete the now-redundant + `.ail.json`**, and regenerate the JSON-AST per `ail parse` at + build / test time. Same for inline JSON-AST blocks in `docs/` + markdown (~7 in DESIGN.md plus ~29 other md files) — convert to + Form-A snippets where the snippet's purpose is to show "what the + language looks like", not "what the schema is". + + After this milestone the working tree contains exactly one + representation per program: the `.ail` source. The JSON-AST is a + build artefact, not a checked-in twin. Tests and benches that + currently glob `*.ail.json` either parse-then-consume or are + rewired to point at `.ail` directly. The round-trip invariant + flips role: today it gates that the two forms agree; afterwards + it gates that `parse` is deterministic. + + Carve-outs that MUST stay JSON-AST (no Form-A counterpart, and + no `.ail` sibling shall be created — these are the only seven + files that remain `.ail.json`-only post-milestone): + - Fixtures that test canonical-form rejection — Form A would + reject them at parse, defeating the test: + `test_ct1_bare_xmod_rejected.ail.json`, + `test_ct1_qualified_class_rejected.ail.json`, + `test_ct1_bad_qualifier.ail.json`, `broken_unbound.ail.json`, + `test_22b2_invalid_superclass_param.ail.json`, + `test_22b2_kind_mismatch.ail.json`, + `test_22b2_unbound_constraint_var.ail.json`. + - DESIGN.md schema documentation blocks where the JSON-AST shape + *is* the point (Decision 11, ParamMode, canonical-form + invariants). + - Any other case where the structured form is the artefact under + discussion, not a vehicle for a program. + + Touches CLAUDE.md ("source of truth is structured data") — the + language doctrine doesn't change (JSON-AST is still the canonical + hashable form), but the *authoring* doctrine does: authors write + `.ail`, the build derives JSON-AST. Sentence in CLAUDE.md needs + rewording in the same milestone. + + Mechanical for the bulk; per-fixture judgement call only for the + carve-out list. Run as one milestone so the corpus flip-over + happens at a single, audit-gated point. + - context: post-Form-A-as-canonical-authoring corollary of the + `examples/prelude.ail` entry above. Generalises the same idea + to the rest of the corpus and follows through on the cross- + model authoring data (textual form cheaper, more first-try + hits) by treating Form A as the privileged surface in the + working tree, not just in flavour text. + - [ ] **\[feature\]** Operator routing through `Eq` / `Ord` — `==`, `<` etc. resolved via the typeclass instead of the built-in primitive comparators. No commitment; gated on bench re-baselining diff --git a/docs/specs/2026-05-13-form-a-default-authoring.md b/docs/specs/2026-05-13-form-a-default-authoring.md new file mode 100644 index 0000000..ca7aae7 --- /dev/null +++ b/docs/specs/2026-05-13-form-a-default-authoring.md @@ -0,0 +1,462 @@ +# Form-A as the Default Authoring Surface — Design Spec + +**Date:** 2026-05-13 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Make Form A (`.ail`) the sole authoring surface for every program in +the working tree. After this milestone, the only `.ail.json` files +that remain in the repository are seven specific negative-test +fixtures whose subject matter cannot be expressed in Form A by +construction. Every other `.ail.json` is rendered to its `.ail` +sibling via `ail render`, and the now-redundant `.ail.json` is +**deleted** — not retained as a generated artefact, not stashed in a +build directory, not regenerated at test time. The working tree +contains exactly one representation per program: the `.ail` source. + +The language doctrine does not change. The canonical, hashable, +content-addressed form remains the JSON-AST (Decision 6, +architectural pin, §"data structure is the source of truth"). What +changes is the **authoring doctrine**: authors write `.ail`; the +build derives the JSON-AST in-process when a consumer needs it, +gated by the round-trip invariant. + +Driving evidence: the cross-model authoring-form study (audit-cma, +audit-ms, 2026-05-12) measured that Form A is cheaper in tokens +(~98k vs ~207k for the same four tasks) and produces strictly +more first-shot green programs than JSON-AST authoring, across two +distinct foreign LLMs (Qwen3-Coder-Next and CodeLlama-13b-Instruct). +The internal authoring practice is now drift from the empirical +"this is the form LLMs reach for unprompted" finding. + +## Architecture + +Four architectural decisions land in this spec; each is the +substantive answer to one of the open questions in the roadmap +entry. + +### A1. In-process parse, no build-time artefacts + +Tests and benches consume `.ail` fixtures by calling +`ailang_surface::parse` (library, in-process) — not by spawning +`ail parse` as a build prelude that emits `.ail.json` to a target +directory. + +**Why.** A build-prelude approach reintroduces a second +representation: `examples/foo.ail` (source) plus +`target/parsed/foo.ail.json` (derived). Even with the derived form +gitignored, it would be a working-tree fact during tests and a +shadow corpus for downstream tools to drift against. The single- +representation discipline that motivates this milestone is broken +the moment a derived `.ail.json` exists on disk anywhere a tool +might read it. + +The in-process path is also already plumbed: since iter ext-cli.1 +(2026-05-12), every path-taking CLI subcommand accepts `.ail` and +parses through `ailang_surface::parse` in-line. The Python bench +drivers (`bench/check.py`, `bench/compile_check.py`, +`bench/cross_lang.py`, `bench/mono_dispatch.py`) call `ail check` +etc. via subprocess and inherit the `.ail` acceptance for free — +they need only a path-suffix change. The integration tests under +`crates/ail/tests/*.rs` and the workspace tests under +`crates/ailang-check/tests/*.rs` currently `fs::read_to_string` a +`.ail.json` and pass the bytes to a loader; they will switch to +reading the `.ail` and calling `ailang_surface::parse` to recover +the same `Module` value. + +### A2. `prelude.ail` ships as iter-0 pilot + +Iteration 0 converts exactly one file: `examples/prelude.ail.json` +→ `examples/prelude.ail`. Nothing else moves. The roundtrip-CI gate +covers it (every `.ail` with a same-stem `.ail.json` counterpart is +already byte-compared by `every_ail_fixture_matches_its_json_counterpart`), +which proves the conversion path end-to-end before the bulk +migration commits to it. + +**Why.** Prelude exercises the widest cross-section of language +features in a single file: typeclasses with superclasses, polymorphic +free functions with constraints, IO-effect signatures, primitive +ctor patterns, and the loader's auto-injection path. If prelude +round-trips identically through `ail render → ail parse → canonical +bytes`, no other fixture in the corpus introduces a feature this +file does not. The pilot validates the migration mechanism on the +hardest single file before the bulk migration commits the test- +infrastructure refactor. + +Iter 0 keeps `prelude.ail.json` on disk alongside `prelude.ail` — +the deletion of redundant JSON-AST starts in iter 1 once the +test-infrastructure refactor lands. This is the *only* iteration +where a fixture is dual-form; it is justified by the existing +`every_ail_fixture_matches_its_json_counterpart` gate, which is +exactly the byte-comparison this iteration wants to leverage as a +witness. + +### A3. Per-file deletion cadence with iteration-end checkpoints + +`.ail.json` files are deleted on the same iteration that materialises +their `.ail` counterpart, not at milestone end. The bulk migration is +sequenced across ~3 iterations of ~30-40 files each, each iteration +fully reaping its own `.ail.json` files before close. There is no +intermediate state where the bulk of fixtures are dual-form. + +The exception is iter 0 (the prelude pilot, A2 above), which is +the singular dual-form window: the JSON counterpart is the witness +the existing CI gate uses, and the gate is the safety net for the +pilot itself. + +**Why.** A milestone-end bulk delete would mean the working tree +spends weeks in a "which form is the source?" ambiguity. Anyone +editing a fixture in the interim — to repair an unrelated bug, +add a regression test, fix a hash drift — has no canonical guide +to which form to edit. The single-representation property must hold +on a per-commit basis, not as a milestone-end snapshot. + +The trade-off is iteration sequencing pressure: every iteration that +ships in this milestone must leave the tree in a coherent +single-representation state. This is enforced by the per-iteration +roundtrip-CI rewrite (see Testing strategy below): once the +infrastructure flips in iter 1, fixtures without `.ail.json` are +the new normal, and fixtures with `.ail.json` are visibly in-flight. + +### A4. CLAUDE.md and DESIGN.md rewording + +Two documents need substantive rewording in this milestone. Both +edits land in the same commit as the iteration that retires the +last non-carve-out `.ail.json`. + +**CLAUDE.md** §"AILang — a language for LLM authors" opening: + +Currently: +> *Machine readability over human readability. Source of truth is +> structured data (`.ail.json`), not text.* + +After this milestone: +> *Machine readability over human readability. The canonical, +> hashable, content-addressed form is structured data (`.ail.json`); +> the authoring projection is Form A (`.ail`). Authors write `.ail`; +> the build derives the JSON-AST in-process via +> `ailang_surface::parse`, gated by the round-trip invariant. The +> two forms are byte-isomorphic — picking either does not change +> the identity of the module.* + +**DESIGN.md** §"What this Decision deliberately does not do" (line +465-466), currently: +> *It does not promote form (A) to the source of truth. Form (A) +> is one projection; the JSON-AST stays canonical.* + +The first sentence is replaced; the second stays. New text: +> *It does not change which form is canonical: the JSON-AST remains +> the hashable, content-addressed representation, and all hashing, +> content-addressing, and cross-module references flow through it +> unchanged. What did change in 2026-05-13 (this milestone) is +> which form is authored: authors now write Form A; the JSON-AST +> is materialised in-process by callers that need it. The two +> forms are byte-isomorphic by the round-trip invariant.* + +The "Form (A) is one projection; the JSON-AST stays canonical" +phrasing is preserved verbatim in the second sentence because it +remains true: Form A and the eventual visual / graphical +projections are still parallel projections of the same AST. What +the milestone changes is which projection is the **entry point**, +not the projection's relation to the AST. + +## Components + +### C1. Migration script + +A one-shot Bash script `scripts/migrate-to-form-a.sh` (new file) +walks `examples/*.ail.json`, skips the seven carve-outs (named +explicitly in the script body), invokes `ail render` to produce +the sibling `.ail`, and `rm`s the JSON counterpart. The script is +idempotent (re-running on a partially migrated tree is a no-op for +already-migrated files). The script is retired at milestone close +— it has no second use; keeping it would invite drift the next +time someone wonders "where did the JSON-AST come from". + +### C2. Test infrastructure refactor + +48 files under `crates/` and `bench/` reference `.ail.json` paths. +The refactor is mechanical but spans three classes of consumer: + +1. **Roundtrip tests** (`crates/ailang-surface/tests/round_trip.rs`, + `crates/ail/tests/roundtrip_cli.rs`). The five-test invariant + restructures (see Testing strategy §T below). +2. **Fixture-loading integration tests** (`crates/ail/tests/*.rs`, + ~35 files). Each test currently reads + `examples/.ail.json` with `fs::read_to_string` and + passes the bytes to `Module::from_json` or the workspace + loader. The refactor reads `examples/.ail` and parses via + `ailang_surface::parse_module_from_str`. The seven carve-outs + continue to use the `.ail.json` path (they have no `.ail` form + by construction). +3. **Bench drivers** (`bench/*.py`, `bench/run.sh`). The drivers + spawn `ail check` etc. with fixture paths. Since ext-cli.1 + the subcommands accept both extensions; the drivers' fixture + walks change from `glob("examples/*.ail.json")` to + `glob("examples/*.ail")` plus an explicit allow-list for the + carve-outs. + +### C3. Roundtrip invariant restatement + +§"Roundtrip Invariant" at DESIGN.md:2027 currently anchors on a +two-way `.ail.json` ↔ `.ail` bijection. After this milestone the +JSON-AST counterpart no longer exists for the bulk of the corpus, +so the invariant restates as: + +- **Parse-determinism.** For every well-formed `.ail` text `t`, + `parse(t)` produces a unique AST (the loader is a pure function; + no randomness, no time dependence, no environment leak). +- **Idempotency under print.** + `canonical(parse(t)) == canonical(parse(print(parse(t))))` — + printing and re-parsing is a no-op on the canonical bytes. +- **CLI-pipeline idempotency.** The public CLI pipeline `ail render + | ail parse` (one round) is byte-identical to direct + `ail parse` of the source `.ail`. Pins the user-facing CLI + against drift internal tests cannot see. +- **Carve-out anchor.** The seven `.ail.json`-only fixtures are + negative-tests (load-time / typecheck-time rejection); they + participate only in the existing rejection-shape tests, not in + the roundtrip gate. Their presence is the *only* surviving + reason `.ail.json` remains a first-class file extension. + +The "JSON → text → JSON" direction (Direction 1 in the current +spec) loses its corpus once the JSON files are deleted. The +property is replaced by parse-determinism plus idempotency — the +two together discharge the load-bearing concern (hashing is well- +defined for any `.ail` source). + +### C4. The seven carve-outs (immutable list) + +These seven files stay `.ail.json` post-milestone and gain no `.ail` +sibling: + +``` +examples/test_ct1_bare_xmod_rejected.ail.json +examples/test_ct1_qualified_class_rejected.ail.json +examples/test_ct1_bad_qualifier.ail.json +examples/broken_unbound.ail.json +examples/test_22b2_invalid_superclass_param.ail.json +examples/test_22b2_kind_mismatch.ail.json +examples/test_22b2_unbound_constraint_var.ail.json +``` + +Justification per file: +- `test_ct1_*` (3): assert canonical-form rejection of malformed + type references. The Form-A parser would either accept the + reference (test loses its subject) or reject it at parse time + with a different diagnostic (test loses its meaning). The + fixture's purpose is to test the **loader/checker** rejection + shape, which requires the input to bypass the parser. +- `broken_unbound`: asserts unbound-variable rejection at typecheck. + The `.ail.json` form encodes a structurally well-formed but + semantically broken module that the Form-A parser would also + parse cleanly; the file is retained in JSON-form not because the + surface cannot express it but because the test already pins the + exact JSON bytes, and re-deriving from `.ail` would add no value + while adding a regeneration step. +- `test_22b2_*` (3): assert class-definition rejection shapes + (invalid superclass param, kind mismatch, unbound constraint var) + that the Form-A parser would either reject at parse time + (changing the diagnostic) or refuse to encode at all (changing + the test subject). + +The orchestrator agrees these are the only legitimate carve-outs. +Any future addition to this list requires brainstorm-level +justification on the same semantic grounds. + +## Data flow + +Before this milestone: +``` +author writes: examples/foo.ail.json + | + v + [loader / serde_json] + | + v + Module value -> typecheck, codegen, ... +``` + +After iter 0 (prelude pilot only): +``` +author writes: examples/prelude.ail (new) + | + v + [ailang_surface::parse] + | + v + Module value -> typecheck, codegen, ... + + [byte-witness] + examples/prelude.ail.json ----+ (retained + | | for iter 0, + v | deleted iter 1) + [loader / serde_json] | + | | + v | + Module value -----> canonical -------+ +``` + +After milestone close (bulk migration done, JSON counterparts +deleted): +``` +author writes: examples/.ail (every program) + | + v + [ailang_surface::parse] + | + v + Module value -> typecheck, codegen, ... + +except for the seven carve-outs: + examples/.ail.json + | + v + [loader / serde_json] (negative-test only) +``` + +Cross-module imports continue to use module names, not file paths; +the workspace loader walks `examples/*` and accepts either extension +for any module it loads. Since the bulk of the corpus is single- +extension post-migration, the loader's extension-dispatch is exercised +only by the seven carve-outs (and by any test that explicitly mixes +the two forms for migration testing). + +## Error handling + +The migration is mechanical and the round-trip invariant +discharges correctness. There are three failure modes the spec +must name explicitly because they could escape the existing CI net: + +**E1. A fixture round-trips before migration but the migrated `.ail` +fails to parse.** This would be a bug in `ail render` or +`ail parse`; the round-trip CI gate +(`every_ail_fixture_matches_its_json_counterpart`) already covers +it for every fixture with a hand-written `.ail` sibling. The +migration script extends the same property to every fixture by +construction (the script's output is exactly the witness the gate +demands). If iter 0 (prelude pilot) fails this gate, the bug is +in render/parse; the milestone halts and the bug fix takes +precedence. + +**E2. A test that depends on the exact byte shape of `.ail.json` +breaks after migration.** Some tests (e.g. snapshot tests) compare +byte-equal canonical JSON to a checked-in expected blob. These +tests do not break — the canonical bytes are derived from the AST, +and the AST is form-agnostic. The migration touches only the +fixture-loading path, not the canonical-encoding path. + +**E3. A bench driver receives a `.ail` path and emits stale +`.ail.json` paths in its report.** The bench drivers' report format +echoes the fixture name; the suffix change from `.ail.json` to +`.ail` propagates into report bytes. The audit-* JSONs under +`bench/orchestrator-stats/` will start carrying `.ail` paths after +the next milestone-close audit; this is intended and not a +regression. + +A failure mode the spec explicitly does *not* worry about: parse +non-determinism. The roundtrip-CI gate has been green across 9 +consecutive audits (audit-cma onward); the parser is well- +exercised. + +## Testing strategy + +### T1. Roundtrip-CI test restatement + +`crates/ailang-surface/tests/round_trip.rs` currently runs three +tests against the `.ail.json` corpus. Post-migration, the three +restate: + +| Pre-migration test | Post-migration test | Behaviour | +|--------------------|---------------------|-----------| +| `print_then_parse_round_trips_every_fixture` (over `.ail.json`) | replaced by `parse_is_deterministic_over_every_ail_fixture` (over `.ail`) | Reads every `.ail`, parses twice, asserts canonical-byte equality between the two parses. Asserts pure-function-of-input. | +| `parse_then_print_then_parse_is_idempotent_on_every_ail_fixture` | unchanged | Reads every `.ail`, asserts `parse == parse ∘ print ∘ parse`. | +| `every_ail_fixture_matches_its_json_counterpart` | retired | Loses its corpus. The property it pinned (no drift between two hand-authored forms) is no longer expressible because only one form is hand-authored. | + +`crates/ail/tests/roundtrip_cli.rs::cli_render_then_parse_preserves_canonical_bytes_on_every_fixture` +restates as `cli_parse_then_render_then_parse_is_idempotent`: +reads every `.ail`, runs `ail parse` (to canonical JSON tempfile), +runs `ail render` (back to `.ail` tempfile), runs `ail parse` +again, and asserts byte equality across the two parses. + +`crates/ailang-core/tests/schema_coverage.rs::every_ast_variant_is_observed_in_the_fixture_corpus` +flips its corpus walk from `.ail.json` to `.ail` (load via +`ailang_surface::parse_module_from_str`); the assertion is +unchanged. The schema-coverage property is the strictest gate +that survives: a new AST variant cannot land without a fixture +that uses it. + +### T2. Per-iteration regression discipline + +Every iteration in this milestone must leave `cargo test +--workspace` green and `bench/compile_check.py` exit-0. The +migration is mechanical; non-green at iteration close is a defect +in the iteration's test-infrastructure refactor, not in the +language. The iteration does not close until the refactor is +green. + +### T3. Test-naming discipline for the migration + +A few existing tests carry `.ail.json` in their test-function names +(e.g. `mono_recursive_fn_loads_from_json_ast`). These names are +renamed in the same iteration that migrates the corresponding +fixture: the new name should not lie about what file it reads. +This is mechanical, but it has to land in the migration commit, not +as a follow-up rename. + +### T4. Carve-out coverage check + +A new test `crates/ailang-core/tests/carve_out_inventory.rs` +asserts that exactly the seven named carve-out files exist under +`examples/*.ail.json` after milestone close. The test fails if a +new `.ail.json` is added or a carve-out goes missing. The list +is hardcoded in the test body — this is intentional: the carve-out +list is part of the language's negative-test surface and any change +to it is a deliberate, brainstorm-level decision. + +## Acceptance criteria + +The milestone closes when all of the following are true: + +1. `ls examples/*.ail.json | wc -l` returns `7`, and the seven + files match the carve-out list verbatim. +2. Every program in `examples/` other than the seven carve-outs + exists as a `.ail` file with no `.ail.json` sibling. +3. The roundtrip-CI restatement (T1) is green: parse is + deterministic, idempotent under print, CLI-pipeline idempotent, + and the carve-out inventory test (T4) passes. +4. `cargo test --workspace` is green at milestone close (no + regressions introduced by the test-infrastructure refactor). +5. `bench/compile_check.py` and `bench/cross_lang.py` exit 0 + against the migrated corpus. +6. `bench/check.py` is either exit 0 or exit 1 with audit- + ratifiable noise only (consistent with the 9-audit lineage + since audit-cma); a new regression class attributable to the + migration triggers brainstorm-level escalation. +7. CLAUDE.md and DESIGN.md carry the reworded paragraphs (§A4 + above) verbatim. +8. The migration script `scripts/migrate-to-form-a.sh` is deleted + from the working tree (one-shot, retired). +9. `docs/WhatsNew.md` carries a milestone-close entry framed for + the user-as-reader: what changed in the authoring experience, + not what changed in the internal representation. +10. The `[milestone] Form-A as the default authoring surface` + entry in `docs/roadmap.md` is checked off and removed. + +## Iteration scope + +The first iteration covers **§A2 (prelude pilot)** only: +- Render `examples/prelude.ail.json` → `examples/prelude.ail` via + `ail render`. +- Do not delete `prelude.ail.json` yet (iter 1 deletes it + alongside the bulk migration of test infrastructure). +- Verify the existing + `every_ail_fixture_matches_its_json_counterpart` test is green + on the new `.ail`. +- No test-infrastructure changes; no CLAUDE.md / DESIGN.md edits + yet. + +Iter 1 onward is out of scope for the first plan; the planner +re-derives iter scope from this spec each iteration.