diff --git a/docs/plans/cma.1.md b/docs/plans/cma.1.md new file mode 100644 index 0000000..d075d35 --- /dev/null +++ b/docs/plans/cma.1.md @@ -0,0 +1,1226 @@ +# cma.1 — Implementation Plan + +> **Parent spec:** `docs/specs/2026-05-12-cross-model-authoring-form-test.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use `skills/implement` +> to run this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Stand up the master mini-spec source and the renderer binary +under a new top-level `experiments/2026-05-12-cross-model-authoring/` +directory, with three green test gates (example roundtrip, token +balance, spec completeness) and the two rendered mini-spec files +checked into the working tree. + +**Architecture:** A new top-level dir parallels `crates/`, `bench/`, +`examples/`, `runtime/`, `docs/`. Inside it, `render/` is a standalone +Cargo crate that is **not** a member of the root workspace; it depends +on `ailang-core` and `ailang-surface` by relative path. The renderer +consumes `master/spec.md` (markdown with `{form-only: X}` blocks and +`{example: id}` markers) plus `master/examples/*.ail.json` (curated +AST fixtures) and emits `rendered/json.md` + `rendered/ailx.md`. +Examples are printed in JSON form by emitting the on-disk +canonical-key-order bytes, and in AILX form by `ailang_core::load_module` +→ `ailang_surface::print`. + +**Tech Stack:** Rust 2021, `ailang-core::{load_module, canonical, +module_hash}`, `ailang-surface::{print, parse}`, `serde_json`, +`anyhow`. Dev-only: `tiktoken-rs` (used by `token_balance` test). + +**Files this plan creates or modifies:** + +- Create: `experiments/2026-05-12-cross-model-authoring/README.md` — purpose + how to run the renderer +- Create: `experiments/2026-05-12-cross-model-authoring/render/Cargo.toml` — standalone crate manifest +- Create: `experiments/2026-05-12-cross-model-authoring/render/src/main.rs` — renderer entry point +- Create: `experiments/2026-05-12-cross-model-authoring/render/src/splitter.rs` — directive parser module +- Create: `experiments/2026-05-12-cross-model-authoring/render/src/examples.rs` — example-id → form-projection +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/splitter_unit.rs` — directive-parser tests +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/example_roundtrip.rs` — mirrors `round_trip_one` for `master/examples/` +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/spec_completeness.rs` — borrowed schema-coverage visitor +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/token_balance.rs` — ±5% form-only block balance +- Create: `experiments/2026-05-12-cross-model-authoring/master/spec.md` — vollumfänglich mini-spec source +- Create: `experiments/2026-05-12-cross-model-authoring/master/examples/*.ail.json` — curated AST fixtures (one per variant cluster; final filename list emerges from Task 4) +- Create: `experiments/2026-05-12-cross-model-authoring/rendered/json.md` — generated; checked in +- Create: `experiments/2026-05-12-cross-model-authoring/rendered/ailx.md` — generated; checked in +- Modify: none. Root `Cargo.toml` is **not** touched (spec §Architecture lines 90–95). + +--- + +## Task 1: Bootstrap experiment dir + Cargo skeleton + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/README.md` +- Create: `experiments/2026-05-12-cross-model-authoring/render/Cargo.toml` +- Create: `experiments/2026-05-12-cross-model-authoring/render/src/main.rs` + +- [ ] **Step 1.1: Create the directory structure** + +Run: +``` +mkdir -p experiments/2026-05-12-cross-model-authoring/render/src +mkdir -p experiments/2026-05-12-cross-model-authoring/render/tests +mkdir -p experiments/2026-05-12-cross-model-authoring/master/examples +mkdir -p experiments/2026-05-12-cross-model-authoring/rendered +``` +Expected: four empty directories created, no errors. + +- [ ] **Step 1.2: Write README.md** + +Write `experiments/2026-05-12-cross-model-authoring/README.md`: + +```markdown +# Cross-model authoring-form test + +Empirical measurement of whether `.ail.json` or `.ailx` is the form a +foreign LLM author reaches for and succeeds with. Single subject for +v1: Qwen3-Coder-Next via IONOS. Two blind cohorts; same four tasks. + +Parent spec: `docs/specs/2026-05-12-cross-model-authoring-form-test.md`. + +## Layout + +- `master/spec.md` — canonical mini-spec source (form-agnostic prose + + `{form-only: X}` blocks + `{example: id}` markers). +- `master/examples/*.ail.json` — AST source-of-truth; each example + prints to either form via the existing roundtrip machinery. +- `master/tasks/*.task.json` — task definitions consumed by the + harness. **Authored in cma.2**, not cma.1. +- `render/` — standalone Cargo crate, outside the root workspace, + builds the renderer binary. +- `rendered/json.md`, `rendered/ailx.md` — projected mini-specs, + checked into the repo for review. +- `harness/` — **Authored in cma.2**. +- `runs/-/` — populated by `harness` during a live run. + +## Running the renderer + +``` +cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml -- \ + --master experiments/2026-05-12-cross-model-authoring/master \ + --rendered experiments/2026-05-12-cross-model-authoring/rendered +``` + +## Running the tests + +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml +``` + +Three integration tests: `example_roundtrip` (each example loads, +prints to AILX, reparses to the same canonical bytes), `spec_completeness` +(every AST variant in `ailang_core::ast` is exercised by at least +one example), `token_balance` (form-only blocks balanced within ±5% +across the two rendered files). +``` + +- [ ] **Step 1.3: Write render/Cargo.toml** + +Write `experiments/2026-05-12-cross-model-authoring/render/Cargo.toml`: + +```toml +[package] +name = "xmodel-render" +version = "0.0.1" +edition = "2021" +publish = false + +[[bin]] +name = "xmodel-render" +path = "src/main.rs" + +[dependencies] +ailang-core = { path = "../../../crates/ailang-core" } +ailang-surface = { path = "../../../crates/ailang-surface" } +serde_json = "1" +anyhow = "1" + +[dev-dependencies] +tiktoken-rs = "0.6" +``` + +- [ ] **Step 1.4: Write skeleton src/main.rs** + +Write `experiments/2026-05-12-cross-model-authoring/render/src/main.rs`: + +```rust +//! xmodel-render — renders the master mini-spec into two form-specific projections. + +use anyhow::Result; + +mod splitter; +mod examples; + +fn main() -> Result<()> { + eprintln!("xmodel-render: not yet wired (Task 7)"); + Ok(()) +} +``` + +Create the two empty `.rs` files referenced (will be filled in by +later tasks): + +``` +touch experiments/2026-05-12-cross-model-authoring/render/src/splitter.rs +touch experiments/2026-05-12-cross-model-authoring/render/src/examples.rs +``` + +- [ ] **Step 1.5: Verify the skeleton builds** + +Run: +``` +cargo build --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml +``` +Expected: succeeds. Two warnings expected (unused `mod splitter;` and +unused `mod examples;`) — these will go once Task 2 and Task 4 wire +the modules in. + +--- + +## Task 2: Directive splitter (TDD) + +**Files:** +- Modify: `experiments/2026-05-12-cross-model-authoring/render/src/splitter.rs` +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/splitter_unit.rs` + +The splitter recognises three directives in `master/spec.md`: + +- `{form-only: json}` … `{/form-only}` — block visible only in + JSON-rendered output. +- `{form-only: ailx}` … `{/form-only}` — block visible only in + AILX-rendered output. +- `{example: }` — single-line marker that the example renderer + resolves to a fenced code block of the matching form. + +Directives appear on their own lines. The parser is line-oriented +(no nested directives, no inline directives). + +- [ ] **Step 2.1: Write the splitter unit tests (RED)** + +Write `experiments/2026-05-12-cross-model-authoring/render/tests/splitter_unit.rs`: + +```rust +use xmodel_render::splitter::{split, Form, Segment}; + +#[test] +fn plain_text_passes_through_unchanged() { + let input = "hello\nworld\n"; + let segments = split(input); + assert_eq!(segments, vec![Segment::Prose("hello\nworld\n".to_string())]); +} + +#[test] +fn form_only_block_is_isolated() { + let input = "before\n{form-only: json}\nschema rule\n{/form-only}\nafter\n"; + let segments = split(input); + assert_eq!( + segments, + vec![ + Segment::Prose("before\n".to_string()), + Segment::FormOnly { form: Form::Json, body: "schema rule\n".to_string() }, + Segment::Prose("after\n".to_string()), + ] + ); +} + +#[test] +fn ailx_form_only_block_uses_ailx_form() { + let input = "{form-only: ailx}\ngrammar rule\n{/form-only}\n"; + let segments = split(input); + assert_eq!( + segments, + vec![ + Segment::FormOnly { form: Form::Ailx, body: "grammar rule\n".to_string() }, + ] + ); +} + +#[test] +fn example_marker_is_isolated() { + let input = "before\n{example: empty_module}\nafter\n"; + let segments = split(input); + assert_eq!( + segments, + vec![ + Segment::Prose("before\n".to_string()), + Segment::Example("empty_module".to_string()), + Segment::Prose("after\n".to_string()), + ] + ); +} + +#[test] +fn multiple_directives_in_sequence() { + let input = "p1\n{example: a}\n{form-only: json}\njs\n{/form-only}\n{form-only: ailx}\nax\n{/form-only}\n{example: b}\np2\n"; + let segments = split(input); + assert_eq!( + segments, + vec![ + Segment::Prose("p1\n".to_string()), + Segment::Example("a".to_string()), + Segment::FormOnly { form: Form::Json, body: "js\n".to_string() }, + Segment::FormOnly { form: Form::Ailx, body: "ax\n".to_string() }, + Segment::Example("b".to_string()), + Segment::Prose("p2\n".to_string()), + ] + ); +} + +#[test] +fn unknown_form_name_is_an_error() { + let input = "{form-only: yaml}\nx\n{/form-only}\n"; + let result = std::panic::catch_unwind(|| split(input)); + assert!(result.is_err(), "split should panic on unknown form name"); +} + +#[test] +fn unclosed_form_only_is_an_error() { + let input = "{form-only: json}\nopen\n"; + let result = std::panic::catch_unwind(|| split(input)); + assert!(result.is_err(), "split should panic on unclosed form-only block"); +} +``` + +The test file imports `xmodel_render::splitter::{split, Form, Segment}`, +which forces `src/main.rs` to expose the splitter module via a library +target. Without a library target this fails to compile. + +- [ ] **Step 2.2: Add a library target to render/Cargo.toml + lib.rs** + +Add to `experiments/2026-05-12-cross-model-authoring/render/Cargo.toml`, +after the existing `[[bin]]` block: + +```toml +[lib] +name = "xmodel_render" +path = "src/lib.rs" +``` + +Create `experiments/2026-05-12-cross-model-authoring/render/src/lib.rs`: + +```rust +//! Library surface for xmodel-render's internal modules — exposed so +//! integration tests can reach them. + +pub mod splitter; +pub mod examples; +``` + +Update `experiments/2026-05-12-cross-model-authoring/render/src/main.rs` +to use the library (replace the existing `mod splitter;` / `mod examples;` +lines with): + +```rust +//! xmodel-render — renders the master mini-spec into two form-specific projections. + +use anyhow::Result; +use xmodel_render::{splitter, examples}; + +fn main() -> Result<()> { + let _ = (&splitter::Form::Json, &examples::Form::Json); // silence unused until Task 7 + eprintln!("xmodel-render: not yet wired (Task 7)"); + Ok(()) +} +``` + +(Two `Form` enums avoid silent aliasing later; the bin reconciles them +at wire-up in Task 7. For now, just silence unused-import warnings.) + +- [ ] **Step 2.3: Run the tests, verify RED** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test splitter_unit +``` +Expected: **compilation fails** because `splitter::{split, Form, Segment}` +do not exist yet. Output contains "cannot find function `split` in module +`splitter`" or equivalent. + +- [ ] **Step 2.4: Write the splitter implementation (GREEN)** + +Replace `experiments/2026-05-12-cross-model-authoring/render/src/splitter.rs` +content with: + +```rust +//! Directive splitter for master/spec.md. +//! +//! Recognises three directives, one per line, no nesting: +//! {form-only: json} … {/form-only} +//! {form-only: ailx} … {/form-only} +//! {example: } + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Form { + Json, + Ailx, +} + +impl Form { + fn parse(name: &str) -> Option
{ + match name { + "json" => Some(Form::Json), + "ailx" => Some(Form::Ailx), + _ => None, + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Segment { + Prose(String), + FormOnly { form: Form, body: String }, + Example(String), +} + +/// Split a master/spec.md text into a sequence of segments. +/// +/// Panics on malformed input (unknown form name, unclosed form-only block). +/// The renderer treats these as authoring bugs that should fail the test +/// suite rather than silently produce diverging mini-specs. +pub fn split(input: &str) -> Vec { + let mut out: Vec = Vec::new(); + let mut current_prose = String::new(); + let mut current_block: Option<(Form, String)> = None; + + for line in input.split_inclusive('\n') { + let trimmed = line.trim_end_matches('\n'); + + // Inside a form-only block: collect until the closing marker. + if let Some((form, body)) = current_block.as_mut() { + if trimmed == "{/form-only}" { + let body_owned = std::mem::take(body); + out.push(Segment::FormOnly { form: *form, body: body_owned }); + current_block = None; + } else { + body.push_str(line); + } + continue; + } + + // {form-only: } + if let Some(rest) = trimmed.strip_prefix("{form-only: ").and_then(|r| r.strip_suffix("}")) { + flush_prose(&mut current_prose, &mut out); + let form = Form::parse(rest) + .unwrap_or_else(|| panic!("unknown form name in directive: {}", rest)); + current_block = Some((form, String::new())); + continue; + } + + // {example: } + if let Some(id) = trimmed.strip_prefix("{example: ").and_then(|r| r.strip_suffix("}")) { + flush_prose(&mut current_prose, &mut out); + out.push(Segment::Example(id.to_string())); + continue; + } + + // Stray closer with no opener. + if trimmed == "{/form-only}" { + panic!("stray {{/form-only}} without matching opener"); + } + + // Plain prose line. + current_prose.push_str(line); + } + + if current_block.is_some() { + panic!("unclosed form-only block at end of input"); + } + flush_prose(&mut current_prose, &mut out); + out +} + +fn flush_prose(buf: &mut String, out: &mut Vec) { + if !buf.is_empty() { + out.push(Segment::Prose(std::mem::take(buf))); + } +} +``` + +- [ ] **Step 2.5: Run the tests, verify GREEN** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test splitter_unit +``` +Expected: `test result: ok. 7 passed; 0 failed`. + +--- + +## Task 3: spec_completeness test (RED on empty examples/) + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/spec_completeness.rs` + +This task lifts the AST-variant visitor from +`crates/ailang-core/tests/schema_coverage.rs` verbatim, adjusts the +`examples_dir()` resolver to point at `experiments/.../master/examples/`, +and runs the test to confirm it fails on the empty examples directory. +Task 4 will then add fixtures to drive it green. + +- [ ] **Step 3.1: Copy the visitor verbatim from schema_coverage.rs** + +Read `crates/ailang-core/tests/schema_coverage.rs` lines 1–399 (the +entire file). + +Copy the contents into a new file +`experiments/2026-05-12-cross-model-authoring/render/tests/spec_completeness.rs` +verbatim with one adjustment: replace the body of the `examples_dir()` +function (originally at source lines 324–327) with: + +```rust +fn examples_dir() -> std::path::PathBuf { + // CARGO_MANIFEST_DIR for this test is + // experiments/2026-05-12-cross-model-authoring/render/ + // Master examples live one parent up + master/examples/. + let manifest = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); + manifest.parent().unwrap().join("master").join("examples") +} +``` + +Rename the driver test from `every_ast_variant_is_observed_in_the_fixture_corpus` +to `every_ast_variant_is_observed_in_master_examples` so the failure +output names the right scope. + +The file ends up structurally identical to schema_coverage.rs except +for `examples_dir()` and the test name. The visitor logic, the +`VariantTag` enum, the `EXPECTED_VARIANTS` constant, and the +`visit_*` functions are unchanged. + +- [ ] **Step 3.2: Run the test, verify RED** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test spec_completeness +``` +Expected: the test fails. Failure mode is one of: +- "no fixtures found in `experiments/.../master/examples/`" if the + visitor pre-checks fixture count (verify behaviour against + `schema_coverage.rs:329–344` — `list_json_fixtures` returns an empty + Vec without erroring, but the variant-observation set is empty so + the assertion at lines 380+ fires with "missing variants: …"). +- "missing variants: " — the body fires + because `observed` is empty. + +Either way: the test is RED. Record the exact failure message in the +implement journal so Task 4 can confirm the change in failure shape +once examples land. + +--- + +## Task 4: master/examples/*.ail.json fixtures + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/master/examples/.ail.json` (multiple files; see Step 4.1 for the curated list) + +The goal is the smallest set of curated, didactic `.ail.json` files +that together exercise all 33 AST variants the `EXPECTED_VARIANTS` +constant lists (5 Def + 13 Term + 4 Pattern + 5 Literal + 4 Type + 3 +ParamMode — note the visitor counts 33 because `Module` itself is +implicit on every file). Each fixture must be a complete, well-formed +AILang module that loads cleanly via `ailang_core::load_module`. + +**Common shape constraints** (apply to every fixture): + +- Top-level JSON object with `"schema": "ailang/v0"`, `"name": + "_module"`, `"imports": []` (unless explicitly noted), + `"defs": [...]`. +- Canonical key order — fields appear in the order the existing + workspace fixtures use (the implementer reads any one of + `examples/*.ail.json` for the precedent). +- Every `Def::Fn` carries mandatory `params`, `param_tys` (with + ParamMode), `ret_ty`, and `effects` per DESIGN.md §"Data model". +- The fixture must load cleanly via `ailang_core::load_module` (the + spec_completeness test calls it on every file). +- The fixture must roundtrip via `ailang_surface::print` → + `ailang_surface::parse` → canonical-byte equality (Task 5 enforces + this). If a fixture cannot roundtrip, the .ail.json embeds + something the surface printer cannot faithfully reproduce — fix + the fixture, not the test. + +The implementer reads `crates/ailang-core/src/ast.rs` to confirm the +exact ParamMode variant names (the planner does not pin those +because the recon report did not enumerate them); the structural +template for each fixture's shape can be lifted from a small +existing fixture in `examples/` (e.g. `box.ail.json` for the +Data/TermCtor shape; `list_map_poly.ail.json` for Forall + Match; +the workspace has 136 candidates). + +- [ ] **Step 4.1: Author `fn_returns_int.ail.json`** + +Single fn `f : () -> Int = 42` (or pick a non-zero integer the +implementer prefers). Exercises: Module, Def::Fn, Term::Lit, Type::Con, +Literal::Int, one ParamMode tag on the (empty) param list — verify +the empty-params shape carries a ParamMode marker; if it doesn't, +fall to `Step 4.13`'s coverage for ParamMode. + +- [ ] **Step 4.2: Author `fn_calls_prelude.ail.json`** + +A fn that takes two Int parameters and returns their sum by calling +a prelude function (the implementer picks the actual symbol — likely +`int_add` or its current canonical name — by reading +`crates/ailang-check/src/lib.rs` or the prelude module). Exercises: +Term::App, Term::Var, a populated ParamMode set on the params. + +- [ ] **Step 4.3: Author `fn_with_lambda.ail.json`** + +A fn that returns a lambda value (e.g. `make_adder : (Int) -> +((Int) -> Int) = \x. \y. int_add(x, y)`). Exercises: Term::Lam, +Type::Fn as a return type. + +- [ ] **Step 4.4: Author `fn_with_do_seq.ail.json`** + +A fn `main : () -> () !IO` whose body is a `do { … }`-style sequence +of two `io_print_int` calls and a unit return. Exercises: Term::Do, +Term::Seq, the IO effect on the fn's effects set, Literal::Unit (if +that variant is the discriminator the visitor recognises; otherwise +it is covered by the unit return value). + +- [ ] **Step 4.5: Author `data_simple.ail.json`** + +A `data Box a where MkBox a` declaration plus a fn that constructs +a `Box Int` via `MkBox(1)`. Exercises: Def::Data, Term::TermCtor, +Type::Var in the ctor's argument position. + +- [ ] **Step 4.6: Author `data_with_match.ail.json`** + +A `data List a where Nil | Cons a (List a)` declaration plus a fn +that pattern-matches on a value: `match xs of Nil -> 0 | Cons _ _ +-> 1`. Exercises: Term::Match with two arms, Pattern::Ctor (both +nullary `Nil` and binary `Cons`), Pattern::Wildcard (on the two +Cons-field positions). + +- [ ] **Step 4.7: Author `match_literal_pattern.ail.json`** + +A fn that matches an Int parameter against literal patterns: +`match n of 0 -> "zero" | _ -> "other"`. Exercises: Pattern::Lit and +Pattern::Wildcard. If the visitor treats variable patterns separately +from wildcard, extend one arm to `| n2 -> …` to also cover +Pattern::Var (if not already covered by Step 4.6). + +- [ ] **Step 4.8: Author `forall_polymorphic.ail.json`** + +A polymorphic identity fn: `fn id : forall a. (a) -> a = \x. x`. +Exercises: Type::Forall, Type::Var in the fn signature. + +- [ ] **Step 4.9: Author `class_def.ail.json`** + +A `class MyShow a where show : (a) -> Str` declaration. Exercises: +Def::Class, the class method's type signature. + +- [ ] **Step 4.10: Author `instance_def.ail.json`** + +An instance `instance MyShow Int where show = \x. "an Int"` (or the +real `MyShow` defined in Step 4.9; the implementer ensures the two +files agree on the class name). If AILang's instance form requires +a constraint on a polymorphic instance, add a fn-type with a +Constraint in this fixture; otherwise add a separate trivial fn +`use_eq : forall a. (Eq a) => (a, a) -> Bool = \x y. eq(x, y)` to +exercise the Constraint type-form. Exercises: Def::Instance, the +Constraint variant of Type. + +- [ ] **Step 4.11: Author `floats.ail.json`** + +A fn returning a specific Float literal: `fn pi : () -> Float = +3.14`. Exercises: Literal::Float, Type::Con (Float). Note: the +authored decimal is what the model writes; the toolchain handles +the bit-hex encoding internally (this is what the spec.md will +teach in Section 9 — see Task 6). + +- [ ] **Step 4.12: Author `bool_str.ail.json`** + +Two trivial fns: `is_true : () -> Bool = true` and `greeting : () +-> Str = "hi"`. Exercises: Literal::Bool, Literal::Str. + +- [ ] **Step 4.13: Author `param_modes_all.ail.json`** + +Three fns, each with one parameter exercising one of the three +ParamMode variants. The implementer reads +`crates/ailang-core/src/ast.rs` to extract the exact mode names +(planner not pinning, recon did not enumerate). Each fn is trivial +(`fn f_ : (Int) -> Int = \x. x`); the load-bearing thing is +the ParamMode tag on each parameter. + +- [ ] **Step 4.14: Run spec_completeness, verify GREEN** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test spec_completeness +``` +Expected: `test result: ok. 1 passed; 0 failed`. + +If specific variants are still missing, the failure message names +them. Extend the fixture closest in scope (e.g. add a Term::Seq use +to Step 4.4's fixture, add a new pattern arm to Step 4.6's fixture), +do not weaken the test. If a variant is *structurally unreachable* +through any fixture an LLM-author would produce, that is a real +finding worth flagging in the implement journal — but the audit-rt +journal records all 34 variants are observed by the existing +schema_coverage corpus, so the curated subset should reach them +too with one fixture each. + +--- + +## Task 5: example_roundtrip test + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/example_roundtrip.rs` + +This test mirrors `crates/ailang-surface/tests/round_trip.rs::round_trip_one` +for the subset of fixtures under `master/examples/`. It is the local +gate ensuring every example printed by the renderer is faithful. + +- [ ] **Step 5.1: Write the roundtrip test** + +Write `experiments/2026-05-12-cross-model-authoring/render/tests/example_roundtrip.rs`: + +```rust +//! Roundtrip gate for master/examples/. +//! +//! Mirrors crates/ailang-surface/tests/round_trip.rs but scoped to +//! the master examples directory. For each .ail.json fixture: load, +//! print via ailang_surface::print, reparse, canonicalise both sides, +//! assert byte equality. + +use std::path::{Path, PathBuf}; + +fn examples_dir() -> PathBuf { + let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + manifest.parent().unwrap().join("master").join("examples") +} + +fn list_examples() -> Vec { + let mut paths: Vec = std::fs::read_dir(examples_dir()) + .expect("master/examples/ should exist") + .filter_map(|entry| entry.ok()) + .map(|entry| entry.path()) + .filter(|path| path.extension().and_then(|s| s.to_str()) == Some("json")) + .filter(|path| { + path.file_name() + .and_then(|s| s.to_str()) + .map(|name| name.ends_with(".ail.json")) + .unwrap_or(false) + }) + .collect(); + paths.sort(); + paths +} + +fn round_trip_one(path: &Path) -> Result<(), String> { + let original = ailang_core::load_module(path) + .map_err(|e| format!("{}: load_module failed: {e}", path.display()))?; + let text = ailang_surface::print(&original); + let parsed = ailang_surface::parse(&text) + .map_err(|e| format!("{}: surface::parse failed: {e}", path.display()))?; + let bytes_original = ailang_core::canonical::to_bytes(&original); + let bytes_parsed = ailang_core::canonical::to_bytes(&parsed); + if bytes_original != bytes_parsed { + return Err(format!( + "{}: roundtrip diverged.\n--- printed AILX ---\n{}\n--- original bytes len {} vs parsed bytes len {} ---", + path.display(), + text, + bytes_original.len(), + bytes_parsed.len(), + )); + } + Ok(()) +} + +#[test] +fn every_example_roundtrips_via_ailx() { + let examples = list_examples(); + assert!( + !examples.is_empty(), + "master/examples/ should contain at least one .ail.json fixture" + ); + let mut failures: Vec = Vec::new(); + for path in &examples { + if let Err(msg) = round_trip_one(path) { + failures.push(msg); + } + } + if !failures.is_empty() { + panic!( + "{} of {} examples failed roundtrip:\n\n{}", + failures.len(), + examples.len(), + failures.join("\n\n") + ); + } +} +``` + +- [ ] **Step 5.2: Run the roundtrip test, verify GREEN** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test example_roundtrip +``` +Expected: `test result: ok. 1 passed; 0 failed`. + +If any fixture fails the roundtrip: that's a fixture authoring bug +(the .ail.json embeds something the surface printer can't faithfully +reproduce). Fix the fixture; do not weaken the test. + +--- + +## Task 6: master/spec.md — the mini-spec content + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/master/spec.md` + +The mini-spec is the load-bearing artifact: this is what the model +sees as its system prompt. It must be vollumfänglich (cover every +authoring construct the model could possibly need) and +form-agnostic in the prose body. Form-specific scaffolding lives in +`{form-only: X}` blocks; code samples are `{example: id}` markers +resolving to fixtures from Task 4. + +Token budget guideline: total per-form rendered length ~6k–10k +tokens (well below the 28k DESIGN.md baseline; the rendering will +be measured in Task 8). + +**Authoring rules common to every section:** + +- Each section is 200–600 words of prose plus example markers as + specified. The implementer writes the prose; the planner pins the + *structure* and what each section must cover. +- Any section with a `{form-only: json}` block must also carry a + `{form-only: ailx}` block of comparable length (the token-balance + test in Task 8 enforces the aggregate ±5%; per-section symmetry is + the easiest way to land there). +- Every `{example: }` marker must reference a file authored in + Task 4. If the marker resolves to a missing fixture the renderer + fails at runtime in Step 7.3. +- The spec.md is content authoring; quality gates are Task 7's + end-to-end render, Task 8's token balance, and the orchestrator's + end-to-end read before any cma.3 live run. + +- [ ] **Step 6.1: Write Section 1 — Header** + +One-paragraph orientation: "AILang is a small data-as-source +functional language designed for an LLM author. This document is +the complete authoring reference. Every well-formed program is a +module." No form-only blocks. No example markers. + +- [ ] **Step 6.2: Write Section 2 — Modules and the on-disk form** + +Explain what a module is — a top-level container of `imports` plus +`defs`. Include one `{form-only: json}` block listing schema +requirements: `schema` field equal to `"ailang/v0"`, `name` field, the +`imports` array, the `defs` array, canonical key order. Include one +`{form-only: ailx}` block listing the grammar conventions: +parenthesised tagged-head form, the `(module NAME …)` opener, the +relationship between AILX surface and the on-disk JSON. Close with +`{example: fn_returns_int}` to anchor the simplest concrete module. + +- [ ] **Step 6.3: Write Section 3 — Types** + +Cover Type::Con (named base types like `Int`, `Bool`, `Str`, +`Float`), Type::Var (polymorphic type variables introduced by +`forall`), Type::Fn (function types with parameter list, return +type, and effect set), Type::Forall (universal quantification at +the fn signature). State that every fn signature is mandatory at +top level — no implicit return-type inference at the binding form. +Form-only blocks: JSON shape of each type variant, AILX surface form +of each. Close with `{example: forall_polymorphic}`. + +- [ ] **Step 6.4: Write Section 4 — Mode annotations** + +Cover the three ParamMode variants. The implementer reads +`crates/ailang-core/src/ast.rs` to extract the exact mode names +(planner not pinning — recon did not enumerate). State that mode is +mandatory on every fn parameter; the schema rejects unannotated +parameters. Form-only blocks: JSON shape of a fn's `param_tys` +field; AILX surface syntax for mode annotations. Close with +`{example: param_modes_all}`. + +- [ ] **Step 6.5: Write Section 5 — Functions** + +Cover Def::Fn shape: `name`, type signature, parameter list, body. +Walk through the term forms inside a function body: Term::Var +(referencing a binding), Term::App (function application), +Term::Lam (anonymous lambda), Term::Lit (literal value). Form-only +blocks: JSON shape of each Term variant; AILX surface for each. +Examples: `{example: fn_calls_prelude}` and `{example: fn_with_lambda}`. + +- [ ] **Step 6.6: Write Section 6 — Algebraic data types** + +Cover Def::Data (declaring a new ADT with its ctors), Term::TermCtor +(constructing a value of that ADT). Note that ctor names are +distinct from fn names. Form-only blocks: JSON shape of `Def::Data` +with `params` (type-var list) and `ctors` (ctor name + arg-type +list); AILX surface for `(data NAME (vars …) (ctor …) …)`. Close +with `{example: data_simple}`. + +- [ ] **Step 6.7: Write Section 7 — Pattern matching** + +Cover Term::Match and the four Pattern variants: Pattern::Ctor (a +constructor pattern with sub-patterns), Pattern::Var (binding the +matched value to a name), Pattern::Wildcard (`_`), Pattern::Lit +(matching against a literal value). Form-only blocks: JSON shape +of a `match` term with arms; AILX surface for `(match SCRUT (case +PAT BODY) …)`. Examples: `{example: data_with_match}` and +`{example: match_literal_pattern}`. + +- [ ] **Step 6.8: Write Section 8 — Effects** + +Cover the effect set on a fn type. The two currently wired effects +are `IO` (required to call `io_print_int` and similar) and +`Diverge` (for non-terminating programs). Term::Do introduces an +effectful sequence; Term::Seq is the sequence node. Form-only +blocks: JSON shape of the `effects` field on a fn type; AILX +surface for `!IO` annotations and `(do …)` blocks. Close with +`{example: fn_with_do_seq}`. + +- [ ] **Step 6.9: Write Section 9 — Literals** + +Cover the five literal variants. Literal::Int (decimal integer +syntax). Literal::Float — note that the model writes the decimal +form (`3.14`); the toolchain encodes the IEEE-754 bit pattern as a +16-lowercase-hex string on parse, per the Roundtrip Invariant. The +model never authors the hex encoding. Literal::Bool (`true` / +`false`). Literal::Str (double-quoted, ASCII-only by Decision 6 +Constraint 3). Literal::Unit (`()`). Form-only blocks: JSON shape +of `Lit` variants; AILX surface for each. Examples: +`{example: floats}` and `{example: bool_str}`. + +- [ ] **Step 6.10: Write Section 10 — Typeclasses** + +Cover Def::Class (declaring a class with method signatures), +Def::Instance (providing an implementation of a class for a +specific type), the Constraint variant of Type (used in +constraint-polymorphic fn signatures like `forall a. (Eq a) => +…`). Mention the four prelude classes: `Eq`, `Ord`, `Num`, +`Bounded`. State that method invocation is by name; the compiler +monomorphises class methods at the call site. Form-only blocks: +JSON shape of class / instance / constraint; AILX surface for each. +Examples: `{example: class_def}` and `{example: instance_def}`. + +- [ ] **Step 6.11: Write Section 11 — The prelude** + +Tabulate the names the model can call without importing anything. +The implementer extracts the canonical list at the time of +authoring from the prelude module (likely under +`crates/ailang-check/src/prelude.rs` or `lib.rs`). One row per +function: name, type signature (form-agnostic in the prose; both +forms inside form-only blocks), one-line description. Minimum +content: integer arithmetic (`int_add`, `int_sub`, `int_mul`, +`int_div` or whatever the canonical names are), integer comparison +or its typeclass-routed equivalent (`Eq Int` / `Ord Int`), and +`io_print_int` (plus `io_print_str` only if it currently ships). +State explicitly that `int_to_str` is OUT — it is type-installed +but codegen-deferred pending the heap-Str ABI milestone (roadmap +P1 entry). No example marker (the table itself is the content). + +- [ ] **Step 6.12: Write Section 12 — Content addressing** + +Short paragraph: AILang has content-addressed identity (Decision +2). Hashes are computed by the toolchain on `ail parse` / `ail +check` from the canonical bytes of each definition. The author +writes no hash literal in either form; the JSON form has no `hash` +field; the AILX surface has no hash production. This is symmetric +across forms and removes hash as a form-distinguishing factor. No +example marker; no form-only blocks (because there is no form +asymmetry to teach). + +- [ ] **Step 6.13: Write Section 13 — Out of scope + closing** + +Short list of what this mini-spec does not teach: cross-module +imports (MVP tasks live in single modules; the `imports` array is +empty), refinement types (reserved in the schema but pass-through +in MVP), point-free style, operator overloading, syntactic sugar +of any kind. Reason: the four MVP tasks do not need them, and +including them would pad the spec. + +Close the file with one paragraph: "When asked to write a module, +return the complete module text and nothing else. No markdown +fences. No prose explanation." This is the directive the harness +relies on to keep response parsing trivial. + +--- + +## Task 7: Wire render/src/main.rs end-to-end + +**Files:** +- Modify: `experiments/2026-05-12-cross-model-authoring/render/src/main.rs` +- Modify: `experiments/2026-05-12-cross-model-authoring/render/src/examples.rs` + +The renderer assembles the two `rendered/*.md` files from +`master/spec.md` + `master/examples/`. + +- [ ] **Step 7.1: Implement the example renderer module** + +Replace `experiments/2026-05-12-cross-model-authoring/render/src/examples.rs` +content with: + +```rust +//! Per-form rendering of a single master example. +//! +//! - JSON form: read the on-disk canonical bytes verbatim, emit a +//! ```json``` fenced block. +//! - AILX form: load the example via ailang_core::load_module, print +//! via ailang_surface::print, emit a ```ailx``` fenced block. + +use anyhow::{anyhow, Context, Result}; +use std::path::Path; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Form { + Json, + Ailx, +} + +pub fn render_example(examples_dir: &Path, id: &str, form: Form) -> Result { + let path = examples_dir.join(format!("{id}.ail.json")); + if !path.exists() { + return Err(anyhow!( + "example id `{id}` references missing fixture at {}", + path.display() + )); + } + match form { + Form::Json => { + let bytes = std::fs::read_to_string(&path) + .with_context(|| format!("reading {}", path.display()))?; + Ok(format!("```json\n{}\n```\n", bytes.trim_end())) + } + Form::Ailx => { + let module = ailang_core::load_module(&path) + .with_context(|| format!("load_module on {}", path.display()))?; + let text = ailang_surface::print(&module); + Ok(format!("```ailx\n{}\n```\n", text.trim_end())) + } + } +} +``` + +- [ ] **Step 7.2: Implement the renderer in main.rs** + +Replace `experiments/2026-05-12-cross-model-authoring/render/src/main.rs` +content with: + +```rust +//! xmodel-render — projects master/spec.md into rendered/json.md and rendered/ailx.md. + +use anyhow::{anyhow, Context, Result}; +use std::path::{Path, PathBuf}; + +use xmodel_render::splitter::{self, Segment}; +use xmodel_render::examples; + +fn main() -> Result<()> { + let (master_dir, rendered_dir) = parse_args()?; + let spec_path = master_dir.join("spec.md"); + let examples_dir = master_dir.join("examples"); + let spec_text = std::fs::read_to_string(&spec_path) + .with_context(|| format!("reading {}", spec_path.display()))?; + let segments = splitter::split(&spec_text); + + let json_out = render_for(&segments, &examples_dir, examples::Form::Json, splitter::Form::Json)?; + let ailx_out = render_for(&segments, &examples_dir, examples::Form::Ailx, splitter::Form::Ailx)?; + + std::fs::create_dir_all(&rendered_dir) + .with_context(|| format!("creating {}", rendered_dir.display()))?; + let json_path = rendered_dir.join("json.md"); + let ailx_path = rendered_dir.join("ailx.md"); + std::fs::write(&json_path, json_out) + .with_context(|| format!("writing {}", json_path.display()))?; + std::fs::write(&ailx_path, ailx_out) + .with_context(|| format!("writing {}", ailx_path.display()))?; + eprintln!("xmodel-render: wrote {} and {}", json_path.display(), ailx_path.display()); + Ok(()) +} + +fn render_for( + segments: &[Segment], + examples_dir: &Path, + example_form: examples::Form, + keep_form: splitter::Form, +) -> Result { + let mut out = String::new(); + for seg in segments { + match seg { + Segment::Prose(text) => out.push_str(text), + Segment::FormOnly { form, body } if *form == keep_form => out.push_str(body), + Segment::FormOnly { .. } => {} // drop the other form's block + Segment::Example(id) => { + let rendered = examples::render_example(examples_dir, id, example_form)?; + out.push_str(&rendered); + } + } + } + Ok(out) +} + +fn parse_args() -> Result<(PathBuf, PathBuf)> { + let mut master: Option = None; + let mut rendered: Option = None; + let mut args = std::env::args().skip(1); + while let Some(arg) = args.next() { + match arg.as_str() { + "--master" => master = Some(PathBuf::from(args.next().ok_or_else(|| anyhow!("--master needs a value"))?)), + "--rendered" => rendered = Some(PathBuf::from(args.next().ok_or_else(|| anyhow!("--rendered needs a value"))?)), + other => return Err(anyhow!("unknown argument: {}", other)), + } + } + let master = master.ok_or_else(|| anyhow!("--master is required"))?; + let rendered = rendered.ok_or_else(|| anyhow!("--rendered is required"))?; + Ok((master, rendered)) +} +``` + +Note the two `Form` enums: `splitter::Form` is the marker found in +`{form-only: X}`; `examples::Form` is the renderer-side mode for an +example. They are intentionally separate types (different semantics) +and the bin's `render_for` reconciles them. The reconciliation is +trivial because there are only two forms, but keeping the types +separate avoids accidentally feeding one to the other. + +- [ ] **Step 7.3: Run the renderer** + +Run: +``` +cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml -- \ + --master experiments/2026-05-12-cross-model-authoring/master \ + --rendered experiments/2026-05-12-cross-model-authoring/rendered +``` +Expected: +- exit code 0, +- stderr line "xmodel-render: wrote …json.md and …ailx.md", +- `experiments/2026-05-12-cross-model-authoring/rendered/json.md` exists, non-empty, +- `experiments/2026-05-12-cross-model-authoring/rendered/ailx.md` exists, non-empty, +- spot-check: open both files; JSON file's code blocks are + ```json fences; AILX file's are ```ailx fences; the prose body is + the same in both. + +If the renderer errors on a missing example id: that's a bug in +master/spec.md (a marker references an id with no fixture). Fix the +spec.md or add the fixture, re-run. + +--- + +## Task 8: token_balance test + +**Files:** +- Create: `experiments/2026-05-12-cross-model-authoring/render/tests/token_balance.rs` + +The test runs the splitter against `master/spec.md`, collects all +`{form-only: json}` block bodies into one buffer and all +`{form-only: ailx}` block bodies into another, tokenises both with +`tiktoken-rs::cl100k_base`, and asserts the totals are within ±5%. + +- [ ] **Step 8.1: Write the token-balance test** + +Write `experiments/2026-05-12-cross-model-authoring/render/tests/token_balance.rs`: + +```rust +//! Token-balance gate for master/spec.md form-only blocks. +//! +//! The mini-spec is the experimental treatment; if the JSON-cohort's +//! form-only scaffolding is meaningfully longer than the AILX-cohort's +//! (or vice versa), the experiment is biased before the model ever +//! sees the prompts. This test catches gross imbalance. + +use std::path::PathBuf; +use xmodel_render::splitter::{self, Form, Segment}; + +fn master_dir() -> PathBuf { + let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + manifest.parent().unwrap().join("master") +} + +#[test] +fn form_only_block_token_totals_within_five_percent() { + let spec_text = std::fs::read_to_string(master_dir().join("spec.md")) + .expect("master/spec.md must exist"); + let segments = splitter::split(&spec_text); + + let mut json_buf = String::new(); + let mut ailx_buf = String::new(); + for seg in &segments { + if let Segment::FormOnly { form, body } = seg { + match form { + Form::Json => json_buf.push_str(body), + Form::Ailx => ailx_buf.push_str(body), + } + } + } + + let bpe = tiktoken_rs::cl100k_base().expect("cl100k_base tokenizer should load"); + let json_count = bpe.encode_with_special_tokens(&json_buf).len(); + let ailx_count = bpe.encode_with_special_tokens(&ailx_buf).len(); + assert!(json_count > 0, "JSON-form blocks produced zero tokens"); + assert!(ailx_count > 0, "AILX-form blocks produced zero tokens"); + + let max = json_count.max(ailx_count) as f64; + let diff = (json_count as i64 - ailx_count as i64).unsigned_abs() as f64; + let ratio = diff / max; + assert!( + ratio <= 0.05, + "form-only block token imbalance: json={json_count}, ailx={ailx_count}, ratio={ratio:.4} > 0.05", + ); +} +``` + +- [ ] **Step 8.2: Run the test** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml --test token_balance +``` +Expected: `test result: ok. 1 passed; 0 failed`. + +If the test fails: revise master/spec.md to lengthen the shorter +form's blocks (typically by adding examples to the shorter-form's +rules section or factually-equivalent prose). Do not pad with +filler — the rebalancing should be substantive (one form being +shorter usually means it's underexplained). Re-run. + +--- + +## Task 9: Run renderer + commit outputs + +**Files:** +- Modify (regenerate): `experiments/2026-05-12-cross-model-authoring/rendered/json.md` +- Modify (regenerate): `experiments/2026-05-12-cross-model-authoring/rendered/ailx.md` + +- [ ] **Step 9.1: Regenerate rendered outputs** + +Run: +``` +cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml -- \ + --master experiments/2026-05-12-cross-model-authoring/master \ + --rendered experiments/2026-05-12-cross-model-authoring/rendered +``` +Expected: exit 0, both rendered files written. + +- [ ] **Step 9.2: Full test sweep** + +Run: +``` +cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/render/Cargo.toml +``` +Expected: all four test suites green — +`splitter_unit` (7 passed), `spec_completeness` (1 passed), +`example_roundtrip` (1 passed), `token_balance` (1 passed). +Total: 10 passed; 0 failed. + +- [ ] **Step 9.3: Confirm working-tree state** + +Run: +``` +git status -- experiments/2026-05-12-cross-model-authoring/ +``` +Expected: the entire `experiments/2026-05-12-cross-model-authoring/` +tree is untracked. The Boss will inspect and decide commit shape at +iter close; the implement step leaves everything unstaged. + +--- + +## Out of scope for cma.1 (mirror of spec) + +- `master/tasks/*.task.json` and their reference solutions — cma.2. +- `harness/` Cargo project, the HTTP client, the per-task loop, + the location-stripping regex, scoring — cma.2. +- Live IONOS run, raw dataset emission, `runs/-/` — cma.3. +- DESIGN.md §"Decision 6" empirical addendum — cma.3. +- Journal entry under `docs/journals/` summarising the run — cma.3. +- Roadmap edits removing the P2 entry — cma.3.