From ea2c485459786736e0cb22f1cd23dbc5fb2db19b Mon Sep 17 00:00:00 2001 From: Brummel Date: Tue, 12 May 2026 09:19:01 +0200 Subject: [PATCH] =?UTF-8?q?spec:=20roundtrip=20invariant=20milestone=20?= =?UTF-8?q?=E2=80=94=20bidirectional=20.ail.json/.ailx=20bijection,=20sche?= =?UTF-8?q?ma-coverage,=20CLI=20roundtrip,=20hard-fail,=20dynamic=20fixtur?= =?UTF-8?q?e=20collection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/specs/2026-05-12-roundtrip-invariant.md | 251 +++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 docs/specs/2026-05-12-roundtrip-invariant.md diff --git a/docs/specs/2026-05-12-roundtrip-invariant.md b/docs/specs/2026-05-12-roundtrip-invariant.md new file mode 100644 index 0000000..17c53f7 --- /dev/null +++ b/docs/specs/2026-05-12-roundtrip-invariant.md @@ -0,0 +1,251 @@ +# Roundtrip Invariant — Design Spec + +**Date:** 2026-05-12 +**Status:** Draft — awaiting user spec review +**Authors:** Brummel (orchestrator) + Claude + +## Goal + +Establish that `.ail.json` ↔ `.ailx` form a structurally lossless +bijection on the module level, and lift this property from a +Decision-6-internal constraint to a prominently anchored top-level +invariant of the language. Enforce it workspace-wide by a coordinated +test suite that collects fixtures dynamically (no hardcoded fixture +lists) and verifies that every AST schema variant is exercised by +at least one fixture. + +Two motivating uses: + +1. **Independent value.** Per Decision 6, `.ailx` is "the AI authoring + projection". If programs are expressible in one form but not the + other, the projection claim is partial — a first-class bug class + that the language design admits no allowance for. +2. **Prerequisite for the cross-model authoring-form test** (P2 + roadmap entry). Measured form-preference across multiple + downstream LLM subjects is only interpretable if both forms are + exactly equipotent in expressive power. Without this milestone + the cross-model test confounds expressive-power gaps with + authoring ergonomics. + +## Architecture + +Three coordinated test layers plus a DESIGN.md anchor: + +1. **DESIGN.md anchor.** A new top-level section *"Roundtrip + Invariant"*, positioned between the Decisions block and the + schema spec block, on the same visibility level as the existing + "Feature-acceptance criterion" section. It states both directions + of the bijection explicitly, names the Float `bits`-hex encoding + as a mechanism *inside* the invariant (not an exception to it), + and references the test files below as the enforcement mechanism. + Decision 6 Constraint 2 stays in place and gains an upward + reference to the new section. + +2. **Machine roundtrip** (existing, unchanged): the current + `print_then_parse_round_trips_every_fixture` in + `crates/ailang-surface/tests/round_trip.rs` already iterates + dynamically over every `.ail.json` fixture under `examples/` and + verifies `JSON → print → parse → canonical-bytes-equal`. Provides + the JSON-anchored half of the bijection. + +3. **Hand cross-check** (replacing the 3-pair static test): a new + dynamic test `every_ailx_fixture_matches_its_json_counterpart` + replaces `handwritten_exhibits_match_json_fixtures`. It iterates + over every `.ailx` file in `examples/`, locates the same-stem + `.ail.json` counterpart, and asserts + `canonical_bytes(parse(ailx_text)) == canonical_bytes(load_module(json_path))`. + For a `.ailx` file without a counterpart (today: none; reserved + for future), only `parse` success is required — drift detection + needs both sides. + +4. **Schema-coverage audit** (new): a new test + `crates/ailang-core/tests/schema_coverage.rs` iterates over every + module loaded from `examples/`, runs an exhaustive AST visitor + that — via `match` arms *without* a wildcard `_ => ...` — populates + a `HashSet` of observed variants. The test asserts + that every expected variant (one per enum case in `Def` / `Term` / + `Pattern` / `Literal` / `Type` / `ParamMode`) appears at least + once. The Rust compiler enforces synchronization between the AST + definition and the visitor: any new AST enum variant will not + compile until the visitor and `VariantTag` are extended, + guaranteeing that the coverage table cannot silently fall behind + the schema. + +5. **Workspace CLI roundtrip** (new): a new integration test + `crates/ail/tests/roundtrip_cli.rs` iterates over every `.ail.json` + fixture in `examples/`, runs the fixture through the public + `ail render` → tempfile → `ail parse` CLI flow (two subprocesses, + tempfile-mediated because `ail parse` reads from a file path, + not stdin), and verifies BLAKE3 hash identity between the + original canonicalised JSON and the round-tripped canonical JSON + captured from `ail parse`'s stdout. This is the user-facing- + surface defence line: it catches drift in the CLI layer that + crate-internal tests miss. + +## Components + +### DESIGN.md edit + +A new top-level section approximately 40–70 lines long. Outline: + +- **Statement** (both directions, explicit). Every valid AILang + module has both a `.ail.json` and a `.ailx` representation; + `print` and `parse` are mutual inverses on the canonical-bytes + level. +- **Float-bits-hex encoding** as the mechanism that keeps Float + literals (including NaN, ±Inf, subnormals) *inside* the invariant + rather than outside it. +- **Enforcement points** — the four test files below, by path. +- **Why this is top-level**: the property is load-bearing on the + language identity, not just on Decision 6's surface-design + rationale. +- **Cross-reference** from Decision 6 Constraint 2 up to this + section, replacing the inline form of the same claim. + +### Test files + +| Path | Status | Purpose | +|------|--------|---------| +| `crates/ailang-surface/tests/round_trip.rs::print_then_parse_round_trips_every_fixture` | Unchanged | Machine roundtrip (`.ail.json → .ailx → .ail.json`) | +| `crates/ailang-surface/tests/round_trip.rs::every_ailx_fixture_matches_its_json_counterpart` | Replaces hardcoded-pair test; dynamic | Hand cross-check (`.ailx → .ail.json` ≡ static JSON counterpart) | +| `crates/ailang-core/tests/schema_coverage.rs` | New | Schema-variant coverage audit | +| `crates/ail/tests/roundtrip_cli.rs` | New | CLI roundtrip — BLAKE3 hash identity through `ail render` → tempfile → `ail parse` | + +## Data flow + +**Machine roundtrip** (existing): + +``` +.ail.json + → ailang_core::load_module + → ailang_surface::print + → ailang_surface::parse + → ailang_core::canonical::to_bytes + ≡ canonical::to_bytes() +``` + +**Hand cross-check** (new, dynamic): + +``` +for every .ailx where .ail.json exists: + .ailx (file-read) + → ailang_surface::parse + → ailang_core::canonical::to_bytes + ≡ canonical::to_bytes(ailang_core::load_module(.ail.json)) +``` + +**Schema coverage** (new): + +``` +for every .ail.json: + load_module + visit AST exhaustively (no `_` wildcard arms) + accumulate observed: HashSet +assert: expected_set ⊆ observed_set +``` + +The `expected_set` is computed at runtime by enumerating one +`VariantTag` per AST enum variant. The visitor's exhaustive matches +guarantee the two stay synchronized via the compiler. + +**CLI roundtrip** (new): + +``` +for every .ail.json: + bytes_orig = canonical::to_bytes(load_module(.ail.json)) + h_orig = blake3(bytes_orig) + + # subprocess 1: render JSON fixture to .ailx text on stdout + ailx_text = run("ail render .ail.json") + write(/round.ailx, ailx_text) + + # subprocess 2: parse the .ailx back to canonical JSON bytes + # (ail parse writes canonical bytes + a trailing newline) + raw_stdout = run("ail parse /round.ailx") + bytes_round = strip_trailing_newline(raw_stdout) + h_round = blake3(bytes_round) + + assert h_orig == h_round +``` + +The `ail parse` subcommand emits the canonical bytes via +`stdout.write_all(&bytes)` followed by a `println!()`; the test +strips the trailing newline before hashing. + +## Error handling + +- Each test failure prints the offending fixture path and a concrete + diff: canonical-byte diff for tests 1 and 2; missing-variant list + for test 3; hex-hash mismatch plus captured stderr for test 4. +- Failures are aggregated per test: a single `cargo test` run lists + *all* failing fixtures, not just the first. Existing test 1 + already follows this pattern; the new tests adopt it. +- Subprocess errors in the CLI test (non-zero exit) propagate the + captured stderr verbatim into the panic message. + +## Testing strategy + +- **Hard-fail mode from iteration 1.** No allowlist, no aspirational + state. The first iteration that adds the new tests will surface + every existing roundtrip gap as a test failure. Each gap is a bug + to fix in subsequent iterations; the milestone closes only when + all four tests are green. +- **Dynamic fixture collection.** Every test uses `read_dir` over + `examples/`. No hardcoded fixture list. New fixtures are picked up + automatically. +- **Float coverage.** The schema coverage test includes + `Literal::Float` as a required variant, so the corpus must contain + at least one Float-bearing fixture (it already does: + `examples/floats.ail.json` and others). The roundtrip tests then + exercise the bits-hex encoding path on those fixtures. NaN / ±Inf + are not currently in the corpus; if the audit reveals no fixture + carries these bit patterns, a minimal Float-edge-case fixture is + added. +- **Independence.** The four tests are intentionally overlapping in + places (e.g. test 1 and test 4 both cover the JSON-anchored + direction, at crate-internal vs CLI-surface levels). This + redundancy is the point: each test guards a different drift + source. + +## Acceptance criteria + +1. `docs/DESIGN.md` carries a top-level section *"Roundtrip + Invariant"* with the structure outlined above; Decision 6 + Constraint 2 carries an upward cross-reference to it. +2. `crates/ailang-surface/tests/round_trip.rs` has + `every_ailx_fixture_matches_its_json_counterpart` in place of the + hardcoded-pair `handwritten_exhibits_match_json_fixtures`, + sourcing all `.ailx`/`.ail.json` pairs dynamically. +3. `crates/ailang-core/tests/schema_coverage.rs` exists; its visitor + matches each of `Def` / `Term` / `Pattern` / `Literal` / `Type` / + `ParamMode` exhaustively without wildcard arms; every variant + appears in at least one fixture. +4. `crates/ail/tests/roundtrip_cli.rs` exists; iterates dynamically + over `.ail.json` fixtures; verifies BLAKE3 hash identity through + the `ail render | ail parse` subprocess pipeline. +5. `cargo test --workspace` is green with all four tests active. +6. Any roundtrip gaps surfaced by the new tests are closed in render + or parse code, not by relaxing the tests. +7. **Tests are pure readers of the repo.** None of the four tests + writes a file inside the workspace, regenerates a fixture, + patches the AST, or otherwise mutates committed content. The + only filesystem side effect permitted is the use of a temp + directory (e.g. `tempfile::TempDir`) for the CLI roundtrip's + intermediate `.ailx` file, which lives outside the repo and is + cleaned up by the test harness. If a test surfaces a gap, the + diagnostic is fail-loud; the fix lands in render/parse code via + a separate iteration, never inline by the test. + +## Scope notes + +- **In scope:** roundtrip property in both directions; DESIGN.md + anchor; dynamic test coverage; schema-variant coverage; fixing + any gaps surfaced. +- **Out of scope:** the cross-model authoring-form test (P2 + roadmap, gated on this milestone). Property-based generation of + random AST instances (the schema-coverage test plus the existing + fixture corpus provides equivalent assurance without the + generator-authoring burden). The two open P2 todos around + `.ailx`-aware CLI (`ail check foo.ailx`) and workspace-search + beyond the entry module's directory — those are orthogonal and + remain on the roadmap.