spec: roundtrip invariant milestone — bidirectional .ail.json/.ailx bijection, schema-coverage, CLI roundtrip, hard-fail, dynamic fixture collection

2026-05-12 09:19:01 +02:00
parent daf9f4fdc9
commit ea2c485459
1 changed files with 251 additions and 0 deletions
@@ -0,0 +1,251 @@
+# Roundtrip Invariant — Design Spec
+
+**Date:** 2026-05-12
+**Status:** Draft — awaiting user spec review
+**Authors:** Brummel (orchestrator) + Claude
+
+## Goal
+
+Establish that `.ail.json` ↔ `.ailx` form a structurally lossless
+bijection on the module level, and lift this property from a
+Decision-6-internal constraint to a prominently anchored top-level
+invariant of the language. Enforce it workspace-wide by a coordinated
+test suite that collects fixtures dynamically (no hardcoded fixture
+lists) and verifies that every AST schema variant is exercised by
+at least one fixture.
+
+Two motivating uses:
+
+1. **Independent value.** Per Decision 6, `.ailx` is "the AI authoring
+   projection". If programs are expressible in one form but not the
+   other, the projection claim is partial — a first-class bug class
+   that the language design admits no allowance for.
+2. **Prerequisite for the cross-model authoring-form test** (P2
+   roadmap entry). Measured form-preference across multiple
+   downstream LLM subjects is only interpretable if both forms are
+   exactly equipotent in expressive power. Without this milestone
+   the cross-model test confounds expressive-power gaps with
+   authoring ergonomics.
+
+## Architecture
+
+Three coordinated test layers plus a DESIGN.md anchor:
+
+1. **DESIGN.md anchor.** A new top-level section *"Roundtrip
+   Invariant"*, positioned between the Decisions block and the
+   schema spec block, on the same visibility level as the existing
+   "Feature-acceptance criterion" section. It states both directions
+   of the bijection explicitly, names the Float `bits`-hex encoding
+   as a mechanism *inside* the invariant (not an exception to it),
+   and references the test files below as the enforcement mechanism.
+   Decision 6 Constraint 2 stays in place and gains an upward
+   reference to the new section.
+
+2. **Machine roundtrip** (existing, unchanged): the current
+   `print_then_parse_round_trips_every_fixture` in
+   `crates/ailang-surface/tests/round_trip.rs` already iterates
+   dynamically over every `.ail.json` fixture under `examples/` and
+   verifies `JSON → print → parse → canonical-bytes-equal`. Provides
+   the JSON-anchored half of the bijection.
+
+3. **Hand cross-check** (replacing the 3-pair static test): a new
+   dynamic test `every_ailx_fixture_matches_its_json_counterpart`
+   replaces `handwritten_exhibits_match_json_fixtures`. It iterates
+   over every `.ailx` file in `examples/`, locates the same-stem
+   `.ail.json` counterpart, and asserts
+   `canonical_bytes(parse(ailx_text)) == canonical_bytes(load_module(json_path))`.
+   For a `.ailx` file without a counterpart (today: none; reserved
+   for future), only `parse` success is required — drift detection
+   needs both sides.
+
+4. **Schema-coverage audit** (new): a new test
+   `crates/ailang-core/tests/schema_coverage.rs` iterates over every
+   module loaded from `examples/`, runs an exhaustive AST visitor
+   that — via `match` arms *without* a wildcard `_ => ...` — populates
+   a `HashSet<VariantTag>` of observed variants. The test asserts
+   that every expected variant (one per enum case in `Def` / `Term` /
+   `Pattern` / `Literal` / `Type` / `ParamMode`) appears at least
+   once. The Rust compiler enforces synchronization between the AST
+   definition and the visitor: any new AST enum variant will not
+   compile until the visitor and `VariantTag` are extended,
+   guaranteeing that the coverage table cannot silently fall behind
+   the schema.
+
+5. **Workspace CLI roundtrip** (new): a new integration test
+   `crates/ail/tests/roundtrip_cli.rs` iterates over every `.ail.json`
+   fixture in `examples/`, runs the fixture through the public
+   `ail render` → tempfile → `ail parse` CLI flow (two subprocesses,
+   tempfile-mediated because `ail parse` reads from a file path,
+   not stdin), and verifies BLAKE3 hash identity between the
+   original canonicalised JSON and the round-tripped canonical JSON
+   captured from `ail parse`'s stdout. This is the user-facing-
+   surface defence line: it catches drift in the CLI layer that
+   crate-internal tests miss.
+
+## Components
+
+### DESIGN.md edit
+
+A new top-level section approximately 40–70 lines long. Outline:
+
+- **Statement** (both directions, explicit). Every valid AILang
+  module has both a `.ail.json` and a `.ailx` representation;
+  `print` and `parse` are mutual inverses on the canonical-bytes
+  level.
+- **Float-bits-hex encoding** as the mechanism that keeps Float
+  literals (including NaN, ±Inf, subnormals) *inside* the invariant
+  rather than outside it.
+- **Enforcement points** — the four test files below, by path.
+- **Why this is top-level**: the property is load-bearing on the
+  language identity, not just on Decision 6's surface-design
+  rationale.
+- **Cross-reference** from Decision 6 Constraint 2 up to this
+  section, replacing the inline form of the same claim.
+
+### Test files
+
+| Path | Status | Purpose |
+|------|--------|---------|
+| `crates/ailang-surface/tests/round_trip.rs::print_then_parse_round_trips_every_fixture` | Unchanged | Machine roundtrip (`.ail.json → .ailx → .ail.json`) |
+| `crates/ailang-surface/tests/round_trip.rs::every_ailx_fixture_matches_its_json_counterpart` | Replaces hardcoded-pair test; dynamic | Hand cross-check (`.ailx → .ail.json` ≡ static JSON counterpart) |
+| `crates/ailang-core/tests/schema_coverage.rs` | New | Schema-variant coverage audit |
+| `crates/ail/tests/roundtrip_cli.rs` | New | CLI roundtrip — BLAKE3 hash identity through `ail render` → tempfile → `ail parse` |
+
+## Data flow
+
+**Machine roundtrip** (existing):
+
+```
+<stem>.ail.json
+  → ailang_core::load_module
+  → ailang_surface::print
+  → ailang_surface::parse
+  → ailang_core::canonical::to_bytes
+  ≡ canonical::to_bytes(<original>)
+```
+
+**Hand cross-check** (new, dynamic):
+
+```
+for every <stem>.ailx where <stem>.ail.json exists:
+    <stem>.ailx (file-read)
+      → ailang_surface::parse
+      → ailang_core::canonical::to_bytes
+      ≡ canonical::to_bytes(ailang_core::load_module(<stem>.ail.json))
+```
+
+**Schema coverage** (new):
+
+```
+for every <stem>.ail.json:
+    load_module
+    visit AST exhaustively (no `_` wildcard arms)
+    accumulate observed: HashSet<VariantTag>
+assert: expected_set ⊆ observed_set
+```
+
+The `expected_set` is computed at runtime by enumerating one
+`VariantTag` per AST enum variant. The visitor's exhaustive matches
+guarantee the two stay synchronized via the compiler.
+
+**CLI roundtrip** (new):
+
+```
+for every <stem>.ail.json:
+    bytes_orig = canonical::to_bytes(load_module(<stem>.ail.json))
+    h_orig = blake3(bytes_orig)
+
+    # subprocess 1: render JSON fixture to .ailx text on stdout
+    ailx_text = run("ail render <stem>.ail.json")
+    write(<tmpdir>/round.ailx, ailx_text)
+
+    # subprocess 2: parse the .ailx back to canonical JSON bytes
+    # (ail parse writes canonical bytes + a trailing newline)
+    raw_stdout = run("ail parse <tmpdir>/round.ailx")
+    bytes_round = strip_trailing_newline(raw_stdout)
+    h_round = blake3(bytes_round)
+
+    assert h_orig == h_round
+```
+
+The `ail parse` subcommand emits the canonical bytes via
+`stdout.write_all(&bytes)` followed by a `println!()`; the test
+strips the trailing newline before hashing.
+
+## Error handling
+
+- Each test failure prints the offending fixture path and a concrete
+  diff: canonical-byte diff for tests 1 and 2; missing-variant list
+  for test 3; hex-hash mismatch plus captured stderr for test 4.
+- Failures are aggregated per test: a single `cargo test` run lists
+  *all* failing fixtures, not just the first. Existing test 1
+  already follows this pattern; the new tests adopt it.
+- Subprocess errors in the CLI test (non-zero exit) propagate the
+  captured stderr verbatim into the panic message.
+
+## Testing strategy
+
+- **Hard-fail mode from iteration 1.** No allowlist, no aspirational
+  state. The first iteration that adds the new tests will surface
+  every existing roundtrip gap as a test failure. Each gap is a bug
+  to fix in subsequent iterations; the milestone closes only when
+  all four tests are green.
+- **Dynamic fixture collection.** Every test uses `read_dir` over
+  `examples/`. No hardcoded fixture list. New fixtures are picked up
+  automatically.
+- **Float coverage.** The schema coverage test includes
+  `Literal::Float` as a required variant, so the corpus must contain
+  at least one Float-bearing fixture (it already does:
+  `examples/floats.ail.json` and others). The roundtrip tests then
+  exercise the bits-hex encoding path on those fixtures. NaN / ±Inf
+  are not currently in the corpus; if the audit reveals no fixture
+  carries these bit patterns, a minimal Float-edge-case fixture is
+  added.
+- **Independence.** The four tests are intentionally overlapping in
+  places (e.g. test 1 and test 4 both cover the JSON-anchored
+  direction, at crate-internal vs CLI-surface levels). This
+  redundancy is the point: each test guards a different drift
+  source.
+
+## Acceptance criteria
+
+1. `docs/DESIGN.md` carries a top-level section *"Roundtrip
+   Invariant"* with the structure outlined above; Decision 6
+   Constraint 2 carries an upward cross-reference to it.
+2. `crates/ailang-surface/tests/round_trip.rs` has
+   `every_ailx_fixture_matches_its_json_counterpart` in place of the
+   hardcoded-pair `handwritten_exhibits_match_json_fixtures`,
+   sourcing all `.ailx`/`.ail.json` pairs dynamically.
+3. `crates/ailang-core/tests/schema_coverage.rs` exists; its visitor
+   matches each of `Def` / `Term` / `Pattern` / `Literal` / `Type` /
+   `ParamMode` exhaustively without wildcard arms; every variant
+   appears in at least one fixture.
+4. `crates/ail/tests/roundtrip_cli.rs` exists; iterates dynamically
+   over `.ail.json` fixtures; verifies BLAKE3 hash identity through
+   the `ail render | ail parse` subprocess pipeline.
+5. `cargo test --workspace` is green with all four tests active.
+6. Any roundtrip gaps surfaced by the new tests are closed in render
+   or parse code, not by relaxing the tests.
+7. **Tests are pure readers of the repo.** None of the four tests
+   writes a file inside the workspace, regenerates a fixture,
+   patches the AST, or otherwise mutates committed content. The
+   only filesystem side effect permitted is the use of a temp
+   directory (e.g. `tempfile::TempDir`) for the CLI roundtrip's
+   intermediate `.ailx` file, which lives outside the repo and is
+   cleaned up by the test harness. If a test surfaces a gap, the
+   diagnostic is fail-loud; the fix lands in render/parse code via
+   a separate iteration, never inline by the test.
+
+## Scope notes
+
+- **In scope:** roundtrip property in both directions; DESIGN.md
+  anchor; dynamic test coverage; schema-variant coverage; fixing
+  any gaps surfaced.
+- **Out of scope:** the cross-model authoring-form test (P2
+  roadmap, gated on this milestone). Property-based generation of
+  random AST instances (the schema-coverage test plus the existing
+  fixture corpus provides equivalent assurance without the
+  generator-authoring burden). The two open P2 todos around
+  `.ailx`-aware CLI (`ail check foo.ailx`) and workspace-search
+  beyond the entry module's directory — those are orthogonal and
+  remain on the roadmap.