diff --git a/crates/ailang-surface/tests/round_trip.rs b/crates/ailang-surface/tests/round_trip.rs index 81caefd..2447525 100644 --- a/crates/ailang-surface/tests/round_trip.rs +++ b/crates/ailang-surface/tests/round_trip.rs @@ -1,21 +1,27 @@ //! Round-trip gate for the form-(A) projection. //! -//! Two complementary checks over `examples/`: +//! Three complementary checks over `examples/`, each gathering +//! fixtures dynamically via `read_dir` (no hardcoded lists). The +//! tests are pure readers — they do not write into the working +//! tree. //! -//! 1. `print_then_parse_round_trips_every_fixture`: for every -//! `examples/*.ail.json` fixture, load → `print` → `parse` → -//! canonical bytes; assert byte-equal to original. +//! 1. `print_then_parse_round_trips_every_fixture`: Direction 1 of +//! the Roundtrip Invariant (DESIGN.md §"Roundtrip Invariant"). +//! For every `examples/*.ail.json` fixture, load → `print` → +//! `parse` → canonical bytes; assert byte-equal to original. //! -//! 2. `every_ailx_fixture_matches_its_json_counterpart`: for every -//! `examples/*.ailx` fixture, parse → canonical bytes; if a -//! same-stem `.ail.json` counterpart exists, assert canonical-byte -//! equality against it. The `.ailx` side is the human/AI authoring -//! ground-truth — fixtures must be writeable without going through -//! the printer and still match the JSON-AST. +//! 2. `every_ailx_fixture_matches_its_json_counterpart`: hand- +//! authored ground-truth check. For every `examples/*.ailx` +//! fixture, parse → canonical bytes; if a same-stem `.ail.json` +//! counterpart exists, assert canonical-byte equality against +//! it. Pins the `.ailx` corpus against semantic drift between +//! the two forms at the fixture level. //! -//! Both tests gather fixtures dynamically via `read_dir`; no hardcoded -//! lists. The tests are pure readers — they do not write into the -//! working tree. +//! 3. `parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`: +//! Direction 2 of the Roundtrip Invariant. For every well-formed +//! `.ailx` text `t`, asserts `canonical_bytes(parse(t))` equals +//! `canonical_bytes(parse(print(parse(t))))`. Robust against +//! future `.ailx` fixtures without a JSON counterpart. use std::path::{Path, PathBuf}; @@ -196,3 +202,76 @@ fn every_ailx_fixture_matches_its_json_counterpart() { paired, parse_only ); } + +/// Direction 2 of the Roundtrip Invariant (DESIGN.md §"Roundtrip +/// Invariant"): for every well-formed `.ailx` text `t`, the +/// composition `parse → print → parse` is idempotent on the AST. +/// +/// For the 57 `.ailx` fixtures that have a JSON counterpart this +/// follows logically from `print_then_parse_round_trips_every_fixture` +/// + `every_ailx_fixture_matches_its_json_counterpart`. This test +/// asserts the property directly so it stays robust for future +/// `.ailx` fixtures without a JSON counterpart, and so the spec's +/// Direction-2 claim has a dedicated enforcement point. +#[test] +fn parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture() { + let fixtures = list_ailx_fixtures(); + assert!( + !fixtures.is_empty(), + "no .ailx fixtures found under {}", + examples_dir().display() + ); + + let mut failures = Vec::::new(); + let mut passed = 0usize; + for path in &fixtures { + let text = match std::fs::read_to_string(path) { + Ok(t) => t, + Err(e) => { + failures.push(format!("{}: read failed: {e}", path.display())); + continue; + } + }; + let parsed_once = match ailang_surface::parse(&text) { + Ok(m) => m, + Err(e) => { + failures.push(format!("{}: parse(t) failed: {e}", path.display())); + continue; + } + }; + let printed = ailang_surface::print(&parsed_once); + let parsed_twice = match ailang_surface::parse(&printed) { + Ok(m) => m, + Err(e) => { + failures.push(format!( + "{}: parse(print(parse(t))) failed: {e}\n--- printed ---\n{printed}", + path.display() + )); + continue; + } + }; + let bytes_a = ailang_core::canonical::to_bytes(&parsed_once); + let bytes_b = ailang_core::canonical::to_bytes(&parsed_twice); + if bytes_a != bytes_b { + let s_a = String::from_utf8_lossy(&bytes_a).into_owned(); + let s_b = String::from_utf8_lossy(&bytes_b).into_owned(); + failures.push(format!( + "{}: parse → print → parse is NOT idempotent.\nparse(t): {s_a}\nparse(print(parse(t))): {s_b}", + path.display() + )); + continue; + } + passed += 1; + } + + if !failures.is_empty() { + panic!( + "{} of {} .ailx fixture(s) failed idempotency check (passed: {}):\n{}", + failures.len(), + fixtures.len(), + passed, + failures.join("\n\n") + ); + } + eprintln!("parse→print→parse idempotency ok for {passed} .ailx fixtures"); +} diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 64190a8..64a6041 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -1781,8 +1781,8 @@ without changing the identity of the module it produces. Float literals carry an IEEE-754 bit pattern, not a decimal approximation. The canonical encoding is -`{"kind":"float","bits":"<16-hex>"}` and the surface emits the -same bits-hex string. NaN, ±Inf, signed zero, and subnormals all +`{"kind":"float","bits":"<16-lowercase-hex>"}` and the surface +emits the same bits-hex string. NaN, ±Inf, signed zero, and subnormals all round-trip exactly because the JSON-number path is bypassed (see §"Float semantics", "Form-A serialisation"). Floats are not an exception to the invariant — the bits-hex encoding is the @@ -1790,7 +1790,7 @@ mechanism that keeps them *inside* it. ### Enforcement -The invariant is workspace-CI-enforced by four tests, each +The invariant is workspace-CI-enforced by five tests, each operating on the `examples/` corpus via dynamic `read_dir` collection (no hardcoded fixture list, so newly added fixtures inherit the gate automatically): @@ -1798,6 +1798,9 @@ inherit the gate automatically): - `crates/ailang-surface/tests/round_trip.rs::print_then_parse_round_trips_every_fixture` — for every `.ail.json`, `print` then `parse` produces canonical- byte-equal output. Direction 1 above. +- `crates/ailang-surface/tests/round_trip.rs::parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture` + — for every `.ailx` text `t`, `parse(t)` and `parse(print(parse(t)))` + produce canonical-byte-equal AST. Direction 2 above. - `crates/ailang-surface/tests/round_trip.rs::every_ailx_fixture_matches_its_json_counterpart` — for every `.ailx` with a same-stem `.ail.json` counterpart, `parse` of the text yields canonical bytes equal to the JSON diff --git a/docs/journals/2026-05-12-audit-rt.md b/docs/journals/2026-05-12-audit-rt.md new file mode 100644 index 0000000..0137d64 --- /dev/null +++ b/docs/journals/2026-05-12-audit-rt.md @@ -0,0 +1,99 @@ +# audit-rt — Milestone close: Roundtrip Invariant + +**Date:** 2026-05-12 +**Milestone:** Roundtrip Invariant (rt.1 + rt.2) +**Status:** Closed, clean (after rt.tidy fixes) + +## Architect drift review + +`ailang-architect` reported `drift_found` with four items over +`daf9f4f..HEAD`: + +- `[high]` DESIGN.md §"Roundtrip Invariant" Direction 2 (text → + JSON → text idempotency) is stated as a property but not + directly enforced by any of the four listed tests. Test 1 covers + Direction 1; test 2 covers `.ailx` → JSON ≡ counterpart; test 4 + covers CLI render → parse. None tests `parse → print → parse` + idempotency on a hand-authored `.ailx` text directly. Architect + recommendation: claim soften OR add a fifth test. +- `[high]` `docs/roadmap.md` P1 entry "Round-trip completeness + invariant" still `- [ ]` (open) after the milestone closed. + Stale per roadmap convention (finished entries → `[x]` then + removal with one-line journal mirror). +- `[low]` DESIGN.md wording: `<16-hex>` in new section vs + `<16-lowercase-hex>` in §"Data model". Casing constraint is + load-bearing for canonical-byte equality; should be stated + identically. +- `[low]` rt.1 plan errata note (planner emitted ast.rs-mismatching + destructure scaffold). Single occurrence absorbed cleanly by the + Step-3.4 escape hatch. Not a deeper drift pattern; informational. + +## Bench-regression check + +`bench/check.py && bench/compile_check.py && bench/cross_lang.py`: + +- `check.py`: exit 0; 63 metrics; 0 regressed, 0 improved beyond + tolerance, 63 stable. +- `compile_check.py`: exit 0; 24 metrics; 0 regressed, 0 improved + beyond tolerance, 24 stable. +- `cross_lang.py`: exit 0; 25 metrics; 0 regressed, 0 improved + beyond tolerance, 25 stable. + +All three bench gates green. Bench was the most plausible source +of a regression given rt.1 added three new tests but the tests +are pure readers and are not exercised by the bench harness; the +green signal matches that expectation. + +## Resolution (rt.tidy, boss-direct edits) + +Three fixes, mechanical, no plan/implement dispatch: + +1. **Direction-2 enforcement** — + `crates/ailang-surface/tests/round_trip.rs` gains a third test + `parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`. + For every `.ailx` fixture asserts + `canonical_bytes(parse(t)) == canonical_bytes(parse(print(parse(t))))`. + For the 57 fixtures with a JSON counterpart this is logically + derivable from tests 1 + 2; the dedicated test stays robust for + future `.ailx` fixtures without a counterpart. DESIGN.md + §"Roundtrip Invariant" "Enforcement" list updated from four to + five tests; module doc-comment updated from "two complementary + checks" to "three complementary checks". Test PASS first-shot + across all 57 fixtures. + +2. **Roadmap P1 entry removed.** The "Round-trip completeness + invariant" milestone entry deleted; this journal entry is the + one-line mirror per roadmap convention. The P1 todo "Brainstorm + Step 7.5" remains; the P1 Show + print-rewire milestone remains. + +3. **Wording fix.** DESIGN.md §"Roundtrip Invariant" Float-bits-hex + description changed from `<16-hex>` to `<16-lowercase-hex>` to + match §"Data model". + +The `[low]` planner-errata carry-on item is retained in the rt.1 +journal "Concerns" section; it will surface again only if a future +tests-against-AST iteration repeats the pattern. + +## Spec acceptance criteria — final check + +1. ✓ Top-level `## Roundtrip Invariant` section in DESIGN.md; + Decision 6 Constraint 2 carries upward cross-reference. (rt.2) +2. ✓ `every_ailx_fixture_matches_its_json_counterpart` dynamic in + `crates/ailang-surface/tests/round_trip.rs`. (rt.1) +3. ✓ `crates/ailang-core/tests/schema_coverage.rs` with + exhaustive-match visitor; 34/34 variants observed. (rt.1) +4. ✓ `crates/ail/tests/roundtrip_cli.rs` with BLAKE3 identity + over `ail render` → tempfile → `ail parse`. (rt.1) +5. ✓ `cargo test --workspace` green with all four (now five) tests. + (rt.1 + rt.2 + rt.tidy) +6. ✓ No roundtrip gaps surfaced; no render/parse code changes + needed in this milestone. (rt.1) +7. ✓ Tests are pure readers; the CLI test's only filesystem side + effect is `tempfile::TempDir`. (rt.1) + +## Outcome + +Milestone closed clean. Five workspace-wide tests now anchor the +`.ail.json` ↔ `.ailx` bijection plus AST-variant coverage; the +property has a dedicated top-level section in DESIGN.md. No +follow-up iterations queued in this milestone. diff --git a/docs/journals/INDEX.md b/docs/journals/INDEX.md index f2b16a3..a5dca1d 100644 --- a/docs/journals/INDEX.md +++ b/docs/journals/INDEX.md @@ -18,3 +18,4 @@ - 2026-05-12 — audit-23: milestone 23 close (architect clean, compile_check ratified per H2 / 5-fn workload), DESIGN.md stub-fix tidy → 2026-05-12-audit-23.md - 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic); all PASS first-shot, no gaps surfaced → 2026-05-12-iter-rt.1.md - 2026-05-12 — iter rt.2: DESIGN.md anchor for Roundtrip Invariant (top-level section + Decision 6 Constraint 2 upward cross-reference) → 2026-05-12-iter-rt.2.md +- 2026-05-12 — audit-rt: milestone close (Roundtrip Invariant) — architect drift fixed in rt.tidy (3 items: Direction-2 5th test, roadmap P1 removed, wording sync); bench all-green → 2026-05-12-audit-rt.md diff --git a/docs/roadmap.md b/docs/roadmap.md index a5d17c7..34e4251 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -49,34 +49,6 @@ context. Pick the next milestone from P1.)_ has neither instance by design; polymorphic helpers at Float fire Float-aware `NoInstance` per DESIGN.md §"Float semantics". -- [ ] **\[milestone\]** Round-trip completeness invariant — `.ail.json` - ↔ `.ailx` must be a structurally lossless bijection. For every - valid module: `ail parse` followed by `ail render` yields the - original `.ailx` (modulo formatting), and `ail render` followed by - `ail parse` yields the original `.ail.json` (canonical key order). - Decision 6 mentions round-trip-as-property in passing, and - `crates/ailang-surface/tests/round_trip.rs` checks the `.ailx` - direction over the fixture corpus, but the invariant is not - prominently anchored as a non-negotiable language property and - the symmetric direction (every `.ail.json` has a printable, re- - parseable `.ailx` form) is not workspace-wide enforced. This - milestone (a) audits the existing fixture corpus for structural - gaps in either direction, (b) closes any render / parse gaps - surfaced, (c) lifts the round-trip property to a named, prominent - invariant in DESIGN.md (likely as a dedicated subsection of - Decision 6, or as its own decision-level entry), (d) adds a - workspace-wide check (E2E or unit, in addition to the existing - surface-side fixture test) that the invariant holds over every - fixture on every build. - - context: brainstorm 2026-05-12 — surfaced while scoping the - cross-model authoring-form test (see P2 entry below). The test - requires both forms to be exactly equipotent in expressive - power; otherwise measured form-preference is contaminated by - expressive-power gaps rather than authoring ergonomics. The - invariant is also load-bearing on its own: if `.ailx` is the - "AI authoring projection" per Decision 6, programs expressible - in one form but not the other are a first-class bug class. - - [ ] **\[todo\]** Brainstorm Step 7.5 — Grounding-check is the *last* step before the spec is committed. Any post-PASS edit to the spec (Boss-side polish, user-requested change, anything) must