audit-rt: milestone close — Direction-2 5th test + roadmap entry retired + wording sync
Milestone-close audit found three drift items (architect) plus two carry-on items; bench all-green. Fixes (rt.tidy, boss-direct): - 5th test parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture enforces Direction 2 of the Roundtrip Invariant directly. The DESIGN.md enforcement list grows from four to five tests. - docs/roadmap.md P1 'Round-trip completeness invariant' entry retired with one-line journal mirror per roadmap convention. - DESIGN.md wording sync: <16-hex> → <16-lowercase-hex> for the Float-bits-hex spelling in the new section (matches §Data model). Milestone closed clean; five workspace-wide tests anchor the .ail.json ↔ .ailx bijection plus AST-variant coverage.
This commit is contained in:
@@ -1,21 +1,27 @@
|
|||||||
//! Round-trip gate for the form-(A) projection.
|
//! Round-trip gate for the form-(A) projection.
|
||||||
//!
|
//!
|
||||||
//! Two complementary checks over `examples/`:
|
//! Three complementary checks over `examples/`, each gathering
|
||||||
|
//! fixtures dynamically via `read_dir` (no hardcoded lists). The
|
||||||
|
//! tests are pure readers — they do not write into the working
|
||||||
|
//! tree.
|
||||||
//!
|
//!
|
||||||
//! 1. `print_then_parse_round_trips_every_fixture`: for every
|
//! 1. `print_then_parse_round_trips_every_fixture`: Direction 1 of
|
||||||
//! `examples/*.ail.json` fixture, load → `print` → `parse` →
|
//! the Roundtrip Invariant (DESIGN.md §"Roundtrip Invariant").
|
||||||
//! canonical bytes; assert byte-equal to original.
|
//! For every `examples/*.ail.json` fixture, load → `print` →
|
||||||
|
//! `parse` → canonical bytes; assert byte-equal to original.
|
||||||
//!
|
//!
|
||||||
//! 2. `every_ailx_fixture_matches_its_json_counterpart`: for every
|
//! 2. `every_ailx_fixture_matches_its_json_counterpart`: hand-
|
||||||
//! `examples/*.ailx` fixture, parse → canonical bytes; if a
|
//! authored ground-truth check. For every `examples/*.ailx`
|
||||||
//! same-stem `.ail.json` counterpart exists, assert canonical-byte
|
//! fixture, parse → canonical bytes; if a same-stem `.ail.json`
|
||||||
//! equality against it. The `.ailx` side is the human/AI authoring
|
//! counterpart exists, assert canonical-byte equality against
|
||||||
//! ground-truth — fixtures must be writeable without going through
|
//! it. Pins the `.ailx` corpus against semantic drift between
|
||||||
//! the printer and still match the JSON-AST.
|
//! the two forms at the fixture level.
|
||||||
//!
|
//!
|
||||||
//! Both tests gather fixtures dynamically via `read_dir`; no hardcoded
|
//! 3. `parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`:
|
||||||
//! lists. The tests are pure readers — they do not write into the
|
//! Direction 2 of the Roundtrip Invariant. For every well-formed
|
||||||
//! working tree.
|
//! `.ailx` text `t`, asserts `canonical_bytes(parse(t))` equals
|
||||||
|
//! `canonical_bytes(parse(print(parse(t))))`. Robust against
|
||||||
|
//! future `.ailx` fixtures without a JSON counterpart.
|
||||||
|
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
@@ -196,3 +202,76 @@ fn every_ailx_fixture_matches_its_json_counterpart() {
|
|||||||
paired, parse_only
|
paired, parse_only
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Direction 2 of the Roundtrip Invariant (DESIGN.md §"Roundtrip
|
||||||
|
/// Invariant"): for every well-formed `.ailx` text `t`, the
|
||||||
|
/// composition `parse → print → parse` is idempotent on the AST.
|
||||||
|
///
|
||||||
|
/// For the 57 `.ailx` fixtures that have a JSON counterpart this
|
||||||
|
/// follows logically from `print_then_parse_round_trips_every_fixture`
|
||||||
|
/// + `every_ailx_fixture_matches_its_json_counterpart`. This test
|
||||||
|
/// asserts the property directly so it stays robust for future
|
||||||
|
/// `.ailx` fixtures without a JSON counterpart, and so the spec's
|
||||||
|
/// Direction-2 claim has a dedicated enforcement point.
|
||||||
|
#[test]
|
||||||
|
fn parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture() {
|
||||||
|
let fixtures = list_ailx_fixtures();
|
||||||
|
assert!(
|
||||||
|
!fixtures.is_empty(),
|
||||||
|
"no .ailx fixtures found under {}",
|
||||||
|
examples_dir().display()
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut failures = Vec::<String>::new();
|
||||||
|
let mut passed = 0usize;
|
||||||
|
for path in &fixtures {
|
||||||
|
let text = match std::fs::read_to_string(path) {
|
||||||
|
Ok(t) => t,
|
||||||
|
Err(e) => {
|
||||||
|
failures.push(format!("{}: read failed: {e}", path.display()));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let parsed_once = match ailang_surface::parse(&text) {
|
||||||
|
Ok(m) => m,
|
||||||
|
Err(e) => {
|
||||||
|
failures.push(format!("{}: parse(t) failed: {e}", path.display()));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let printed = ailang_surface::print(&parsed_once);
|
||||||
|
let parsed_twice = match ailang_surface::parse(&printed) {
|
||||||
|
Ok(m) => m,
|
||||||
|
Err(e) => {
|
||||||
|
failures.push(format!(
|
||||||
|
"{}: parse(print(parse(t))) failed: {e}\n--- printed ---\n{printed}",
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let bytes_a = ailang_core::canonical::to_bytes(&parsed_once);
|
||||||
|
let bytes_b = ailang_core::canonical::to_bytes(&parsed_twice);
|
||||||
|
if bytes_a != bytes_b {
|
||||||
|
let s_a = String::from_utf8_lossy(&bytes_a).into_owned();
|
||||||
|
let s_b = String::from_utf8_lossy(&bytes_b).into_owned();
|
||||||
|
failures.push(format!(
|
||||||
|
"{}: parse → print → parse is NOT idempotent.\nparse(t): {s_a}\nparse(print(parse(t))): {s_b}",
|
||||||
|
path.display()
|
||||||
|
));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
passed += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if !failures.is_empty() {
|
||||||
|
panic!(
|
||||||
|
"{} of {} .ailx fixture(s) failed idempotency check (passed: {}):\n{}",
|
||||||
|
failures.len(),
|
||||||
|
fixtures.len(),
|
||||||
|
passed,
|
||||||
|
failures.join("\n\n")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
eprintln!("parse→print→parse idempotency ok for {passed} .ailx fixtures");
|
||||||
|
}
|
||||||
|
|||||||
+6
-3
@@ -1781,8 +1781,8 @@ without changing the identity of the module it produces.
|
|||||||
|
|
||||||
Float literals carry an IEEE-754 bit pattern, not a decimal
|
Float literals carry an IEEE-754 bit pattern, not a decimal
|
||||||
approximation. The canonical encoding is
|
approximation. The canonical encoding is
|
||||||
`{"kind":"float","bits":"<16-hex>"}` and the surface emits the
|
`{"kind":"float","bits":"<16-lowercase-hex>"}` and the surface
|
||||||
same bits-hex string. NaN, ±Inf, signed zero, and subnormals all
|
emits the same bits-hex string. NaN, ±Inf, signed zero, and subnormals all
|
||||||
round-trip exactly because the JSON-number path is bypassed (see
|
round-trip exactly because the JSON-number path is bypassed (see
|
||||||
§"Float semantics", "Form-A serialisation"). Floats are not an
|
§"Float semantics", "Form-A serialisation"). Floats are not an
|
||||||
exception to the invariant — the bits-hex encoding is the
|
exception to the invariant — the bits-hex encoding is the
|
||||||
@@ -1790,7 +1790,7 @@ mechanism that keeps them *inside* it.
|
|||||||
|
|
||||||
### Enforcement
|
### Enforcement
|
||||||
|
|
||||||
The invariant is workspace-CI-enforced by four tests, each
|
The invariant is workspace-CI-enforced by five tests, each
|
||||||
operating on the `examples/` corpus via dynamic `read_dir`
|
operating on the `examples/` corpus via dynamic `read_dir`
|
||||||
collection (no hardcoded fixture list, so newly added fixtures
|
collection (no hardcoded fixture list, so newly added fixtures
|
||||||
inherit the gate automatically):
|
inherit the gate automatically):
|
||||||
@@ -1798,6 +1798,9 @@ inherit the gate automatically):
|
|||||||
- `crates/ailang-surface/tests/round_trip.rs::print_then_parse_round_trips_every_fixture`
|
- `crates/ailang-surface/tests/round_trip.rs::print_then_parse_round_trips_every_fixture`
|
||||||
— for every `.ail.json`, `print` then `parse` produces canonical-
|
— for every `.ail.json`, `print` then `parse` produces canonical-
|
||||||
byte-equal output. Direction 1 above.
|
byte-equal output. Direction 1 above.
|
||||||
|
- `crates/ailang-surface/tests/round_trip.rs::parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`
|
||||||
|
— for every `.ailx` text `t`, `parse(t)` and `parse(print(parse(t)))`
|
||||||
|
produce canonical-byte-equal AST. Direction 2 above.
|
||||||
- `crates/ailang-surface/tests/round_trip.rs::every_ailx_fixture_matches_its_json_counterpart`
|
- `crates/ailang-surface/tests/round_trip.rs::every_ailx_fixture_matches_its_json_counterpart`
|
||||||
— for every `.ailx` with a same-stem `.ail.json` counterpart,
|
— for every `.ailx` with a same-stem `.ail.json` counterpart,
|
||||||
`parse` of the text yields canonical bytes equal to the JSON
|
`parse` of the text yields canonical bytes equal to the JSON
|
||||||
|
|||||||
@@ -0,0 +1,99 @@
|
|||||||
|
# audit-rt — Milestone close: Roundtrip Invariant
|
||||||
|
|
||||||
|
**Date:** 2026-05-12
|
||||||
|
**Milestone:** Roundtrip Invariant (rt.1 + rt.2)
|
||||||
|
**Status:** Closed, clean (after rt.tidy fixes)
|
||||||
|
|
||||||
|
## Architect drift review
|
||||||
|
|
||||||
|
`ailang-architect` reported `drift_found` with four items over
|
||||||
|
`daf9f4f..HEAD`:
|
||||||
|
|
||||||
|
- `[high]` DESIGN.md §"Roundtrip Invariant" Direction 2 (text →
|
||||||
|
JSON → text idempotency) is stated as a property but not
|
||||||
|
directly enforced by any of the four listed tests. Test 1 covers
|
||||||
|
Direction 1; test 2 covers `.ailx` → JSON ≡ counterpart; test 4
|
||||||
|
covers CLI render → parse. None tests `parse → print → parse`
|
||||||
|
idempotency on a hand-authored `.ailx` text directly. Architect
|
||||||
|
recommendation: claim soften OR add a fifth test.
|
||||||
|
- `[high]` `docs/roadmap.md` P1 entry "Round-trip completeness
|
||||||
|
invariant" still `- [ ]` (open) after the milestone closed.
|
||||||
|
Stale per roadmap convention (finished entries → `[x]` then
|
||||||
|
removal with one-line journal mirror).
|
||||||
|
- `[low]` DESIGN.md wording: `<16-hex>` in new section vs
|
||||||
|
`<16-lowercase-hex>` in §"Data model". Casing constraint is
|
||||||
|
load-bearing for canonical-byte equality; should be stated
|
||||||
|
identically.
|
||||||
|
- `[low]` rt.1 plan errata note (planner emitted ast.rs-mismatching
|
||||||
|
destructure scaffold). Single occurrence absorbed cleanly by the
|
||||||
|
Step-3.4 escape hatch. Not a deeper drift pattern; informational.
|
||||||
|
|
||||||
|
## Bench-regression check
|
||||||
|
|
||||||
|
`bench/check.py && bench/compile_check.py && bench/cross_lang.py`:
|
||||||
|
|
||||||
|
- `check.py`: exit 0; 63 metrics; 0 regressed, 0 improved beyond
|
||||||
|
tolerance, 63 stable.
|
||||||
|
- `compile_check.py`: exit 0; 24 metrics; 0 regressed, 0 improved
|
||||||
|
beyond tolerance, 24 stable.
|
||||||
|
- `cross_lang.py`: exit 0; 25 metrics; 0 regressed, 0 improved
|
||||||
|
beyond tolerance, 25 stable.
|
||||||
|
|
||||||
|
All three bench gates green. Bench was the most plausible source
|
||||||
|
of a regression given rt.1 added three new tests but the tests
|
||||||
|
are pure readers and are not exercised by the bench harness; the
|
||||||
|
green signal matches that expectation.
|
||||||
|
|
||||||
|
## Resolution (rt.tidy, boss-direct edits)
|
||||||
|
|
||||||
|
Three fixes, mechanical, no plan/implement dispatch:
|
||||||
|
|
||||||
|
1. **Direction-2 enforcement** —
|
||||||
|
`crates/ailang-surface/tests/round_trip.rs` gains a third test
|
||||||
|
`parse_then_print_then_parse_is_idempotent_on_every_ailx_fixture`.
|
||||||
|
For every `.ailx` fixture asserts
|
||||||
|
`canonical_bytes(parse(t)) == canonical_bytes(parse(print(parse(t))))`.
|
||||||
|
For the 57 fixtures with a JSON counterpart this is logically
|
||||||
|
derivable from tests 1 + 2; the dedicated test stays robust for
|
||||||
|
future `.ailx` fixtures without a counterpart. DESIGN.md
|
||||||
|
§"Roundtrip Invariant" "Enforcement" list updated from four to
|
||||||
|
five tests; module doc-comment updated from "two complementary
|
||||||
|
checks" to "three complementary checks". Test PASS first-shot
|
||||||
|
across all 57 fixtures.
|
||||||
|
|
||||||
|
2. **Roadmap P1 entry removed.** The "Round-trip completeness
|
||||||
|
invariant" milestone entry deleted; this journal entry is the
|
||||||
|
one-line mirror per roadmap convention. The P1 todo "Brainstorm
|
||||||
|
Step 7.5" remains; the P1 Show + print-rewire milestone remains.
|
||||||
|
|
||||||
|
3. **Wording fix.** DESIGN.md §"Roundtrip Invariant" Float-bits-hex
|
||||||
|
description changed from `<16-hex>` to `<16-lowercase-hex>` to
|
||||||
|
match §"Data model".
|
||||||
|
|
||||||
|
The `[low]` planner-errata carry-on item is retained in the rt.1
|
||||||
|
journal "Concerns" section; it will surface again only if a future
|
||||||
|
tests-against-AST iteration repeats the pattern.
|
||||||
|
|
||||||
|
## Spec acceptance criteria — final check
|
||||||
|
|
||||||
|
1. ✓ Top-level `## Roundtrip Invariant` section in DESIGN.md;
|
||||||
|
Decision 6 Constraint 2 carries upward cross-reference. (rt.2)
|
||||||
|
2. ✓ `every_ailx_fixture_matches_its_json_counterpart` dynamic in
|
||||||
|
`crates/ailang-surface/tests/round_trip.rs`. (rt.1)
|
||||||
|
3. ✓ `crates/ailang-core/tests/schema_coverage.rs` with
|
||||||
|
exhaustive-match visitor; 34/34 variants observed. (rt.1)
|
||||||
|
4. ✓ `crates/ail/tests/roundtrip_cli.rs` with BLAKE3 identity
|
||||||
|
over `ail render` → tempfile → `ail parse`. (rt.1)
|
||||||
|
5. ✓ `cargo test --workspace` green with all four (now five) tests.
|
||||||
|
(rt.1 + rt.2 + rt.tidy)
|
||||||
|
6. ✓ No roundtrip gaps surfaced; no render/parse code changes
|
||||||
|
needed in this milestone. (rt.1)
|
||||||
|
7. ✓ Tests are pure readers; the CLI test's only filesystem side
|
||||||
|
effect is `tempfile::TempDir`. (rt.1)
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Milestone closed clean. Five workspace-wide tests now anchor the
|
||||||
|
`.ail.json` ↔ `.ailx` bijection plus AST-variant coverage; the
|
||||||
|
property has a dedicated top-level section in DESIGN.md. No
|
||||||
|
follow-up iterations queued in this milestone.
|
||||||
@@ -18,3 +18,4 @@
|
|||||||
- 2026-05-12 — audit-23: milestone 23 close (architect clean, compile_check ratified per H2 / 5-fn workload), DESIGN.md stub-fix tidy → 2026-05-12-audit-23.md
|
- 2026-05-12 — audit-23: milestone 23 close (architect clean, compile_check ratified per H2 / 5-fn workload), DESIGN.md stub-fix tidy → 2026-05-12-audit-23.md
|
||||||
- 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic); all PASS first-shot, no gaps surfaced → 2026-05-12-iter-rt.1.md
|
- 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic); all PASS first-shot, no gaps surfaced → 2026-05-12-iter-rt.1.md
|
||||||
- 2026-05-12 — iter rt.2: DESIGN.md anchor for Roundtrip Invariant (top-level section + Decision 6 Constraint 2 upward cross-reference) → 2026-05-12-iter-rt.2.md
|
- 2026-05-12 — iter rt.2: DESIGN.md anchor for Roundtrip Invariant (top-level section + Decision 6 Constraint 2 upward cross-reference) → 2026-05-12-iter-rt.2.md
|
||||||
|
- 2026-05-12 — audit-rt: milestone close (Roundtrip Invariant) — architect drift fixed in rt.tidy (3 items: Direction-2 5th test, roadmap P1 removed, wording sync); bench all-green → 2026-05-12-audit-rt.md
|
||||||
|
|||||||
@@ -49,34 +49,6 @@ context. Pick the next milestone from P1.)_
|
|||||||
has neither instance by design; polymorphic helpers at Float fire
|
has neither instance by design; polymorphic helpers at Float fire
|
||||||
Float-aware `NoInstance` per DESIGN.md §"Float semantics".
|
Float-aware `NoInstance` per DESIGN.md §"Float semantics".
|
||||||
|
|
||||||
- [ ] **\[milestone\]** Round-trip completeness invariant — `.ail.json`
|
|
||||||
↔ `.ailx` must be a structurally lossless bijection. For every
|
|
||||||
valid module: `ail parse` followed by `ail render` yields the
|
|
||||||
original `.ailx` (modulo formatting), and `ail render` followed by
|
|
||||||
`ail parse` yields the original `.ail.json` (canonical key order).
|
|
||||||
Decision 6 mentions round-trip-as-property in passing, and
|
|
||||||
`crates/ailang-surface/tests/round_trip.rs` checks the `.ailx`
|
|
||||||
direction over the fixture corpus, but the invariant is not
|
|
||||||
prominently anchored as a non-negotiable language property and
|
|
||||||
the symmetric direction (every `.ail.json` has a printable, re-
|
|
||||||
parseable `.ailx` form) is not workspace-wide enforced. This
|
|
||||||
milestone (a) audits the existing fixture corpus for structural
|
|
||||||
gaps in either direction, (b) closes any render / parse gaps
|
|
||||||
surfaced, (c) lifts the round-trip property to a named, prominent
|
|
||||||
invariant in DESIGN.md (likely as a dedicated subsection of
|
|
||||||
Decision 6, or as its own decision-level entry), (d) adds a
|
|
||||||
workspace-wide check (E2E or unit, in addition to the existing
|
|
||||||
surface-side fixture test) that the invariant holds over every
|
|
||||||
fixture on every build.
|
|
||||||
- context: brainstorm 2026-05-12 — surfaced while scoping the
|
|
||||||
cross-model authoring-form test (see P2 entry below). The test
|
|
||||||
requires both forms to be exactly equipotent in expressive
|
|
||||||
power; otherwise measured form-preference is contaminated by
|
|
||||||
expressive-power gaps rather than authoring ergonomics. The
|
|
||||||
invariant is also load-bearing on its own: if `.ailx` is the
|
|
||||||
"AI authoring projection" per Decision 6, programs expressible
|
|
||||||
in one form but not the other are a first-class bug class.
|
|
||||||
|
|
||||||
- [ ] **\[todo\]** Brainstorm Step 7.5 — Grounding-check is the *last*
|
- [ ] **\[todo\]** Brainstorm Step 7.5 — Grounding-check is the *last*
|
||||||
step before the spec is committed. Any post-PASS edit to the spec
|
step before the spec is committed. Any post-PASS edit to the spec
|
||||||
(Boss-side polish, user-requested change, anything) must
|
(Boss-side polish, user-requested change, anything) must
|
||||||
|
|||||||
Reference in New Issue
Block a user