iter rt.1: roundtrip-invariant audit tests — 3 new tests, all PASS first-shot
Three new reader-only tests pin the .ail.json / .ailx bijection plus AST-variant coverage. All three passed first observation across the current corpus — no roundtrip, schema-coverage, or CLI-drift gaps surfaced. - every_ailx_fixture_matches_its_json_counterpart (replaces the 3-pair handwritten exhibits): 57 paired .ailx fixtures pass. - every_ast_variant_is_observed_in_the_fixture_corpus (new): 34/34 AST variants exercised across 136 fixtures; exhaustive matches without _ wildcard so AST drift fails the build. - cli_render_then_parse_preserves_canonical_bytes_on_every_fixture (new): 136 fixtures round-trip through ail render → tempfile → ail parse with BLAKE3 identity on canonical bytes. No production code, no DESIGN.md changes (those follow in later iterations). Tests are pure readers of the repo per spec acceptance #7 — tempfile crate added as workspace dep for the CLI roundtrip's intermediate file outside the repo.
This commit is contained in:
@@ -0,0 +1,128 @@
|
||||
# iter rt.1 — Roundtrip-Invariant audit tests
|
||||
|
||||
**Date:** 2026-05-12
|
||||
**Started from:** 10ccd1406c886270d105cd7d5a08ec4b447030cd
|
||||
**Status:** DONE
|
||||
**Tasks completed:** 4 of 4
|
||||
|
||||
## Summary
|
||||
|
||||
Three new tests landed; one existing test (handwritten exhibits)
|
||||
was replaced with a dynamic counterpart. No production code or
|
||||
DESIGN.md changes (those would be later iterations if any gap
|
||||
surfaced). All three new tests passed on first observation —
|
||||
the corpus carries enough breadth that the spec-intended audit
|
||||
deliverable is the clean signal "no current gap" rather than a
|
||||
defect list.
|
||||
|
||||
## Per-task notes
|
||||
|
||||
- iter rt.1.1: `crates/ailang-surface/tests/round_trip.rs` —
|
||||
replaced static three-pair `handwritten_exhibits_match_json_fixtures`
|
||||
with dynamic `every_ailx_fixture_matches_its_json_counterpart`
|
||||
(and helper `list_ailx_fixtures`). Module doc rewritten to drop
|
||||
the "three hand-written exhibits" phrasing.
|
||||
- iter rt.1.2: `Cargo.toml` — `tempfile = "3"` added to
|
||||
`[workspace.dependencies]`. `crates/ail/Cargo.toml` — new
|
||||
`[dev-dependencies]` section with `tempfile.workspace = true`
|
||||
and `blake3.workspace = true`.
|
||||
- iter rt.1.3: `crates/ailang-core/tests/schema_coverage.rs` — new
|
||||
file. Exhaustive AST visitor (`Def`, `Term`, `Pattern`, `Literal`,
|
||||
`Type`, `ParamMode`) against the fixture corpus; flat `VariantTag`
|
||||
enum + `EXPECTED_VARIANTS` array; assertion that every variant is
|
||||
observed at least once. No `_` wildcards in outer match arms —
|
||||
AST drift fails the build, not the test.
|
||||
- iter rt.1.4: `crates/ail/tests/roundtrip_cli.rs` — new file.
|
||||
Subprocess pipeline `ail render → tempfile → ail parse`, BLAKE3
|
||||
hash identity on canonical bytes, for every `examples/*.ail.json`.
|
||||
|
||||
## Audit findings
|
||||
|
||||
- `every_ailx_fixture_matches_its_json_counterpart`:
|
||||
PASS. `.ailx cross-check ok (57 paired, 0 parse-only)`.
|
||||
All 57 `.ailx` fixtures parse and produce canonical bytes
|
||||
byte-equal to their same-stem `.ail.json` counterpart. Today
|
||||
no `.ailx` exists without a JSON counterpart.
|
||||
|
||||
- `every_ast_variant_is_observed_in_the_fixture_corpus`:
|
||||
PASS. `schema coverage ok: 34 variants observed across 136 fixtures`.
|
||||
All 34 expected `VariantTag` entries (5 `Def` + 13 `Term` + 4
|
||||
`Pattern` + 5 `Literal` + 4 `Type` + 3 `ParamMode`) are exercised
|
||||
by at least one fixture. No load failures across the 136-file
|
||||
corpus.
|
||||
|
||||
- `cli_render_then_parse_preserves_canonical_bytes_on_every_fixture`:
|
||||
PASS. `CLI roundtrip ok for 136 fixtures`. Every `examples/*.ail.json`
|
||||
survives `ail render → tempfile → ail parse` with BLAKE3 hash
|
||||
identity on canonical bytes.
|
||||
|
||||
- `cargo test --workspace --quiet` (Step 4.5): PASS, no
|
||||
regressions in the pre-existing test suite.
|
||||
|
||||
## Concerns
|
||||
|
||||
- Plan-text errata in Task 3's visitor scaffold. The plan in
|
||||
`docs/plans/rt.1.md` Step 3.2 destructured AST nodes with field
|
||||
names that do not match `crates/ailang-core/src/ast.rs` (the
|
||||
authoritative source). Step 3.4 of the plan explicitly anticipated
|
||||
this and prescribed inline correction by reading `ast.rs` and
|
||||
adjusting the destructuring. The corrections applied:
|
||||
- `Def::Fn(fd)`: dropped the `for m in &fd.params { visit_param_mode(m, …) }`
|
||||
loop (`fd.params: Vec<String>` is parameter names, not modes).
|
||||
- `Def::Const(cd)`: walks `cd.value` (the actual field name), not
|
||||
`cd.body`.
|
||||
- `Def::Class(cd)`: walks `m.ty` plus optional `m.default: Option<Term>`
|
||||
on each `ClassMethod`; no `m.body`/`m.params` fields exist on
|
||||
`ClassMethod`.
|
||||
- `Def::Instance(id)`: walks `id.type_: Type` (singular) and
|
||||
`id.methods`; no `id.args: Vec<Type>` field exists.
|
||||
`InstanceMethod` has only `name` and `body`.
|
||||
- `Term::App`: `{ callee, args, .. }` (not `{ f, args }`).
|
||||
- `Term::Let`: `{ value, body, .. }` (not `{ bound, body }`).
|
||||
- `Term::LetRec`: `{ ty, body, in_term, .. }` (per actual AST:
|
||||
`name, ty, params, body, in_term`), not the plan's
|
||||
`{ bindings, body }`.
|
||||
- `Term::If`: `{ cond, then, else_ }` (not `{ cond, then_b, else_b }`).
|
||||
- `Term::Do`: `{ args, .. }` only (no `stmts`/`ret` fields).
|
||||
- `Term::Lam`: `{ param_tys, ret_ty, body, .. }` (no `modes` field
|
||||
exists on `Lam`).
|
||||
- `Term::Seq`: `{ lhs, rhs }` (not `{ stmts, ret }`).
|
||||
- ParamMode discovery routed through `visit_type` on
|
||||
`Type::Fn { param_modes, ret_mode, .. }` — the only place
|
||||
`ParamMode` appears in the AST. The plan tried to visit
|
||||
ParamMode from `Def::Fn.params` (wrong type) and `Term::Lam.modes`
|
||||
(non-existent field).
|
||||
|
||||
The spec-compliance intent of Step 3.2 ("every AST enum variant is
|
||||
visited at least once via an exhaustive match with no `_` wildcard")
|
||||
is preserved; only the destructuring patterns adapted to the actual
|
||||
AST. All 34 expected variants observe coverage in the corpus, and
|
||||
the build remains a tripwire for future AST additions.
|
||||
|
||||
- Plan template note: a future `planner`/`brainstorm` pass on
|
||||
tests-against-AST might cheaply read `ast.rs` and emit a
|
||||
destructuring-correct scaffold instead of relying on the
|
||||
implementer to repair under the Step 3.4 escape hatch. Not
|
||||
rt.1 work.
|
||||
|
||||
## Known debt
|
||||
|
||||
- None added by this iter. The three new tests are pure readers
|
||||
and add no production-code surface; their existence pins three
|
||||
invariants that future iters can use to detect regression
|
||||
rather than rediscover.
|
||||
|
||||
## Files touched
|
||||
|
||||
- `Cargo.toml` — workspace dep `tempfile = "3"`.
|
||||
- `Cargo.lock` — auto-updated by cargo for `tempfile`.
|
||||
- `crates/ail/Cargo.toml` — new `[dev-dependencies]` section.
|
||||
- `crates/ailang-surface/tests/round_trip.rs` — module doc
|
||||
rewrite; `handwritten_exhibits_match_json_fixtures` →
|
||||
`every_ailx_fixture_matches_its_json_counterpart` (+ helper).
|
||||
- `crates/ailang-core/tests/schema_coverage.rs` — new file.
|
||||
- `crates/ail/tests/roundtrip_cli.rs` — new file.
|
||||
|
||||
## Stats
|
||||
|
||||
bench/orchestrator-stats/2026-05-12-iter-rt.1.json
|
||||
@@ -16,3 +16,4 @@
|
||||
- 2026-05-11 — iter 23.4: mono-pass unification (class methods + polymorphic free fns in one fixpoint; codegen-time specialiser removed) → 2026-05-11-iter-23.4.md
|
||||
- 2026-05-12 — iter 23.5: prelude free fns (ne/lt/le/gt/ge) + E2E (positive / user-ADT / Float-NoInstance); milestone 23 closed → 2026-05-12-iter-23.5.md
|
||||
- 2026-05-12 — audit-23: milestone 23 close (architect clean, compile_check ratified per H2 / 5-fn workload), DESIGN.md stub-fix tidy → 2026-05-12-audit-23.md
|
||||
- 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic); all PASS first-shot, no gaps surfaced → 2026-05-12-iter-rt.1.md
|
||||
|
||||
Reference in New Issue
Block a user