All 176 files in the four accumulating directories now use a zero-padded 4-digit counter prefix that reflects creation order (`NNNN-slug.md`). The counter is assigned per directory in strict git-log creation order; ties broken alphabetically by original name. The old `YYYY-MM-DD-` prefix on docs/specs/ and docs/plans/ files is dropped — the date is recoverable from git log and the counter carries the ordering. A file's counter is stable for the life of the file: never reassigned, never reused, never compacted. Deleted files retire their counter; subsequent files do not fill the gap. This is the property that lets cross-references stay literal — refs use the full filename including the counter (`design/contracts/0007-honesty-rule.md`) so they grep cleanly and resolve directly without a glob step. 313 cross-references updated across .md/.rs/.toml/.c/.json files (test pins, include_str! paths, design-INDEX entries, baseline notes, runtime C comments, inter-contract markdown links incl. bare basename and `../models/foo.md` forms). CLAUDE.md gets a new "File-naming convention" section spelling out the rule and rationale. skills/brainstorm/SKILL.md and skills/planner/SKILL.md updated so new spec/plan creation produces counter-prefixed names from the start. The full test suite (cargo test --workspace) passes.
30 KiB
rt.1 — Roundtrip Invariant Audit Tests — Implementation Plan
Parent spec:
docs/specs/0015-roundtrip-invariant.mdFor agentic workers: REQUIRED SUB-SKILL: use
skills/implementto run this plan. Steps use- [ ]checkboxes for tracking.
Goal: Add three new test files that together verify the
.ail.json ↔ .ailx bijection plus AST-variant coverage of the
fixture corpus, all dynamically iterating examples/ (no hardcoded
fixture lists). Surface any roundtrip / coverage gaps as fail-RED
test output for later iterations to fix.
Architecture: Three reader-only test files added to existing
crates: replace one static-pair test in ailang-surface with a
dynamic-pair one; add new tests/schema_coverage.rs to
ailang-core (exhaustive-match AST visitor against
fixture-corpus observed variants); add new tests/roundtrip_cli.rs
to ail (subprocess render → tempfile → parse, BLAKE3 identity).
One dev-deps Cargo edit on the ail crate plus one workspace dep
(tempfile) — no production code changes.
Tech Stack: ailang-core (AST + canonical bytes), ailang-surface
(parse/print), ail (CLI binary under test), tempfile = "3",
blake3 (already workspace).
Files this plan creates or modifies:
- Modify:
crates/ailang-surface/tests/round_trip.rs:1-14, 93-133— replacehandwritten_exhibits_match_json_fixtureswith the dynamicevery_ailx_fixture_matches_its_json_counterpart; update the module-level doc comment to drop the "three hand-written exhibits" wording. - Modify:
Cargo.toml:19-26— addtempfile = "3"to[workspace.dependencies]. - Modify:
crates/ail/Cargo.toml:11-19— append a[dev-dependencies]section withtempfile.workspace = trueandblake3.workspace = true. - Create:
crates/ailang-core/tests/schema_coverage.rs— exhaustive AST visitor over fixture-loaded modules; flatVariantTagenum; asserts every variant ofDef/Term/Pattern/Literal/Type/ParamModeis observed at least once. - Create:
crates/ail/tests/roundtrip_cli.rs— dynamic iteration overexamples/*.ail.json, subprocess pipelineail render→tempfile::TempDir→ail parse, BLAKE3 hash of canonical bytes on both sides.
Task 1: Replace static-pair handwritten test with dynamic pair cross-check
Files:
-
Modify:
crates/ailang-surface/tests/round_trip.rs:1-14, 93-133 -
Step 1.1: Update the module-level doc comment
Replace the existing
crates/ailang-surface/tests/round_trip.rs:1-14doc block with://! Round-trip gate for the form-(A) projection. //! //! Two complementary checks over `examples/`: //! //! 1. `print_then_parse_round_trips_every_fixture`: for every //! `examples/*.ail.json` fixture, load → `print` → `parse` → //! canonical bytes; assert byte-equal to original. //! //! 2. `every_ailx_fixture_matches_its_json_counterpart`: for every //! `examples/*.ailx` fixture, parse → canonical bytes; if a //! same-stem `.ail.json` counterpart exists, assert canonical-byte //! equality against it. The `.ailx` side is the human/AI authoring //! ground-truth — fixtures must be writeable without going through //! the printer and still match the JSON-AST. //! //! Both tests gather fixtures dynamically via `read_dir`; no hardcoded //! lists. The tests are pure readers — they do not write into the //! working tree. -
Step 1.2: Replace the test function
Delete
handwritten_exhibits_match_json_fixturesatcrates/ailang-surface/tests/round_trip.rs:93-133and replace it with:fn list_ailx_fixtures() -> Vec<PathBuf> { let dir = examples_dir(); let mut paths: Vec<PathBuf> = std::fs::read_dir(&dir) .unwrap_or_else(|e| panic!("read_dir({}): {e}", dir.display())) .filter_map(|entry| entry.ok()) .map(|e| e.path()) .filter(|p| { p.file_name() .and_then(|n| n.to_str()) .map(|n| n.ends_with(".ailx")) .unwrap_or(false) }) .collect(); paths.sort(); paths } #[test] fn every_ailx_fixture_matches_its_json_counterpart() { let fixtures = list_ailx_fixtures(); assert!( !fixtures.is_empty(), "no .ailx fixtures found under {}", examples_dir().display() ); let mut failures = Vec::<String>::new(); let mut paired = 0usize; let mut parse_only = 0usize; for ailx_path in &fixtures { let text = match std::fs::read_to_string(ailx_path) { Ok(t) => t, Err(e) => { failures.push(format!("{}: read failed: {e}", ailx_path.display())); continue; } }; let parsed = match ailang_surface::parse(&text) { Ok(m) => m, Err(e) => { failures.push(format!("{}: parse failed: {e}", ailx_path.display())); continue; } }; // Same-stem counterpart lookup. `<stem>.ailx` → `<stem>.ail.json`. let stem = ailx_path .file_name() .and_then(|n| n.to_str()) .and_then(|n| n.strip_suffix(".ailx")) .unwrap_or(""); let json_path = ailx_path.with_file_name(format!("{stem}.ail.json")); if !json_path.exists() { // Spec: parse success alone is sufficient for `.ailx` files // without a JSON counterpart. (Today: none expected.) parse_only += 1; continue; } let original = match ailang_core::load_module(&json_path) { Ok(m) => m, Err(e) => { failures.push(format!( "{}: load_module({}) failed: {e}", ailx_path.display(), json_path.display() )); continue; } }; let bytes_orig = ailang_core::canonical::to_bytes(&original); let bytes_parsed = ailang_core::canonical::to_bytes(&parsed); if bytes_orig != bytes_parsed { let s_orig = String::from_utf8_lossy(&bytes_orig).into_owned(); let s_round = String::from_utf8_lossy(&bytes_parsed).into_owned(); failures.push(format!( "{} vs {}: canonical bytes differ.\noriginal: {s_orig}\nparsed: {s_round}", ailx_path.display(), json_path.display() )); continue; } paired += 1; } if !failures.is_empty() { panic!( "{} of {} .ailx fixture(s) failed cross-check (paired ok: {}, parse-only: {}):\n{}", failures.len(), fixtures.len(), paired, parse_only, failures.join("\n\n") ); } eprintln!( ".ailx cross-check ok ({} paired, {} parse-only)", paired, parse_only ); } -
Step 1.3: Run the updated test
Run:
cargo test --quiet -p ailang-surface --test round_tripExpected: either PASS for both tests (all 57
.ailxfixtures pair correctly with their.ail.jsoncounterparts), or a structured panic listing every mismatching fixture pair with canonical-byte diffs. A failure here is iteration deliverable, not a regression: capture the list verbatim into the per-iter journal as the audit output for later iterations.
Task 2: Workspace dep tempfile, ail dev-deps for tempfile + blake3
Files:
-
Modify:
Cargo.toml:19-26 -
Modify:
crates/ail/Cargo.toml:11-19 -
Step 2.1: Add
tempfileto workspace dependenciesEdit
Cargo.toml. After theindexmapline in[workspace.dependencies](currently line 26), append:tempfile = "3"The full
[workspace.dependencies]block after the edit reads:[workspace.dependencies] serde = { version = "1", features = ["derive"] } serde_json = { version = "1", features = ["preserve_order"] } blake3 = "1" anyhow = "1" thiserror = "1" clap = { version = "4", features = ["derive"] } indexmap = { version = "2", features = ["serde"] } tempfile = "3" ailang-core = { path = "crates/ailang-core" } ... -
Step 2.2: Add dev-deps to ail crate
Edit
crates/ail/Cargo.toml. Append after the existing[dependencies]block (after line 19):[dev-dependencies] tempfile.workspace = true blake3.workspace = true -
Step 2.3: Verify the workspace still builds and tests still pass
Run:
cargo test --workspace --quietExpected: PASS — all existing tests still green. New deps are declared but not yet used (Task 4 will use them).
Task 3: New crates/ailang-core/tests/schema_coverage.rs
Files:
-
Create:
crates/ailang-core/tests/schema_coverage.rs -
Step 3.1: Create the file with the visitor scaffold
Write
crates/ailang-core/tests/schema_coverage.rswith this initial scaffold (imports +VariantTagenum + helper signatures)://! Schema-coverage audit: every AST enum variant of `Def`, `Term`, //! `Pattern`, `Literal`, `Type`, `ParamMode` is exercised by at //! least one fixture under `examples/`. //! //! Why this exists: the round-trip tests in `ailang-surface` are //! only as strong as the fixture corpus that drives them. A schema //! variant with zero fixture coverage round-trips trivially — there //! is nothing to round-trip. This test makes the corpus's coverage //! observable, so a gap surfaces as a fail-loud test, not as silent //! confidence. //! //! How it stays in sync with the AST: every match arm below is //! exhaustive (no `_ => ...` wildcard). When a new AST variant //! lands, the compiler rejects this file until the visitor and the //! `VariantTag` enum are extended in lockstep. //! //! Pure reader: this test loads fixtures but does not modify any //! committed content. A failure means a fixture must be added (in //! a separate iteration); the test must not be relaxed. use std::collections::HashSet; use std::path::PathBuf; use ailang_core::ast::{ Def, InstanceMethod, Literal, Module, ParamMode, Pattern, Term, Type, }; #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] enum VariantTag { // Def DefFn, DefConst, DefType, DefClass, DefInstance, // Term TermLit, TermVar, TermApp, TermLet, TermLetRec, TermIf, TermDo, TermCtor, TermMatch, TermLam, TermSeq, TermClone, TermReuseAs, // Pattern PatternWild, PatternVar, PatternLit, PatternCtor, // Literal LiteralInt, LiteralBool, LiteralStr, LiteralUnit, LiteralFloat, // Type TypeCon, TypeFn, TypeVar, TypeForall, // ParamMode ParamModeImplicit, ParamModeOwn, ParamModeBorrow, } /// Lists every expected variant. Stored as a const array so that /// adding a new `VariantTag` here and forgetting to add the /// corresponding match arm in the visitor is a Rust compile error /// (and vice versa). const EXPECTED_VARIANTS: &[VariantTag] = &[ VariantTag::DefFn, VariantTag::DefConst, VariantTag::DefType, VariantTag::DefClass, VariantTag::DefInstance, VariantTag::TermLit, VariantTag::TermVar, VariantTag::TermApp, VariantTag::TermLet, VariantTag::TermLetRec, VariantTag::TermIf, VariantTag::TermDo, VariantTag::TermCtor, VariantTag::TermMatch, VariantTag::TermLam, VariantTag::TermSeq, VariantTag::TermClone, VariantTag::TermReuseAs, VariantTag::PatternWild, VariantTag::PatternVar, VariantTag::PatternLit, VariantTag::PatternCtor, VariantTag::LiteralInt, VariantTag::LiteralBool, VariantTag::LiteralStr, VariantTag::LiteralUnit, VariantTag::LiteralFloat, VariantTag::TypeCon, VariantTag::TypeFn, VariantTag::TypeVar, VariantTag::TypeForall, VariantTag::ParamModeImplicit, VariantTag::ParamModeOwn, VariantTag::ParamModeBorrow, ]; -
Step 3.2: Add the per-enum visitor functions
Append to
crates/ailang-core/tests/schema_coverage.rs:fn visit_def(def: &Def, observed: &mut HashSet<VariantTag>) { match def { Def::Fn(fd) => { observed.insert(VariantTag::DefFn); for m in &fd.params { visit_param_mode(m, observed); } visit_type(&fd.ty, observed); visit_term(&fd.body, observed); } Def::Const(cd) => { observed.insert(VariantTag::DefConst); visit_type(&cd.ty, observed); visit_term(&cd.body, observed); } Def::Type(td) => { observed.insert(VariantTag::DefType); for ctor in &td.ctors { for field_ty in &ctor.fields { visit_type(field_ty, observed); } } } Def::Class(cd) => { observed.insert(VariantTag::DefClass); for m in &cd.methods { visit_type(&m.ty, observed); } } Def::Instance(id) => { observed.insert(VariantTag::DefInstance); for arg in &id.args { visit_type(arg, observed); } for m in &id.methods { visit_instance_method(m, observed); } } } } fn visit_instance_method(m: &InstanceMethod, observed: &mut HashSet<VariantTag>) { for mode in &m.params { visit_param_mode(mode, observed); } visit_term(&m.body, observed); } fn visit_term(t: &Term, observed: &mut HashSet<VariantTag>) { match t { Term::Lit { lit } => { observed.insert(VariantTag::TermLit); visit_literal(lit, observed); } Term::Var { .. } => { observed.insert(VariantTag::TermVar); } Term::App { f, args } => { observed.insert(VariantTag::TermApp); visit_term(f, observed); for a in args { visit_term(a, observed); } } Term::Let { bound, body, .. } => { observed.insert(VariantTag::TermLet); visit_term(bound, observed); visit_term(body, observed); } Term::LetRec { bindings, body } => { observed.insert(VariantTag::TermLetRec); for (_n, _ty, bterm) in bindings { visit_term(bterm, observed); } visit_term(body, observed); } Term::If { cond, then_b, else_b } => { observed.insert(VariantTag::TermIf); visit_term(cond, observed); visit_term(then_b, observed); visit_term(else_b, observed); } Term::Do { stmts, ret } => { observed.insert(VariantTag::TermDo); for s in stmts { visit_term(s, observed); } visit_term(ret, observed); } Term::Ctor { args, .. } => { observed.insert(VariantTag::TermCtor); for a in args { visit_term(a, observed); } } Term::Match { scrutinee, arms } => { observed.insert(VariantTag::TermMatch); visit_term(scrutinee, observed); for arm in arms { visit_pattern(&arm.pat, observed); visit_term(&arm.body, observed); } } Term::Lam { modes, body, .. } => { observed.insert(VariantTag::TermLam); for m in modes { visit_param_mode(m, observed); } visit_term(body, observed); } Term::Seq { stmts, ret } => { observed.insert(VariantTag::TermSeq); for s in stmts { visit_term(s, observed); } visit_term(ret, observed); } Term::Clone { value } => { observed.insert(VariantTag::TermClone); visit_term(value, observed); } Term::ReuseAs { source, body } => { observed.insert(VariantTag::TermReuseAs); visit_term(source, observed); visit_term(body, observed); } } } fn visit_pattern(p: &Pattern, observed: &mut HashSet<VariantTag>) { match p { Pattern::Wild => { observed.insert(VariantTag::PatternWild); } Pattern::Var { .. } => { observed.insert(VariantTag::PatternVar); } Pattern::Lit { lit } => { observed.insert(VariantTag::PatternLit); visit_literal(lit, observed); } Pattern::Ctor { fields, .. } => { observed.insert(VariantTag::PatternCtor); for f in fields { visit_pattern(f, observed); } } } } fn visit_literal(l: &Literal, observed: &mut HashSet<VariantTag>) { match l { Literal::Int { .. } => { observed.insert(VariantTag::LiteralInt); } Literal::Bool { .. } => { observed.insert(VariantTag::LiteralBool); } Literal::Str { .. } => { observed.insert(VariantTag::LiteralStr); } Literal::Unit => { observed.insert(VariantTag::LiteralUnit); } Literal::Float { .. } => { observed.insert(VariantTag::LiteralFloat); } } } fn visit_type(t: &Type, observed: &mut HashSet<VariantTag>) { match t { Type::Con { args, .. } => { observed.insert(VariantTag::TypeCon); for a in args { visit_type(a, observed); } } Type::Fn { params, ret, .. } => { observed.insert(VariantTag::TypeFn); for p in params { visit_type(p, observed); } visit_type(ret, observed); } Type::Var { .. } => { observed.insert(VariantTag::TypeVar); } Type::Forall { body, .. } => { observed.insert(VariantTag::TypeForall); visit_type(body, observed); } } } fn visit_param_mode(m: &ParamMode, observed: &mut HashSet<VariantTag>) { match m { ParamMode::Implicit => { observed.insert(VariantTag::ParamModeImplicit); } ParamMode::Own => { observed.insert(VariantTag::ParamModeOwn); } ParamMode::Borrow => { observed.insert(VariantTag::ParamModeBorrow); } } } fn visit_module(m: &Module, observed: &mut HashSet<VariantTag>) { for def in &m.defs { visit_def(def, observed); } } -
Step 3.3: Add fixture discovery + the top-level test
Append to
crates/ailang-core/tests/schema_coverage.rs:fn examples_dir() -> PathBuf { let crate_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); crate_dir.parent().unwrap().parent().unwrap().join("examples") } fn list_json_fixtures() -> Vec<PathBuf> { let dir = examples_dir(); let mut paths: Vec<PathBuf> = std::fs::read_dir(&dir) .unwrap_or_else(|e| panic!("read_dir({}): {e}", dir.display())) .filter_map(|entry| entry.ok()) .map(|e| e.path()) .filter(|p| { p.file_name() .and_then(|n| n.to_str()) .map(|n| n.ends_with(".ail.json")) .unwrap_or(false) }) .collect(); paths.sort(); paths } #[test] fn every_ast_variant_is_observed_in_the_fixture_corpus() { let fixtures = list_json_fixtures(); assert!( !fixtures.is_empty(), "no .ail.json fixtures found under {}", examples_dir().display() ); let mut observed: HashSet<VariantTag> = HashSet::new(); let mut load_failures = Vec::<String>::new(); for path in &fixtures { match ailang_core::load_module(path) { Ok(m) => visit_module(&m, &mut observed), Err(e) => load_failures.push(format!("{}: {e}", path.display())), } } // Load failures fail the test on their own — a fixture that // does not load is a corpus bug, surface it loud. if !load_failures.is_empty() { panic!( "{} fixture(s) failed to load during coverage scan:\n{}", load_failures.len(), load_failures.join("\n") ); } let missing: Vec<VariantTag> = EXPECTED_VARIANTS .iter() .copied() .filter(|v| !observed.contains(v)) .collect(); if !missing.is_empty() { let names: Vec<String> = missing.iter().map(|v| format!("{v:?}")).collect(); panic!( "{} AST variant(s) have NO fixture coverage in {}:\n {}\n\n\ every variant must be exercised by at least one fixture; \ add a fixture that uses each missing variant (do NOT \ weaken this test).", missing.len(), examples_dir().display(), names.join("\n ") ); } eprintln!( "schema coverage ok: {} variants observed across {} fixtures", observed.len(), fixtures.len() ); } -
Step 3.4: Verify the visitor field names match the AST
Run:
cargo build --quiet -p ailang-core --testsExpected: PASS. If the build fails, the visitor's destructuring pattern does not match the current AST struct field names — fix the field names by reading
crates/ailang-core/src/ast.rsand adjusting the destructuring pattern in the offending arm. Do NOT add_wildcards in the outermatcharms; field-level destructuring may use..to ignore irrelevant fields. -
Step 3.5: Run the schema-coverage test
Run:
cargo test --quiet -p ailang-core --test schema_coverageExpected: either PASS (every AST variant has at least one fixture-exemplar in
examples/), or a structured panic listing the missing variants. A failure here is iteration deliverable — capture the missing-variant list into the per-iter journal as audit output.
Task 4: New crates/ail/tests/roundtrip_cli.rs
Files:
-
Create:
crates/ail/tests/roundtrip_cli.rs -
Step 4.1: Create the file with imports, helpers, and binary lookup
Write
crates/ail/tests/roundtrip_cli.rs://! CLI roundtrip: every `examples/*.ail.json` survives //! `ail render` → tempfile → `ail parse` with BLAKE3 hash //! identity on canonical bytes. //! //! This is the user-facing-surface defence line. Crate-internal //! roundtrip tests in `ailang-surface` cover the same property //! at the library level; this test additionally protects the //! `ail render` and `ail parse` CLI wrappers from drift (output //! formatting, exit codes, stdout framing). //! //! Pure reader: the test writes only to a `tempfile::TempDir` //! that lives outside the repo and is cleaned up by Drop. No //! committed content is mutated. use std::path::{Path, PathBuf}; use std::process::Command; fn ail_bin() -> &'static str { env!("CARGO_BIN_EXE_ail") } fn workspace_root() -> PathBuf { let manifest_dir = env!("CARGO_MANIFEST_DIR"); Path::new(manifest_dir).parent().unwrap().parent().unwrap().to_path_buf() } fn examples_dir() -> PathBuf { workspace_root().join("examples") } fn list_json_fixtures() -> Vec<PathBuf> { let dir = examples_dir(); let mut paths: Vec<PathBuf> = std::fs::read_dir(&dir) .unwrap_or_else(|e| panic!("read_dir({}): {e}", dir.display())) .filter_map(|entry| entry.ok()) .map(|e| e.path()) .filter(|p| { p.file_name() .and_then(|n| n.to_str()) .map(|n| n.ends_with(".ail.json")) .unwrap_or(false) }) .collect(); paths.sort(); paths } fn strip_trailing_newlines(mut v: Vec<u8>) -> Vec<u8> { while v.last() == Some(&b'\n') { v.pop(); } v } -
Step 4.2: Add the per-fixture roundtrip helper
Append to
crates/ail/tests/roundtrip_cli.rs:/// Run the full render → parse pipeline for one fixture. Returns /// Ok(()) on hash-identity, Err(reason) otherwise. fn roundtrip_one(fixture: &Path, tmpdir: &Path) -> Result<(), String> { // Step A: load the original fixture and compute canonical bytes // (independent of how the JSON file is formatted on disk). let original_module = ailang_core::load_module(fixture) .map_err(|e| format!("load_module: {e}"))?; let bytes_orig = ailang_core::canonical::to_bytes(&original_module); let h_orig = blake3::hash(&bytes_orig); // Step B: `ail render <fixture>` → captured stdout = ailx text. let render_out = Command::new(ail_bin()) .args(["render", fixture.to_str().unwrap()]) .output() .map_err(|e| format!("spawn ail render: {e}"))?; if !render_out.status.success() { return Err(format!( "ail render exit={:?}\nstderr: {}", render_out.status.code(), String::from_utf8_lossy(&render_out.stderr), )); } // Step C: write the rendered .ailx into the tempdir. let stem = fixture .file_name() .and_then(|n| n.to_str()) .and_then(|n| n.strip_suffix(".ail.json")) .unwrap_or("fixture"); let tmp_ailx = tmpdir.join(format!("{stem}.round.ailx")); std::fs::write(&tmp_ailx, &render_out.stdout) .map_err(|e| format!("write tempfile {}: {e}", tmp_ailx.display()))?; // Step D: `ail parse <tmp_ailx>` → captured stdout = canonical // bytes + trailing newline. let parse_out = Command::new(ail_bin()) .args(["parse", tmp_ailx.to_str().unwrap()]) .output() .map_err(|e| format!("spawn ail parse: {e}"))?; if !parse_out.status.success() { return Err(format!( "ail parse exit={:?}\nstderr: {}", parse_out.status.code(), String::from_utf8_lossy(&parse_out.stderr), )); } let bytes_round = strip_trailing_newlines(parse_out.stdout); let h_round = blake3::hash(&bytes_round); if h_orig != h_round { let s_orig = String::from_utf8_lossy(&bytes_orig).into_owned(); let s_round = String::from_utf8_lossy(&bytes_round).into_owned(); return Err(format!( "BLAKE3 mismatch:\n orig: {}\n round: {}\noriginal bytes: {s_orig}\nround bytes: {s_round}", h_orig.to_hex(), h_round.to_hex(), )); } Ok(()) } -
Step 4.3: Add the top-level dynamic test
Append to
crates/ail/tests/roundtrip_cli.rs:#[test] fn cli_render_then_parse_preserves_canonical_bytes_on_every_fixture() { let fixtures = list_json_fixtures(); assert!( !fixtures.is_empty(), "no .ail.json fixtures found under {}", examples_dir().display() ); let tmpdir = tempfile::TempDir::new().expect("create tempdir"); let mut failures = Vec::<String>::new(); let mut passed = 0usize; for fixture in &fixtures { match roundtrip_one(fixture, tmpdir.path()) { Ok(()) => passed += 1, Err(msg) => failures.push(format!("{}: {msg}", fixture.display())), } } if !failures.is_empty() { panic!( "CLI roundtrip failed for {} of {} fixtures (passed: {}):\n{}", failures.len(), fixtures.len(), passed, failures.join("\n\n") ); } eprintln!("CLI roundtrip ok for {passed} fixtures"); } -
Step 4.4: Run the CLI roundtrip test
Run:
cargo test --quiet -p ail --test roundtrip_cliExpected: either PASS (every
.ail.jsonround-trips through the CLI), or a structured panic listing each fixture that failed with its diagnostic. A failure here is iteration deliverable — capture the failing-fixture list into the per-iter journal as audit output. -
Step 4.5: Final workspace test run
Run:
cargo test --workspace --quietExpected: every existing test still PASS plus the three new tests report their status. If any of the three new tests fail, the failing-fixture lists are the audit deliverable; do NOT relax the tests to make them green. Capture the full failure output into the per-iter journal entry.
Per-iter journal entry
After Task 4 completes, append a per-iter journal file at
docs/journals/2026-05-12-iter-rt.1.md with this skeleton (filled
in with the actual audit findings from steps 1.3, 3.5, 4.4, 4.5):
# iter rt.1 — Roundtrip-Invariant audit tests
Three new tests landed; one existing test (handwritten exhibits)
was replaced with a dynamic counterpart. No production code or
DESIGN.md changes (those are later iterations).
## Audit findings
- `every_ailx_fixture_matches_its_json_counterpart`: <PASS or list>
- `every_ast_variant_is_observed_in_the_fixture_corpus`: <PASS or list of missing variants>
- `cli_render_then_parse_preserves_canonical_bytes_on_every_fixture`: <PASS or list of failing fixtures>
## Follow-up
If any of the three reported failures, queue rt.2..N to fix each
gap class in render or parse code (never by relaxing tests).
Also append a one-line entry to docs/journals/INDEX.md:
- 2026-05-12 — iter rt.1: roundtrip invariant audit tests (3 new tests + ailx-pair dynamic) → 2026-05-12-iter-rt.1.md