Inline pinning test in pipeline.rs constructs the same chain
shape module_name_from_json produces on serde-parse failure
(leaf wrapped with "parsing JSON in <path>" context) and
asserts both layers survive the formatter. RED→GREEN against
the property, not against production line bytes — for a
two-character fix, extracting a helper just for testability
would be over-engineering.
The strip_locations regex collapses \nCaused by: chains; the
{:#} alternate Display joins with ': ' on one line, so the
collapser is a no-op on the fixed output. AILX cohort
unaffected — only the harness-side anyhow path (module
rename pre-check) was dropping cause information.
Harness suite: 14/14 green (was 13/13 + 1 new pin).
Architect drift_found, both [low]:
- INDEX backfill for commit 90512d5 (brainstorm Step 7.5 SKILL.md
edit + roadmap P1 todo removed without a paired INDEX mirror).
- README "Total 8 passed" understated the actual 13/13 sweep.
Both fixed inline (mechanical, ≤30 LOC) per CLAUDE.md trivial-edits
carve-out; no implementer dispatch.
Bench: all three scripts exit 0, 0 regressed, 5 unexplained
latency.explicit_at_rc improvements not coupled to milestone work
(no in-workspace crate touched). Baseline left pristine — future
audits will reveal whether the improvements hold.
All 9 spec acceptance criteria green. Milestone closed clean.
Milestone "Cross-model authoring-form test" closed. Live IONOS run
against Qwen/Qwen3-Coder-Next executed at temperature=0, max-turns=5,
token-budget=500000. Raw dataset under runs/2026-05-12-df7531/:
RUN_STATUS=ok, scores.csv with 8 rows (4 tasks × 2 cohorts),
summary.md, plus 156 per-turn raw artefacts (request/response/program/
stderr/stdout) preserved for any future analysis.
Headline numbers:
- AILX cohort: 2/4 green, 1/4 first-attempt, ~95k prompt + ~2.6k completion
- JSON cohort: 1/4 green, 0/4 first-attempt, ~192k prompt + ~15k completion
- Only first-attempt success was AILX on t3_main_prints (~5 lines of
tagged s-expr vs ~23 lines of structured JSON for the JSON cohort's
turn-1 attempt that needed correction).
- Per spec, single subject + single deterministic run = data point,
not verdict.
DESIGN.md §"Decision 6: authoring surface" gains an "Empirical
addendum (2026-05-12)" subsection: 6-row metric table, illustrative
t3 contrast, explicit single-subject scope note pointing at the
roadmap follow-up entry for multi-subject expansion.
Roadmap: P2 entry "Cross-model authoring-form test" removed (this
journal is the convention's one-line mirror); P3 entry
"Multi-subject expansion (cross-model authoring-form follow-up)"
added with the run dir named as baseline.
Milestone artefacts span three iters (cma.1 master mini-spec +
renderer; cma.2 harness + tasks + reference solutions; cma.3 this
live run + DESIGN.md addendum + close).
Sibling standalone Cargo crate `harness/` under
experiments/2026-05-12-cross-model-authoring/ (out-of-workspace
idiom carried forward from cma.1 verbatim). Six modules:
strip_locations (regex pass for form-asymmetric location info,
calibrated against five real `ail check`/`ail parse` stderr
captures), pipeline (subprocess wrapper for parse|check|build +
5s-timeout exec; preflight on ail+clang), ionos (blocking reqwest
client + retry policy per spec), mock (canned-response loader),
scoring (CSV + summary.md), tasks (definition struct + loader).
main.rs ties them into the per-(cohort,task) loop with budget
accounting and per-turn artefact recording.
Four MVP tasks land with reference solutions that compile, build,
and execute green through the actual ail+clang pipeline:
t1_add_three (chained `+` + io/print_int), t2_length (polymorphic
List + recursion), t3_main_prints (minimal IO module), t4_count_zeros
(prelude Eq Int + branched if). Reference solutions stay in canonical
AILang form — param_modes is omitted when every parameter is the
Implicit default, consistent with the existing examples/ corpus.
13/13 tests green: 5 lib unit (strip_locations) + 5 integration
(strip_locations against verbatim captured fixtures) + 1
verify_references (drives every reference through the real
ail+clang pipeline) + 1 mock_full_run (full eight-row sweep with
mixed green-on-turn-2 + turn-limit cycles) + 1 budget_abort
(synthetic budget exhaustion with budget_abort rows + run_status).
Two implementer-phase repairs beyond the plan, both small and
surfaced in the iter journal Concerns:
1. pipeline.rs renames the program file to `<module-name>.ail.json`
between parse and check because `ail check` enforces filename
stem == module name (compiler contract the plan did not anticipate).
2. main.rs fills synthetic budget_abort rows for tasks the outer
loop never reached so scores.csv preserves the expected eight-row
shape on budget exhaustion.
One plan/text mismatch carried over: README "Total 8 passed" reflects
the plan's four-suite count; actual sweep produces 13. Doc-fix
candidate for cma.3.
cma.3 (live IONOS run + DESIGN.md addendum + roadmap edits) remains
out of scope.
New top-level experiments/2026-05-12-cross-model-authoring/ (out of
root Cargo workspace; nested-crate standalone via empty [workspace]
table per Cargo idiom). The renderer projects one canonical
master/spec.md into two form-specific files (rendered/json.md and
rendered/ailx.md) by walking three directive forms ({form-only:
json}, {form-only: ailx}, {example: <id>}) and emitting per-example
fenced blocks sourced from 13 curated .ail.json fixtures.
Four test gates green (10/10 total):
- splitter_unit (7) — directive parser, malformed inputs panic
- spec_completeness (1) — AST-variant visitor lifted verbatim from
schema_coverage.rs; 34/34 variants covered by the curated subset
- example_roundtrip (1) — every fixture roundtrips through
ailang_surface::{print, parse} to identical canonical bytes
- token_balance (1) — form-only block totals balanced to 3.22% under
tiktoken-rs::cl100k_base (gate is ±5%)
Six of the 34 variants did not have a dedicated fixture topic in the
plan and were absorbed into the closest existing fixture per the
plan's Step 4.14 escape hatch (Let/Clone → fn_calls_prelude; LetRec
→ data_with_match; If → match_literal_pattern; ReuseAs →
data_simple; Const → floats). Surfaced as a journal note, not a
plan revision.
Two ancillary additions vs. plan, both structural Cargo necessities,
spec-compatible: render/Cargo.toml carries an empty [workspace] table
so the nested crate refuses parent-workspace discovery, and
render/.gitignore drops /target + Cargo.lock. Both surfaced inline
in the per-iter journal's Concerns section.
cma.2 (harness) and cma.3 (live IONOS run + DESIGN.md addendum +
journal + roadmap edits) remain out of scope per the spec.
Brummel