AILang/docs/journals/2026-05-12-audit-cma.md

audit-cma — Milestone close: Cross-model authoring-form test

Date: 2026-05-12 Milestone: Cross-model authoring-form test (cma.1 + cma.2 + cma.3) Status: Closed, clean (after two [low] tidy fixes)

Architect drift review

ailang-architect reported drift_found with two items over 44c6e56..HEAD:

• [low] docs/journals/INDEX.md — commit 90512d5 ("brainstorm Step 7.5 re-dispatch on any post-PASS spec edit") removed the corresponding P1 todo from roadmap.md but did not add an INDEX mirror. CLAUDE.md's roadmap convention requires a one-line mirror in the journals when a roadmap entry is removed.
• [low] experiments/.../README.md — the "Total 8 passed" sentence inherited from the cma.2 plan understates the actual harness test sweep (13 passed across --lib unit + 4 integration suites). Surface cosmetic; flagged in cma.3 known-debt.

Both items confined to record-keeping, neither blocks milestone close. Architect recommendation: carry on as planned.

Tidy fixes (boss-direct, inline)

1. INDEX backfill for 90512d5. Added a single-line entry in docs/journals/INDEX.md pointing at the commit and noting that no per-iter journal accompanies it (a small skill-discipline tweak rather than an iter). The format mirrors existing entries so future agents can navigate from INDEX to the commit history without reconstructing.

2. README test-count correction. The five-suite layout under experiments/2026-05-12-cross-model-authoring/README.md's "Running the harness" section now lists all five suites explicitly (inline --lib unit tests for strip_locations, plus the four integration files) and reports the correct total (13 passed).

Both edits are ≤30 LOC and require no design judgement (CLAUDE.md "trivial mechanical edits" carve-out); no planner+implement dispatch needed.

Bench-regression check

bench/check.py && bench/compile_check.py && bench/cross_lang.py:

• check.py: exit 0; 63 metrics; 0 regressed, 5 improved beyond tolerance, 58 stable.
• compile_check.py: exit 0; 24 metrics; 0 regressed, 0 improved beyond tolerance, 24 stable.
• cross_lang.py: exit 0; 25 metrics; 0 regressed, 0 improved beyond tolerance, 25 stable.

All three exit-code gates green. The five improvements on check.py cluster on latency.explicit_at_rc (p99 -36.5%, p99_9 -41.8%, max -41.7%, p99_over_median -38.0%) plus throughput.bench_list_sum.gc_over_bump -8.7%.

Decision: no baseline update (carry-on). Rationale:

• The cma milestone touched no in-workspace crate. Every line of code change lives under experiments/2026-05-12-cross-model-authoring/ which is out of the root Cargo workspace by design. No mechanism in the milestone could plausibly improve runtime latency or GC-over-bump throughput.
• The improvements therefore reflect either (a) runtime variance / scheduler / clang version drift between the prior audit-rt baseline measurement and now, or (b) genuine system-level improvement uncorrelated with the milestone.
• Ratifying an unexplained improvement would lock it into the baseline and lose the ability to detect a future regression back to the prior level. Better to leave the baseline pristine; if the improvement holds across the next two or three audits, the next audit can ratify with confidence.
• This is the audit-rt pattern carried forward: don't ratify what the milestone didn't intentionally move.

Spec acceptance criteria — final check

Reviewing the parent spec docs/specs/2026-05-12-cross-model-authoring-form-test.md §"Acceptance criteria" against landed work:

1. ✓ Directory experiments/2026-05-12-cross-model-authoring/ exists with all four sub-components (master/, render/, harness/, runs/) plus top-level README. (cma.1 + cma.2)
2. ✓ master/spec.md is vollumfänglich; spec_completeness.rs gate green; 34/34 AST variants covered by the curated subset. (cma.1)
3. ✓ Both rendered/json.md and rendered/ailx.md are checked in; all render/ tests green. (cma.1)
4. ✓ harness/ builds, passes 13/13 tests including the mock_full_run end-to-end. (cma.2)
5. ✓ Four task definitions + reference solutions exist; the verify_references integration test confirms each reaches green through the real ail+clang pipeline locally. (cma.2)
6. ✓ Live run executed against IONOS (Qwen3-Coder-Next, temperature=0, top_p=1, max-turns=5) and recorded under runs/2026-05-12-df7531/ with RUN_STATUS=ok, full per-turn raw artefacts for all eight (cohort × task) combinations, scores.csv, and summary.md. (cma.3)
7. ✓ DESIGN.md §"Decision 6" gained an "Empirical addendum (2026-05-12)" subsection — 27 lines, model id + run date + per-cohort means table + illustrative t3 contrast + explicit scope note deferring the universal claim to a multi-subject expansion. (cma.3)
8. ✓ Journal entry 2026-05-12-iter-cma.3.md linked from INDEX.md summarises the data point. (cma.3)
9. ✓ Roadmap P2 entry removed; P3 entry "Multi-subject expansion (cross-model authoring-form follow-up)" added pointing at the run as baseline. (cma.3)

All nine criteria green.

Outcome

Milestone closed clean after the two record-keeping tidies above. The cross-model authoring-form test ships its harness, four MVP tasks with verified reference solutions, a single-subject baseline dataset, and a calibrated DESIGN.md addendum. The next step (P3 roadmap: multi-subject expansion) is queued but not started.