iter cma.3: live Qwen3-Coder-Next run + DESIGN.md §Decision-6 empirical addendum + milestone close

Milestone "Cross-model authoring-form test" closed. Live IONOS run
against Qwen/Qwen3-Coder-Next executed at temperature=0, max-turns=5,
token-budget=500000. Raw dataset under runs/2026-05-12-df7531/:
RUN_STATUS=ok, scores.csv with 8 rows (4 tasks × 2 cohorts),
summary.md, plus 156 per-turn raw artefacts (request/response/program/
stderr/stdout) preserved for any future analysis.

Headline numbers:
- AILX cohort: 2/4 green, 1/4 first-attempt, ~95k prompt + ~2.6k completion
- JSON cohort: 1/4 green, 0/4 first-attempt, ~192k prompt + ~15k completion
- Only first-attempt success was AILX on t3_main_prints (~5 lines of
  tagged s-expr vs ~23 lines of structured JSON for the JSON cohort's
  turn-1 attempt that needed correction).
- Per spec, single subject + single deterministic run = data point,
  not verdict.

DESIGN.md §"Decision 6: authoring surface" gains an "Empirical
addendum (2026-05-12)" subsection: 6-row metric table, illustrative
t3 contrast, explicit single-subject scope note pointing at the
roadmap follow-up entry for multi-subject expansion.

Roadmap: P2 entry "Cross-model authoring-form test" removed (this
journal is the convention's one-line mirror); P3 entry
"Multi-subject expansion (cross-model authoring-form follow-up)"
added with the run dir named as baseline.

Milestone artefacts span three iters (cma.1 master mini-spec +
renderer; cma.2 harness + tasks + reference solutions; cma.3 this
live run + DESIGN.md addendum + close).

docs/roadmap.md

@@ -72,80 +72,6 @@ context. Pick the next milestone from P1.)_
 
 ## P2 — Medium-term
 
-- [ ] **\[milestone\]** Cross-model authoring-form test — empirically
-  measure whether `.ail.json` or `.ailx` is the form a foreign LLM
-  author reaches for and succeeds with. Setup: three configurations
-  per task — (i) JSON-mandated, (ii) `.ailx`-mandated, (iii) free
-  choice (both corpora available, model picks). Multiple subjects to
-  check whether form-preference is model-robust. For each (model ×
-  form × task): system prompt includes DESIGN.md verbatim (~28k
-  tokens); user prompt is the task plus the form mandate; an error-
-  roundtrip loop feeds `ail check` / `ail parse` errors back to the
-  model until green or iter-limit. Measure: first-attempt success
-  (yes/no), iterations to green, total tokens spent, error classes
-  encountered (schema violation, type mismatch, hash mismatch, syntax
-  error in the chosen form); for free-choice runs, which form the
-  model picked unprompted. The output is a JOURNAL'd dataset +
-  interpretation + a concrete DESIGN.md edit on Decision 6 —
-  either ratify "ailx is the AI authoring projection" with empirical
-  backing across N subjects, or retire the form if JSON wins
-  consistently across all subjects.
-  - context: brainstorm 2026-05-12 (aborted in favour of two roadmap
-    entries). The Decision-6 prose currently asserts `.ailx` is the
-    AI authoring projection while §"Feature-acceptance criterion"
-    cites JSON-as-canonical-authoring-surface as a behavioural
-    counterexample to human-driven feature picks. The tension is
-    real; only empirical data over multiple downstream subjects
-    resolves it.
-  - depends on: Round-trip completeness invariant (P1 above) —
-    without it, measured preference is confounded by expressive-
-    power gaps. Strongly benefits from (but is not strictly
-    blocked by): `ail check`/`build`/`run` accept `.ailx`
-    extension (P2 todo below); workspace search beyond entry-
-    module's directory (P2 todo below). Both will be encountered
-    during the test if not closed first.
-  - smoke-test evidence (2026-05-12): Qwen3-Coder-Next reads
-    DESIGN.md and produces a structurally correct AILang program
-    that passes `ail check` in two turns; first turn missed the
-    module-name-matches-file-path rule, second turn fixed it after
-    `ail check` error was fed back. ~28k prompt tokens, ~150
-    completion tokens, ~2-5 s latency per turn.
-  - how to address the IONOS endpoint:
-    - URL: `https://openai.inference.de-txl.ionos.com/v1/chat/completions`
-    - protocol: OpenAI-compatible chat completions API.
-      `/v1/models` returns the catalogue.
-    - auth: HTTP header `Authorization: Bearer <token>`. Store the
-      token outside the repo (e.g. `~/.ionos_token`, `chmod 600`).
-      Tokens are JWTs; the one used in the 2026-05-12 smoke test
-      was IONOS-issued and time-bound — refresh when re-engaging.
-    - model IDs to try (subset of the IONOS catalogue as of
-      2026-05-12, sorted by relevance to this test):
-      - `Qwen/Qwen3-Coder-Next` — code-specialised, smoke-tested.
-      - `meta-llama/Meta-Llama-3.1-405B-Instruct-FP8` — large
-        general instruct, useful as a contrast subject.
-      - `meta-llama/Llama-3.3-70B-Instruct` — mid-size instruct.
-      - `openai/gpt-oss-120b` — OpenAI open-weight 120B.
-      - `mistralai/Mistral-Small-24B-Instruct` — smaller subject,
-        stresses spec comprehension.
-      - `meta-llama/CodeLlama-13b-Instruct-hf` — older code model,
-        useful as a low-capability floor.
-    - request shape: standard `messages: [{role: "system", ...},
-      {role: "user", ...}, ...]`, with DESIGN.md (107 KB / ~28k
-      tokens) in the system message and the task + form mandate in
-      the first user message. Subsequent turns alternate
-      `assistant` / `user` with the `ail check` error verbatim in
-      the user message.
-    - response shape: `choices[0].message.content` carries the
-      generated program (request "no markdown fences, no
-      explanation" in the prompt to skip post-processing).
-      `usage.prompt_tokens` + `usage.completion_tokens` give per-
-      turn token spend.
-    - cost estimate: ~56k tokens per task per (model × form)
-      configuration including a few correction roundtrips; a
-      full matrix (4 tasks × 3 forms × 4 models × 1 run each) is
-      ~2.7M tokens total. Multiply by repetition count if a
-      median-of-3 design is chosen.
-
 - [ ] **\[feature\]** Operator routing through `Eq` / `Ord` — `==`,
   `<` etc. resolved via the typeclass instead of the built-in
   primitive comparators. No commitment; gated on bench re-baselining
@@ -323,3 +249,20 @@ context. Pick the next milestone from P1.)_
 - [ ] **\[idea\]** Richer integration paths between RC and
   uniqueness — deferred from the 21' arc; revisit once the
   uniqueness inference covers more program shapes.
+
+- [ ] **\[milestone\]** Multi-subject expansion (cross-model
+  authoring-form follow-up) — extend the cross-model authoring-form
+  test to ≥2 additional foreign LLMs and ≥4 more tasks (broader
+  surface coverage; current MVP run hit only 3/8 green so the
+  failure clusters dominate the signal). Optional: repetitions for
+  statistical robustness; calibrate strip_locations to also strip
+  `at byte N` from AILX-cohort feedback so the asymmetry observed
+  in the cma.3 baseline is removed before re-measuring. Goal: turn
+  the single-subject data point recorded in DESIGN.md §"Decision 6
+  / Empirical addendum (2026-05-12)" into either a ratification or
+  retirement of Decision 6's universal claim. Baseline run:
+  `experiments/2026-05-12-cross-model-authoring/runs/2026-05-12-df7531/`.
+  - context: JOURNAL 2026-05-12 ("iter cma.3"). Other IONOS catalogue
+    models named in the brainstorm record (Meta-Llama-3.1-405B,
+    Llama-3.3-70B, gpt-oss-120b, Mistral-Small-24B, CodeLlama-13b)
+    are reasonable next subjects.