iter ms.2: Qwen retroactive re-run + first CodeLlama-13b-Instruct run + DESIGN.md §Decision-6 addendum extended to two subjects
Two live IONOS harness invocations with the ms.1 fix in place:
- Qwen/Qwen3-Coder-Next → runs/2026-05-12-080864/ (RUN_STATUS=ok)
- meta-llama/CodeLlama-13b-Instruct-hf
→ runs/2026-05-12-9197fd/ (RUN_STATUS=ok;
2/8 JSON cells terminated as api_failure
on turn 1 zero-token — transient IONOS
terminal error, disclosed in addendum)
Per-cell results (reached/4, prompt tokens, completion tokens):
Qwen JSON 1/4 182,378 14,972
Qwen AILX 1/4 110,575 3,474
CodeLlama JSON 0/4 116,015 2,711 [2 cells api_failure]
CodeLlama AILX 2/4 93,017 2,234
Both subjects agree on direction: AILX cohort cheaper on prompt
tokens (Qwen 61%, CodeLlama 80% of JSON-cohort spend) and on
completion tokens (Qwen 23%, CodeLlama 82%); AILX reached-green
≥ JSON reached-green for each subject. Three of four
first-attempt-green cells across the two subjects are AILX.
Failure classes split symmetric to form: JSON cohorts fail at
typecheck, AILX cohorts at parse — each form's
front-of-pipeline check.
DESIGN.md §"Decision 6 / Empirical addendum (2026-05-12)"
replaced: 2-column single-subject view → 4-column two-subject
view, framing paragraph reports direction agreement and the
api_failure / Qwen-rerun-noise anomalies, scope paragraph
updated to 'two subjects, n=1 each, deterministic; not a
verdict — a universal claim would need ≥3 subjects with
statistical robustness'.
Roadmap P3 'Multi-subject expansion' entry removed; scoped
goal met for two subjects.
Milestone token spend: prompt 501,985 + completion 23,391
= 525,376.
Brummel
65a4f0aa16