iter cma.2: harness binary + 4 tasks + reference solutions + 4 integration tests

Sibling standalone Cargo crate `harness/` under
experiments/2026-05-12-cross-model-authoring/ (out-of-workspace
idiom carried forward from cma.1 verbatim). Six modules:
strip_locations (regex pass for form-asymmetric location info,
calibrated against five real `ail check`/`ail parse` stderr
captures), pipeline (subprocess wrapper for parse|check|build +
5s-timeout exec; preflight on ail+clang), ionos (blocking reqwest
client + retry policy per spec), mock (canned-response loader),
scoring (CSV + summary.md), tasks (definition struct + loader).
main.rs ties them into the per-(cohort,task) loop with budget
accounting and per-turn artefact recording.

Four MVP tasks land with reference solutions that compile, build,
and execute green through the actual ail+clang pipeline:
t1_add_three (chained `+` + io/print_int), t2_length (polymorphic
List + recursion), t3_main_prints (minimal IO module), t4_count_zeros
(prelude Eq Int + branched if). Reference solutions stay in canonical
AILang form — param_modes is omitted when every parameter is the
Implicit default, consistent with the existing examples/ corpus.

13/13 tests green: 5 lib unit (strip_locations) + 5 integration
(strip_locations against verbatim captured fixtures) + 1
verify_references (drives every reference through the real
ail+clang pipeline) + 1 mock_full_run (full eight-row sweep with
mixed green-on-turn-2 + turn-limit cycles) + 1 budget_abort
(synthetic budget exhaustion with budget_abort rows + run_status).

Two implementer-phase repairs beyond the plan, both small and
surfaced in the iter journal Concerns:
1. pipeline.rs renames the program file to `<module-name>.ail.json`
   between parse and check because `ail check` enforces filename
   stem == module name (compiler contract the plan did not anticipate).
2. main.rs fills synthetic budget_abort rows for tasks the outer
   loop never reached so scores.csv preserves the expected eight-row
   shape on budget exhaustion.

One plan/text mismatch carried over: README "Total 8 passed" reflects
the plan's four-suite count; actual sweep produces 13. Doc-fix
candidate for cma.3.

cma.3 (live IONOS run + DESIGN.md addendum + roadmap edits) remains
out of scope.

experiments/2026-05-12-cross-model-authoring/README.md

@@ -40,3 +40,39 @@ prints to AILX, reparses to the same canonical bytes), `spec_completeness`
 (every AST variant in `ailang_core::ast` is exercised by at least
 one example), `token_balance` (form-only blocks balanced within ±5%
 across the two rendered files).
+
+## Running the harness
+
+Live mode (one full eight-run sweep, ~480k tokens budget by default):
+
+```
+export IONOS_API_TOKEN="<token>"   # see roadmap entry for token provenance
+cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
+  --rendered experiments/2026-05-12-cross-model-authoring/rendered \
+  --tasks    experiments/2026-05-12-cross-model-authoring/master/tasks \
+  --out      experiments/2026-05-12-cross-model-authoring/runs \
+  --model    Qwen/Qwen3-Coder-Next
+```
+
+The harness pre-flights `ail --version` and `clang --version` before
+the first API call. Set `AIL_BIN` if `ail` is not on PATH.
+
+Mock mode (offline; CI-friendly; bypasses the IONOS API):
+
+```
+cargo run --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml -- \
+  --rendered experiments/2026-05-12-cross-model-authoring/rendered \
+  --tasks    experiments/2026-05-12-cross-model-authoring/master/tasks \
+  --out      /tmp/mock-runs \
+  --model    mock \
+  --mock     experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/mock_full_run.json
+```
+
+Tests:
+
+```
+cargo test --manifest-path experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml
+```
+
+Four test suites: `strip_locations` (5), `verify_references` (1),
+`mock_full_run` (1), `budget_abort` (1). Total 8 passed.

experiments/2026-05-12-cross-model-authoring/harness/.gitignore

@@ -0,0 +1,2 @@
+/target
+Cargo.lock

experiments/2026-05-12-cross-model-authoring/harness/Cargo.toml

@@ -0,0 +1,34 @@
+# Standalone crate, intentionally outside the root workspace.
+# The empty [workspace] table prevents Cargo's automatic
+# workspace-discovery from attaching this crate to the root manifest.
+# See parent spec §Architecture lines 90–95 ("not added to root
+# workspace members; built locally inside the experiment directory").
+[workspace]
+
+[package]
+name = "xmodel-harness"
+version = "0.0.1"
+edition = "2021"
+publish = false
+
+[[bin]]
+name = "xmodel-harness"
+path = "src/main.rs"
+
+[lib]
+name = "xmodel_harness"
+path = "src/lib.rs"
+
+[dependencies]
+clap       = { version = "4", features = ["derive"] }
+reqwest    = { version = "0.12", features = ["blocking", "json", "rustls-tls"], default-features = false }
+regex      = "1"
+serde      = { version = "1", features = ["derive"] }
+serde_json = "1"
+anyhow     = "1"
+chrono     = { version = "0.4", default-features = false, features = ["clock"] }
+thiserror  = "1"
+tempfile   = "3"
+
+[dev-dependencies]
+tempfile = "3"

experiments/2026-05-12-cross-model-authoring/harness/src/ionos.rs

@@ -0,0 +1,148 @@
+//! Blocking `reqwest` client for the IONOS OpenAI-compatible endpoint.
+//!
+//! Retry policy (parent spec §Error handling):
+//! - HTTP 5xx / connection / read-timeout (30s): exp backoff 1s/4s/16s, give up after 3rd retry.
+//! - HTTP 429: respect `Retry-After`, else 30s, give up after 5th retry.
+//! - HTTP 4xx other than 429: hard fail (typically auth).
+
+use anyhow::{anyhow, Context, Result};
+use serde::{Deserialize, Serialize};
+use std::time::Duration;
+
+const ENDPOINT: &str = "https://openai.inference.de-txl.ionos.com/v1/chat/completions";
+
+#[derive(Debug, Clone, Serialize)]
+pub struct Message {
+    pub role: String,
+    pub content: String,
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct ChatRequest<'a> {
+    pub model: &'a str,
+    pub messages: &'a [Message],
+    pub temperature: f32,
+    pub top_p: f32,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct ChatResponse {
+    pub choices: Vec<Choice>,
+    pub usage: Usage,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct Choice {
+    pub message: ChoiceMessage,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct ChoiceMessage {
+    pub content: String,
+}
+
+#[derive(Debug, Deserialize, Clone, Copy)]
+pub struct Usage {
+    pub prompt_tokens: u64,
+    pub completion_tokens: u64,
+    pub total_tokens: u64,
+}
+
+/// Errors the client treats as terminal (caller should mark the run as api_failure).
+#[derive(Debug, thiserror::Error)]
+pub enum IonosError {
+    #[error("authentication failure (HTTP {0}): check IONOS_API_TOKEN")]
+    Auth(u16),
+    #[error("retry budget exhausted after {tries} attempts: {last_err}")]
+    RetriesExhausted { tries: u32, last_err: String },
+    #[error("transport error: {0}")]
+    Transport(String),
+}
+
+pub struct IonosClient {
+    http: reqwest::blocking::Client,
+    token: String,
+}
+
+impl IonosClient {
+    /// Build from `IONOS_API_TOKEN` env. Errors if the var is unset or empty.
+    pub fn from_env() -> Result<Self> {
+        let token = std::env::var("IONOS_API_TOKEN")
+            .context("IONOS_API_TOKEN env var is unset")?;
+        if token.trim().is_empty() {
+            return Err(anyhow!("IONOS_API_TOKEN is empty"));
+        }
+        let http = reqwest::blocking::Client::builder()
+            .timeout(Duration::from_secs(30))
+            .build()?;
+        Ok(Self { http, token })
+    }
+
+    /// Post a chat request with the retry policy above.
+    pub fn post(&self, req: &ChatRequest<'_>) -> std::result::Result<ChatResponse, IonosError> {
+        let backoffs_5xx = [Duration::from_secs(1), Duration::from_secs(4), Duration::from_secs(16)];
+        let mut tries_5xx = 0u32;
+        let mut tries_429 = 0u32;
+        let mut last_err = String::from("no attempts");
+        loop {
+            let result = self.http
+                .post(ENDPOINT)
+                .bearer_auth(&self.token)
+                .json(req)
+                .send();
+            match result {
+                Ok(resp) => {
+                    let status = resp.status();
+                    if status.is_success() {
+                        return resp.json::<ChatResponse>().map_err(|e| IonosError::Transport(e.to_string()));
+                    }
+                    let code = status.as_u16();
+                    if code == 429 {
+                        let wait = resp
+                            .headers()
+                            .get("retry-after")
+                            .and_then(|v| v.to_str().ok())
+                            .and_then(|s| s.parse::<u64>().ok())
+                            .map(Duration::from_secs)
+                            .unwrap_or(Duration::from_secs(30));
+                        if tries_429 >= 5 {
+                            return Err(IonosError::RetriesExhausted {
+                                tries: tries_429,
+                                last_err: format!("429 rate limit; last wait {}s", wait.as_secs()),
+                            });
+                        }
+                        std::thread::sleep(wait);
+                        tries_429 += 1;
+                        continue;
+                    }
+                    if (400..500).contains(&code) {
+                        return Err(IonosError::Auth(code));
+                    }
+                    // 5xx
+                    if let Some(backoff) = backoffs_5xx.get(tries_5xx as usize) {
+                        std::thread::sleep(*backoff);
+                        tries_5xx += 1;
+                        last_err = format!("HTTP {code}");
+                        continue;
+                    }
+                    return Err(IonosError::RetriesExhausted {
+                        tries: tries_5xx,
+                        last_err: format!("HTTP {code}"),
+                    });
+                }
+                Err(e) => {
+                    if let Some(backoff) = backoffs_5xx.get(tries_5xx as usize) {
+                        std::thread::sleep(*backoff);
+                        tries_5xx += 1;
+                        last_err = e.to_string();
+                        continue;
+                    }
+                    return Err(IonosError::RetriesExhausted {
+                        tries: tries_5xx,
+                        last_err: e.to_string(),
+                    });
+                }
+            }
+        }
+    }
+}

experiments/2026-05-12-cross-model-authoring/harness/src/lib.rs

@@ -0,0 +1,9 @@
+//! Library surface for xmodel-harness — exposed so integration tests
+//! under tests/ can reach the per-module types.
+
+pub mod strip_locations;
+pub mod pipeline;
+pub mod ionos;
+pub mod mock;
+pub mod scoring;
+pub mod tasks;

experiments/2026-05-12-cross-model-authoring/harness/src/main.rs

@@ -0,0 +1,211 @@
+//! xmodel-harness — drives the two-cohort cross-model authoring-form test.
+
+use anyhow::{anyhow, bail, Context, Result};
+use clap::Parser;
+use std::collections::BTreeSet;
+use std::path::PathBuf;
+use xmodel_harness::ionos::{ChatRequest, ChatResponse, IonosClient, IonosError, Message};
+use xmodel_harness::mock::MockResponses;
+use xmodel_harness::pipeline::{preflight, run_pipeline, Cohort};
+use xmodel_harness::scoring::{write_scores_csv, write_summary_md, FinalStatus, ScoreRow};
+use xmodel_harness::strip_locations::strip_locations;
+use xmodel_harness::tasks::{self, Task};
+
+#[derive(Parser, Debug)]
+#[command(name = "xmodel-harness", version, about = "Two-cohort cross-model authoring-form test")]
+struct Args {
+    #[arg(long)] rendered: PathBuf,
+    #[arg(long)] tasks: PathBuf,
+    #[arg(long)] out: PathBuf,
+    #[arg(long)] model: String,
+    #[arg(long, default_value_t = 5)] max_turns: u32,
+    #[arg(long, default_value_t = 500_000)] token_budget: u64,
+    #[arg(long)] mock: Option<PathBuf>,
+}
+
+enum Backend {
+    Live(IonosClient),
+    Mock(MockResponses),
+}
+
+fn main() -> Result<()> {
+    let args = Args::parse();
+    let run_dir = create_run_dir(&args.out)?;
+
+    preflight().context("preflight: ail and clang must be on PATH")?;
+
+    let backend = if let Some(mp) = &args.mock {
+        Backend::Mock(MockResponses::load(mp)?)
+    } else {
+        Backend::Live(IonosClient::from_env().context("IONOS_API_TOKEN")?)
+    };
+
+    let rendered_json = std::fs::read_to_string(args.rendered.join("json.md"))
+        .with_context(|| format!("reading {}", args.rendered.join("json.md").display()))?;
+    let rendered_ailx = std::fs::read_to_string(args.rendered.join("ailx.md"))
+        .with_context(|| format!("reading {}", args.rendered.join("ailx.md").display()))?;
+    let tasks = tasks::load_all(&args.tasks)?;
+    if tasks.is_empty() { bail!("no tasks found in {}", args.tasks.display()); }
+
+    let mut tokens_used: u64 = 0;
+    let mut rows: Vec<ScoreRow> = Vec::new();
+    let mut run_status = "ok";
+
+    let mut completed: std::collections::BTreeSet<(&'static str, String)> = Default::default();
+    'outer: for cohort in [Cohort::Json, Cohort::Ailx] {
+        let system_prompt = match cohort { Cohort::Json => &rendered_json, Cohort::Ailx => &rendered_ailx };
+        for t in &tasks {
+            let (row, consumed) = run_one(
+                &backend, &args, cohort, system_prompt, t,
+                &run_dir, args.token_budget.saturating_sub(tokens_used),
+            )?;
+            tokens_used = tokens_used.saturating_add(consumed);
+            let aborted_budget = matches!(row.final_status, FinalStatus::BudgetAbort);
+            completed.insert((cohort.as_str(), t.id.clone()));
+            rows.push(row);
+            if tokens_used >= args.token_budget {
+                run_status = "budget_exceeded";
+                break 'outer;
+            }
+            let _ = aborted_budget; // continue to next task regardless
+        }
+    }
+
+    // If the budget ran out mid-run, the outer loop broke before
+    // walking every (cohort, task) pair. The pairs we never reached
+    // still need a row, with final_status = budget_abort.
+    if run_status == "budget_exceeded" {
+        for cohort in [Cohort::Json, Cohort::Ailx] {
+            for t in &tasks {
+                if !completed.contains(&(cohort.as_str(), t.id.clone())) {
+                    rows.push(ScoreRow {
+                        cohort: cohort.as_str().to_string(),
+                        task_id: t.id.clone(),
+                        first_attempt_green: false,
+                        turns_to_green: None,
+                        prompt_tokens: 0,
+                        completion_tokens: 0,
+                        error_classes: BTreeSet::new(),
+                        final_status: FinalStatus::BudgetAbort,
+                    });
+                }
+            }
+        }
+    }
+
+    write_scores_csv(&rows, &run_dir.join("scores.csv"))?;
+    write_summary_md(&rows, &run_dir.join("summary.md"))?;
+    std::fs::write(run_dir.join("RUN_STATUS"), run_status)?;
+    eprintln!("xmodel-harness: run complete at {}; status={run_status}", run_dir.display());
+    Ok(())
+}
+
+fn create_run_dir(out: &std::path::Path) -> Result<PathBuf> {
+    let date = chrono::Utc::now().format("%Y-%m-%d");
+    let hash: String = (0..6).map(|_| {
+        let n = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().subsec_nanos();
+        std::char::from_digit((n % 16) as u32, 16).unwrap()
+    }).collect();
+    let dir = out.join(format!("{date}-{hash}"));
+    std::fs::create_dir_all(&dir)?;
+    Ok(dir)
+}
+
+fn call_backend(backend: &Backend, model: &str, messages: &[Message], cohort: &str, task: &str, turn: u32)
+    -> std::result::Result<ChatResponse, IonosError>
+{
+    match backend {
+        Backend::Live(c) => c.post(&ChatRequest { model, messages, temperature: 0.0, top_p: 1.0 }),
+        Backend::Mock(m) => m.response_for(cohort, task, turn).map_err(|e| IonosError::Transport(e.to_string())),
+    }
+}
+
+fn run_one(
+    backend: &Backend, args: &Args, cohort: Cohort, system_prompt: &str, task: &Task,
+    run_dir: &std::path::Path, budget_remaining: u64,
+) -> Result<(ScoreRow, u64)> {
+    let cohort_dir = run_dir.join("per_cohort").join(cohort.as_str()).join(&task.id);
+    std::fs::create_dir_all(&cohort_dir)?;
+    let mut messages = vec![
+        Message { role: "system".into(), content: system_prompt.to_string() },
+        Message { role: "user".into(),   content: task.description.clone() },
+    ];
+    let mut prompt_total: u64 = 0;
+    let mut completion_total: u64 = 0;
+    let mut error_classes: BTreeSet<String> = BTreeSet::new();
+    let mut first_attempt_green = false;
+    let mut turns_to_green: Option<u32> = None;
+    let mut final_status = FinalStatus::TurnLimit;
+
+    for turn in 1..=args.max_turns {
+        if prompt_total.saturating_add(completion_total) >= budget_remaining {
+            final_status = FinalStatus::BudgetAbort;
+            break;
+        }
+        let resp = match call_backend(backend, &args.model, &messages, cohort.as_str(), &task.id, turn) {
+            Ok(r) => r,
+            Err(IonosError::Auth(_)) | Err(IonosError::Transport(_)) | Err(IonosError::RetriesExhausted{..}) => {
+                final_status = FinalStatus::ApiFailure;
+                break;
+            }
+        };
+        prompt_total += resp.usage.prompt_tokens;
+        completion_total += resp.usage.completion_tokens;
+        let program = resp.choices.first()
+            .ok_or_else(|| anyhow!("empty choices array"))?
+            .message.content.clone();
+
+        std::fs::write(
+            cohort_dir.join(format!("turn_{turn}_program.{}", cohort.extension())),
+            &program,
+        )?;
+        std::fs::write(
+            cohort_dir.join(format!("turn_{turn}_request.json")),
+            serde_json::to_string_pretty(&messages)?,
+        )?;
+        std::fs::write(
+            cohort_dir.join(format!("turn_{turn}_response.json")),
+            serde_json::to_string_pretty(&serde_json::json!({
+                "usage": { "prompt_tokens": resp.usage.prompt_tokens, "completion_tokens": resp.usage.completion_tokens, "total_tokens": resp.usage.total_tokens },
+                "content": program,
+            }))?,
+        )?;
+
+        let workdir = tempfile::Builder::new().prefix("cma2-run-").tempdir()?;
+        let cap = run_pipeline(cohort, &program, &task.expected_stdout, workdir.path())?;
+        std::fs::write(cohort_dir.join(format!("turn_{turn}_check_stderr.txt")), &cap.stderr)?;
+        std::fs::write(cohort_dir.join(format!("turn_{turn}_run_stdout.txt")), &cap.stdout)?;
+
+        match cap.error {
+            None => {
+                if turn == 1 { first_attempt_green = true; }
+                turns_to_green = Some(turn);
+                final_status = FinalStatus::Green;
+                break;
+            }
+            Some(err) => {
+                if let Some((class, _)) = err.split_once(':') {
+                    error_classes.insert(class.to_string());
+                }
+                let stripped = strip_locations(&err);
+                messages.push(Message { role: "assistant".into(), content: program });
+                messages.push(Message { role: "user".into(),      content: stripped });
+            }
+        }
+    }
+
+    let consumed = prompt_total + completion_total;
+    Ok((
+        ScoreRow {
+            cohort: cohort.as_str().to_string(),
+            task_id: task.id.clone(),
+            first_attempt_green,
+            turns_to_green,
+            prompt_tokens: prompt_total,
+            completion_tokens: completion_total,
+            error_classes,
+            final_status,
+        },
+        consumed,
+    ))
+}

experiments/2026-05-12-cross-model-authoring/harness/src/mock.rs

@@ -0,0 +1,61 @@
+//! `--mock <file>` canned-response loader.
+//!
+//! Mock file shape (JSON):
+//! ```json
+//! {
+//!   "json": {
+//!     "t1_add_three": { "1": { "content": "...", "usage": {...} } },
+//!     "t2_length":    { "1": {...}, "2": {...} }
+//!   },
+//!   "ailx": { ... }
+//! }
+//! ```
+//! Where each `"<turn>"` entry has `"content"` (the program the
+//! mocked model would emit) and `"usage"` (matching IONOS Usage shape).
+
+use crate::ionos::{ChatResponse, Choice, ChoiceMessage, Usage};
+use anyhow::{anyhow, Context, Result};
+use serde::Deserialize;
+use std::collections::BTreeMap;
+use std::path::Path;
+
+#[derive(Debug, Deserialize)]
+struct MockTurn {
+    content: String,
+    usage: Usage,
+}
+
+#[derive(Debug, Deserialize)]
+pub struct MockFile {
+    /// cohort_name -> task_id -> turn -> response
+    #[serde(flatten)]
+    by_cohort: BTreeMap<String, BTreeMap<String, BTreeMap<String, MockTurn>>>,
+}
+
+pub struct MockResponses(MockFile);
+
+impl MockResponses {
+    pub fn load(path: &Path) -> Result<Self> {
+        let text = std::fs::read_to_string(path)
+            .with_context(|| format!("reading mock file {}", path.display()))?;
+        let file: MockFile = serde_json::from_str(&text)
+            .with_context(|| format!("parsing mock file {}", path.display()))?;
+        Ok(MockResponses(file))
+    }
+
+    pub fn response_for(&self, cohort: &str, task: &str, turn: u32) -> Result<ChatResponse> {
+        let turn_str = turn.to_string();
+        let mt = self
+            .0
+            .by_cohort.get(cohort)
+            .and_then(|t| t.get(task))
+            .and_then(|t| t.get(&turn_str))
+            .ok_or_else(|| anyhow!("mock has no entry for {cohort}/{task}/turn={turn}"))?;
+        Ok(ChatResponse {
+            choices: vec![Choice {
+                message: ChoiceMessage { content: mt.content.clone() },
+            }],
+            usage: mt.usage,
+        })
+    }
+}

experiments/2026-05-12-cross-model-authoring/harness/src/pipeline.rs

@@ -0,0 +1,219 @@
+//! Subprocess wrapper around `ail parse | check | build` plus the
+//! model's compiled binary. The harness shells out to the system
+//! `ail` binary (resolved via `AIL_BIN` env var or PATH lookup of
+//! "ail") and to the binary that `ail build` emits.
+
+use anyhow::{anyhow, Context, Result};
+use std::path::{Path, PathBuf};
+use std::process::{Command, Stdio};
+use std::time::Duration;
+
+/// Which authoring form the program was produced in.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Cohort {
+    Json,
+    Ailx,
+}
+
+impl Cohort {
+    pub fn extension(self) -> &'static str {
+        match self {
+            Cohort::Json => "ail.json",
+            Cohort::Ailx => "ailx",
+        }
+    }
+    pub fn as_str(self) -> &'static str {
+        match self {
+            Cohort::Json => "json",
+            Cohort::Ailx => "ailx",
+        }
+    }
+}
+
+/// Outcome of one pipeline run; `Ok(None)` is the success path.
+#[derive(Debug)]
+pub struct PipelineCapture {
+    pub error: Option<String>,
+    pub stdout: String,
+    pub stderr: String,
+}
+
+/// Resolve the `ail` binary path: `AIL_BIN` env var, else bare "ail".
+pub fn ail_bin() -> PathBuf {
+    std::env::var("AIL_BIN")
+        .ok()
+        .map(PathBuf::from)
+        .unwrap_or_else(|| PathBuf::from("ail"))
+}
+
+/// Pre-flight check: `ail` and `clang` both runnable.
+/// Returns an error with a clear "missing dependency" message if either is absent.
+pub fn preflight() -> Result<()> {
+    for (label, bin) in [("ail", ail_bin()), ("clang", PathBuf::from("clang"))] {
+        let out = Command::new(&bin)
+            .arg("--version")
+            .stdout(Stdio::null())
+            .stderr(Stdio::null())
+            .status();
+        match out {
+            Ok(st) if st.success() => {}
+            Ok(st) => return Err(anyhow!("{label} ({}) --version exited {st}", bin.display())),
+            Err(e) => return Err(anyhow!("{label} ({}) not runnable: {e}", bin.display())),
+        }
+    }
+    Ok(())
+}
+
+/// Run the per-cohort pipeline on `program`. Returns `Ok(None)` on
+/// success, `Ok(Some(error_string))` on a pipeline failure that the
+/// harness will feed back to the model (un-stripped — stripping is
+/// the caller's responsibility), `Err(...)` only on harness-level
+/// faults (missing `ail` binary, IO error).
+pub fn run_pipeline(
+    cohort: Cohort,
+    program: &str,
+    expected_stdout: &str,
+    workdir: &Path,
+) -> Result<PipelineCapture> {
+    std::fs::create_dir_all(workdir).with_context(|| format!("creating {}", workdir.display()))?;
+    let prog_path = workdir.join(format!("prog.{}", cohort.extension()));
+    std::fs::write(&prog_path, program).with_context(|| format!("writing {}", prog_path.display()))?;
+
+    // For AILX cohort, parse to JSON first.
+    let json_path_initial = if matches!(cohort, Cohort::Ailx) {
+        let out = workdir.join("prog.ail.json");
+        let parse_out = Command::new(ail_bin())
+            .arg("parse").arg(&prog_path)
+            .arg("-o").arg(&out)
+            .output()
+            .with_context(|| format!("running ail parse on {}", prog_path.display()))?;
+        if !parse_out.status.success() {
+            return Ok(PipelineCapture {
+                error: Some(format!("parse: {}", String::from_utf8_lossy(&parse_out.stderr).trim())),
+                stdout: String::new(),
+                stderr: String::from_utf8_lossy(&parse_out.stderr).into_owned(),
+            });
+        }
+        out
+    } else {
+        prog_path.clone()
+    };
+
+    // `ail check` enforces filename stem == module name. Extract the module
+    // name from the JSON and rename the file accordingly before checking.
+    // Faults reading/parsing the JSON are reported as a check-class error
+    // (the same diagnostic the model would see if it submitted a JSON with
+    // a malformed top-level shape).
+    let json_path = match module_name_from_json(&json_path_initial) {
+        Ok(name) => {
+            let renamed = workdir.join(format!("{name}.ail.json"));
+            if renamed != json_path_initial {
+                std::fs::rename(&json_path_initial, &renamed)
+                    .with_context(|| format!("renaming {} -> {}", json_path_initial.display(), renamed.display()))?;
+            }
+            renamed
+        }
+        Err(e) => {
+            return Ok(PipelineCapture {
+                error: Some(format!("check: {e}")),
+                stdout: String::new(),
+                stderr: e.to_string(),
+            });
+        }
+    };
+
+    // Type check.
+    let check_out = Command::new(ail_bin()).arg("check").arg(&json_path).output()
+        .with_context(|| format!("running ail check on {}", json_path.display()))?;
+    if !check_out.status.success() {
+        return Ok(PipelineCapture {
+            error: Some(format!("check: {}", String::from_utf8_lossy(&check_out.stderr).trim())),
+            stdout: String::new(),
+            stderr: String::from_utf8_lossy(&check_out.stderr).into_owned(),
+        });
+    }
+
+    // Build to a native binary.
+    let bin_path = workdir.join("prog.bin");
+    let build_out = Command::new(ail_bin())
+        .arg("build").arg(&json_path)
+        .arg("-o").arg(&bin_path)
+        .output()
+        .with_context(|| format!("running ail build on {}", json_path.display()))?;
+    if !build_out.status.success() {
+        return Ok(PipelineCapture {
+            error: Some(format!("build: {}", String::from_utf8_lossy(&build_out.stderr).trim())),
+            stdout: String::new(),
+            stderr: String::from_utf8_lossy(&build_out.stderr).into_owned(),
+        });
+    }
+
+    // Execute with a 5-second timeout.
+    let run_out = run_with_timeout(&bin_path, Duration::from_secs(5))?;
+    let actual_stdout = String::from_utf8_lossy(&run_out.stdout).into_owned();
+    if !run_out.timed_out {
+        if actual_stdout != expected_stdout {
+            return Ok(PipelineCapture {
+                error: Some(format!(
+                    "output: expected {:?}, got {:?}",
+                    expected_stdout, actual_stdout
+                )),
+                stdout: actual_stdout,
+                stderr: String::from_utf8_lossy(&run_out.stderr).into_owned(),
+            });
+        }
+        return Ok(PipelineCapture {
+            error: None,
+            stdout: actual_stdout,
+            stderr: String::from_utf8_lossy(&run_out.stderr).into_owned(),
+        });
+    }
+    Ok(PipelineCapture {
+        error: Some("runtime: timeout after 5s".to_string()),
+        stdout: actual_stdout,
+        stderr: String::from_utf8_lossy(&run_out.stderr).into_owned(),
+    })
+}
+
+/// Extract the top-level `"name"` field from a JSON module file.
+/// Returns a check-class error string if the JSON cannot be read or
+/// the `name` field is absent / not a string.
+fn module_name_from_json(path: &Path) -> Result<String> {
+    let text = std::fs::read_to_string(path)
+        .with_context(|| format!("reading {}", path.display()))?;
+    let v: serde_json::Value = serde_json::from_str(&text)
+        .with_context(|| format!("parsing JSON in {}", path.display()))?;
+    let name = v.get("name")
+        .and_then(|n| n.as_str())
+        .ok_or_else(|| anyhow!("top-level `name` field missing or not a string in {}", path.display()))?;
+    Ok(name.to_string())
+}
+
+struct RunOutput {
+    stdout: Vec<u8>,
+    stderr: Vec<u8>,
+    timed_out: bool,
+}
+
+fn run_with_timeout(bin: &Path, timeout: Duration) -> Result<RunOutput> {
+    use std::io::Read;
+    let mut child = Command::new(bin)
+        .stdout(Stdio::piped()).stderr(Stdio::piped())
+        .spawn().with_context(|| format!("spawning {}", bin.display()))?;
+    let start = std::time::Instant::now();
+    loop {
+        if let Some(_status) = child.try_wait()? {
+            let mut so = Vec::new();
+            let mut se = Vec::new();
+            if let Some(mut s) = child.stdout.take() { s.read_to_end(&mut so).ok(); }
+            if let Some(mut s) = child.stderr.take() { s.read_to_end(&mut se).ok(); }
+            return Ok(RunOutput { stdout: so, stderr: se, timed_out: false });
+        }
+        if start.elapsed() >= timeout {
+            let _ = child.kill();
+            let _ = child.wait();
+            return Ok(RunOutput { stdout: Vec::new(), stderr: Vec::new(), timed_out: true });
+        }
+        std::thread::sleep(Duration::from_millis(25));
+    }
+}

experiments/2026-05-12-cross-model-authoring/harness/src/scoring.rs

@@ -0,0 +1,101 @@
+//! `scores.csv` + `summary.md` emitters.
+//!
+//! Columns (parent spec §Scoring):
+//!   cohort, task_id, first_attempt_green, turns_to_green,
+//!   prompt_tokens, completion_tokens, error_classes, final_status
+
+use anyhow::{Context, Result};
+use serde::Serialize;
+use std::collections::BTreeSet;
+use std::path::Path;
+
+#[derive(Debug, Clone, Serialize)]
+pub struct ScoreRow {
+    pub cohort: String,
+    pub task_id: String,
+    pub first_attempt_green: bool,
+    /// `None` means INF (never reached green).
+    pub turns_to_green: Option<u32>,
+    pub prompt_tokens: u64,
+    pub completion_tokens: u64,
+    pub error_classes: BTreeSet<String>,
+    pub final_status: FinalStatus,
+}
+
+#[derive(Debug, Clone, Copy, Serialize)]
+#[serde(rename_all = "snake_case")]
+pub enum FinalStatus {
+    Green,
+    TurnLimit,
+    BudgetAbort,
+    ApiFailure,
+}
+
+impl FinalStatus {
+    fn as_csv(self) -> &'static str {
+        match self {
+            FinalStatus::Green => "green",
+            FinalStatus::TurnLimit => "turn_limit",
+            FinalStatus::BudgetAbort => "budget_abort",
+            FinalStatus::ApiFailure => "api_failure",
+        }
+    }
+}
+
+pub fn write_scores_csv(rows: &[ScoreRow], path: &Path) -> Result<()> {
+    use std::io::Write;
+    let mut f = std::fs::File::create(path)
+        .with_context(|| format!("creating {}", path.display()))?;
+    writeln!(f, "cohort,task_id,first_attempt_green,turns_to_green,prompt_tokens,completion_tokens,error_classes,final_status")?;
+    for r in rows {
+        let turns = match r.turns_to_green {
+            Some(n) => n.to_string(),
+            None => "INF".to_string(),
+        };
+        let errs = r.error_classes.iter().cloned().collect::<Vec<_>>().join(";");
+        writeln!(
+            f,
+            "{},{},{},{},{},{},{},{}",
+            r.cohort, r.task_id, r.first_attempt_green, turns, r.prompt_tokens,
+            r.completion_tokens, errs, r.final_status.as_csv()
+        )?;
+    }
+    Ok(())
+}
+
+pub fn write_summary_md(rows: &[ScoreRow], path: &Path) -> Result<()> {
+    use std::io::Write;
+    let mut f = std::fs::File::create(path)
+        .with_context(|| format!("creating {}", path.display()))?;
+    writeln!(f, "# Cross-model authoring-form test — run summary\n")?;
+    for cohort in ["json", "ailx"] {
+        writeln!(f, "## Cohort: {cohort}\n")?;
+        let cohort_rows: Vec<&ScoreRow> = rows.iter().filter(|r| r.cohort == cohort).collect();
+        if cohort_rows.is_empty() {
+            writeln!(f, "_(no rows)_\n")?;
+            continue;
+        }
+        let n_total = cohort_rows.len();
+        let n_green = cohort_rows.iter().filter(|r| matches!(r.final_status, FinalStatus::Green)).count();
+        let n_first = cohort_rows.iter().filter(|r| r.first_attempt_green).count();
+        let mean_turns: f64 = {
+            let xs: Vec<f64> = cohort_rows.iter().filter_map(|r| r.turns_to_green.map(|n| n as f64)).collect();
+            if xs.is_empty() { f64::NAN } else { xs.iter().sum::<f64>() / xs.len() as f64 }
+        };
+        let total_prompt: u64 = cohort_rows.iter().map(|r| r.prompt_tokens).sum();
+        let total_completion: u64 = cohort_rows.iter().map(|r| r.completion_tokens).sum();
+        let mut error_freq: std::collections::BTreeMap<String, u32> = Default::default();
+        for r in &cohort_rows {
+            for e in &r.error_classes { *error_freq.entry(e.clone()).or_default() += 1; }
+        }
+        let top_err = error_freq.iter().max_by_key(|(_, n)| **n).map(|(k, n)| format!("{} (x{})", k, n)).unwrap_or_else(|| "(none)".to_string());
+        writeln!(f, "- tasks: {n_total}")?;
+        writeln!(f, "- reached green: {n_green}")?;
+        writeln!(f, "- first-attempt green: {n_first}")?;
+        writeln!(f, "- mean turns-to-green (green-only): {mean_turns:.2}")?;
+        writeln!(f, "- total prompt tokens: {total_prompt}")?;
+        writeln!(f, "- total completion tokens: {total_completion}")?;
+        writeln!(f, "- most common error class: {top_err}\n")?;
+    }
+    Ok(())
+}

experiments/2026-05-12-cross-model-authoring/harness/src/strip_locations.rs

@@ -0,0 +1,94 @@
+//! Regex pass that removes form-asymmetric location info from
+//! compiler-error strings before they are fed back to the model.
+//!
+//! The intent (parent spec §strip_locations) is **symmetric
+//! degradation**: both cohorts lose the localisation information
+//! their compiler natively produces. The JSON cohort would
+//! otherwise get JSON-pointer fragments; the AILX cohort would get
+//! `at byte N` offsets. Neither survives this pass.
+
+use regex::Regex;
+use std::sync::OnceLock;
+
+struct Patterns {
+    json_pointer: Regex,
+    byte_offset: Regex,
+    line: Regex,
+    column: Regex,
+    line_col: Regex,
+    file_prefix: Regex,
+    caused_by_chain: Regex,
+}
+
+fn patterns() -> &'static Patterns {
+    static P: OnceLock<Patterns> = OnceLock::new();
+    P.get_or_init(|| Patterns {
+        json_pointer: Regex::new(r"\$\.[A-Za-z0-9._\[\]]+").unwrap(),
+        byte_offset: Regex::new(r"\bat byte \d+\b").unwrap(),
+        line: Regex::new(r"\bline \d+\b").unwrap(),
+        column: Regex::new(r"\bcolumn \d+\b").unwrap(),
+        line_col: Regex::new(r"\bat \d+:\d+\b").unwrap(),
+        file_prefix: Regex::new(r"(?m)^[^:\s]+:\d+:\d+:\s*").unwrap(),
+        // Captures from "\nCaused by:" (inclusive) through end of string.
+        caused_by_chain: Regex::new(r"(?s)\nCaused by:.*$").unwrap(),
+    })
+}
+
+/// Strip form-asymmetric location info from a compiler-error string.
+///
+/// Order matters: collapse the anyhow `Caused by:` chain first so the
+/// per-line regexes operate only on the leading message line.
+pub fn strip_locations(s: &str) -> String {
+    let p = patterns();
+    let mut out = p.caused_by_chain.replace(s, "").into_owned();
+    out = p.file_prefix.replace_all(&out, "").into_owned();
+    out = p.json_pointer.replace_all(&out, "").into_owned();
+    out = p.byte_offset.replace_all(&out, "").into_owned();
+    out = p.line_col.replace_all(&out, "").into_owned();
+    out = p.line.replace_all(&out, "").into_owned();
+    out = p.column.replace_all(&out, "").into_owned();
+    // Collapse runs of whitespace introduced by removed location tokens.
+    let ws = Regex::new(r" {2,}").unwrap();
+    ws.replace_all(out.trim_end(), " ").into_owned()
+}
+
+#[cfg(test)]
+mod tests {
+    use super::strip_locations;
+
+    #[test]
+    fn json_pointer_is_removed() {
+        let input = "type mismatch at $.defs[0].fn.body.app.fun";
+        assert_eq!(strip_locations(input), "type mismatch at");
+    }
+
+    #[test]
+    fn byte_offset_is_removed() {
+        let input = "parse error: expected `)` (end of fn-def), got `(` at byte 28";
+        assert_eq!(
+            strip_locations(input),
+            "parse error: expected `)` (end of fn-def), got `(`",
+        );
+    }
+
+    #[test]
+    fn line_and_column_are_removed() {
+        let input = "schema/parse error: missing field `type` at line 7 column 3";
+        let stripped = strip_locations(input);
+        assert!(!stripped.contains("line "));
+        assert!(!stripped.contains("column "));
+        assert!(stripped.contains("missing field `type`"));
+    }
+
+    #[test]
+    fn caused_by_chain_is_collapsed() {
+        let input = "Error: top message\n\nCaused by:\n    0: lower message\n    1: even lower";
+        assert_eq!(strip_locations(input), "Error: top message");
+    }
+
+    #[test]
+    fn passthrough_when_no_locations() {
+        let input = "error: [unbound-var] main: unknown identifier: `does_not_exist`";
+        assert_eq!(strip_locations(input), input);
+    }
+}

experiments/2026-05-12-cross-model-authoring/harness/src/tasks.rs

@@ -0,0 +1,42 @@
+//! Task definition struct + loader for `master/tasks/*.task.json`.
+
+use anyhow::{Context, Result};
+use serde::Deserialize;
+use std::path::{Path, PathBuf};
+
+#[derive(Debug, Clone, Deserialize)]
+pub struct Task {
+    pub id: String,
+    pub title: String,
+    pub description: String,
+    pub expected_stdout: String,
+    pub reference_solution: PathBuf,
+}
+
+impl Task {
+    pub fn load(path: &Path) -> Result<Task> {
+        let text = std::fs::read_to_string(path)
+            .with_context(|| format!("reading {}", path.display()))?;
+        let t: Task = serde_json::from_str(&text)
+            .with_context(|| format!("parsing {}", path.display()))?;
+        Ok(t)
+    }
+}
+
+/// Load every `*.task.json` file from a directory, sorted by id.
+pub fn load_all(dir: &Path) -> Result<Vec<Task>> {
+    let mut tasks: Vec<Task> = std::fs::read_dir(dir)
+        .with_context(|| format!("reading {}", dir.display()))?
+        .filter_map(|e| e.ok())
+        .map(|e| e.path())
+        .filter(|p| {
+            p.file_name()
+                .and_then(|s| s.to_str())
+                .map(|n| n.ends_with(".task.json"))
+                .unwrap_or(false)
+        })
+        .map(|p| Task::load(&p))
+        .collect::<Result<Vec<_>>>()?;
+    tasks.sort_by(|a, b| a.id.cmp(&b.id));
+    Ok(tasks)
+}

experiments/2026-05-12-cross-model-authoring/harness/tests/budget_abort.rs

@@ -0,0 +1,39 @@
+//! Budget-abort test (parent spec §Error handling / Budget-level).
+//! Mock run with deliberately tiny budget; asserts the harness
+//! stops with RUN_STATUS=budget_exceeded and that the affected rows
+//! carry final_status=budget_abort.
+
+use std::path::PathBuf;
+use std::process::Command;
+
+fn harness_bin() -> PathBuf { PathBuf::from(env!("CARGO_BIN_EXE_xmodel-harness")) }
+
+fn experiment_root() -> PathBuf {
+    PathBuf::from(env!("CARGO_MANIFEST_DIR")).parent().unwrap().to_path_buf()
+}
+
+#[test]
+fn tiny_budget_aborts_run_cleanly() {
+    let out = tempfile::Builder::new().prefix("cma2-budget-").tempdir().unwrap();
+    let status = Command::new(harness_bin())
+        .arg("--rendered").arg(experiment_root().join("rendered"))
+        .arg("--tasks").arg(experiment_root().join("master").join("tasks"))
+        .arg("--out").arg(out.path())
+        .arg("--model").arg("mock")
+        .arg("--mock").arg(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests").join("fixtures").join("mock_full_run.json"))
+        .arg("--token-budget").arg("1500")  // less than one task's worth (~1200 per turn)
+        .status()
+        .expect("running xmodel-harness");
+    assert!(status.success(), "harness should exit 0 on budget exhaustion");
+
+    let mut entries = std::fs::read_dir(out.path()).unwrap();
+    let run_dir = entries.next().unwrap().unwrap().path();
+    let status_str = std::fs::read_to_string(run_dir.join("RUN_STATUS")).unwrap();
+    assert_eq!(status_str, "budget_exceeded");
+    let scores = std::fs::read_to_string(run_dir.join("scores.csv")).unwrap();
+    // At least one row must carry budget_abort.
+    assert!(
+        scores.lines().any(|l| l.ends_with(",budget_abort")),
+        "expected at least one budget_abort row; got:\n{}", scores,
+    );
+}

experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_bare_xmod.stderr

@@ -0,0 +1 @@
+Error: module `test_ct1_bare_xmod_rejected` contains bare type name `Ordering` that does not resolve to a local type. AILang's `.ail.json` requires cross-module type references to be qualified. Candidates from imports: ["prelude.Ordering"]. Run `ail migrate-canonical-types` to fix legacy fixtures.

experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_schema_missing_field.stderr

@@ -0,0 +1,5 @@
+Error: schema/parse error in /tmp/airecon/missing_field.ail.json: json: missing field `type` at line 7 column 3
+
+Caused by:
+    0: json: missing field `type` at line 7 column 3
+    1: missing field `type` at line 7 column 3

experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_type_mismatch.stderr

@@ -0,0 +1 @@
+error: [type-mismatch] main: type mismatch: expected Int, got Bool

experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/check_unbound_var.stderr

@@ -0,0 +1 @@
+error: [unbound-var] main: unknown identifier: `does_not_exist`

experiments/2026-05-12-cross-model-authoring/harness/tests/fixtures/parse_unclosed.stderr

@@ -0,0 +1 @@
+parse error: parse error: expected `)` (end of fn-def), got `(` at byte 28

experiments/2026-05-12-cross-model-authoring/harness/tests/mock_full_run.rs

@@ -0,0 +1,44 @@
+//! End-to-end mock-mode test (parent spec §Testing strategy).
+//! Runs the harness binary against a canned response file that
+//! makes (json, t3_main_prints) green on turn 2 and
+//! (ailx, t1_add_three) run to the turn limit. Asserts the
+//! scores.csv is well-formed and the right per-(cohort,task)
+//! artefacts land on disk.
+
+use std::path::PathBuf;
+use std::process::Command;
+
+fn harness_bin() -> PathBuf { PathBuf::from(env!("CARGO_BIN_EXE_xmodel-harness")) }
+
+fn experiment_root() -> PathBuf {
+    PathBuf::from(env!("CARGO_MANIFEST_DIR")).parent().unwrap().to_path_buf()
+}
+
+#[test]
+fn mock_full_run_produces_scores_and_artefacts() {
+    let out = tempfile::Builder::new().prefix("cma2-mock-").tempdir().unwrap();
+    let status = Command::new(harness_bin())
+        .arg("--rendered").arg(experiment_root().join("rendered"))
+        .arg("--tasks").arg(experiment_root().join("master").join("tasks"))
+        .arg("--out").arg(out.path())
+        .arg("--model").arg("mock")
+        .arg("--mock").arg(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests").join("fixtures").join("mock_full_run.json"))
+        .status()
+        .expect("running xmodel-harness");
+    assert!(status.success(), "harness exited non-zero");
+
+    let mut entries = std::fs::read_dir(out.path()).unwrap();
+    let run_dir = entries.next().unwrap().unwrap().path();
+    let status_file = run_dir.join("RUN_STATUS");
+    assert_eq!(std::fs::read_to_string(&status_file).unwrap(), "ok");
+    let scores = std::fs::read_to_string(run_dir.join("scores.csv")).unwrap();
+    assert!(scores.starts_with("cohort,task_id,first_attempt_green,"));
+    assert_eq!(scores.lines().count(), 1 + 8, "header + 8 rows expected");
+    assert!(scores.contains("json,t3_main_prints,false,2"));
+    assert!(scores.contains("ailx,t1_add_three,false,INF"));
+
+    // Spot-check one per-cohort artefact tree.
+    let pc = run_dir.join("per_cohort").join("ailx").join("t1_add_three");
+    assert!(pc.join("turn_1_program.ailx").exists());
+    assert!(pc.join("turn_5_program.ailx").exists());
+}

experiments/2026-05-12-cross-model-authoring/harness/tests/strip_locations.rs

@@ -0,0 +1,59 @@
+//! Integration test for strip_locations against captured stderr from
+//! real `ail check` / `ail parse` failures (parent spec §Testing strategy).
+//! Each fixture is verbatim from a current-HEAD invocation; goldens
+//! are the strip_locations output the model should see.
+
+use std::path::PathBuf;
+use xmodel_harness::strip_locations::strip_locations;
+
+fn fixtures_dir() -> PathBuf {
+    PathBuf::from(env!("CARGO_MANIFEST_DIR"))
+        .join("tests").join("fixtures")
+}
+
+fn read_fixture(name: &str) -> String {
+    let path = fixtures_dir().join(name);
+    std::fs::read_to_string(&path)
+        .unwrap_or_else(|e| panic!("read {}: {e}", path.display()))
+}
+
+#[test]
+fn check_unbound_var_strips_to_class_plus_message() {
+    let input = read_fixture("check_unbound_var.stderr");
+    let out = strip_locations(input.trim_end());
+    assert_eq!(out, "error: [unbound-var] main: unknown identifier: `does_not_exist`");
+}
+
+#[test]
+fn check_type_mismatch_strips_to_class_plus_message() {
+    let input = read_fixture("check_type_mismatch.stderr");
+    let out = strip_locations(input.trim_end());
+    assert_eq!(out, "error: [type-mismatch] main: type mismatch: expected Int, got Bool");
+}
+
+#[test]
+fn check_bare_xmod_passes_through_unchanged() {
+    let input = read_fixture("check_bare_xmod.stderr");
+    let out = strip_locations(input.trim_end());
+    // No location info to strip; passthrough.
+    assert!(out.contains("bare type name `Ordering`"));
+    assert!(!out.contains("Caused by:"));
+}
+
+#[test]
+fn check_schema_missing_field_strips_locations_and_collapses_chain() {
+    let input = read_fixture("check_schema_missing_field.stderr");
+    let out = strip_locations(input.trim_end());
+    assert!(!out.contains("line "));
+    assert!(!out.contains("column "));
+    assert!(!out.contains("Caused by:"));
+    assert!(out.contains("missing field `type`"));
+}
+
+#[test]
+fn parse_unclosed_strips_byte_offset() {
+    let input = read_fixture("parse_unclosed.stderr");
+    let out = strip_locations(input.trim_end());
+    assert!(!out.contains("at byte"));
+    assert!(out.contains("expected `)`"));
+}

experiments/2026-05-12-cross-model-authoring/harness/tests/verify_references.rs

@@ -0,0 +1,35 @@
+//! Pre-flight: every reference solution under master/tasks/ must
+//! reach green through the harness pipeline locally (no API call).
+//! This is the parent spec §Pre-flight item 4 enforced as a test.
+
+use std::path::PathBuf;
+use xmodel_harness::pipeline::{run_pipeline, Cohort};
+use xmodel_harness::tasks;
+
+fn master_tasks_dir() -> PathBuf {
+    let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+    manifest.parent().unwrap().join("master").join("tasks")
+}
+
+#[test]
+fn every_reference_solution_reaches_green() {
+    let tasks_dir = master_tasks_dir();
+    let tasks = tasks::load_all(&tasks_dir).expect("loading tasks");
+    assert!(!tasks.is_empty(), "no tasks loaded from {}", tasks_dir.display());
+
+    let mut failures: Vec<String> = Vec::new();
+    for t in &tasks {
+        let ref_path = tasks_dir.join(&t.reference_solution);
+        let program = std::fs::read_to_string(&ref_path)
+            .unwrap_or_else(|e| panic!("read {}: {e}", ref_path.display()));
+        let workdir = tempfile::Builder::new().prefix("cma2-ref-").tempdir().unwrap();
+        let cap = run_pipeline(Cohort::Json, &program, &t.expected_stdout, workdir.path())
+            .unwrap_or_else(|e| panic!("pipeline failed for {}: {e}", t.id));
+        if let Some(err) = cap.error {
+            failures.push(format!("{}: {}", t.id, err));
+        }
+    }
+    if !failures.is_empty() {
+        panic!("{} reference(s) failed pipeline:\n  {}", failures.len(), failures.join("\n  "));
+    }
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t1_add_three.reference.ail.json

@@ -0,0 +1,77 @@
+{
+  "schema": "ailang/v0",
+  "name": "t1_add_three",
+  "imports": [],
+  "defs": [
+    {
+      "kind": "fn",
+      "name": "add_three",
+      "type": {
+        "k": "fn",
+        "params": [
+          { "k": "con", "name": "Int" },
+          { "k": "con", "name": "Int" },
+          { "k": "con", "name": "Int" }
+        ],
+        "ret": { "k": "con", "name": "Int" },
+        "effects": []
+      },
+      "params": ["a", "b", "c"],
+      "body": {
+        "t": "app",
+        "fn": { "t": "var", "name": "+" },
+        "args": [
+          { "t": "var", "name": "a" },
+          {
+            "t": "app",
+            "fn": { "t": "var", "name": "+" },
+            "args": [
+              { "t": "var", "name": "b" },
+              { "t": "var", "name": "c" }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "kind": "fn",
+      "name": "main",
+      "type": {
+        "k": "fn",
+        "params": [],
+        "ret": { "k": "con", "name": "Unit" },
+        "effects": ["IO"]
+      },
+      "params": [],
+      "body": {
+        "t": "seq",
+        "lhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "add_three" },
+            "args": [
+              { "t": "lit", "lit": { "kind": "int", "value": 1 } },
+              { "t": "lit", "lit": { "kind": "int", "value": 2 } },
+              { "t": "lit", "lit": { "kind": "int", "value": 3 } }
+            ]
+          }]
+        },
+        "rhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "add_three" },
+            "args": [
+              { "t": "lit", "lit": { "kind": "int", "value": 10 } },
+              { "t": "lit", "lit": { "kind": "int", "value": 20 } },
+              { "t": "lit", "lit": { "kind": "int", "value": 30 } }
+            ]
+          }]
+        }
+      }
+    }
+  ]
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t1_add_three.task.json

@@ -0,0 +1,7 @@
+{
+  "id": "t1_add_three",
+  "title": "Three-argument addition",
+  "description": "Write a complete AILang module named t1_add_three. It must export a top-level function add_three that takes three Int parameters and returns their sum. It must also export main : () -> () !IO that prints add_three(1, 2, 3) and then add_three(10, 20, 30), each on its own line.",
+  "expected_stdout": "6\n60\n",
+  "reference_solution": "t1_add_three.reference.ail.json"
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t2_length.reference.ail.json

@@ -0,0 +1,121 @@
+{
+  "schema": "ailang/v0",
+  "name": "t2_length",
+  "imports": [],
+  "defs": [
+    {
+      "kind": "type",
+      "name": "List",
+      "vars": ["a"],
+      "ctors": [
+        { "name": "Nil", "fields": [] },
+        {
+          "name": "Cons",
+          "fields": [
+            { "k": "var", "name": "a" },
+            { "k": "con", "name": "List", "args": [{ "k": "var", "name": "a" }] }
+          ]
+        }
+      ]
+    },
+    {
+      "kind": "fn",
+      "name": "length",
+      "type": {
+        "k": "forall",
+        "vars": ["a"],
+        "body": {
+          "k": "fn",
+          "params": [
+            { "k": "con", "name": "List", "args": [{ "k": "var", "name": "a" }] }
+          ],
+          "ret": { "k": "con", "name": "Int" },
+          "effects": []
+        }
+      },
+      "params": ["xs"],
+      "body": {
+        "t": "match",
+        "scrutinee": { "t": "var", "name": "xs" },
+        "arms": [
+          {
+            "pat": { "p": "ctor", "ctor": "Nil", "fields": [] },
+            "body": { "t": "lit", "lit": { "kind": "int", "value": 0 } }
+          },
+          {
+            "pat": {
+              "p": "ctor",
+              "ctor": "Cons",
+              "fields": [
+                { "p": "wild" },
+                { "p": "var", "name": "rest" }
+              ]
+            },
+            "body": {
+              "t": "app",
+              "fn": { "t": "var", "name": "+" },
+              "args": [
+                { "t": "lit", "lit": { "kind": "int", "value": 1 } },
+                {
+                  "t": "app",
+                  "fn": { "t": "var", "name": "length" },
+                  "args": [{ "t": "var", "name": "rest" }]
+                }
+              ]
+            }
+          }
+        ]
+      }
+    },
+    {
+      "kind": "fn",
+      "name": "main",
+      "type": {
+        "k": "fn",
+        "params": [],
+        "ret": { "k": "con", "name": "Unit" },
+        "effects": ["IO"]
+      },
+      "params": [],
+      "body": {
+        "t": "seq",
+        "lhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "length" },
+            "args": [{ "t": "ctor", "type": "List", "ctor": "Nil", "args": [] }]
+          }]
+        },
+        "rhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "length" },
+            "args": [{
+              "t": "ctor", "type": "List", "ctor": "Cons",
+              "args": [
+                { "t": "lit", "lit": { "kind": "int", "value": 7 } },
+                {
+                  "t": "ctor", "type": "List", "ctor": "Cons",
+                  "args": [
+                    { "t": "lit", "lit": { "kind": "int", "value": 8 } },
+                    {
+                      "t": "ctor", "type": "List", "ctor": "Cons",
+                      "args": [
+                        { "t": "lit", "lit": { "kind": "int", "value": 9 } },
+                        { "t": "ctor", "type": "List", "ctor": "Nil", "args": [] }
+                      ]
+                    }
+                  ]
+                }
+              ]
+            }]
+          }]
+        }
+      }
+    }
+  ]
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t2_length.task.json

@@ -0,0 +1,7 @@
+{
+  "id": "t2_length",
+  "title": "List length (polymorphic, locally-defined List)",
+  "description": "Write a complete AILang module named t2_length. It must define a local algebraic data type List a with constructors Nil and Cons a (List a). It must export a top-level function length : forall a. (List a) -> Int that returns the number of elements in the list. It must also export main : () -> () !IO that prints length(Nil) and then length(Cons(7, Cons(8, Cons(9, Nil)))), each on its own line.",
+  "expected_stdout": "0\n3\n",
+  "reference_solution": "t2_length.reference.ail.json"
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t3_main_prints.reference.ail.json

@@ -0,0 +1,31 @@
+{
+  "schema": "ailang/v0",
+  "name": "t3_main_prints",
+  "imports": [],
+  "defs": [
+    {
+      "kind": "fn",
+      "name": "main",
+      "type": {
+        "k": "fn",
+        "params": [],
+        "ret": { "k": "con", "name": "Unit" },
+        "effects": ["IO"]
+      },
+      "params": [],
+      "body": {
+        "t": "seq",
+        "lhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{ "t": "lit", "lit": { "kind": "int", "value": 42 } }]
+        },
+        "rhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{ "t": "lit", "lit": { "kind": "int", "value": 1337 } }]
+        }
+      }
+    }
+  ]
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t3_main_prints.task.json

@@ -0,0 +1,7 @@
+{
+  "id": "t3_main_prints",
+  "title": "main prints two fixed integers",
+  "description": "Write a complete AILang module named t3_main_prints. It must export main : () -> () !IO that prints 42 and then 1337, each on its own line. No other definitions are required.",
+  "expected_stdout": "42\n1337\n",
+  "reference_solution": "t3_main_prints.reference.ail.json"
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t4_count_zeros.reference.ail.json

@@ -0,0 +1,145 @@
+{
+  "schema": "ailang/v0",
+  "name": "t4_count_zeros",
+  "imports": [],
+  "defs": [
+    {
+      "kind": "type",
+      "name": "List",
+      "vars": ["a"],
+      "ctors": [
+        { "name": "Nil", "fields": [] },
+        {
+          "name": "Cons",
+          "fields": [
+            { "k": "var", "name": "a" },
+            { "k": "con", "name": "List", "args": [{ "k": "var", "name": "a" }] }
+          ]
+        }
+      ]
+    },
+    {
+      "kind": "fn",
+      "name": "count_zeros",
+      "type": {
+        "k": "fn",
+        "params": [
+          { "k": "con", "name": "List", "args": [{ "k": "con", "name": "Int" }] }
+        ],
+        "ret": { "k": "con", "name": "Int" },
+        "effects": []
+      },
+      "params": ["xs"],
+      "body": {
+        "t": "match",
+        "scrutinee": { "t": "var", "name": "xs" },
+        "arms": [
+          {
+            "pat": { "p": "ctor", "ctor": "Nil", "fields": [] },
+            "body": { "t": "lit", "lit": { "kind": "int", "value": 0 } }
+          },
+          {
+            "pat": {
+              "p": "ctor",
+              "ctor": "Cons",
+              "fields": [
+                { "p": "var", "name": "h" },
+                { "p": "var", "name": "rest" }
+              ]
+            },
+            "body": {
+              "t": "if",
+              "cond": {
+                "t": "app",
+                "fn": { "t": "var", "name": "eq" },
+                "args": [
+                  { "t": "var", "name": "h" },
+                  { "t": "lit", "lit": { "kind": "int", "value": 0 } }
+                ]
+              },
+              "then": {
+                "t": "app",
+                "fn": { "t": "var", "name": "+" },
+                "args": [
+                  { "t": "lit", "lit": { "kind": "int", "value": 1 } },
+                  {
+                    "t": "app",
+                    "fn": { "t": "var", "name": "count_zeros" },
+                    "args": [{ "t": "var", "name": "rest" }]
+                  }
+                ]
+              },
+              "else": {
+                "t": "app",
+                "fn": { "t": "var", "name": "count_zeros" },
+                "args": [{ "t": "var", "name": "rest" }]
+              }
+            }
+          }
+        ]
+      }
+    },
+    {
+      "kind": "fn",
+      "name": "main",
+      "type": {
+        "k": "fn",
+        "params": [],
+        "ret": { "k": "con", "name": "Unit" },
+        "effects": ["IO"]
+      },
+      "params": [],
+      "body": {
+        "t": "seq",
+        "lhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "count_zeros" },
+            "args": [{ "t": "ctor", "type": "List", "ctor": "Nil", "args": [] }]
+          }]
+        },
+        "rhs": {
+          "t": "do",
+          "op": "io/print_int",
+          "args": [{
+            "t": "app",
+            "fn": { "t": "var", "name": "count_zeros" },
+            "args": [{
+              "t": "ctor", "type": "List", "ctor": "Cons",
+              "args": [
+                { "t": "lit", "lit": { "kind": "int", "value": 0 } },
+                {
+                  "t": "ctor", "type": "List", "ctor": "Cons",
+                  "args": [
+                    { "t": "lit", "lit": { "kind": "int", "value": 5 } },
+                    {
+                      "t": "ctor", "type": "List", "ctor": "Cons",
+                      "args": [
+                        { "t": "lit", "lit": { "kind": "int", "value": 0 } },
+                        {
+                          "t": "ctor", "type": "List", "ctor": "Cons",
+                          "args": [
+                            { "t": "lit", "lit": { "kind": "int", "value": 3 } },
+                            {
+                              "t": "ctor", "type": "List", "ctor": "Cons",
+                              "args": [
+                                { "t": "lit", "lit": { "kind": "int", "value": 0 } },
+                                { "t": "ctor", "type": "List", "ctor": "Nil", "args": [] }
+                              ]
+                            }
+                          ]
+                        }
+                      ]
+                    }
+                  ]
+                }
+              ]
+            }]
+          }]
+        }
+      }
+    }
+  ]
+}

experiments/2026-05-12-cross-model-authoring/master/tasks/t4_count_zeros.task.json

@@ -0,0 +1,7 @@
+{
+  "id": "t4_count_zeros",
+  "title": "Count zeros in a list using Eq Int",
+  "description": "Write a complete AILang module named t4_count_zeros. It must define a local algebraic data type List a with constructors Nil and Cons a (List a). It must export a top-level function count_zeros : (List Int) -> Int that returns the number of elements in the list that equal 0, using the prelude Eq Int instance to compare against 0. It must also export main : () -> () !IO that prints count_zeros(Nil) and then count_zeros(Cons(0, Cons(5, Cons(0, Cons(3, Cons(0, Nil)))))), each on its own line.",
+  "expected_stdout": "0\n3\n",
+  "reference_solution": "t4_count_zeros.reference.ail.json"
+}