AILang/docs/plans/0085-embedding-abi-m2.1.md

Embedding ABI — M2.1: per-thread runtime context + concurrency safety — Implementation Plan

Parent spec: docs/specs/0042-embedding-abi-m2.md

For agentic workers: REQUIRED SUB-SKILL: use skills/implement to run this plan. Steps use - [ ] checkboxes for tracking.

Goal: Make the compiled scalar kernel callable concurrently from a host thread swarm without an RC-runtime data race, by threading a mandatory ailang_ctx_t* first parameter through the C @<sym> forwarder (TLS transport, internal convention byte-untouched) and de-globalising the RC accounting onto the ctx for the swarm path while retaining g_rc_*+atexit as the null-ctx executable fallback.

Architecture: Three code layers change — runtime/rc.c (new ctx struct + lifecycle + __thread slot + conditional accounting split), crates/ailang-codegen Target::StaticLib forwarder only (leading ptr %ctx, TLS save/store/restore around the unchanged internal call), crates/ail build_staticlib (reject --alloc != rc). Plus DESIGN.md current-state update. No schema field, no ailang-check change, no Form-A change; the internal @ail_<mod>_<fn> convention and the _adapter/_clos pair stay byte-identical.

Tech Stack: C11 (runtime/rc.c, __thread, pthread test harnesses, -fsanitize=thread), Rust (ailang-codegen IR emission, ailang CLI, integration tests), LLVM IR text, clang/ar.

---

Files this plan creates or modifies

• Modify: runtime/rc.c:86–103,113–127,161–177 — ailang_ctx_t struct + ailang_ctx_new/ailang_ctx_free + __thread __ail_tls_ctx; conditional if (ctx) ctx->… else g_rc_… accounting split; statics+atexit+constructor retained as null-ctx fallback.
• Modify: crates/ailang-codegen/src/lib.rs:600–638 — Target::StaticLib forwarder: leading ptr %ctx, one @__ail_tls_ctx = external thread_local global ptr decl, TLS save/store/restore around the unchanged call @ail_<mname>_<fn>.
• Modify: crates/ailang-codegen/src/lib.rs:165–176 — Target::StaticLib doc comment: narrow "provisional until M3" to the value/record layout.
• Modify: crates/ail/src/main.rs:2485–2493 — build_staticlib: reject alloc != AllocStrategy::Rc with the RC-only diagnostic before lowering.
• Modify: docs/DESIGN.md:2263–2307 — §"Embedding ABI (M1)" updated in place to post-M2 current state (ctx mandatory; per-thread RC-only swarm; accounting per ctx; "provisional until M3" narrowed to value/record layout). The "Export parameters are written bare…" sentence + the canonical (con Int) snippet preserved verbatim.
• Modify: docs/DESIGN.md:1528–1529 — Decision 10 atomicity clause: one-line per-thread-ctx note.
• Modify: crates/ailang-codegen/tests/embed_staticlib_lowering.rs:20–33 — staticlib_emits_forwarder_no_main extended for the ctx forwarder shape.
• Modify: crates/ail/tests/embed/host.c — migrated to the ctx ABI (M1 value oracle, single-threaded).
• Modify: crates/ail/tests/embed_e2e.rs:17–46 — c_host_calls_exported_scalar_kernel migrated to drive the ctx-ABI host.c.
• Create: crates/ail/tests/embed/rc_accounting_tsan.c — direct ailang_rc_alloc/ailang_rc_dec+ailang_ctx_* N-thread harness (links libailang_rt.a only; the de-globalisation teeth — item 1).
• Create: crates/ail/tests/embed/swarm.c — N-pthread scalar-kernel swarm, per-thread-ctx capability demo ONLY (no SHARED_CTX; a non-allocating scalar kernel cannot race even under a shared ctx — item 2).
• Create: crates/ail/tests/embed_swarm_tsan.rs — driver: builds the staticlib, links swarm.c under -fsanitize=thread, asserts the per-ctx positive (item 2).
• Modify: crates/ail/tests/embed_rc_accounting_tsan.rs (Task 5 Step 3) — add the -DSHARED_CTX de-globalisation negative-control standing test (item 3, in Task 2's rc-accounting harness where a shared ctx genuinely races).
• Create: crates/ail/tests/embed_staticlib_alloc_guard.rs — RED-first pin: --emit=staticlib --alloc=gc|bump rejected (Task 1).
• Test (regression, unmodified): crates/ail/tests/print_no_leak_pin.rs, crates/ail/tests/e2e.rs (leak-stat helper), crates/ailang-core/tests/embed_export_hash_stable.rs, crates/ailang-core/tests/design_schema_drift.rs, crates/ailang-check/tests/embed_export_gate.rs, crates/ailang-core/tests/docs_honesty_pin.rs.

---

Task 1: Baseline pins (FIXED FIRST — spec §Iteration sequencing)

Ships before any runtime/codegen/CLI change. (a) names the existing executable-path readback tests as the standing null-ctx regression guard; (b) RED-first pins the --alloc-guard final behaviour (RED now ⇒ proves current success).

Files:

• Create: crates/ail/tests/embed_staticlib_alloc_guard.rs

• Step 1: Confirm the executable-path readback baseline is green (0a guard, no new test)

Run: cargo test -p ail --test print_no_leak_pin -- alloc_rc_print_int_does_not_leak_show_result_str Expected: PASS (test result: ok. 1 passed)

Run: cargo test -p ail --test e2e -- alloc_rc_explicit_mode_tail_sum_does_not_leak_outer_cells Expected: PASS (test result: ok. 1 passed)

These two named tests are the null-ctx-fallback regression guard. The plan's runtime task (Task 2) must keep them green unmodified; Task 8 re-asserts.

• Step 2: Write the RED-first alloc-guard pin

Create crates/ail/tests/embed_staticlib_alloc_guard.rs:

//! M2: `ail build --emit=staticlib` is RC-only. `--alloc=gc`/`--alloc=bump`
//! must fail the build (the shared Boehm collector / bench stub are not
//! swarm-safe). RED until the Task-4 CLI guard lands: today the alloc
//! strategy is passed straight through and the build SUCCEEDS.
use std::process::Command;

fn ail_bin() -> &'static str { env!("CARGO_BIN_EXE_ail") }

fn build_staticlib_with_alloc(alloc: &str, outdir: &str) -> std::process::Output {
    Command::new(ail_bin())
        .args([
            "build", "examples/embed_backtest_step.ail",
            "--emit=staticlib", &format!("--alloc={alloc}"), "-o", outdir,
        ])
        .current_dir(env!("CARGO_MANIFEST_DIR").to_string() + "/../..")
        .output()
        .expect("spawn ail build")
}

#[test]
fn staticlib_gc_is_rejected() {
    let out = build_staticlib_with_alloc("gc", "/tmp/ail_m2_guard_gc");
    assert!(
        !out.status.success(),
        "expected `--emit=staticlib --alloc=gc` to FAIL the build; exit was success"
    );
    let stderr = String::from_utf8_lossy(&out.stderr);
    assert!(
        stderr.contains("staticlib (swarm) artefact is RC-only"),
        "expected the RC-only diagnostic; stderr was:\n{stderr}"
    );
}

#[test]
fn staticlib_bump_is_rejected() {
    let out = build_staticlib_with_alloc("bump", "/tmp/ail_m2_guard_bump");
    assert!(
        !out.status.success(),
        "expected `--emit=staticlib --alloc=bump` to FAIL the build; exit was success"
    );
    let stderr = String::from_utf8_lossy(&out.stderr);
    assert!(
        stderr.contains("staticlib (swarm) artefact is RC-only"),
        "expected the RC-only diagnostic; stderr was:\n{stderr}"
    );
}

• Step 3: Run the pin to verify it is RED (proves current success — spec item 0b)

Run: cargo test -p ail --test embed_staticlib_alloc_guard Expected: FAIL — both tests fail at the first assertion with expected ... to FAIL the build; exit was success (today build_staticlib passes the strategy through and the build succeeds, producing the two archives). This RED state IS the "currently succeeds" baseline capture.

---

Task 2: Runtime — ailang_ctx_t + lifecycle + __ail_tls_ctx + conditional accounting (de-globalisation teeth)

runtime/rc.c is C — no cargo compile gate. Its regression coverage is the direct N-thread rc-accounting tsan harness (spec item 1), which this task ships RED-first alongside the change. Harness links only libailang_rt.a (never the kernel) — keeps the de-globalisation proof distinct from the capability demo.

Files:

• Create: crates/ail/tests/embed/rc_accounting_tsan.c

• Modify: runtime/rc.c:86–103,113–127,161–177

• (driver test fn added to the new crates/ail/tests/embed_swarm_tsan.rs in Task 5; this task adds only the rc-accounting C file + its standalone build/run inside a #[test] here)

• Create: crates/ail/tests/embed_rc_accounting_tsan.rs

• Step 1: Write the direct rc-accounting tsan C harness (RED — symbols absent today)

Create crates/ail/tests/embed/rc_accounting_tsan.c:

/* M2 item 1: the de-globalisation teeth. Exercises ailang_rc_alloc /
 * ailang_rc_dec DIRECTLY (not through any kernel) from N threads, each
 * owning its own ailang_ctx_t. Per-ctx counters must be individually
 * correct and -fsanitize=thread-clean. Build -DSHARED_CTX for the
 * global-hammering negative control (must make tsan report a race). */
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <assert.h>

typedef struct ailang_ctx ailang_ctx_t;
extern ailang_ctx_t *ailang_ctx_new(void);
extern void          ailang_ctx_free(ailang_ctx_t *);
extern void         *ailang_rc_alloc(size_t);
extern void          ailang_rc_dec(void *);
/* set the per-call ctx the runtime accounts into (defined in rc.c) */
extern __thread ailang_ctx_t *__ail_tls_ctx;

#define NTHREADS 8
#define NCYCLES  200000

#ifdef SHARED_CTX
static ailang_ctx_t *g_shared; /* negative control: one ctx, all threads */
#endif

static void *worker(void *_a) {
    (void)_a;
#ifdef SHARED_CTX
    ailang_ctx_t *ctx = g_shared;
#else
    ailang_ctx_t *ctx = ailang_ctx_new();
#endif
    __ail_tls_ctx = ctx;
    for (int i = 0; i < NCYCLES; i++) {
        void *p = ailang_rc_alloc(16);
        ailang_rc_dec(p); /* refcount 1 -> 0, frees, free_count++ */
    }
#ifndef SHARED_CTX
    /* per-ctx balance must hold exactly */
    /* (read via the same struct layout the runtime defines) */
    ailang_ctx_free(ctx);
#endif
    return NULL;
}

int main(void) {
#ifdef SHARED_CTX
    g_shared = ailang_ctx_new();
#endif
    pthread_t t[NTHREADS];
    for (int i = 0; i < NTHREADS; i++) pthread_create(&t[i], NULL, worker, NULL);
    for (int i = 0; i < NTHREADS; i++) pthread_join(t[i], NULL);
#ifdef SHARED_CTX
    ailang_ctx_free(g_shared);
#endif
    printf("rc_accounting_tsan: ok\n");
    return 0;
}

• Step 2: Write the driver and run it to verify RED

Create crates/ail/tests/embed_rc_accounting_tsan.rs:

//! M2 item 1: direct rc-accounting de-globalisation proof. Compiles
//! runtime/rc.c + runtime/str.c into libailang_rt.a, links the
//! rc_accounting_tsan.c harness against it under -fsanitize=thread.
//! Per-thread-ctx build: tsan-clean + exit 0. RED until rc.c gains
//! ailang_ctx_* + __ail_tls_ctx (link error: undefined symbols).
use std::process::Command;
use std::path::PathBuf;

fn ws_root() -> PathBuf {
    PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..").canonicalize().unwrap()
}

fn cc(args: &[&str], ctx: &str) {
    let st = Command::new("clang").args(args).status().expect("spawn clang");
    assert!(st.success(), "clang failed: {ctx}");
}

#[test]
fn rc_accounting_per_ctx_is_tsan_clean() {
    let root = ws_root();
    let tmp = std::env::temp_dir().join("ail_m2_rcacct");
    std::fs::create_dir_all(&tmp).unwrap();
    let rc_o = tmp.join("rc.o");
    let str_o = tmp.join("str.o");
    let rt_a = tmp.join("libailang_rt.a");
    cc(&["-O1", "-g", "-fsanitize=thread", "-c", "-o", rc_o.to_str().unwrap(),
         root.join("runtime/rc.c").to_str().unwrap()], "rc.c");
    cc(&["-O1", "-g", "-fsanitize=thread", "-c", "-o", str_o.to_str().unwrap(),
         root.join("runtime/str.c").to_str().unwrap()], "str.c");
    let st = Command::new("ar").args(["rcs", rt_a.to_str().unwrap(),
        rc_o.to_str().unwrap(), str_o.to_str().unwrap()]).status().unwrap();
    assert!(st.success(), "ar failed");
    let bin = tmp.join("rcacct");
    cc(&["-O1", "-g", "-fsanitize=thread", "-pthread",
         "-o", bin.to_str().unwrap(),
         root.join("crates/ail/tests/embed/rc_accounting_tsan.c").to_str().unwrap(),
         rt_a.to_str().unwrap()], "link rc_accounting_tsan");
    let out = Command::new(&bin).env("TSAN_OPTIONS", "halt_on_error=1")
        .output().expect("run harness");
    assert!(out.status.success(),
        "per-ctx rc-accounting harness must exit 0 tsan-clean; stderr:\n{}",
        String::from_utf8_lossy(&out.stderr));
}

Run: cargo test -p ail --test embed_rc_accounting_tsan Expected: FAIL — clang failed: link rc_accounting_tsan (undefined references to ailang_ctx_new, ailang_ctx_free, __ail_tls_ctx — absent from today's rc.c).

• Step 3: Implement the rc.c ctx struct + lifecycle + TLS slot

In runtime/rc.c, immediately after the g_rc_free_count static (currently rc.c:87), insert:

/* M2: per-thread embedding context. A scalar kernel allocates nothing,
 * so this carries only the de-globalised RC accounting. Set by the
 * generated C `@<sym>` forwarder into __ail_tls_ctx for the synchronous
 * duration of one call (never held across a suspension point). The
 * g_rc_* statics + atexit below are RETAINED as the null-ctx
 * (single-threaded executable) fallback. */
typedef struct ailang_ctx {
    uint64_t alloc_count;
    uint64_t free_count;
} ailang_ctx_t;

__thread ailang_ctx_t *__ail_tls_ctx = NULL;

ailang_ctx_t *ailang_ctx_new(void) {
    return (ailang_ctx_t *)calloc(1, sizeof(ailang_ctx_t));
}

void ailang_ctx_free(ailang_ctx_t *ctx) {
    if (ctx != NULL) {
        const char *flag = getenv("AILANG_RC_STATS");
        if (flag != NULL && flag[0] != '\0') {
            fprintf(stderr,
                    "ailang_rc_stats: allocs=%llu frees=%llu live=%lld\n",
                    (unsigned long long)ctx->alloc_count,
                    (unsigned long long)ctx->free_count,
                    (long long)(ctx->alloc_count - ctx->free_count));
        }
    }
    free(ctx);
}

• Step 4: Split the two increment sites (swarm → ctx, else → global)

In ailang_rc_alloc, replace the line g_rc_alloc_count++; (currently rc.c:125) with:

    ailang_ctx_t *_ctx = __ail_tls_ctx;
    if (_ctx != NULL) _ctx->alloc_count++; else g_rc_alloc_count++;

In ailang_rc_dec, inside the if (*hdr == 0) block, replace the line g_rc_free_count++; (currently rc.c:175) with:

        ailang_ctx_t *_ctx = __ail_tls_ctx;
        if (_ctx != NULL) _ctx->free_count++; else g_rc_free_count++;

Leave g_rc_alloc_count/g_rc_free_count (:86–87), ailang_rc_stats_atexit (:89–95), and ailang_rc_stats_install (:97–103) verbatim — they are now the null-ctx fallback only.

• Step 5: Run the item-1 harness to verify GREEN

Run: cargo test -p ail --test embed_rc_accounting_tsan Expected: PASS (test result: ok. 1 passed) — links cleanly, per-ctx counters correct, zero tsan reports.

---

Task 3: Codegen — Target::StaticLib forwarder gains ptr %ctx + TLS save/store/restore

lower_workspace_staticlib* Rust signatures are unchanged (recon-confirmed) — no E0061 caller cascade; the only Rust gate is the codegen crate's own build + the extended IR-shape test. embed_e2e.rs is expected RED between this task and Task 6 (M1 host.c calls the now-ptr-prefixed symbol); its migration is Task 6 — do NOT add embed_e2e to this task's GREEN gate.

Files:

• Modify: crates/ailang-codegen/src/lib.rs:600–638 and :165–176

• Modify: crates/ailang-codegen/tests/embed_staticlib_lowering.rs:20–33

• Step 1: Extend the RED IR-shape pin

In crates/ailang-codegen/tests/embed_staticlib_lowering.rs, in staticlib_emits_forwarder_no_main, after the existing assertions on the forwarder, add (keep all existing assertions, including the no-@main one and the _adapter/_clos ones, unchanged):

    // M2: forwarder takes a leading ptr %ctx and routes it through TLS
    // around the UNCHANGED internal call.
    assert!(ir.contains("@__ail_tls_ctx = external thread_local global ptr"),
        "staticlib module must declare the external thread-local ctx slot;\nIR:\n{ir}");
    assert!(ir.contains("define i64 @backtest_step(ptr %ctx, i64 %a0, i64 %a1)"),
        "forwarder must take a leading ptr %ctx;\nIR:\n{ir}");
    assert!(ir.contains("store ptr %ctx, ptr @__ail_tls_ctx"),
        "forwarder must publish ctx into the TLS slot;\nIR:\n{ir}");
    assert!(ir.contains("call i64 @ail_embed_backtest_step_step(i64 %a0, i64 %a1)"),
        "internal call must stay byte-unchanged (no ctx arg);\nIR:\n{ir}");

• Step 2: Run to verify RED

Run: cargo test -p ailang-codegen --test embed_staticlib_lowering -- staticlib_emits_forwarder_no_main Expected: FAIL — first new assertion fails: staticlib module must declare the external thread-local ctx slot (today's forwarder is the no-ctx M1 shape).

• Step 3: Emit the external thread-local decl once in the StaticLib arm

In crates/ailang-codegen/src/lib.rs, in the Target::StaticLib => { arm (currently opens at :600), immediately after the arm opens and before the for (mname, m) in &ws.modules { loop, insert:

            // M2: the per-thread embedding ctx slot. Defined by
            // runtime/rc.c (__thread ailang_ctx_t *__ail_tls_ctx);
            // the forwarder publishes its explicit ctx param here for
            // the synchronous duration of the internal call.
            out.push_str("\n@__ail_tls_ctx = external thread_local global ptr\n");

• Step 4: Thread ptr %ctx through the forwarder body

In the same arm, replace the forwarder format! block (currently lib.rs:626–633, the out.push_str(&format!("\ndefine {cret} @{sym}(... … ret {cret} %r\n}}\n", …))) with:

                            let mut sig_params = vec!["ptr %ctx".to_string()];
                            sig_params.extend(
                                param_tys.iter().enumerate()
                                    .map(|(i, t)| format!("{} %a{i}", llvm_scalar(t))));
                            out.push_str(&format!(
                                "\ndefine {cret} @{sym}({}) {{\n  \
                                 %saved = load ptr, ptr @__ail_tls_ctx\n  \
                                 store ptr %ctx, ptr @__ail_tls_ctx\n  \
                                 %r = call {cret} @ail_{mname}_{}({})\n  \
                                 store ptr %saved, ptr @__ail_tls_ctx\n  \
                                 ret {cret} %r\n}}\n",
                                sig_params.join(", "),
                                f.name,
                                args.join(", "),
                            ));

(args — the internal-call argument vector built at lib.rs:623–625 — is left exactly as is: the internal @ail_<mname>_<fn> call takes the scalars only, byte-unchanged. _adapter/_clos emission is untouched.)

• Step 5: Narrow the provisional doc on the Target enum

In crates/ailang-codegen/src/lib.rs:165–176, in the Target::StaticLib doc comment, replace the phrase that calls the signature "provisional until M3" with wording that scopes provisionality to the value/record layout. Exact replacement — change the sentence reading (around :173) to:

    /// emit one external C entrypoint per `(export …)` fn forwarding
    /// to the internal `@ail_<module>_<def>`. M2: the entrypoint takes
    /// a mandatory leading `ptr %ctx` (per-thread embedding context)
    /// published via `@__ail_tls_ctx` around the unchanged internal
    /// call. The ctx-threaded C signature is the M2 shape; only the
    /// value/record layout remains provisional until M3.

(Match and replace only the existing StaticLib doc lines; do not alter the Executable variant or its doc.)

• Step 6: Run to verify GREEN (codegen IR-shape only)

Run: cargo test -p ailang-codegen --test embed_staticlib_lowering Expected: PASS (test result: ok. N passed) — both staticlib_emits_forwarder_no_main (extended) and executable_target_still_emits_main (unmodified) pass.

Run: cargo build -p ailang-codegen Expected: Finished (0 errors).

---

Task 4: CLI — build_staticlib --alloc != rc reject guard

RED test already shipped in Task 1 (embed_staticlib_alloc_guard). This task is the GREEN side: the guard.

Files:

• Modify: crates/ail/src/main.rs:2485–2493

• Step 1: Insert the reject guard before lowering

In crates/ail/src/main.rs, in fn build_staticlib, immediately before the let ir = ailang_codegen::lower_workspace_staticlib_with_alloc(&ws, alloc)?; line (currently :2493), insert:

    if !matches!(alloc, ailang_codegen::AllocStrategy::Rc) {
        anyhow::bail!(
            "staticlib (swarm) artefact is RC-only — `--alloc=gc` links the \
             shared Boehm collector, which is not swarm-safe; use `--alloc=rc`"
        );
    }

(Placed after the has_export check and before lowering, so a non-RC staticlib request fails before any .ll/archive work. The --alloc clap default is already rc, so the default path is unaffected; this rejects only an explicit --alloc=gc/--alloc=bump.)

• Step 2: Run the Task-1 pin to verify RED→GREEN

Run: cargo test -p ail --test embed_staticlib_alloc_guard Expected: PASS (test result: ok. 2 passed) — both staticlib_gc_is_rejected and staticlib_bump_is_rejected now see a non-zero exit with the staticlib (swarm) artefact is RC-only string.

• Step 3: Confirm the default (rc) staticlib build still works

Run: cargo test -p ail --test embed_staticlib_cli Expected: PASS (test result: ok. N passed) — default-alloc two-archive build path unaffected by the guard.

---

Task 5: Scalar-swarm capability demo (per-thread-ctx) + de-globalisation negative-control standing test

Boss correction (2026-05-18, post-first-dispatch): the original Task 5 paired a -DSHARED_CTX negative control onto swarm.c. That is structurally impossible and was the spec defect the first implement dispatch correctly surfaced: examples/embed_backtest_step.ail is a non-allocating scalar kernel — it never calls ailang_rc_alloc/ailang_rc_dec, never writes a ctx field — and @__ail_tls_ctx is __thread (per-thread, never shared even when the ailang_ctx_t* value is), so a shared-ctx scalar swarm has no shared-memory write to race on and tsan correctly stays clean (exactly the §Testing-item-1 honesty point). The negative control belongs to the item-1 direct rc-accounting harness (rc_accounting_tsan.c, which does write ctx->alloc_count concurrently under a shared ctx — a genuine race). swarm.c is the per-thread-ctx capability demo ONLY. Spec amended in lockstep (§Goal coherent-stop, §must-fail-axis item 2, §Testing item 3, §Acceptance).

Two deliverables: (5a) the per-thread-ctx scalar-swarm capability demo (swarm.c, links both archives); (5b) the de-globalisation negative-control standing test, added to Task 2's item-1 rc_accounting_tsan.c harness (which already carries the #ifdef SHARED_CTX block) so the teeth are a reproducible CI test, not a one-off manual check.

Files:

• Create: crates/ail/tests/embed/swarm.c (per-thread-ctx capability demo, no SHARED_CTX)

• Create: crates/ail/tests/embed_swarm_tsan.rs (one green positive test)

• Modify: crates/ail/tests/embed_rc_accounting_tsan.rs (Task 2's driver — add the -DSHARED_CTX negative-control standing test)

• Step 1: Write the capability-demo swarm C file (per-thread-ctx only)

Create crates/ail/tests/embed/swarm.c:

/* M2 item 2: scalar-kernel capability demo. N threads, each owning
 * its own ailang_ctx_t, drive the headline scalar kernel; must be
 * -fsanitize=thread-clean with every thread's result == the serial
 * oracle. NO negative control here: a non-allocating scalar kernel
 * writes no shared mutable state even under a shared ctx, so a
 * shared-ctx variant cannot race (the de-globalisation negative
 * control lives in the rc_accounting harness — Task 5 Step 3). */
#include <stdint.h>
#include <stdio.h>
#include <pthread.h>

typedef struct ailang_ctx ailang_ctx_t;
extern ailang_ctx_t *ailang_ctx_new(void);
extern void          ailang_ctx_free(ailang_ctx_t *);
extern int64_t backtest_step(ailang_ctx_t *ctx, int64_t state, int64_t sample);

#define NTHREADS 8
#define NITERS   200000

static int64_t serial_oracle(void) {
    /* mirrors examples/embed_backtest_step.ail: state += sample*sample,
     * sample = (i & 7) over NITERS iterations. */
    int64_t s = 0;
    for (int i = 0; i < NITERS; i++) { int64_t x = i & 7; s = s + x * x; }
    return s;
}

static void *worker(void *out) {
    ailang_ctx_t *ctx = ailang_ctx_new();
    int64_t s = 0;
    for (int i = 0; i < NITERS; i++) s = backtest_step(ctx, s, (i & 7));
    ailang_ctx_free(ctx);
    *(int64_t *)out = s;
    return NULL;
}

int main(void) {
    pthread_t t[NTHREADS];
    int64_t res[NTHREADS];
    for (int i = 0; i < NTHREADS; i++) pthread_create(&t[i], NULL, worker, &res[i]);
    for (int i = 0; i < NTHREADS; i++) pthread_join(t[i], NULL);
    int64_t oracle = serial_oracle();
    for (int i = 0; i < NTHREADS; i++) {
        if (res[i] != oracle) { fprintf(stderr, "thread %d: %lld != %lld\n",
            i, (long long)res[i], (long long)oracle); return 1; }
    }
    printf("swarm: ok (%lld)\n", (long long)oracle);
    return 0;
}

• Step 2: Write the capability-demo driver (one green positive test)

Create crates/ail/tests/embed_swarm_tsan.rs:

//! M2 item 2 (capability demo). Builds the staticlib (rc) via the
//! `ail` CLI, links swarm.c against lib<entry>.a + libailang_rt.a
//! under -fsanitize=thread. Per-thread-ctx: tsan-clean, exit 0,
//! every thread's result == serial oracle. (The de-globalisation
//! negative control is in embed_rc_accounting_tsan.rs — item 1.)
use std::process::Command;
use std::path::PathBuf;

fn ws_root() -> PathBuf {
    PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..").canonicalize().unwrap()
}
fn ail_bin() -> &'static str { env!("CARGO_BIN_EXE_ail") }

fn build_staticlib(outdir: &PathBuf) {
    std::fs::create_dir_all(outdir).unwrap();
    let st = Command::new(ail_bin())
        .args(["build", "examples/embed_backtest_step.ail",
               "--emit=staticlib", "--alloc=rc", "-o", outdir.to_str().unwrap()])
        .current_dir(ws_root()).status().expect("ail build");
    assert!(st.success(), "ail build --emit=staticlib failed");
}

#[test]
fn scalar_swarm_per_ctx_is_tsan_clean_and_matches_oracle() {
    let outdir = std::env::temp_dir().join("ail_m2_swarm_ok");
    build_staticlib(&outdir);
    let root = ws_root();
    let bin = outdir.join("swarm_ok");
    let st = Command::new("clang").args([
        "-O1", "-g", "-fsanitize=thread", "-pthread",
        "-o", bin.to_str().unwrap(),
        root.join("crates/ail/tests/embed/swarm.c").to_str().unwrap(),
        &format!("-L{}", outdir.to_string_lossy()),
        "-lembed_backtest_step", "-lailang_rt",
    ]).status().expect("clang");
    assert!(st.success(), "linking swarm.c failed");
    let out = Command::new(&bin).env("TSAN_OPTIONS", "halt_on_error=1")
        .output().expect("run swarm");
    assert!(out.status.success(),
        "per-thread-ctx swarm must exit 0 tsan-clean; stderr:\n{}",
        String::from_utf8_lossy(&out.stderr));
}

• Step 3: Add the de-globalisation negative-control standing test (item-1 harness)

crates/ail/tests/embed/rc_accounting_tsan.c already carries the #ifdef SHARED_CTX block (Task 2). Append this test to the existing crates/ail/tests/embed_rc_accounting_tsan.rs (reuse its ws_root() + cc() helpers defined in Task 2):

#[test]
fn rc_accounting_shared_ctx_is_flagged_by_tsan() {
    // Negative control: all N threads share one ailang_ctx_t* while
    // calling ailang_rc_alloc/ailang_rc_dec → concurrent
    // ctx->alloc_count++ is a genuine data race tsan MUST report.
    // Proves the de-globalisation property is not vacuous.
    let root = ws_root();
    let tmp = std::env::temp_dir().join("ail_m2_rcacct_neg");
    std::fs::create_dir_all(&tmp).unwrap();
    let rc_o = tmp.join("rc.o");
    let str_o = tmp.join("str.o");
    let rt_a = tmp.join("libailang_rt.a");
    cc(&["-O1", "-g", "-fsanitize=thread", "-c", "-o", rc_o.to_str().unwrap(),
         root.join("runtime/rc.c").to_str().unwrap()], "rc.c");
    cc(&["-O1", "-g", "-fsanitize=thread", "-c", "-o", str_o.to_str().unwrap(),
         root.join("runtime/str.c").to_str().unwrap()], "str.c");
    let st = Command::new("ar").args(["rcs", rt_a.to_str().unwrap(),
        rc_o.to_str().unwrap(), str_o.to_str().unwrap()]).status().unwrap();
    assert!(st.success(), "ar failed");
    let bin = tmp.join("rcacct_neg");
    cc(&["-O1", "-g", "-fsanitize=thread", "-pthread", "-DSHARED_CTX",
         "-o", bin.to_str().unwrap(),
         root.join("crates/ail/tests/embed/rc_accounting_tsan.c").to_str().unwrap(),
         rt_a.to_str().unwrap()], "link rc_accounting_tsan -DSHARED_CTX");
    let out = Command::new(&bin).env("TSAN_OPTIONS", "halt_on_error=1")
        .output().expect("run rcacct neg");
    let stderr = String::from_utf8_lossy(&out.stderr);
    assert!(
        !out.status.success() || stderr.contains("WARNING: ThreadSanitizer"),
        "shared-ctx rc-accounting negative control MUST be tsan-flagged \
         (the de-globalisation teeth); exit={:?} stderr:\n{stderr}",
        out.status.code()
    );
}

• Step 4: Run both halves

Run: cargo test -p ail --test embed_swarm_tsan Expected: PASS (test result: ok. 1 passed) — scalar_swarm_per_ctx_is_tsan_clean_and_matches_oracle exits 0 tsan-clean, every thread == oracle.

Run: cargo test -p ail --test embed_rc_accounting_tsan Expected: PASS (test result: ok. 2 passed) — the per-ctx positive (Task 2) stays green AND rc_accounting_shared_ctx_is_flagged_by_tsan observes the genuine tsan race on ctx->alloc_count (teeth proven).

---

Task 6: Migrate M1 host.c + embed_e2e.rs to the ctx ABI (item 5)

After Task 3 the M1 host.c (no-ctx backtest_step) is RED (wrong call ABI). Migrate it; keep the M1 value assertion (s == 25).

Files:

• Modify: crates/ail/tests/embed/host.c

• Modify: crates/ail/tests/embed_e2e.rs:17–46

• Step 1: Confirm embed_e2e is RED post-Task-3 (expected)

Run: cargo test -p ail --test embed_e2e -- c_host_calls_exported_scalar_kernel Expected: FAIL — the M1 host.c calls backtest_step(s, 3) against the now-(ptr,i64,i64) symbol (garbage ctx; s != 25 or crash). This is the expected RED that Task 3 deliberately left for this task.

• Step 2: Migrate host.c to the ctx ABI

Replace the entire contents of crates/ail/tests/embed/host.c with:

#include <stdint.h>
#include <assert.h>
typedef struct ailang_ctx ailang_ctx_t;
extern ailang_ctx_t *ailang_ctx_new(void);
extern void          ailang_ctx_free(ailang_ctx_t *);
extern int64_t backtest_step(ailang_ctx_t *ctx, int64_t state, int64_t sample);
int main(void) {
    ailang_ctx_t *ctx = ailang_ctx_new();
    int64_t s = 0;
    s = backtest_step(ctx, s, 3);   /* 0 + 9  */
    s = backtest_step(ctx, s, 4);   /* 9 + 16 */
    ailang_ctx_free(ctx);
    assert(s == 25);
    return 0;
}

• Step 3: Update embed_e2e.rs link line for the runtime archive

In crates/ail/tests/embed_e2e.rs, in c_host_calls_exported_scalar_kernel, ensure the harness links libailang_rt.a alongside lib<entry>.a (M1 already produces both archives; the host now references ailang_ctx_new/_free which live in libailang_rt.a). If the existing link command already passes -lailang_rt (M1 two-archive harness), no change is needed beyond Step 2; otherwise add -lailang_rt to the clang link args next to the existing -lembed_backtest_step (the exact existing arg vector is in this fn — append -lailang_rt if absent). Do not change the asserted value (s == 25) or exit-0 expectation.

• Step 4: Run to verify GREEN

Run: cargo test -p ail --test embed_e2e -- c_host_calls_exported_scalar_kernel Expected: PASS (test result: ok. 1 passed) — ctx-ABI host links, runs, s == 25 holds.

---

Task 7: DESIGN.md current-state update (preserve docs_honesty_pin.rs:135)

docs_honesty_pin.rs:135 asserts (via the norm() whitespace-collapse helper) that DESIGN.md contains Export parameters are written **bare**: a scalar type carries no own/borrow mode. M2's §"Embedding ABI (M1)" rewrite MUST keep that sentence and the canonical (con Int) snippet verbatim; edit only the provisional-until-M3 sentence and add the ctx/swarm prose. The pin then stays green unmodified.

Files:

• Modify: docs/DESIGN.md:2282–2285

• Modify: docs/DESIGN.md:1528–1529

• Step 1: Replace only the provisional-until-M3 sentence

In docs/DESIGN.md, in §"Embedding ABI (M1)", replace the sentence currently at :2282–2285 reading:

M1 ships no runtime lifecycle API: the per-thread
context parameter that M2 threads through every exported call will
change this C signature; do not treat the M1 signature as frozen.
The value/record layout freeze is M3.

with:

Every exported entrypoint takes a mandatory leading `ailang_ctx_t*`
(M2): a per-thread embedding context created by `ailang_ctx_new()`
and released by `ailang_ctx_free()`, owned by the calling thread for
its lifetime. The host links one `ailang_ctx_t` per OS worker thread
and the runtime accounts RC alloc/free into it (no shared mutable
runtime state — the swarm artefact is data-race-free, sanitiser-
verified). The staticlib swarm artefact is **RC-only**:
`ail build --emit=staticlib` rejects `--alloc=gc`/`--alloc=bump`
(the shared Boehm collector is not swarm-safe). Only the value/record
layout remains provisional until M3; the ctx-threaded C signature is
the M2 shape.

Do NOT touch the paragraph at :2287–2289 ("Export parameters are written bare: a scalar type carries no own/borrow mode …") or the canonical (con Int) snippet at :2292–2302 — they are pinned and orthogonal to M2.

• Step 2: Add the Decision-10 per-thread-ctx note

In docs/DESIGN.md, the Decision 10 atomicity bullet currently at :1528–1529:

- **Does not commit to atomic refcounts.** AILang is
  single-threaded; refcounts are non-atomic.

Replace with:

- **Does not commit to atomic refcounts.** AILang is
  single-threaded; refcounts are non-atomic. The embedding ABI's
  per-thread `ailang_ctx_t` keeps this correct under a host swarm:
  each thread's allocations are private to its ctx, no RC cell
  crosses a thread, so non-atomic refcounts stay sound.

• Step 3: Verify the honesty pin + schema-drift stay green unmodified

Run: cargo test -p ailang-core --test docs_honesty_pin Expected: PASS (test result: ok. N passed) — the preserved "Export parameters are written bare…" sentence keeps the :135 assertion green; no pin edit.

Run: cargo test -p ailang-core --test design_schema_drift Expected: PASS — M2 edits DESIGN.md only outside the ## Data model scan window (2263–2307 and 1528–1529); anchors unaffected.

---

Task 8: Executable-path + schema/hash invariance regression gate (items 6, 7)

Pure verification — no code. Asserts the null-ctx fallback and the no-schema-change promises hold.

Files: (none modified — regression gate)

• Step 1: Executable-path AILANG_RC_STATS readback unchanged

Run: cargo test -p ail --test print_no_leak_pin Expected: PASS (test result: ok. N passed) — unmodified; the g_rc_*+atexit null-ctx fallback prints exactly as before.

Run: cargo test -p ail --test e2e -- rc_stats Expected: PASS — the leak-stat helper consumers (alloc_rc_explicit_mode_tail_sum_does_not_leak_outer_cells and the sibling *rc_stats* tests) green; the ailang_rc_stats: atexit line shape preserved.

• Step 2: No schema field — hash + drift invariant

Run: cargo test -p ailang-core --test embed_export_hash_stable Expected: PASS (fn_without_export_hash_is_unchanged green) — M2 adds no schema field.

Run: cargo test -p ailang-check --test embed_export_gate Expected: PASS (test result: ok. 8 passed) — the M1 scalar/effect export gate is unchanged by M2.

• Step 3: Full workspace gate

Run: cargo test --workspace 2>&1 | tail -5 Expected: every suite test result: ok; 0 failed. (Diff against the pre-M2 baseline: the only test files added are embed_staticlib_alloc_guard, embed_rc_accounting_tsan, embed_swarm_tsan; the only behaviour-migrated test is embed_e2e::c_host_calls_exported_scalar_kernel.)

---

Task 9: Bench-trio carry-on + null-ctx-fallback bench-neutrality (item 9)

The null-ctx fallback adds load __thread ptr; brif null ahead of the executable path's g_rc_*++. Demonstrate bench-neutral, not hand-waved.

Files: (none modified — regression gate)

• Step 1: Run the regression trio

Run: python3 bench/check.py; echo "exit=$?" Expected: exit=0. If exit=1 on a *.bump_s or latency.*.max_us metric, that is a tracked-P2 noise family (roadmap P2) — proceed to Step 2 for causal exoneration; do NOT ratify a baseline.

Run: python3 bench/compile_check.py; echo "exit=$?" Expected: exit=0.

Run: python3 bench/cross_lang.py; echo "exit=$?" Expected: exit=0.

• Step 2: Causal exoneration if a tracked-noise metric fires

If and only if check.py exited 1 on a tracked-noise metric, build the benched binary at the pre-M2 commit and at HEAD and byte-compare:

Run:

git stash; git rev-parse HEAD > /tmp/m2_head
cargo build --release -p ail 2>/dev/null
./target/release/ail build examples/list_sum.ail --alloc=rc -o /tmp/bench_head
git stash pop
cargo build --release -p ail 2>/dev/null
./target/release/ail build examples/list_sum.ail --alloc=rc -o /tmp/bench_m2
cmp /tmp/bench_head/list_sum /tmp/bench_m2/list_sum && echo "BYTE-IDENTICAL: causally exonerated (no ratify)"

Expected: for the executable path, list_sum does not allocate across a ctx (null-ctx fallback), and the rc.c change adds only the load/brif null ahead of g_rc_*++; if the metric is a tracked noise family the binaries differ only by the fallback branch — record the exoneration in the iter journal, do NOT update any baseline. (If the binaries are NOT byte-identical AND a non-noise metric regressed, STOP and bounce to the Boss: that is a real codegen regression, not carry-on.)

---

Self-review (Step 5 — completed inline)

1. Spec coverage: §Architecture → Tasks 2–4,7; §Concrete code shapes (impl) → Tasks 2,3,4; §Components table → every row mapped (rc.c T2; forwarder T3; CLI T4; DESIGN.md T7; embed_e2e T6; tests T1,T2,T5,T8); §Testing items 0→T1, 1→T2, 2/3→T5, 4→T3, 5→T6, 6→T8, 7→T8, 8→T8/architect-at-audit, 9→T9; §Acceptance/§Iteration sequencing → T1 fixed-first honoured. ✓
2. Placeholder scan: no "TBD/TODO/implement later/similar to/add appropriate" — grep-clean. ✓
3. Type/name consistency: ailang_ctx_t, ailang_ctx_new, ailang_ctx_free, __ail_tls_ctx, @__ail_tls_ctx, backtest_step, staticlib_gc_is_rejected, embed_swarm_tsan used identically across T1–T9 and the file-map. ✓
4. Step granularity: every step is one 2–5 min action (one file edit / one cargo run). ✓
5. No commit steps: none present (Boss commits at iter end). ✓
6. Pin/replacement contiguity: the only presence-pin paired with a verbatim doc edit is docs_honesty_pin.rs:135 ↔ DESIGN.md §"Embedding ABI (M1)". Task 7 Step 1 edits ONLY :2282–2285 and explicitly forbids touching :2287–2289 (the pinned sentence) — the pinned substring is never inside the replacement body, so no soft-wrap split is possible; norm() additionally collapses whitespace. ✓
7. Compile-gate vs deferred-caller: no task changes a Rust fn signature (lower_workspace_staticlib* unchanged; build_staticlib adds an internal guard, signature unchanged). The one cross-task RED interval (Task 3 leaves embed_e2e RED until Task 6) is a link/runtime gate, explicitly named in both Task 3's preamble and Task 6 Step 1, and Task 3's GREEN gate scopes to embed_staticlib_lowering only — no silent resequencing. ✓
8. Verification filter strings resolve: existing-test filters use recon-verified names (alloc_rc_print_int_does_not_leak_show_result_str, alloc_rc_explicit_mode_tail_sum_does_not_leak_outer_cells, staticlib_emits_forwarder_no_main, executable_target_still_emits_main, c_host_calls_exported_scalar_kernel, fn_without_export_hash_is_unchanged); new-test filters resolve by construction (defined in the same task: scalar_swarm_per_ctx_is_tsan_clean_and_matches_oracle, rc_accounting_per_ctx_is_tsan_clean, rc_accounting_shared_ctx_is_flagged_by_tsan, staticlib_gc_is_rejected/staticlib_bump_is_rejected); the -- rc_stats filter (T8 S1) is backed by the unfiltered print_no_leak_pin run in the same step and e2e.rs:2178 helper consumers are *rc_stats*-named (recon-confirmed) — and T8 S3 runs the unfiltered cargo test --workspace so "nothing ran" cannot masquerade as green. The Task-5 split (capability demo with no negative control vs. de-globalisation teeth in the rc-accounting harness) is the resolution of the spec defect the first dispatch surfaced; no harness asserts a structurally-unreachable race. ✓