# iter embedding-abi-m2.1 — per-thread runtime context + concurrency safety

**Date:** 2026-05-18
**Started from:** c9a84b33b3e7806491153a5bd3037be1b8ef9d31 (Tasks 1–4 committed)
**Original start (first dispatch):** b3388c83506a314fbd87f3354f80ae8cc212f501
**Status:** DONE
**Tasks completed:** 9 of 9 (Tasks 1–4 in the first dispatch, committed at c9a84b3; Tasks 5–9 in this re-dispatch)

## Summary

The per-thread embedding-ctx ABI is complete. Tasks 1–4 (first
dispatch, committed at c9a84b3): the FIXED-FIRST alloc-guard baseline
pin, the de-globalised `ailang_ctx_t` runtime (per-thread TLS slot +
conditional accounting, `g_rc_*`+atexit retained verbatim as the
null-ctx fallback), the codegen forwarder gaining a leading
`ptr %ctx` + `@__ail_tls_ctx` save/store/restore around the
byte-unchanged internal call, and the CLI RC-only guard. Tasks 5–9
(this re-dispatch): the per-thread-ctx scalar-swarm capability demo
(`swarm.c`, no SHARED_CTX, GREEN), the de-globalisation
negative-control standing test in the rc-accounting harness
(`rc_accounting_shared_ctx_is_flagged_by_tsan` — shared-ctx genuinely
raced under tsan, teeth proven), the M1 `host.c` migration to the ctx
ABI (`s == 25` holds through the ctx-threaded forwarder), the
DESIGN.md current-state update (provisional-until-M3 narrowed to
value/record layout; Decision-10 per-thread-ctx note; pinned
"Export parameters are written **bare**…" sentence + `(con Int)`
snippet preserved verbatim), and the full regression + bench gate
(workspace 0 failed; bench-trio causally exonerated). The milestone's
concurrency-safety invariant is sanitiser-verified.

## Boss spec-defect repair (2026-05-18)

The first dispatch (Tasks 1–4 clean) correctly **BLOCKED on Task 5**,
surfacing a genuine spec defect: the original Task 5 paired a
`-DSHARED_CTX` negative control onto `swarm.c`. That is structurally
impossible — `examples/embed_backtest_step.ail` is a non-allocating
scalar kernel (`state += sample*sample`); it never calls
`ailang_rc_alloc`/`ailang_rc_dec`, never writes a `ctx` field, and
`@__ail_tls_ctx` is `__thread` (per-thread, never shared even when the
`ailang_ctx_t*` value is passed to a shared ctx). A shared-ctx scalar
swarm therefore has no shared-memory write to race on; tsan correctly
stays clean. Asserting a structurally-unreachable race would have been
a vacuous (and dishonest) test. The Boss adjudicated: the
de-globalisation negative-control teeth belong to the **item-1
rc-accounting harness** (`rc_accounting_tsan.c -DSHARED_CTX`, where a
shared ctx genuinely races on `ctx->alloc_count`), NOT `swarm.c`. The
spec was amended in lockstep and the plan's Task 5 restructured:
`swarm.c` is the per-thread-ctx capability demo ONLY (one green
positive test); the negative control became
`rc_accounting_shared_ctx_is_flagged_by_tsan` added to the committed
`embed_rc_accounting_tsan.rs`. This re-dispatch implemented the
corrected Task 5 — no swarm.c negative control reintroduced. The
plan's Task 3 expected-string transcription error (recorded below)
was likewise Boss-corrected (`@ail_embed_backtest_step_step`, the
module-qualified convention-correct symbol).

## Per-task notes

- iter embedding-abi-m2.1.1 (first dispatch, committed): FIXED-FIRST
  alloc-guard pin. New `crates/ail/tests/embed_staticlib_alloc_guard.rs`
  (verbatim from plan); RED captured (build currently succeeds), the
  deliverable. RED→GREEN across the iter (Task 4 closes it).
- iter embedding-abi-m2.1.2 (first dispatch, committed): `runtime/rc.c`
  gains `ailang_ctx_t` {alloc_count, free_count} + `ailang_ctx_new`/
  `ailang_ctx_free` + `__thread __ail_tls_ctx`; the two increment
  sites become `if (_ctx) _ctx->… else g_rc_…`. `g_rc_*` statics +
  atexit + constructor retained verbatim. New `rc_accounting_tsan.c` +
  `embed_rc_accounting_tsan.rs`: RED (link error) → GREEN (per-ctx,
  8×200000, tsan-clean).
- iter embedding-abi-m2.1.3 (first dispatch, committed): codegen
  `Target::StaticLib` arm emits `@__ail_tls_ctx = external
  thread_local global ptr` once; forwarder signature gains leading
  `ptr %ctx`; body does load-save / store-ctx / unchanged
  `call @ail_<mname>_<fn>(args)` / store-restore. `_adapter`/`_clos`
  + internal arg vector byte-untouched. Doc-comment narrowed to
  value/record-layout provisionality. `embed_e2e` left RED by design
  (Task 6 migrates it).
- iter embedding-abi-m2.1.4 (first dispatch, committed):
  `crates/ail/src/main.rs build_staticlib` rejects `--alloc != Rc`
  after the has_export check, before lowering. Task-1 pin RED→GREEN;
  `embed_staticlib_cli` default path unaffected.
- iter embedding-abi-m2.1.5 (this re-dispatch): per-thread-ctx
  capability demo. New `crates/ail/tests/embed/swarm.c` (no
  SHARED_CTX) + `crates/ail/tests/embed_swarm_tsan.rs`;
  `scalar_swarm_per_ctx_is_tsan_clean_and_matches_oracle` GREEN
  (8 threads × 200k, tsan-clean, every result == serial oracle).
  De-globalisation negative control
  `rc_accounting_shared_ctx_is_flagged_by_tsan` appended to the
  committed `embed_rc_accounting_tsan.rs` (reuses its `ws_root()`/
  `cc()` helpers); GREEN — shared-ctx build genuinely raced under
  tsan (teeth proven, not vacuous). No swarm.c negative control
  reintroduced (the Boss spec-defect repair).
- iter embedding-abi-m2.1.6 (this re-dispatch): `crates/ail/tests/
  embed/host.c` replaced with the ctx-ABI version (opaque
  `ailang_ctx_t`, `ailang_ctx_new`/`_free` lifecycle, ctx-prefixed
  `backtest_step`). Confirmed expected RED post-Task-3 first
  (`s != 25` from the garbage-ctx arg shift). `embed_e2e.rs`
  intentionally NOT modified: the M1 harness already links
  `libailang_rt.a` by absolute path (`.arg(outdir.join(
  "libailang_rt.a"))`), so the ctx symbols resolve without an
  `-lailang_rt` add — the task block's own "no change is needed
  beyond Step 2" conditional branch. GREEN, `s == 25` holds.
- iter embedding-abi-m2.1.7 (this re-dispatch): `docs/DESIGN.md` —
  provisional-until-M3 sentence (~:2282) replaced with the M2 ctx
  prose (mandatory leading `ailang_ctx_t*`, per-thread, RC-only
  swarm artefact, value/record layout still provisional);
  Decision-10 atomicity bullet (~:1528) gains the per-thread-ctx
  soundness note. Pinned "Export parameters are written **bare**…"
  sentence + canonical `(con Int)` snippet preserved verbatim (the
  edit ends before them; no soft-wrap split risk). `docs_honesty_pin`
  GREEN unmodified (5 passed), `design_schema_drift` GREEN (8
  passed) — edits outside the `## Data model` scan window.
- iter embedding-abi-m2.1.8 (this re-dispatch): regression gate, no
  code. `print_no_leak_pin` (1) + `e2e -- rc_stats` (4) GREEN —
  null-ctx fallback prints unchanged. `embed_export_hash_stable`
  (1, `fn_without_export_hash_is_unchanged`) + `embed_export_gate`
  (6, all pass) GREEN — no schema field, export gate unchanged.
  Full `cargo test --workspace`: every suite ok, 0 failing suites.
- iter embedding-abi-m2.1.9 (this re-dispatch): bench gate, no code.
  `compile_check.py` exit=0 (24/24 stable); `cross_lang.py` exit=0
  (25/25 stable); `check.py` exit=1 on exactly 4 tracked-P2 noise
  metrics (`bench_list_sum.bump_s` +16.53%,
  `latency.explicit_at_rc.max_us` +40.41%,
  `latency.implicit_at_rc.p99_9_us` +31.66%,
  `latency.implicit_at_rc.max_us` +92.35%). Causally exonerated by
  the source-level invariant (see Concerns); no baseline ratified.

## Concerns

- iter embedding-abi-m2.1.3 (first dispatch — carried forward):
  PLAN TRANSCRIPTION ERROR. Task 3 Step 1's plan code block asserted
  `ir.contains("call i64 @ail_backtest_step(i64 %a0, i64 %a1)")`.
  The internal symbol is `@ail_embed_backtest_step_step`
  (module-qualified `@ail_<mname>_<fn>`), which the plan's own Step-4
  parenthetical and Boss constraint 3 require to stay byte-unchanged.
  Written instead with the convention-correct symbol (Boss-corrected
  in the re-dispatched plan). The plan file should remain corrected.
- iter embedding-abi-m2.1.6: `embed_e2e.rs` is in the task's `Files:`
  block but was intentionally left unchanged — the M1 harness already
  links `libailang_rt.a` by absolute path, so the ctx symbols resolve
  without an `-lailang_rt` add. This is the task block's own sanctioned
  "no change is needed beyond Step 2" conditional branch, not a missed
  requirement (observation, not correctness).
- iter embedding-abi-m2.1.7: the section header is still
  `## Embedding ABI (M1)` while the body now describes the M2 ctx ABI
  as current state — a header/content honesty mismatch. NOT fixed:
  renaming the header is outside the task block's named edit regions
  (`:2282–2285`, `:1528–1529`) and could perturb a doc-honesty pin
  tracking the `(M1)` token; the controller curates scope, not the
  implementer. Routed here for Boss adjudication — a candidate
  follow-up (header rename + fieldtest/pin audit) but explicitly NOT
  in this iter's plan scope.
- iter embedding-abi-m2.1.8: the task block expected
  `embed_export_gate` to be "8 passed"; the suite at HEAD c9a84b3 has
  exactly 6 tests, all passing, 0 filtered. The literal "8" is a
  plan-transcription artefact (pre-existing — M2 added/removed no test
  in this suite). The protected invariant (export gate unchanged by
  M2) holds unambiguously; the implementer correctly did NOT pad the
  suite to 8 (that would fabricate coverage). Plan number vs reality,
  not a regression.
- iter embedding-abi-m2.1.9: `check.py` exit=1 on 4 metrics, all in
  the tracked-P2 noise families (`*.bump_s`, `latency.*.max_us`,
  `latency.*.p99_9_us`). CAUSALLY EXONERATED by a source-level
  invariant stronger than the plan's binary `cmp`: this re-dispatch's
  entire working-tree footprint is test fixtures + test drivers +
  `docs/DESIGN.md` — ZERO files under `crates/*/src/` or `runtime/`.
  The `ail` codegen binary and the linked `rc.c`/`str.c` runtime are
  byte-identical to HEAD c9a84b3 by construction, so the firing
  metrics cannot be caused by this iter. The plan's `git stash`/
  `git stash pop` recipe was deliberately NOT run: on a re-dispatch
  it would discard the iter's uncommitted Tasks 5–8 work and violate
  the working-tree-as-handoff contract. No baseline ratified
  (consistent with roadmap-P2 noise policy + the 2026-05-14 audit-pd
  "baseline pristine, Nth consecutive observation" envelope).

## Known debt

- `## Embedding ABI (M1)` section header in DESIGN.md still says
  "(M1)" though its body is now M2 current-state. Out of this iter's
  plan scope (Task 7 named only `:2282–2285` and `:1528–1529`).
  Candidate follow-up: header rename with a doc-honesty-pin audit
  for any `(M1)` token dependency. Not touched to keep the diff
  scope-compliant and avoid perturbing a pin.

## Blocked detail

None. All 9 tasks completed (Tasks 1–4 committed at c9a84b3 in the
first dispatch; Tasks 5–9 in this re-dispatch, all reaching
`approved` quality with zero review re-loops). The first dispatch's
Task-5 BLOCK was a genuine spec defect, Boss-adjudicated and repaired
before this re-dispatch (see "Boss spec-defect repair").

## Files touched (this re-dispatch — Tasks 5–9 working tree)

Modified:
- `crates/ail/tests/embed/host.c` (Task 6: ctx-ABI migration)
- `crates/ail/tests/embed_rc_accounting_tsan.rs` (Task 5: +
  `rc_accounting_shared_ctx_is_flagged_by_tsan` negative control)
- `docs/DESIGN.md` (Task 7: M2 current-state + Decision-10 note)

Created:
- `crates/ail/tests/embed/swarm.c` (Task 5: per-thread-ctx
  capability demo, no SHARED_CTX)
- `crates/ail/tests/embed_swarm_tsan.rs` (Task 5: one green
  positive test)

(Tasks 1–4's files — `embed_staticlib_alloc_guard.rs`, `rc.c`,
codegen `lib.rs`, `embed_staticlib_lowering.rs`, `main.rs`,
`rc_accounting_tsan.c`, the Task-2 `embed_rc_accounting_tsan.rs`
base — are already committed at c9a84b3.)

## Stats

bench/orchestrator-stats/2026-05-18-iter-embedding-abi-m2.1.json
