iter embedding-abi-m5.1 (DONE 3/3): lean ail-embed core + build.rs + hermetic smoke

M5 iteration 1 (spec ae905de, plan 22f02aa). Stands up the
workspace-excluded `ail-embed` crate:

- zero-dependency embedding core (`ail-embed/src/lib.rs`): extern "C"
  to the M3-frozen ABI + frozen-layout State/Tick box helpers + a
  Kernel price fold; the Rust port of the audited
  crates/ail/tests/embed/tick_roundtrip.c. Raw pointers never escape
  the type.
- build.rs (no in-repo precedent): AIL_BIN env override else nested
  `cargo build -p ail` against the parent workspace (separate target
  dir → no cargo-lock deadlock), `ail build --emit=staticlib`, link
  directives.
- hermetic data-server smoke (ail-embed/tests/smoke.rs): synthetic
  Pepperstone-format ZIP fixture via data-server's own public
  RawTickRecord type → real DataServer → Kernel, bit-exact vs a
  same-order host reference fold; runs with no /mnt.
- `ail-embed` is its own cargo workspace root (empty [workspace]
  table); data-server is a dev-dependency only. Root Cargo.toml gains
  only a 4-line non-membership comment.

Invariant 1 Boss-verified independently: full+no-deps cargo metadata
on the AILang workspace shows data-server count 0; git status path
filter empty (zero diff to crates/ailang-*, crates/ail/, runtime/,
examples/*.ail); src/lib.rs zero code-level data_server; AILang
cargo build --workspace still clean. ail-embed suite 2/2 green
(kernel_run_sums_prices unit RED-first + hermetic_smoke integration),
verified by me, not just the agent report.

Two toolchain-forced corrections to the plan's verbatim
ail-embed/Cargo.toml (added empty [workspace] table; sibling dev-dep
path ../libs -> ../../libs, manifest-relative) — confined to the
plan-created manifest, no acceptance gate altered, 0 review re-loops.
Journal Concerns records the planner-recon implication for the next
workspace-excluded-nested-crate plan. Adapter API + thread-swarm
deferred to M5 iter 2+ per spec/plan.

Includes the per-iter journal, stats, and the INDEX.md line.

Cargo.toml

@@ -8,6 +8,10 @@ members = [
     "crates/ailang-prose",
     "crates/ail",
 ]
+# `ail-embed/` is intentionally NOT a member: it depends on the
+# external `../libs/data-server` and is the sole data-server↔AILang
+# meeting point (Invariant 1, docs/specs/2026-05-19-embedding-abi-m5.md).
+# Adding it here would couple the compiler workspace to a sibling dir.
 
 [workspace.package]
 version = "0.0.1"

ail-embed/.gitignore

@@ -0,0 +1 @@
+/target

ail-embed/Cargo.toml

@@ -0,0 +1,26 @@
+# Empty table: marks `ail-embed` as its own workspace root so the
+# physically-enclosing AILang workspace (../Cargo.toml) does not
+# auto-absorb this nested package. This is cargo's documented
+# mechanism for a deliberately workspace-excluded crate and keeps
+# `ail-embed` out of the AILang dependency graph entirely
+# (Invariant 1, docs/specs/2026-05-19-embedding-abi-m5.md).
+[workspace]
+
+[package]
+name = "ail-embed"
+version = "0.0.1"
+edition = "2021"
+license = "MIT"
+publish = false
+description = "Lean embedding of an AILang M3-frozen staticlib kernel into a Rust host. Not an AILang [workspace] member (Invariant 1)."
+
+# Iter m5.1: the library itself has ZERO dependencies — the embedding
+# core only touches the frozen C ABI. `data-server` is dev-only,
+# exercised solely by the hermetic smoke test, so Invariant 1 holds
+# in the dependency graph, not just on paper.
+[dependencies]
+
+[dev-dependencies]
+data-server = { path = "../../libs/data-server" }
+zip = "2"
+tempfile = "3"

ail-embed/build.rs

@@ -0,0 +1,55 @@
+//! Emits the M3-frozen kernel staticlib and links it.
+//!
+//! Resolution of the `ail` binary has no in-repo precedent (the
+//! in-test mechanism `env!("CARGO_BIN_EXE_ail")` is unavailable to a
+//! workspace-excluded crate's build script): use `$AIL_BIN` if set,
+//! else `cargo build --manifest-path <repo>/Cargo.toml -p ail` and
+//! use `<repo>/target/debug/ail`. The AILang workspace target dir
+//! (`<repo>/target`) is distinct from this crate's
+//! (`ail-embed/target`), so the nested `cargo build` cannot deadlock
+//! on the outer build's lock.
+use std::env;
+use std::path::PathBuf;
+use std::process::Command;
+
+fn main() {
+    let manifest = PathBuf::from(env::var("CARGO_MANIFEST_DIR").unwrap());
+    let repo = manifest.parent().unwrap().to_path_buf(); // /home/brummel/dev/ailang
+    let kernel = repo.join("examples/embed_backtest_step_tick.ail");
+    let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
+
+    println!("cargo:rerun-if-changed={}", kernel.display());
+    println!("cargo:rerun-if-changed=build.rs");
+    println!("cargo:rerun-if-env-changed=AIL_BIN");
+
+    let ail_bin: PathBuf = match env::var("AIL_BIN") {
+        Ok(p) => PathBuf::from(p),
+        Err(_) => {
+            let status = Command::new("cargo")
+                .args(["build", "--manifest-path"])
+                .arg(repo.join("Cargo.toml"))
+                .args(["-p", "ail"])
+                .status()
+                .expect("spawn `cargo build -p ail`");
+            assert!(status.success(), "building the `ail` CLI failed");
+            repo.join("target/debug/ail")
+        }
+    };
+
+    let build = Command::new(&ail_bin)
+        .arg("build")
+        .arg(&kernel)
+        .args(["--emit=staticlib", "-o"])
+        .arg(&out_dir)
+        .output()
+        .expect("spawn `ail build --emit=staticlib`");
+    assert!(
+        build.status.success(),
+        "ail build --emit=staticlib failed:\n{}",
+        String::from_utf8_lossy(&build.stderr)
+    );
+
+    println!("cargo:rustc-link-search=native={}", out_dir.display());
+    println!("cargo:rustc-link-lib=static=embed_backtest_step_tick");
+    println!("cargo:rustc-link-lib=static=ailang_rt");
+}

ail-embed/src/lib.rs

@@ -0,0 +1,122 @@
+//! Lean embedding of an AILang M3-frozen staticlib kernel into a
+//! Rust host. The Rust analogue of the audited C host
+//! `crates/ail/tests/embed/tick_roundtrip.c`: same frozen value
+//! layout (DESIGN.md §"Frozen value layout", :2322-2360), same
+//! `own`-mode contract. ZERO `data_server`/finance knowledge.
+
+use std::ffi::c_void;
+
+// The M3-frozen C ABI. Symbol names + signatures mirror
+// `crates/ail/tests/embed/tick_roundtrip.c:31-36`. `backtest_step_tick`
+// is the author-chosen export of `examples/embed_backtest_step_tick.ail`.
+unsafe extern "C" {
+    fn ailang_ctx_new() -> *mut c_void;
+    fn ailang_ctx_free(c: *mut c_void);
+    fn ailang_rc_alloc(n: usize) -> *mut c_void;
+    fn ailang_rc_dec(p: *mut c_void);
+    fn backtest_step_tick(c: *mut c_void, st: *mut c_void, tick: *mut c_void)
+        -> *mut c_void;
+}
+
+/// A per-thread embedding context (M2 `ailang_ctx_t`). `!Send` by
+/// construction (raw pointer) — one ctx per OS thread, as the ABI
+/// requires.
+struct Ctx(*mut c_void);
+
+impl Ctx {
+    fn new() -> Self {
+        Self(unsafe { ailang_ctx_new() })
+    }
+
+    /// `State` box, frozen layout (DESIGN.md :2328-2336):
+    /// payload `8 + 2*8` = 24 bytes — tag:i64@0, acc:f64@8, n:i64@16.
+    /// `ailang_rc_alloc` sets the `p-8` rc header to 1 and zeroes the
+    /// payload, so the tag write of 0 is the single-ctor tag.
+    fn make_state(acc: f64, n: i64) -> *mut c_void {
+        unsafe {
+            let p = ailang_rc_alloc(8 + 2 * 8) as *mut u8;
+            (p.add(0) as *mut i64).write_unaligned(0);
+            (p.add(8) as *mut f64).write_unaligned(acc);
+            (p.add(16) as *mut i64).write_unaligned(n);
+            p as *mut c_void
+        }
+    }
+
+    /// `Tick` box, frozen layout: payload `8 + 1*8` = 16 bytes —
+    /// tag:i64@0, px:f64@8.
+    fn make_tick(px: f64) -> *mut c_void {
+        unsafe {
+            let p = ailang_rc_alloc(8 + 1 * 8) as *mut u8;
+            (p.add(0) as *mut i64).write_unaligned(0);
+            (p.add(8) as *mut f64).write_unaligned(px);
+            p as *mut c_void
+        }
+    }
+
+    /// Reads back `(acc, n)` from a `State` payload pointer.
+    fn read_state(st: *mut c_void) -> (f64, i64) {
+        unsafe {
+            let p = st as *mut u8;
+            let acc = (p.add(8) as *const f64).read_unaligned();
+            let n = (p.add(16) as *const i64).read_unaligned();
+            (acc, n)
+        }
+    }
+}
+
+impl Drop for Ctx {
+    fn drop(&mut self) {
+        unsafe { ailang_ctx_free(self.0) }
+    }
+}
+
+/// Embeds the M3-frozen `(State, Tick) -> State` kernel and folds a
+/// price stream through it. Raw ABI pointers never escape this type
+/// (spec §"Error handling": no public way to hand the kernel a raw
+/// pointer).
+pub struct Kernel {
+    ctx: Ctx,
+}
+
+impl Kernel {
+    pub fn new() -> Self {
+        Self { ctx: Ctx::new() }
+    }
+
+    /// Folds `prices` via the kernel and returns `(Σ px, count)`.
+    /// `own` discipline (DESIGN.md :2346-2352): each `Tick` is freshly
+    /// `ailang_rc_alloc`'d and consumed by the kernel; `st` is
+    /// consumed and replaced by the return each step; only the final
+    /// return is host-`dec`'d.
+    pub fn run<I: IntoIterator<Item = f64>>(&self, prices: I) -> (f64, i64) {
+        let mut st = Ctx::make_state(0.0, 0);
+        for px in prices {
+            let tick = Ctx::make_tick(px);
+            st = unsafe { backtest_step_tick(self.ctx.0, st, tick) };
+        }
+        let out = Ctx::read_state(st);
+        unsafe { ailang_rc_dec(st) };
+        out
+    }
+}
+
+impl Default for Kernel {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::Kernel;
+
+    /// The kernel folds `acc += px; n += 1` over the price stream.
+    /// Same-order f64 sum ⇒ bit-exact: 3.0 + 4.0 + 5.0 == 12.0, n==3.
+    #[test]
+    fn kernel_run_sums_prices() {
+        let k = Kernel::new();
+        let (acc, n) = k.run([3.0_f64, 4.0, 5.0]);
+        assert_eq!(acc, 12.0);
+        assert_eq!(n, 3);
+    }
+}

ail-embed/tests/smoke.rs

@@ -0,0 +1,80 @@
+//! Hermetic smoke (Testing strategy §1): synthesize a
+//! Pepperstone-format `.tick` ZIP, point the REAL `DataServer` at it,
+//! fold the stream through the embedding `Kernel`, assert the result
+//! equals a host reference fold. Proves "adapter compiles, links,
+//! ABI handshake holds" with no `/mnt` dependency. The fixture writer
+//! dumps `data_server::records::RawTickRecord` bytes (the crate's own
+//! public on-disk type) into a `.bin` ZIP entry — correct by
+//! construction, mirroring `data-server/src/loader.rs:110-124`.
+
+use std::io::Write;
+use std::sync::Arc;
+
+use ail_embed::Kernel;
+use data_server::records::RawTickRecord;
+use data_server::DataServer;
+
+/// Writes `recs` as a Pepperstone tick file `<sym>_2017_01.tick`
+/// (ZIP holding one packed-`RawTickRecord` `.bin`) into `dir`.
+/// Filename satisfies the scanner regex `^(.+)_(\d{4})_(\d{2})\.tick$`
+/// (`data-server/src/lib.rs:88`); the `.bin` entry name is arbitrary
+/// as long as it ends in `.bin` (`loader.rs:50-55`).
+fn write_tick_fixture(dir: &std::path::Path, sym: &str, recs: &[RawTickRecord]) {
+    let path = dir.join(format!("{sym}_2017_01.tick"));
+    let f = std::fs::File::create(&path).unwrap();
+    let mut zip = zip::ZipWriter::new(f);
+    zip.start_file::<String, ()>("TICK.bin".into(), Default::default())
+        .unwrap();
+    for r in recs {
+        // SAFETY: RawTickRecord is #[repr(C, packed)], 24 bytes; this
+        // is exactly data-server's own test serialisation
+        // (loader.rs:117-120).
+        let bytes: &[u8] = unsafe {
+            std::slice::from_raw_parts(r as *const RawTickRecord as *const u8, 24)
+        };
+        zip.write_all(bytes).unwrap();
+    }
+    zip.finish().unwrap();
+}
+
+#[test]
+fn hermetic_smoke_data_server_roundtrip() {
+    // 10 deterministic ticks. mid = (ask+bid)/2; with ask==bid==i+1,
+    // mid == i+1. Σ_{i=0..9}(i+1) == 55.0 exactly in f64; n == 10.
+    // `time` is any monotonic Delphi-day value (no window ⇒ unused by
+    // the fold; only the filename drives symbol/format scanning).
+    let recs: Vec<RawTickRecord> = (0..10)
+        .map(|i| RawTickRecord {
+            time: 42795.0 + i as f64 * 1e-4,
+            ask: (i + 1) as f64,
+            bid: (i + 1) as f64,
+        })
+        .collect();
+
+    let dir = tempfile::tempdir().unwrap();
+    write_tick_fixture(dir.path(), "TEST", &recs);
+
+    let server = Arc::new(DataServer::new(dir.path()));
+    assert!(server.has_symbol("TEST"), "fixture symbol not scanned");
+
+    // Collect the mid-price stream in data-server order.
+    let mut it = server.stream_tick("TEST").expect("TEST tick stream");
+    let mut prices: Vec<f64> = Vec::new();
+    while let Some(chunk) = it.next_chunk() {
+        for r in chunk.iter() {
+            prices.push((r.ask + r.bid) / 2.0);
+        }
+    }
+    assert_eq!(prices.len(), 10, "all ticks streamed");
+
+    // Host reference fold, same order, same ops ⇒ bit-exact.
+    let ref_acc: f64 = prices.iter().copied().fold(0.0, |a, p| a + p);
+    let ref_n = prices.len() as i64;
+
+    let (acc, n) = Kernel::new().run(prices.iter().copied());
+
+    assert_eq!(acc, ref_acc, "kernel acc bit-exact vs host reference");
+    assert_eq!(acc, 55.0, "deterministic fixture sum");
+    assert_eq!(n, ref_n);
+    assert_eq!(n, 10);
+}

bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.1.json

@@ -0,0 +1,13 @@
+{
+  "iter_id": "embedding-abi-m5.1",
+  "date": "2026-05-19",
+  "mode": "standard",
+  "outcome": "DONE",
+  "tasks_total": 3,
+  "tasks_completed": 3,
+  "reloops_per_task": { "1": 0, "2": 0, "3": 0 },
+  "review_loops_spec": 0,
+  "review_loops_quality": 0,
+  "blocked_reason": null,
+  "notes": "Task 1 DONE_WITH_CONCERNS: two toolchain-forced corrections to the plan's verbatim ail-embed/Cargo.toml (added empty [workspace] table; dev-dep path ../libs/data-server -> ../../libs/data-server). Both confined to the in-scope plan-created manifest; no review re-loop incurred (caught and resolved within the implementer phase, spec+quality both first-pass green). RED-first observed for Task 2 (unresolved import super::Kernel). Invariant-1 gates: cargo metadata grep == 0, git status path filter empty."
+}

docs/journals/2026-05-19-iter-embedding-abi-m5.1.md

@@ -0,0 +1,112 @@
+# iter embedding-abi-m5.1 — lean `ail-embed` core + build.rs + hermetic smoke
+
+**Date:** 2026-05-19
+**Started from:** 22f02aa26b57c478760265dfc6468b54d35c7cdc
+**Status:** DONE
+**Tasks completed:** 3 of 3
+
+## Summary
+
+Stood up the workspace-excluded `ail-embed` crate: a zero-dependency
+embedding core that links the M3-frozen staticlib via a `build.rs`
+and folds a price stream through the frozen `(State, Tick) -> State`
+C ABI, plus a hermetic `data-server` smoke test proving the real
+adapter wiring without `/mnt`. The crate is its own cargo workspace
+root (not an AILang `[workspace]` member); `data-server` is a
+dev-dependency only, so Invariant 1 holds in the dependency graph,
+mechanically verified by both the `cargo metadata` grep (Task 1
+Step 7 → `0`) and the `git status` path filter (Task 3 Step 4 →
+empty). Zero diff to `crates/ailang-*`, `crates/ail/`, `runtime/`,
+`examples/*.ail`. Both ratifying tests are green
+(`kernel_run_sums_prices` unit + `hermetic_smoke_data_server_roundtrip`
+integration; 2/0). Two minimal, toolchain-forced corrections to the
+plan's verbatim `ail-embed/Cargo.toml` were required (see Concerns) —
+neither alters the iter's intent or any acceptance gate.
+
+## Per-task notes
+
+- iter embedding-abi-m5.1.1: Scaffold the workspace-excluded crate +
+  build.rs. Created `ail-embed/{Cargo.toml,.gitignore,src/lib.rs
+  (stub),build.rs}`; appended the deliberate-non-membership comment
+  to the root `Cargo.toml` members block (verbatim). Build gate
+  PASS, `cargo metadata … grep -c data-server` == 0. RED-first N/A
+  (build-infra de-risking task; verification is the build+metadata
+  gate, which the plan scripts).
+- iter embedding-abi-m5.1.2: The lean embedding core. RED-first
+  honored: wrote the test-only `src/lib.rs` first, observed RED
+  (`unresolved import \`super::Kernel\``, exactly the plan's
+  expected reason), then prepended the verbatim impl block
+  (`extern "C"` ABI, `Ctx` + frozen-layout box helpers + `Drop`,
+  `Kernel::new`/`run`, `Default`). GREEN: `kernel_run_sums_prices`
+  1/0.
+- iter embedding-abi-m5.1.3: Hermetic data-server smoke. Created
+  `ail-embed/tests/smoke.rs` verbatim (synthetic Pepperstone ZIP
+  fixture → real `DataServer` → `Kernel` fold → bit-exact assertion
+  vs host reference and against the closed-form `55.0`). Test-only
+  task (no separate RED). Full suite 2/0; Invariant-1 path filter
+  empty.
+
+## Concerns
+
+- iter .1 (DONE_WITH_CONCERNS): the plan's verbatim
+  `ail-embed/Cargo.toml` could not satisfy its own Step 6 build gate
+  as written. Two minimal toolchain-forced corrections, both confined
+  to the in-scope plan-created `ail-embed/Cargo.toml`, neither
+  changing iter intent or any acceptance gate:
+  1. Added an empty `[workspace]` table. Cargo hard-errors on a
+     nested package that "believes it's in a workspace when it's
+     not"; the empty table is cargo's documented mechanism for a
+     deliberately workspace-excluded nested crate (and is what makes
+     the Step 7 metadata grep return `0` — the AILang root manifest
+     no longer traverses into `ail-embed`). A comment at the table
+     records the rationale.
+  2. Corrected the `data-server` dev-dep path from
+     `../libs/data-server` to `../../libs/data-server`. The crate
+     lives at `/home/brummel/dev/libs/data-server` — a sibling of
+     the repo *directory*, one level above the repo root. From
+     `ail-embed/Cargo.toml`, `../` is the repo root, so the plan's
+     literal resolved to the non-existent
+     `repo/libs/data-server`. The plan's read-only cross-reference
+     anchors (`../libs/data-server/...`) are written from the
+     repo-root cwd and are correct *there*; only the manifest path
+     literal, resolved relative to `ail-embed/`, needed the extra
+     `../`. `build.rs` is unaffected (it never references
+     data-server; it computes `repo = manifest.parent()`).
+
+  Plan-template implication for the orchestrator: planner path-recon
+  for a workspace-excluded nested crate should (a) emit the empty
+  `[workspace]` table in the manifest template and (b) resolve
+  sibling-dependency paths relative to the *manifest's* directory,
+  not the repo-root cwd the anchors are written from.
+
+## Known debt
+
+- No `examples/*.ail.json` + `crates/ail/tests/e2e.rs` E2E fixture
+  added. Deliberate, not a gap: this iter ships zero compiler-surface
+  change (mechanically enforced by Task 3 Step 4 — adding a fixture
+  under `examples/` or `crates/ail/tests/` would itself break the
+  iter's Invariant-1 acceptance). The milestone invariant
+  ("M3-frozen staticlib embeds + folds bit-exactly via the C ABI;
+  data-server is a dev-only meeting point") is already protected by
+  the two tests this iter ships. Standard E2E protects compiler
+  invariants; there is none to protect here.
+- Adapter API + thread-swarm are explicitly deferred to embedding-abi
+  iter 2+ per the plan and spec — not touched, not debt of this iter.
+
+## Blocked detail
+
+N/A — DONE.
+
+## Files touched
+
+- Modified: `Cargo.toml` (root — 4-line non-membership comment after
+  the `members` array; no other change)
+- Created: `ail-embed/Cargo.toml`, `ail-embed/.gitignore`,
+  `ail-embed/build.rs`, `ail-embed/src/lib.rs`,
+  `ail-embed/tests/smoke.rs`
+- Untracked build artefacts under `ail-embed/target/` are
+  `.gitignore`d by `ail-embed/.gitignore`.
+
+## Stats
+
+bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.1.json

docs/journals/INDEX.md

@@ -106,3 +106,4 @@
 - 2026-05-18 — iter embedding-abi-m3.tidy (M3 audit [medium]+[low] doc-honesty fix, DONE 3/3, pin-safe): closed the two DRIFT items the M3 milestone-close audit routed here (M2.tidy `[medium]+[low] doc-honesty → tidy` precedent). [medium] docs/DESIGN.md §"Embedding ABI" — surgically replaced ONLY the contradicted M1-era parenthetical "(modes apply only to heap-shaped types, which the scalar-only rule above forbids at an export boundary anyway)" with the present-tense truth "(a single-constructor record export parameter, by contrast, carries `own`/`borrow` — the ownership contract the frozen value layout below specifies)"; the parenthetical shared physical line :2300 with the docs_honesty_pin.rs:135 pinned bare-scalar sentence ("Export parameters are written **bare**: a scalar type carries no `own`/`borrow` mode", norm()-whitespace-collapsed, fn form_a_scalar_param_carveout_present_and_old_rule_absent) — the edit kept every pinned word (line :2299 + the `own`/`borrow` mode` continuation untouched), the planner Step-5 item-6 presence-pin-vs-verbatim-edit collision the M2.tidy precedent ran. [low] crates/ailang-codegen/src/lib.rs:608-610 — comment-only honesty fix ("gate guarantees Int/Float; map Int→i64,Float→double" → "...Int/Float or a single-constructor record of those (M3); map Int→i64, Float→double, a record → ptr"); tree-wide-grep-confirmed no test pins the comment text; the byte-pin (embed_record_layout_pin) + forwarder-IR pin (embed_staticlib_lowering) assert generated IR not source comments → byte-identical before/after (the guard that no codegen moved). Boss-verified independently: both stale fragments grep-ABSENT; the 4 standing pins green at the exact recon baseline (docs_honesty_pin 5/0 ⇒ pin-safety held, design_schema_drift 8/0, embed_record_layout_pin 1/0, embed_staticlib_lowering 3/0); workspace 639/77 byte-unchanged from the M3-DONE baseline (docs/comment tidy, zero behaviour/test delta); diff exactly 2 files (DESIGN.md 5±, codegen/src/lib.rs 7±). No language/checker/codegen behaviour change; no audit/fieldtest gate (the 4 pins + 639/77 ARE the regression coverage, M2.tidy precedent). One non-gating planner-quality defect recorded in Concerns: Task-3 Step-1's `guarantees every param` verification grep is a substring of the plan's own Task-2 replacement text (non-discriminating) — the orchestrator correctly verified the substantive intent via discriminating fragments instead of bending code (same family as planner Step-5 item-8). bench: already carry-on / NO ratify at the M3 audit (causally exonerated by byte-identical generated IR; this tidy touches no executable path). audit source docs/journals/2026-05-18-audit-embedding-abi-m3.md (b8a60b1) → plan docs/plans/embedding-abi-m3.tidy.md (44ced51) → iter this commit. M3 milestone substantively closed + sound; roadmap [~]→[x] follows. → 2026-05-18-iter-embedding-abi-m3.tidy.md
 - 2026-05-18 — brainstorm embedding-abi-m4 → RETIRED, never speced (premise collapsed under its own feature-acceptance gate during Step-2/3 Q&A; no spec, no grounding-check, no planner handoff — the "problem mis-framed → don't ratify a known-unneeded shape" brainstorm path): `/boss` picked top-P0 "Embedding ABI — M4: sequence crossing via `List`"; user green-lit a continue-here brainstorm; recon (`ailang-plan-recon`) returned a full fact sheet; two user forks resolved in Q&A (own-only `List` param; `List Record`-only element) and Approach A (structural list-shaped `is_c_abi_type` arm, no name-match; B name-anchored / C `std_list`-SSOT-first rejected on language/scope grounds) recommended — all now moot. Struck on **feature-acceptance clause 2**: the shipped M3 gate `is_c_abi_type` (`crates/ailang-check/src/lib.rs:1934-1953`) is a per-parameter loop accepting a C scalar OR a single-ctor all-scalar record *independently per param*, and the forwarder's `llvm_scalar` maps every non-scalar `Type::Con`→`ptr` (M3-frozen), so `(State, Tick) -> State` (both single-ctor all-scalar records) is **already gate-accepted + forwarder-supported today** — the minimal data-server binding is M3 (shipped) + a host-side per-tick loop; cons-list crossing would *add* a 2N+1-box-per-chunk host builder + the deferred flat-array perf debt and removes no redundancy, with no named consumer (M5's adapter unrolls each chunk host-side — a clean adapter, the sole data-server↔AILang meeting point per Invariant 1; whole-chunk in-kernel visibility is semantically void since `State` threads across calls regardless of chunk boundaries). Honest mid-Q&A correction recorded: I had asserted "M5 cannot wire data-server without M4" — false (M5 wires it on M3 + `for tick in chunk`); re-deriving against the code rather than defending the roadmap I wrote is what surfaced the clause-2 failure ("user suggestions ≠ directives, form own judgment"). Outcome: M4 retired in `docs/roadmap.md` (struck entry kept one cycle, never `[x]`); M5 reconciled (`depends on:` M4→M3 + Tick-coverage todo; adapter unrolls host-side; friction feeds the host-per-tick-FFI-vs-batch P2 perf decision); residual = a new `[todo]` "Tick-coverage on M3" (E2E+fixture pinning the two-record-param per-tick `(State, Tick) -> State` shape — capability present today but only E2E-proven for a single record param `State`; every shipped M3 fixture pushes a scalar `Float` sample, none a record `Tick`; test backfill, no brainstorm — the actual "minimal data-server binding"). Forward note: the P2 flat-array item's "1024 cons-cells/chunk" framing is now partly stale (cons-list path dropped) — reconcile when picked up, not now. → 2026-05-18-brainstorm-embedding-abi-m4-retired.md
 - 2026-05-18 — iter bugfix-over-strict-mode-ctor-rebuild-consume (RED→GREEN, debug→implement mini, DONE 1/1): fixed a conservative `[over-strict-mode]` false-positive surfaced by the Tick-coverage fixtures. The lint's consume-detection (`any_sub_binder_consumed_for`/`pattern_has_consumed_heap_binder`, `crates/ailang-check/src/linearity.rs`) only recognised a consume of an `(own (con T))` param when a *heap-typed* pattern-binder was moved out of `match p`; when `p` was destructured into purely *primitive* fields fed into a `Term::Ctor` rebuilding `p`'s own ctor, that genuine dismantle+rebuild consume was invisible, so the lint spuriously advised `(borrow ...)`. Real harm: an LLM author "fixing" the spurious warning by flipping an export's declared mode `own`→`borrow` would silently invert the ABI ownership contract. RED-first: debugger disproved the carrier's initial nested-`match` hypothesis (the M3 `embed_backtest_step_record.ail` is silent only because its implicit-mode scalar `Float` param disables the lint via the activation gate, linearity.rs:327 — NOT because it handles the rebuild; the defect reproduces single-param, no nesting), wrote the synthetic RED unit `over_strict_mode_silent_when_ctor_rebuilt_from_primitive_fields` committed as its own audit-trail commit `a11cb7c`. GREEN (implement mini): added a 2nd recognition path to `any_sub_binder_consumed_for` — a `match p` arm that destructures binders out of `p`'s ctor and references any of them (primitive or not) inside a `Term::Ctor`'s args in the arm body genuinely consumes `p`; two pure helpers (`ctor_uses_any_binder` + deep `term_mentions_any_binder`, so `(+ acc px)`-mediated flow counts), conservative toward NOT suppressing (a ctor ignoring `p`'s payload still warns — negative-control proven), over-strict-only (by-name shadowing imprecision is extra-silence, never under-strict; recorded as Known debt). Both stale doc comments that mis-attributed the FP to nested `match` corrected for doc-honesty (debugger concern #2, same code region — in-scope, not opportunistic). +166/−20 in linearity.rs only; check-only, zero codegen/runtime/ABI/schema/DESIGN.md change. Boss-verified independently: RED→GREEN, full `cargo test -p ailang-check` 108/0 lib + every binary 0-failed with NO existing test modified, `ail check embed_backtest_step_tick.ail` no longer over-strict on `st`/`tick` (exit 0), `_tick_borrow.ail` + M3 `embed_backtest_step_record.ail` still clean, the already-green `embed_tick_e2e` + bench posture untouched. No audit/fieldtest gate (lint-precision bugfix; the RED test + the green check-suite ARE the regression coverage). RED `a11cb7c` → GREEN this commit. → 2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md
+- 2026-05-19 — iter embedding-abi-m5.1 (DONE 3/3): M5 iter 1 — stood up the workspace-excluded `ail-embed` crate. A zero-dependency embedding core (Rust port of the audited `crates/ail/tests/embed/tick_roundtrip.c`: `extern "C"` to the M3-frozen ABI + frozen-layout `State`/`Tick` box helpers + a `Kernel` price fold, raw pointers never escaping the type) links the M3 staticlib via a new `build.rs` (no in-repo precedent — `AIL_BIN` env override else nested `cargo build -p ail` against the parent workspace, separate target dir so no cargo-lock deadlock). Plus a hermetic `data-server` smoke: a synthetic Pepperstone-format ZIP fixture written via the crate's own public `RawTickRecord` type (correct-by-construction, mirrors `data-server/src/loader.rs:110-124`) → real `DataServer` → `Kernel`, asserting bit-exact vs a same-order host reference fold and the closed-form `55.0`; runs with no `/mnt`. `ail-embed` is its own cargo workspace root (empty `[workspace]` table) so the AILang compiler workspace owes it nothing; `data-server` is a dev-dependency only ⇒ Invariant 1 holds in the dependency graph, not just on paper. Boss-verified independently: ail-embed suite 2/2 green (`kernel_run_sums_prices` unit RED-first + `hermetic_smoke_data_server_roundtrip` integration), full+no-deps `cargo metadata` on the AILang workspace shows `data-server` count 0, `git status` path filter empty (zero diff to `crates/ailang-*`/`crates/ail/`/`runtime/`/`examples/*.ail`), `src/lib.rs` zero code-level `data_server`, AILang `cargo build --workspace` still clean. Two toolchain-forced corrections to the plan's verbatim `ail-embed/Cargo.toml` (added the empty `[workspace]` table; sibling dev-dep path `../libs`→`../../libs` manifest-relative) — confined to the plan-created manifest, no acceptance gate altered, 0 review re-loops; planner-recon implication recorded in the journal Concerns. Adapter API + thread-swarm explicitly deferred to M5 iter 2+. → 2026-05-19-iter-embedding-abi-m5.1.md