iter embedding-abi-m5.3 (DONE 3/3): time-shard boundary-invisibility proof + friction harvest

Final functional M5 iteration (spec ae905de, plan 67027ab; on
green m5.1 204c171 + m5.2 b724cd1 whose leak-proof resolved via
dbd76e5). Proves the chunk/window-boundary-invisibility claim the
M4 retirement rests on:

- adapter::fold_window — purely additive windowed sibling of
  fold_symbol (same MidPriceStream+Kernel path; stream_tick_windowed).
  fold_symbol/MidPriceStream/Kernel byte-untouched.
- chrono as a DEV-dep only — the time-shard test derives
  (y,m)->Unix-ms bounds (data-server's own precedent) and passes
  raw ms to the bin; the bin is date-math-free; Invariant 1
  unaffected.
- new single-responsibility [[bin]] timeshard_runner — one thread
  per disjoint window, each owning its Kernel (Ctx:!Send =>
  compile-time per-thread-ctx).
- tests/timeshard.rs — spec §3 strength-ordered: (a) per-shard
  (acc,n) BIT-EXACT vs single-thread host fold of that exact window
  in stream order (THE boundary-invisibility proof); (b) Σ partials
  bit-exact by construction; (c) whole-window within REL_TOL=1e-6
  only (cross-shard f64 reassociation is host arithmetic, never a
  kernel property — bit-exact there would be the bug; tolerance
  derived from the recursive-summation error bound). Plus the
  now-deterministic leak-Σ + a journal-only friction timing capture
  (no bench gate, no timing assertion).

Concrete pin: EURUSD 2017-03/04/05 (recon-verified present,
contiguous, no gaps; data-server has no month-listing API). RED was
a genuine deterministic compile failure (declared-but-absent bin
source; Cargo missing-source error vs the plan's predicted env!
error — same cause, no false-green; benign plan-text imprecision in
Concerns).

Boss-verified independently: timeshard determinism 5/5 (no jitter);
full ail-embed suite 0 failed AND 0 ignored across all binaries
(m5.2 symbol_fan_swarm_leak_free stays un-ignored & green — the
saga's resolution intact); isolated embed_rc_global_stats_race
still green; AILang workspace data-server count 0; compiler-surface
diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/,
examples/*.ail, root Cargo.toml, DESIGN.md).

Friction-harvest deliverable (P2 flat-array-decision input):
host-per-tick-FFI ~= 658 ms / 3,192,562 ticks ~= ~206 ns/tick at
real EURUSD volume.

M5 functional work complete; milestone closes next via the
mandatory audit (which also handles the pre-existing
DESIGN.md:2358-2360 "additive M4 concern" drift, out of m5.3 scope).
Includes the per-iter journal, stats, and the INDEX.md line.

docs/journals/2026-05-19-iter-embedding-abi-m5.3.md

@@ -0,0 +1,101 @@
+# iter embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest
+
+**Date:** 2026-05-19
+**Started from:** 67027ab0f98210926ed30ffd551306d3db3645ee
+**Status:** DONE
+**Tasks completed:** 3 of 3
+
+## Summary
+
+Final functional M5 iteration. Proves the chunk/window-boundary-invisibility
+claim the M4 retirement rests on: one symbol (EURUSD), three disjoint
+contiguous single-month windows (2017-03 / 2017-04 / 2017-05), each folded
+on its own thread owning its own `Kernel` (`Ctx: !Send` makes
+one-ctx-per-thread a compile-time guarantee), each shard's `(acc,n)`
+asserted **bit-exact** vs a single-thread host fold of that exact window in
+data-server stream order. Adds the windowed adapter sibling `fold_window`
+(purely additive after `fold_symbol`), a new single-responsibility
+`[[bin]] timeshard_runner`, and `tests/timeshard.rs` carrying the spec §3
+strength-ordered (a)/(b)/(c) assertions plus the deterministic leak-Σ and a
+journal-only friction-timing capture (no bench gate, no timing assertion).
+The shipped m5.2 symbol-fan path (`swarm_runner.rs`, `swarm.rs`) and m5.1
+core stay byte-untouched and green. Invariant 1 holds (the AILang workspace
+graph has zero `data-server`; `chrono` enters only as an `ail-embed`
+dev-dep). Determinism re-verified 6× (5× standalone + 1× in full suite),
+all identical. Friction-harvest observed:
+`TIMING total_ticks=3192562 elapsed_ms=658` → derived per-tick-FFI cost
+≈ 658 ms / 3,192,562 ≈ **~206 ns/tick** (the Boss records this as the P2
+flat-array-decision input in the close-out journal).
+
+## Per-task notes
+
+- iter embedding-abi-m5.3.1: windowed adapter + chrono dev-dep + bin
+  declaration — `fold_window(server, symbol, from_ms, to_ms)` appended to
+  `ail-embed/src/adapter.rs` immediately after `fold_symbol`, before the
+  `#[cfg(test)] mod tests` block (windowed sibling, same
+  `MidPriceStream`+`Kernel` path, differs only by `stream_tick_windowed`);
+  `chrono = "0.4"` added to `[dev-dependencies]` + `[[bin]] timeshard_runner`
+  declared in `ail-embed/Cargo.toml`. `--lib` regression check
+  `2 passed; 0 failed` (exactly the plan's prediction). Spec compliant,
+  quality approved, 0 re-loops.
+- iter embedding-abi-m5.3.2: the time-shard E2E test (RED) —
+  `ail-embed/tests/timeshard.rs` created verbatim from the plan
+  (boundary-invisibility (a)/(b)/(c) + leak-Σ + friction capture).
+  RED observed: the declared-but-absent `[[bin]] timeshard_runner` source
+  makes the test binary fail to build. Spec compliant, quality approved,
+  0 re-loops. DONE_WITH_CONCERNS — see Concerns (RED-text discrepancy,
+  same cause).
+- iter embedding-abi-m5.3.3: the `timeshard_runner` bin (GREEN) —
+  `ail-embed/src/bin/timeshard_runner.rs` created verbatim (one thread per
+  window, each building its own `Kernel`; prints `RESULT`/`TIMING`).
+  Step 2 GREEN (`1 passed; 0 failed`, ran 2.71s against real /mnt data —
+  did NOT skip). Step 3 determinism ×5: 5/5 `ok`. Step 4 full suite:
+  0 failed AND 0 ignored across all binaries (m5.2 `symbol_fan_swarm_*`
+  un-ignored & green). Step 5: `cargo metadata` data-server count = 0;
+  path filter empty. Step 6: `embed_rc_global_stats_race`
+  `1 passed; 0 failed`. Spec compliant, quality approved, 0 re-loops.
+
+## Concerns
+
+- iter embedding-abi-m5.3.2 (DONE_WITH_CONCERNS): the Task 2 Step 2 RED
+  surfaced as `error: couldn't read \`src/bin/timeshard_runner.rs\`: No such
+  file or directory` rather than the plan's predicted `error: environment
+  variable \`CARGO_BIN_EXE_timeshard_runner\` not defined at compile time`.
+  Same root cause (the declared `[[bin]] timeshard_runner` target has no
+  source until Task 3), same trigger, same resolution (disappears exactly
+  when Task 3 lands the bin) — Cargo fails at the missing-source check
+  before reaching the `env!` expansion, so the surface error differs but
+  the RED is genuine, deterministic, and not a false-green. Observation
+  only; the plan's intent (a deterministic compile-failure RED isolating
+  Task 2 from Task 3) is fully satisfied.
+
+## Known debt
+
+- (none introduced by this iter)
+- Pre-existing `docs/DESIGN.md:2358-2360` "additive M4 concern" drift is
+  explicitly out of m5.3 scope (spec §"Out of scope" / Boss decision) and
+  was deliberately not touched — the milestone-close audit handles it,
+  not this iter.
+
+## Blocked detail
+
+(not applicable — Status DONE)
+
+## Files touched
+
+- `ail-embed/src/adapter.rs` (modified — additive `fold_window`)
+- `ail-embed/Cargo.toml` (modified — `chrono` dev-dep + `timeshard_runner` bin)
+- `ail-embed/Cargo.lock` (modified — single line: `chrono` direct-dep edge;
+  automatic, strictly required by the Cargo.toml dep add, already resolved
+  transitively via data-server)
+- `ail-embed/src/bin/timeshard_runner.rs` (created)
+- `ail-embed/tests/timeshard.rs` (created)
+
+All within the plan's declared file set and the Step 5b allowed path scope
+(`ail-embed/**`, `docs/`, `bench/orchestrator-stats/`). Zero diff to
+`crates/ailang-*`, `crates/ail/`, `runtime/`, `examples/*.ail`, root
+`Cargo.toml`, `docs/DESIGN.md`.
+
+## Stats
+
+bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.3.json

docs/journals/INDEX.md

@@ -111,3 +111,4 @@
 - 2026-05-19 — iter bugfix-rc-global-stats-race (RED→GREEN, debug→implement mini, DONE 1/1): the M5 iter-2 bounce-back resolution (user-approved Option A). The two null-ctx fallback RC-stats counters `g_rc_alloc_count`/`g_rc_free_count` (`runtime/rc.c:90-91`) were plain `static uint64_t` with a non-atomic `++` on the `__ail_tls_ctx == NULL` arm of `ailang_rc_alloc` (rc.c:161) / the to-zero branch of `ailang_rc_dec` (rc.c:212); a multi-threaded host driving the global fallback (no `ailang_ctx_new`) raced the read-modify-write and lost increments, so the `AILANG_RC_STATS` atexit Σ under-counted non-deterministically — exactly the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c's own header (rc.c:44-49) names, M5 being AILang's first concurrent consumer. NOT a memory bug (`Ctx: !Send` keeps every box on one thread; the real refcount/free is correct; programs bit-exact). RED (debugger, audit-trail commit `427b687`): a C host — 8 threads × 2M alloc-then-dec, no ctx so the global path is taken, no box crossing a thread — + integration test asserting the atexit Σ is exact; Boss-verified RED `allocs=2141382 expected 16000000, live=-131242` (deterministic-fail under that contention, not flaky). GREEN (implement mini, this commit): the two globals are now `_Atomic uint64_t`, `atomic_fetch_add_explicit(.., memory_order_relaxed)` at the two fallback `++` sites, `atomic_load_explicit(.., relaxed)` in `ailang_rc_stats_atexit` (its sole reader) — relaxed is correct (pure stats, no happens-before; atexit reader runs post-join). Per-ctx counters + the per-object refcount header left non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross threads) — explicitly out of scope, untouched; frozen value layout / ABI offsets / host-free rule untouched; the genuinely-still-single-threaded drop-worklist doc note correctly left unchanged (a false correction was refused); the two genuinely-stale atomicity doc blocks corrected for honesty. Scope held to `runtime/rc.c` ONLY (37+/14−). Boss-verified independently: RED→GREEN deterministically 3/3 (jitter gone, `allocs==frees==16_000_000`), full `cargo test -p ail` green (every binary 0 failed — the per-ctx tsan harnesses embed_swarm_tsan/embed_rc_accounting_tsan + embed_tick_e2e/embed_record_e2e unaffected since the per-ctx path is untouched; rc.c links into every ail binary so this is the strong regression gate), git-status scope = rc.c + journal + stats only, the RED files unchanged. Unblocks resuming the M5 leak-proof (un-`#[ignore]` `ail-embed`'s `symbol_fan_swarm_leak_free`). RED `427b687` → GREEN this commit. → 2026-05-19-iter-bugfix-rc-global-stats-race.md
 - 2026-05-19 — iter embedding-abi-m5.2-resume-attempt (RE-QUARANTINED; honesty fix only): attempted to un-`#[ignore]` the M5 swarm leak-proof `symbol_fan_swarm_leak_free` as the integration-level acceptance of the runtime fix `7bfa11e` (bugfix-rc-global-stats-race). Boss verification FALSIFIED the sufficiency premise: across 4–6 swarm runs the leak Σ is still non-deterministic — `Σfrees` stable-exact (12000003) every run, `Σallocs` short by a jittering ~1600–2260 in ~1/3 of runs (e.g. left:11998383 right:12000003 across 4 stat lines). The atomic-global fix is real, committed, NECESSARY (the isolated 8×2M pure-global RED `embed_rc_global_stats_race` is deterministically green) but NOT sufficient — a second, distinct alloc-side undercount remains that the isolated RED did not model (frees never lose ⇒ not a per-ctx race; not pure global contention ⇒ RED green; localisation beyond that is where Boss speculation stops per the debug Iron Law). Un-ignore reverted (never committed); the only shipped diff is the three now-stale `#[ignore]`-rationale breadcrumbs in `ail-embed/tests/swarm.rs` corrected from "un-ignore when the runtime is atomic" to the accurate necessary-but-insufficient state (a future agent seeing `7bfa11e` must NOT un-ignore on that basis — doc-honesty). Test body + the `#[ignore]` itself unchanged (still quarantined, NOT weakened); `symbol_fan_swarm_bit_exact` stays live+GREEN (the swarm IS actually correct and leak-free — only the accounting is wrong). Residual handed to a fresh debug RED-first cycle; two framings flagged for it (genuine residual runtime defect vs Σ-over-heterogeneous-global+per-ctx-lines methodology unsoundness — the latter, if so, is a post-root-cause design decision). main green at the honesty-fix commit; M5 stays open `[~]`; m5.3 remains downstream of a sound leak-proof. → 2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md
 - 2026-05-19 — iter bugfix-swarm-rc-alloc-undercount (RED→GREEN, debug→implement mini, DONE 1/1): root-caused and fixed the m5.2-resume-attempt residual. The M5 swarm leak-proof was still non-deterministic after the atomic-counter runtime fix `7bfa11e` — NOT a second runtime bug and NOT test-methodology unsoundness, but a **build-dependency staleness gap**: `ail-embed/build.rs` declared `cargo:rerun-if-changed` only for the kernel `.ail` + `build.rs`, NOT the `runtime/` C sources `ail build --emit=staticlib` compiles into `libailang_rt.a`. So Cargo never re-ran `build.rs` after `7bfa11e`; `ail-embed` kept linking a stale pre-`7bfa11e` non-atomic `libailang_rt.a` whose `g_rc_alloc_count++` raced the swarm's concurrent TLS-NULL host allocs (explaining the frees-stable/allocs-jitter asymmetry, and why the isolated `crates/ail` RED — which rebuilds the staticlib fresh per run — was green while the `ail-embed` swarm was not). `runtime/rc.c` at HEAD was always correct; the entire production fix is one directory-level `cargo:rerun-if-changed=<repo>/runtime` line (preferred over a per-file list — the implementer inspected `crates/ail` `build_staticlib` confirming `libailang_rt.a = ar(rc.o,str.o)`, both under `runtime/`; directory tracking is future-proof against any new runtime source). RED (debugger, audit-trail `483117d`): `ail-embed/tests/rt_archive_freshness.rs` pins the *cause* deterministically (objdump the linked archive for `lock`-prefixed atomic increments + a fresh-build cross-check) since the *symptom* is non-deterministic by construction; Boss-verified RED = 0 lock insns ("STALE"). GREEN (implement mini, this commit): the build.rs fix + the cohesive consequence — `symbol_fan_swarm_leak_free` un-`#[ignore]`d (body byte-for-byte verbatim — it was always quarantined, never weakened; the three doc breadcrumbs rewritten to the resolved build-dep-staleness rationale), now the integration-level acceptance of both `7bfa11e` and this fix. Boss removed a pre-existing dead `DataFormat` import in swarm.rs inline (trivial; file committed this iter regardless). Boss-verified independently: RED→GREEN; **swarm determinism 5/5 — `symbol_fan_swarm_leak_free` + `symbol_fan_swarm_bit_exact` GREEN every run, 0 ignored, jitter gone, Σallocs==Σfrees==12000003**; isolated `embed_rc_global_stats_race` still GREEN (untouched); Invariant 1 data-server count 0; scope = `ail-embed/build.rs` + `ail-embed/tests/swarm.rs` + journal + stats only. The M5 swarm leak-proof bounce-back is now fully resolved end-to-end (bounce-back → Option A `7bfa11e` → resume-attempt "necessary-but-insufficient" → this build-dep root-cause+fix). RED `483117d` → GREEN this commit. M5 stays open `[~]`; m5.3 (time-shard + friction-harvest + close-out) remains. → 2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md
+- 2026-05-19 — iter embedding-abi-m5.3 (DONE 3/3): final functional M5 iteration — the time-shard boundary-invisibility proof + friction harvest. One symbol (EURUSD), three disjoint contiguous single-month windows (2017-03/04/05), each folded on its own thread owning its own `Kernel` (`Ctx: !Send` ⇒ one-ctx-per-thread compile-time-enforced). Adds the purely-additive windowed adapter sibling `fold_window` (same `MidPriceStream`+`Kernel` path as `fold_symbol`, differs only by `stream_tick_windowed`), a new single-responsibility `[[bin]] timeshard_runner`, `chrono` as a **dev-dep** (the time-shard test derives `(y,m)→Unix-ms` bounds via data-server's own `NaiveDate…timestamp_millis()` precedent and passes raw ms to the bin, keeping the bin date-math-free and Invariant 1 untouched), and `tests/timeshard.rs` carrying the spec §3 strength-ordered assertions: (a) each shard's `(acc,n)` **bit-exact** vs a single-thread host fold of THAT EXACT window in data-server stream order — the first actual proof of the chunk/window-boundary-invisibility claim the entire M4 retirement rests on; (b) host-summed partials bit-exact by construction; (c) the whole-window [2017-03..2017-05] single-thread total within `REL_TOL=1e-6` only (cross-shard f64 re-association is host arithmetic, NOT a kernel property — asserting it bit-exact would be the bug; tolerance derived from the recursive-summation error bound with ~4 orders of safety, shown in the test). Plus the now-deterministic global leak-Σ (post-`dbd76e5`) and a journal-only friction-timing capture (no bench gate, no timing assertion — flake-free). RED was a genuine deterministic compile failure (declared-but-absent `[[bin]] timeshard_runner` source; surfaced as Cargo's missing-source error rather than the plan's predicted `env!` error — same root cause, same trigger, no false-green; recorded as a benign plan-text imprecision in Concerns). The shipped m5.2 symbol-fan path (`swarm_runner.rs`, `swarm.rs`) and m5.1 core stay byte-untouched and green. Boss-verified independently: **timeshard determinism 5/5** (consistent ~2.7s, no jitter); full ail-embed suite **0 failed AND 0 ignored** across all binaries (lib 2, swarm 2 — the m5.2 leak-proof stays un-ignored & green, the whole saga's resolution intact —, smoke 1, rt_archive 1, timeshard 1); isolated `embed_rc_global_stats_race` still green; AILang workspace `cargo metadata` data-server count 0; compiler-surface diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail, root Cargo.toml, DESIGN.md). **Friction-harvest deliverable (P2 flat-array-decision input): host-per-tick-FFI ≈ 658 ms / 3 192 562 ticks ≈ ~206 ns/tick at real EURUSD volume.** M5's functional work is complete; the milestone closes next via the mandatory `audit` (which also handles the pre-existing `docs/DESIGN.md:2358-2360` "additive M4 concern" drift, explicitly out of m5.3 scope). → 2026-05-19-iter-embedding-abi-m5.3.md