audit embedding-abi-m5: milestone close — RATIFY gc_rss trio (evidenced) + DRIFT (one doc-honesty tidy); P2 bench noise causally exonerated, baseline pristine for it

Architect: Invariant 1 PASS (zero data-server in any compiler
crate/runtime; ail-embed its own workspace root); M3 frozen-layout
SSOT unmoved (the lone rc.c change 7bfa11e touched only the global
stats counters); rc.c header-comment honesty accurate. Drift =
[medium] DESIGN.md:2358-2360 stale "additive M4 concern" (PRE-
EXISTING — M4 retired; correctly scoped out of every M5 iter) +
[medium] DESIGN.md §"Embedding ABI" silent-drift (it asserts "no
shared mutable runtime state … data-race-free" but the now-atomic
global RC-stats fallback IS shared mutable state under concurrency;
M5 is AILang's first concurrent consumer) + [low] rc.c:88 "two
unconditional ++" imprecise → one doc-honesty tidy iteration
(embedding-abi-m5.tidy), same honesty axis, not split.

Bencher refuted my pre-audit "gc_rss allocator-orthogonal →
environmental" hypothesis WITH EVIDENCE: the 3x *.gc_rss_kb +32%
are genuine, deterministic, M5-attributable — 7bfa11e's atomic
split a branch in ailang_rc_alloc (reached by GC programs via
str.c int->str at print), spilling %r14/%rbx; a stale interior
list pointer in the new spill slot is conservatively retained by
Boehm, pinning a ~32 MB dead spine (collection #12 freed PRE 32 MB
vs HEAD 32 B); bisects exactly on the commit, IR byte-identical,
ail sha256-identical; RC/bump RSS dead-stable. bump_s + max_us are
pre-existing tracked-P2 noise (bump RC-orthogonal; max_us 1-sample
tail-jitter, PRE often worse, +65->+104% swing between runs) — NOT
M5.

RATIFY the gc_rss trio. Language reason (not effort): Decision 9 —
Boehm is explicitly transitional; AILang's committed model is RC +
uniqueness. The measurable cost of the correct, necessary,
user-approved swarm-safety fix is exclusively a Boehm
conservative-stack-scan false-retention artifact on the
transitional allocator (codegen-layout-fragile; no memory actually
leaked; RC/bump RSS dead-stable). Chasing it with an out-of-scope
codegen change invests in the path being migrated away from; the
committed RC path is unaffected. Selective baseline re-anchor:
bench/baseline.json lines 14/44/64 ONLY -> 137448/137768/137636
(bencher-confirmed steady-state); every other metric byte-pristine
(a wholesale --update-baseline would absorb the P2 noise and break
the established-envelope discipline). Re-run confirms the 3 now ok
at +-0.3%.

bump_s x2 + max_us x2: NO-ratify, causally exonerated, baseline
left pristine — identical disposition to the M2/M3/prelude-decouple
milestone closes.

Friction-harvest recorded (P2 flat-array-decision input):
host-per-tick-FFI ~= ~206 ns/tick at real EURUSD volume.

Ratify discipline: updated baseline JSON + this paired JOURNAL
ratify entry committed together (bench/baseline.json + the audit
journal + INDEX line). M5 closes after the doc-honesty tidy lands
and is Boss-verified.

docs/journals/INDEX.md

@@ -112,3 +112,4 @@
 - 2026-05-19 — iter embedding-abi-m5.2-resume-attempt (RE-QUARANTINED; honesty fix only): attempted to un-`#[ignore]` the M5 swarm leak-proof `symbol_fan_swarm_leak_free` as the integration-level acceptance of the runtime fix `7bfa11e` (bugfix-rc-global-stats-race). Boss verification FALSIFIED the sufficiency premise: across 4–6 swarm runs the leak Σ is still non-deterministic — `Σfrees` stable-exact (12000003) every run, `Σallocs` short by a jittering ~1600–2260 in ~1/3 of runs (e.g. left:11998383 right:12000003 across 4 stat lines). The atomic-global fix is real, committed, NECESSARY (the isolated 8×2M pure-global RED `embed_rc_global_stats_race` is deterministically green) but NOT sufficient — a second, distinct alloc-side undercount remains that the isolated RED did not model (frees never lose ⇒ not a per-ctx race; not pure global contention ⇒ RED green; localisation beyond that is where Boss speculation stops per the debug Iron Law). Un-ignore reverted (never committed); the only shipped diff is the three now-stale `#[ignore]`-rationale breadcrumbs in `ail-embed/tests/swarm.rs` corrected from "un-ignore when the runtime is atomic" to the accurate necessary-but-insufficient state (a future agent seeing `7bfa11e` must NOT un-ignore on that basis — doc-honesty). Test body + the `#[ignore]` itself unchanged (still quarantined, NOT weakened); `symbol_fan_swarm_bit_exact` stays live+GREEN (the swarm IS actually correct and leak-free — only the accounting is wrong). Residual handed to a fresh debug RED-first cycle; two framings flagged for it (genuine residual runtime defect vs Σ-over-heterogeneous-global+per-ctx-lines methodology unsoundness — the latter, if so, is a post-root-cause design decision). main green at the honesty-fix commit; M5 stays open `[~]`; m5.3 remains downstream of a sound leak-proof. → 2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md
 - 2026-05-19 — iter bugfix-swarm-rc-alloc-undercount (RED→GREEN, debug→implement mini, DONE 1/1): root-caused and fixed the m5.2-resume-attempt residual. The M5 swarm leak-proof was still non-deterministic after the atomic-counter runtime fix `7bfa11e` — NOT a second runtime bug and NOT test-methodology unsoundness, but a **build-dependency staleness gap**: `ail-embed/build.rs` declared `cargo:rerun-if-changed` only for the kernel `.ail` + `build.rs`, NOT the `runtime/` C sources `ail build --emit=staticlib` compiles into `libailang_rt.a`. So Cargo never re-ran `build.rs` after `7bfa11e`; `ail-embed` kept linking a stale pre-`7bfa11e` non-atomic `libailang_rt.a` whose `g_rc_alloc_count++` raced the swarm's concurrent TLS-NULL host allocs (explaining the frees-stable/allocs-jitter asymmetry, and why the isolated `crates/ail` RED — which rebuilds the staticlib fresh per run — was green while the `ail-embed` swarm was not). `runtime/rc.c` at HEAD was always correct; the entire production fix is one directory-level `cargo:rerun-if-changed=<repo>/runtime` line (preferred over a per-file list — the implementer inspected `crates/ail` `build_staticlib` confirming `libailang_rt.a = ar(rc.o,str.o)`, both under `runtime/`; directory tracking is future-proof against any new runtime source). RED (debugger, audit-trail `483117d`): `ail-embed/tests/rt_archive_freshness.rs` pins the *cause* deterministically (objdump the linked archive for `lock`-prefixed atomic increments + a fresh-build cross-check) since the *symptom* is non-deterministic by construction; Boss-verified RED = 0 lock insns ("STALE"). GREEN (implement mini, this commit): the build.rs fix + the cohesive consequence — `symbol_fan_swarm_leak_free` un-`#[ignore]`d (body byte-for-byte verbatim — it was always quarantined, never weakened; the three doc breadcrumbs rewritten to the resolved build-dep-staleness rationale), now the integration-level acceptance of both `7bfa11e` and this fix. Boss removed a pre-existing dead `DataFormat` import in swarm.rs inline (trivial; file committed this iter regardless). Boss-verified independently: RED→GREEN; **swarm determinism 5/5 — `symbol_fan_swarm_leak_free` + `symbol_fan_swarm_bit_exact` GREEN every run, 0 ignored, jitter gone, Σallocs==Σfrees==12000003**; isolated `embed_rc_global_stats_race` still GREEN (untouched); Invariant 1 data-server count 0; scope = `ail-embed/build.rs` + `ail-embed/tests/swarm.rs` + journal + stats only. The M5 swarm leak-proof bounce-back is now fully resolved end-to-end (bounce-back → Option A `7bfa11e` → resume-attempt "necessary-but-insufficient" → this build-dep root-cause+fix). RED `483117d` → GREEN this commit. M5 stays open `[~]`; m5.3 (time-shard + friction-harvest + close-out) remains. → 2026-05-19-iter-bugfix-swarm-rc-alloc-undercount.md
 - 2026-05-19 — iter embedding-abi-m5.3 (DONE 3/3): final functional M5 iteration — the time-shard boundary-invisibility proof + friction harvest. One symbol (EURUSD), three disjoint contiguous single-month windows (2017-03/04/05), each folded on its own thread owning its own `Kernel` (`Ctx: !Send` ⇒ one-ctx-per-thread compile-time-enforced). Adds the purely-additive windowed adapter sibling `fold_window` (same `MidPriceStream`+`Kernel` path as `fold_symbol`, differs only by `stream_tick_windowed`), a new single-responsibility `[[bin]] timeshard_runner`, `chrono` as a **dev-dep** (the time-shard test derives `(y,m)→Unix-ms` bounds via data-server's own `NaiveDate…timestamp_millis()` precedent and passes raw ms to the bin, keeping the bin date-math-free and Invariant 1 untouched), and `tests/timeshard.rs` carrying the spec §3 strength-ordered assertions: (a) each shard's `(acc,n)` **bit-exact** vs a single-thread host fold of THAT EXACT window in data-server stream order — the first actual proof of the chunk/window-boundary-invisibility claim the entire M4 retirement rests on; (b) host-summed partials bit-exact by construction; (c) the whole-window [2017-03..2017-05] single-thread total within `REL_TOL=1e-6` only (cross-shard f64 re-association is host arithmetic, NOT a kernel property — asserting it bit-exact would be the bug; tolerance derived from the recursive-summation error bound with ~4 orders of safety, shown in the test). Plus the now-deterministic global leak-Σ (post-`dbd76e5`) and a journal-only friction-timing capture (no bench gate, no timing assertion — flake-free). RED was a genuine deterministic compile failure (declared-but-absent `[[bin]] timeshard_runner` source; surfaced as Cargo's missing-source error rather than the plan's predicted `env!` error — same root cause, same trigger, no false-green; recorded as a benign plan-text imprecision in Concerns). The shipped m5.2 symbol-fan path (`swarm_runner.rs`, `swarm.rs`) and m5.1 core stay byte-untouched and green. Boss-verified independently: **timeshard determinism 5/5** (consistent ~2.7s, no jitter); full ail-embed suite **0 failed AND 0 ignored** across all binaries (lib 2, swarm 2 — the m5.2 leak-proof stays un-ignored & green, the whole saga's resolution intact —, smoke 1, rt_archive 1, timeshard 1); isolated `embed_rc_global_stats_race` still green; AILang workspace `cargo metadata` data-server count 0; compiler-surface diff empty (zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail, root Cargo.toml, DESIGN.md). **Friction-harvest deliverable (P2 flat-array-decision input): host-per-tick-FFI ≈ 658 ms / 3 192 562 ticks ≈ ~206 ns/tick at real EURUSD volume.** M5's functional work is complete; the milestone closes next via the mandatory `audit` (which also handles the pre-existing `docs/DESIGN.md:2358-2360` "additive M4 concern" drift, explicitly out of m5.3 scope). → 2026-05-19-iter-embedding-abi-m5.3.md
+- 2026-05-19 — audit embedding-abi-m5 (milestone close): DRIFT (one doc-honesty tidy) + RATIFY (gc_rss trio, evidenced) + causally-exonerated pre-existing P2 bench noise. Architect: **Invariant 1 PASS** (zero data-server in any compiler crate/runtime; ail-embed its own workspace root; only "finance" hits are the M3 ABI export *symbol name* — ABI not data-server knowledge), M3 frozen-layout SSOT unmoved (the lone rc.c change `7bfa11e` touched only the global stats counters), rc.c header-comment honesty accurate; drift = `[medium]` DESIGN.md:2358-2360 stale "additive M4 concern" (PRE-EXISTING, M4 retired, correctly scoped out of M5) + `[medium]` DESIGN.md §"Embedding ABI" silent-drift (it asserts "no shared mutable runtime state … data-race-free" but the now-atomic global RC-stats fallback IS shared mutable state under concurrency — M5 is AILang's first concurrent consumer) + `[low]` rc.c:88 "two unconditional ++" imprecise — one tidy, same honesty axis, not split. Bencher (my pre-audit "gc_rss is allocator-orthogonal → environmental" hypothesis was **refuted by evidence** — the verification discipline working): the 3× `*.gc_rss_kb +32%` are GENUINE, DETERMINISTIC, M5-attributable — `7bfa11e`'s atomic split a branch in `ailang_rc_alloc` (reached by GC programs via `str.c` int→str at `print`), spilling `%r14/%rbx`; a stale interior list pointer in the new spill slot is conservatively retained by Boehm, pinning a ~32 MB dead spine (Boehm collection #12 freed PRE 32 MB vs HEAD 32 B); bisects exactly on the commit, IR byte-identical, `ail` sha256-identical; RC/bump RSS dead-stable. `bump_s`/`max_us` = pre-existing tracked-P2 (bump RC-orthogonal; max_us 1-sample tail-jitter, PRE often worse, +65→+104% swing between runs minutes apart — dispositively noise), NOT M5. Adjudication: **RATIFY** the gc_rss trio — language reason (Decision 9: Boehm is explicitly transitional; the cost is purely a Boehm conservative-scan false-retention artifact on the transitional allocator of a *correct, necessary, user-approved swarm-safety fix*, codegen-layout-fragile, RC path the committed model and unaffected; chasing it with an out-of-scope codegen change invests in the path being removed) — selective baseline re-anchor `bench/baseline.json` lines 14/44/64 ONLY (137448/137768/137636, bencher-confirmed steady-state; every other metric byte-pristine — a wholesale --update-baseline would have absorbed the P2 noise and broken the established-envelope discipline); re-run confirms the 3 now `ok` at ±0.3%. **NO-ratify / causally exonerated** for bump_s×2 + max_us×2 — baseline pristine, identical disposition to the M2/M3/prelude-decouple closes. Friction-harvest recorded (P2 input): host-per-tick-FFI ≈ ~206 ns/tick at real EURUSD volume. M5 closes after the doc-honesty tidy lands + Boss-verified. Ratify discipline honored: updated baseline JSON + this paired JOURNAL ratify entry committed together. → 2026-05-19-audit-embedding-abi-m5.md