ail-embed: re-quarantine symbol_fan_swarm_leak_free — atomic-global fix necessary but insufficient (doc-honesty + finding record)

Attempted to un-#[ignore] the M5 swarm leak-proof as the
integration-level acceptance of bugfix-rc-global-stats-race
(7bfa11e). Boss verification falsified the sufficiency premise:
across 4-6 swarm runs the leak Σ is STILL non-deterministic —
Σfrees stable-exact (12000003) every run, Σallocs short by a
jittering ~1600-2260 in ~1/3 of runs (e.g. left:11998383
right:12000003, 4 stat lines).

The atomic-global runtime fix (7bfa11e) is real, committed, and
NECESSARY — the isolated 8x2M pure-global RED
embed_rc_global_stats_race is deterministically green. It is NOT
sufficient for the swarm: a second, distinct alloc-side undercount
remains that the isolated RED did not model (Σfrees never loses =>
not a per-ctx race; not pure global contention => RED green).

The un-#[ignore] was reverted (never committed). The ONLY shipped
diff is the three now-stale #[ignore]-rationale breadcrumbs in
ail-embed/tests/swarm.rs corrected from "un-ignore when the runtime
makes the counters atomic" to the accurate necessary-but-insufficient
state — a future agent seeing 7bfa11e must NOT un-ignore on that
basis (doc-honesty rule). The test body and the #[ignore] attribute
itself are unchanged: still quarantined, NOT weakened.
symbol_fan_swarm_bit_exact stays live and GREEN — the swarm IS
actually correct and leak-free; only the accounting is wrong.

Residual handed to a fresh debug RED-first cycle. Two framings
recorded for it (genuine residual runtime defect vs Σ-over-
heterogeneous-lines test-methodology unsoundness; the latter, if so,
is a post-root-cause design decision). main stays green; M5 open
[~]; m5.3 remains downstream of a sound leak-proof.

Includes the resume-attempt journal + INDEX line.
This commit is contained in:
2026-05-19 02:06:43 +02:00
parent 7bfa11e838
commit 6500ca0493
3 changed files with 131 additions and 33 deletions
+39 -33
View File
@@ -11,23 +11,25 @@
//! host reference fold. This is GREEN and live: it proves the
//! real-data-server → adapter → M3-kernel swarm computes correctly.
//!
//! - `symbol_fan_swarm_leak_free` — `#[ignore]`d pending an escalated
//! design decision. The global `Σallocs==Σfrees` measurement
//! (ported from crates/ail/tests/embed_tick_e2e.rs:94-104) is
//! *invalid under a multi-threaded host*: host-side box allocs run
//! with `__ail_tls_ctx == NULL` and fall through to the
//! **non-atomic** global `g_rc_*` counters (runtime/rc.c:90-91,
//! 161,212), which rc.c's own header explicitly documents as
//! single-threaded/non-atomic ("when it acquires [concurrency],
//! atomic-vs-non-atomic becomes a separate decision"). The swarm is
//! *actually* leak-free (no box crosses a thread — `Ctx: !Send`;
//! bit-exact is GREEN every run); only the global *counter* races,
//! losing increments. The assertion body is preserved verbatim and
//! will be re-armed (un-`#[ignore]`d) by the runtime iteration that
//! makes the global fallback counters atomic. See
//! docs/journals/2026-05-19-iter-embedding-abi-m5.2.md and the M5
//! bounce-back. Quarantined, NOT weakened — what it asserts is
//! unchanged; it is gated on a decision the Boss escalated.
//! - `symbol_fan_swarm_leak_free` — STILL `#[ignore]`d. The global
//! `Σallocs==Σfrees` measurement (ported from
//! crates/ail/tests/embed_tick_e2e.rs:94-104) was non-deterministic
//! under the multi-threaded host. The runtime fix
//! `bugfix-rc-global-stats-race` (commit `7bfa11e`) made the global
//! `g_rc_*` fallback counters `_Atomic` — that fix is real,
//! committed, and NECESSARY, but it is **not sufficient**: a Boss
//! verification (4 swarm runs) showed the leak Σ is still
//! non-deterministic — `Σfrees` is stable-exact (12000003) every
//! run while `Σallocs` is short by a jittering ~16002260 in ~1/3
//! of runs (e.g. left:11998383 right:12000003 across the 4 stat
//! lines). A SECOND, distinct alloc-undercount remains that the
//! isolated 8×2M pure-global RED did not model. Un-`#[ignore]` is
//! now gated on that residual's own root-cause/debug cycle, NOT on
//! `7bfa11e`. The swarm is still *actually* leak-free (no box
//! crosses a thread — `Ctx: !Send`; `symbol_fan_swarm_bit_exact`
//! GREEN every run); only the alloc *accounting* is still wrong.
//! Body preserved verbatim — quarantined, NOT weakened. See
//! docs/journals/2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md.
use std::process::Command;
use std::sync::Arc;
@@ -123,23 +125,27 @@ fn symbol_fan_swarm_bit_exact() {
/// embed_tick_e2e.rs:94-104). The invariant is the global Σ, not
/// per-line balance (M2 TLS-ctx cross-attribution).
///
/// `#[ignore]`d: this measurement is invalid under a multi-threaded
/// host. Host-side box allocs run with `__ail_tls_ctx == NULL` and
/// hit the non-atomic global `g_rc_*` counters (runtime/rc.c:90-91,
/// 161,212) — single-threaded/non-atomic *by rc.c's own documented
/// design*. 3 worker threads race the counter, losing increments;
/// the *counter* is unreliable even though the swarm is actually
/// leak-free (no box crosses a thread; `symbol_fan_swarm_bit_exact`
/// is GREEN every run). Un-`#[ignore]` this — body unchanged — once
/// the runtime iteration makes the global fallback counters atomic
/// (the deferred "concurrency arrived" decision rc.c:44-49 names).
/// See docs/journals/2026-05-19-iter-embedding-abi-m5.2.md.
/// STILL `#[ignore]`d. The runtime fix `bugfix-rc-global-stats-race`
/// (commit `7bfa11e`) made the global `g_rc_*` fallback counters
/// `_Atomic` — necessary and committed, but Boss verification proved
/// it INSUFFICIENT for this swarm: `Σfrees` is stable-exact
/// (12000003) every run, `Σallocs` short by a jittering ~16002260
/// in ~1/3 of runs (e.g. left:11998383 right:12000003, 4 stat
/// lines). A second, distinct alloc-undercount remains — NOT pure
/// global-counter contention (the isolated 8×2M pure-global RED
/// `embed_rc_global_stats_race` is deterministically green). The
/// swarm is still actually leak-free (no box crosses a thread —
/// `Ctx: !Send`; bit-exact GREEN every run); only the alloc
/// accounting is wrong. Un-`#[ignore]` is gated on the residual's
/// own root-cause/debug cycle, not on `7bfa11e`. Body unchanged.
/// See docs/journals/2026-05-19-iter-embedding-abi-m5.2-resume-attempt.md.
#[test]
#[ignore = "blocked: global g_rc_* counter is non-atomic by rc.c design; \
invalid under the multi-thread host. Swarm is actually \
leak-free (bit-exact is green; no box crosses a thread). \
Un-ignore when the runtime makes the global fallback \
counters atomic — see the m5.2 journal + M5 bounce-back."]
#[ignore = "still blocked: atomic-global fix 7bfa11e is necessary but \
INSUFFICIENT — a residual swarm alloc-undercount remains \
(Σfrees exact, Σallocs short ~1600 jittering, ~1/3 runs; \
NOT pure global contention — the isolated RED is green). \
Un-ignore on the residual's own debug cycle, not on \
7bfa11e. See the m5.2-resume-attempt journal."]
fn symbol_fan_swarm_leak_free() {
let Some((_stdout, stderr)) = run_swarm_or_skip() else { return };