fix(rc): GREEN — atomic global g_rc_* stats fallback counters (swarm-safe)

bugfix-rc-global-stats-race, GREEN stage. RED is the separate
audit-trail commit 427b687.

g_rc_alloc_count / g_rc_free_count (runtime/rc.c:90-91) were plain
static uint64_t; the __ail_tls_ctx == NULL fallback at rc.c:161
(alloc) and rc.c:212 (dec-to-zero) did a non-atomic ++. A
multi-threaded host using the global fallback path (no
ailang_ctx_new) raced the read-modify-write and lost increments, so
the AILANG_RC_STATS atexit Σ under-counted non-deterministically.
Not a memory bug — Ctx:!Send keeps every box on one thread; only
the global statistics Σ was wrong (the M5 iter-2 symbol-fan
leak-proof finding, b724cd1; user-approved Option A bounce-back
resolution).

Fix (runtime/rc.c only, 37+/14-): the two globals are now
_Atomic uint64_t; atomic_fetch_add_explicit(.., memory_order_relaxed)
at the two null-ctx fallback ++ sites; atomic_load_explicit(..,
relaxed) snapshot in ailang_rc_stats_atexit (its sole reader).
Relaxed is correct and sufficient — pure statistics, no
happens-before obligation, atexit reader runs after all worker
threads joined. Per-ctx counters and the per-object refcount header
stay non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross
threads) — out of scope, untouched. Frozen value layout / ABI
offsets / host-free rule untouched. The genuinely-still-
single-threaded drop-worklist doc note was correctly left unchanged
(false correction refused); the two stale atomicity doc blocks
corrected for honesty.

Boss-verified independently: RED -> GREEN deterministically 3/3
(jitter gone, allocs==frees==16_000_000); full cargo test -p ail
green (every binary 0 failed — per-ctx tsan harnesses + embed
e2e unaffected; rc.c links into every ail binary, strong regression
gate); scope = runtime/rc.c + journal + stats only; RED files
unchanged. Unblocks resuming the M5 leak-proof.

Includes the per-iter journal, stats, and the INDEX.md line.
This commit is contained in:
2026-05-19 01:57:06 +02:00
parent 427b687b95
commit 7bfa11e838
4 changed files with 123 additions and 14 deletions
@@ -0,0 +1,73 @@
# iter bugfix-rc-global-stats-race — atomic global RC-stats fallback counters
**Date:** 2026-05-19
**Started from:** 427b687b9588ac8e1f96070ffc20f2644e9cd28c
**Status:** DONE
**Tasks completed:** 1 of 1
## Summary
The two null-ctx fallback RC-stats counters `g_rc_alloc_count` /
`g_rc_free_count` (`runtime/rc.c`) were plain `static uint64_t` with a
non-atomic `++` at the `__ail_tls_ctx == NULL` arm of `ailang_rc_alloc`
and the to-zero branch of `ailang_rc_dec`. A multi-threaded host that
uses the global fallback path (no `ailang_ctx_new`) raced the
read-modify-write and lost increments, so the `AILANG_RC_STATS` atexit
Σ under-counted non-deterministically. Not a memory bug — no box
crosses a thread; only the global statistics Σ was wrong. Fix
(user-approved Option A, 2026-05-19): the two global counters are now
`_Atomic uint64_t`, incremented via
`atomic_fetch_add_explicit(.., memory_order_relaxed)` and read via
`atomic_load_explicit(.., memory_order_relaxed)` in the atexit printer
(its only reader). Relaxed is correct and sufficient: pure statistics
with no happens-before obligation, and the atexit reader runs after all
worker threads have joined (single-threaded at exit). The per-ctx
counters and the per-object refcount header stay non-atomic by design
(single-thread-per-ctx; boxes never cross threads) — explicitly out of
scope. Scope held to `runtime/rc.c` only.
## Per-task notes
- bugfix-rc-global-stats-race.1: made `g_rc_alloc_count` /
`g_rc_free_count` `_Atomic uint64_t`; added `<stdatomic.h>`;
`atomic_fetch_add_explicit(.., relaxed)` at the two null-ctx
fallback `++` sites; `atomic_load_explicit(.., relaxed)` snapshot
in `ailang_rc_stats_atexit` (the sole reader). Corrected the two
now-stale doc blocks (file header comment + the 18g.0 stats block)
to state the global counters are atomic while per-ctx + header
stay non-atomic by design. Left the rc.c drop-worklist
"Single-threaded; non-atomic" note unchanged — it refers only to
the per-call drop-worklist, which is genuinely still
single-threaded, so correcting it would be a false correction the
constraint explicitly forbids. RED `embed_rc_global_stats_race`
went GREEN deterministically (3/3 runs, allocs == frees ==
16_000_000); full `cargo test -p ail` green (39/39 result lines
ok, all embed_* incl. the per-ctx tsan harnesses unaffected).
## Concerns
(none)
## Known debt
- `runtime/rc.c` comment near the 18g.0 block ("adds only two
unconditional `++` operations to the hot path") is now slightly
imprecise: the global path is a relaxed atomic add and is
conditional on `_ctx == NULL`. Left verbatim deliberately — it is
a performance aside (not an atomicity claim), it predates and was
already imprecise after the M2 per-ctx split, and the atomicity
statement is now correctly carried by the adjacent rewritten
block; editing it would exceed the named doc-honesty drift the
minimal-fix constraint scoped this iter to.
## Blocked detail
(n/a — DONE)
## Files touched
- runtime/rc.c
## Stats
bench/orchestrator-stats/2026-05-19-iter-bugfix-rc-global-stats-race.json
+1
View File
@@ -108,3 +108,4 @@
- 2026-05-18 — iter bugfix-over-strict-mode-ctor-rebuild-consume (RED→GREEN, debug→implement mini, DONE 1/1): fixed a conservative `[over-strict-mode]` false-positive surfaced by the Tick-coverage fixtures. The lint's consume-detection (`any_sub_binder_consumed_for`/`pattern_has_consumed_heap_binder`, `crates/ailang-check/src/linearity.rs`) only recognised a consume of an `(own (con T))` param when a *heap-typed* pattern-binder was moved out of `match p`; when `p` was destructured into purely *primitive* fields fed into a `Term::Ctor` rebuilding `p`'s own ctor, that genuine dismantle+rebuild consume was invisible, so the lint spuriously advised `(borrow ...)`. Real harm: an LLM author "fixing" the spurious warning by flipping an export's declared mode `own``borrow` would silently invert the ABI ownership contract. RED-first: debugger disproved the carrier's initial nested-`match` hypothesis (the M3 `embed_backtest_step_record.ail` is silent only because its implicit-mode scalar `Float` param disables the lint via the activation gate, linearity.rs:327 — NOT because it handles the rebuild; the defect reproduces single-param, no nesting), wrote the synthetic RED unit `over_strict_mode_silent_when_ctor_rebuilt_from_primitive_fields` committed as its own audit-trail commit `a11cb7c`. GREEN (implement mini): added a 2nd recognition path to `any_sub_binder_consumed_for` — a `match p` arm that destructures binders out of `p`'s ctor and references any of them (primitive or not) inside a `Term::Ctor`'s args in the arm body genuinely consumes `p`; two pure helpers (`ctor_uses_any_binder` + deep `term_mentions_any_binder`, so `(+ acc px)`-mediated flow counts), conservative toward NOT suppressing (a ctor ignoring `p`'s payload still warns — negative-control proven), over-strict-only (by-name shadowing imprecision is extra-silence, never under-strict; recorded as Known debt). Both stale doc comments that mis-attributed the FP to nested `match` corrected for doc-honesty (debugger concern #2, same code region — in-scope, not opportunistic). +166/20 in linearity.rs only; check-only, zero codegen/runtime/ABI/schema/DESIGN.md change. Boss-verified independently: RED→GREEN, full `cargo test -p ailang-check` 108/0 lib + every binary 0-failed with NO existing test modified, `ail check embed_backtest_step_tick.ail` no longer over-strict on `st`/`tick` (exit 0), `_tick_borrow.ail` + M3 `embed_backtest_step_record.ail` still clean, the already-green `embed_tick_e2e` + bench posture untouched. No audit/fieldtest gate (lint-precision bugfix; the RED test + the green check-suite ARE the regression coverage). RED `a11cb7c` → GREEN this commit. → 2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md
- 2026-05-19 — iter embedding-abi-m5.1 (DONE 3/3): M5 iter 1 — stood up the workspace-excluded `ail-embed` crate. A zero-dependency embedding core (Rust port of the audited `crates/ail/tests/embed/tick_roundtrip.c`: `extern "C"` to the M3-frozen ABI + frozen-layout `State`/`Tick` box helpers + a `Kernel` price fold, raw pointers never escaping the type) links the M3 staticlib via a new `build.rs` (no in-repo precedent — `AIL_BIN` env override else nested `cargo build -p ail` against the parent workspace, separate target dir so no cargo-lock deadlock). Plus a hermetic `data-server` smoke: a synthetic Pepperstone-format ZIP fixture written via the crate's own public `RawTickRecord` type (correct-by-construction, mirrors `data-server/src/loader.rs:110-124`) → real `DataServer``Kernel`, asserting bit-exact vs a same-order host reference fold and the closed-form `55.0`; runs with no `/mnt`. `ail-embed` is its own cargo workspace root (empty `[workspace]` table) so the AILang compiler workspace owes it nothing; `data-server` is a dev-dependency only ⇒ Invariant 1 holds in the dependency graph, not just on paper. Boss-verified independently: ail-embed suite 2/2 green (`kernel_run_sums_prices` unit RED-first + `hermetic_smoke_data_server_roundtrip` integration), full+no-deps `cargo metadata` on the AILang workspace shows `data-server` count 0, `git status` path filter empty (zero diff to `crates/ailang-*`/`crates/ail/`/`runtime/`/`examples/*.ail`), `src/lib.rs` zero code-level `data_server`, AILang `cargo build --workspace` still clean. Two toolchain-forced corrections to the plan's verbatim `ail-embed/Cargo.toml` (added the empty `[workspace]` table; sibling dev-dep path `../libs``../../libs` manifest-relative) — confined to the plan-created manifest, no acceptance gate altered, 0 review re-loops; planner-recon implication recorded in the journal Concerns. Adapter API + thread-swarm explicitly deferred to M5 iter 2+. → 2026-05-19-iter-embedding-abi-m5.1.md
- 2026-05-19 — iter embedding-abi-m5.2 (PARTIAL 2/3; M5 bounce-back): data-server adapter + symbol-fan swarm; the leak-proof did its job and surfaced a real concurrency finding. Tasks 1+2 landed clean: `data-server` promoted dev-dep→real dep (Invariant-1 sanctioned — compiler workspace graph still data-server-count 0, zero compiler-surface diff), additive `adapter` module (`tick_to_px`/`MidPriceStream` lazy `Iterator`/`fold_symbol`, RED-first), and a `swarm_runner` `[[bin]]` whose clean compile *is* the compile-time per-thread-ctx proof (`Ctx: !Send` ⇒ a shared-ctx swarm is `E0277`, cannot build). The real-data symbol-fan swarm runs (EURUSD/GER40/XAUUSD, ~4s) and is **bit-exact vs an independent same-order host reference every run** (`symbol_fan_swarm_bit_exact`, GREEN, live). Task 3's `Σallocs==Σfrees` leak gate is BLOCKED: Boss independently read `runtime/rc.c` and confirms it is an **instrumentation race, not a memory bug** — no box crosses a thread (`!Send`), the real RC is correct (bit-exact green), only the *global* `g_rc_*` stat counters (rc.c:90-91,161,212) are non-atomic *by rc.c's own documented single-threaded design* and lose `++`s when 3 worker threads hit the null-`__ail_tls_ctx` host-side path. M5 is AILang's first concurrent consumer → the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c:44-49 names is now due. The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (`#[ignore]`, **body verbatim** — quarantined not weakened; un-ignore = the runtime fix's acceptance) so main stays green and the finding is pinned. Prior claim that `embed_swarm_tsan.rs` covers this was corrected (it uses the scalar kernel — zero box allocs — never exercised this path). Escalated as a **bounce-back** (touches M3-frozen `runtime/`; re-frames M5's "zero runtime change"; multiple substantive options); Boss recommendation on the record = Option A (atomic *global-fallback* counters only, standalone RED-first runtime micro-iter, M5 framing amended) — user picks the direction. Time-shard + friction-harvest remain m5.3. → 2026-05-19-iter-embedding-abi-m5.2.md
- 2026-05-19 — iter bugfix-rc-global-stats-race (RED→GREEN, debug→implement mini, DONE 1/1): the M5 iter-2 bounce-back resolution (user-approved Option A). The two null-ctx fallback RC-stats counters `g_rc_alloc_count`/`g_rc_free_count` (`runtime/rc.c:90-91`) were plain `static uint64_t` with a non-atomic `++` on the `__ail_tls_ctx == NULL` arm of `ailang_rc_alloc` (rc.c:161) / the to-zero branch of `ailang_rc_dec` (rc.c:212); a multi-threaded host driving the global fallback (no `ailang_ctx_new`) raced the read-modify-write and lost increments, so the `AILANG_RC_STATS` atexit Σ under-counted non-deterministically — exactly the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c's own header (rc.c:44-49) names, M5 being AILang's first concurrent consumer. NOT a memory bug (`Ctx: !Send` keeps every box on one thread; the real refcount/free is correct; programs bit-exact). RED (debugger, audit-trail commit `427b687`): a C host — 8 threads × 2M alloc-then-dec, no ctx so the global path is taken, no box crossing a thread — + integration test asserting the atexit Σ is exact; Boss-verified RED `allocs=2141382 expected 16000000, live=-131242` (deterministic-fail under that contention, not flaky). GREEN (implement mini, this commit): the two globals are now `_Atomic uint64_t`, `atomic_fetch_add_explicit(.., memory_order_relaxed)` at the two fallback `++` sites, `atomic_load_explicit(.., relaxed)` in `ailang_rc_stats_atexit` (its sole reader) — relaxed is correct (pure stats, no happens-before; atexit reader runs post-join). Per-ctx counters + the per-object refcount header left non-atomic BY DESIGN (single-thread-per-ctx; boxes never cross threads) — explicitly out of scope, untouched; frozen value layout / ABI offsets / host-free rule untouched; the genuinely-still-single-threaded drop-worklist doc note correctly left unchanged (a false correction was refused); the two genuinely-stale atomicity doc blocks corrected for honesty. Scope held to `runtime/rc.c` ONLY (37+/14). Boss-verified independently: RED→GREEN deterministically 3/3 (jitter gone, `allocs==frees==16_000_000`), full `cargo test -p ail` green (every binary 0 failed — the per-ctx tsan harnesses embed_swarm_tsan/embed_rc_accounting_tsan + embed_tick_e2e/embed_record_e2e unaffected since the per-ctx path is untouched; rc.c links into every ail binary so this is the strong regression gate), git-status scope = rc.c + journal + stats only, the RED files unchanged. Unblocks resuming the M5 leak-proof (un-`#[ignore]` `ail-embed`'s `symbol_fan_swarm_leak_free`). RED `427b687` → GREEN this commit. → 2026-05-19-iter-bugfix-rc-global-stats-race.md