iter embedding-abi-m5.2 (PARTIAL 2/3): data-server adapter + symbol-fan swarm; leak-proof surfaced a non-atomic global g_rc_* stats race
M5 iteration 2 (specae905de, plan9cc9d9c). Tasks 1+2 clean and committable; Task 3 BLOCKED on a real finding → M5 bounce-back. Shipped: - data-server promoted [dev-dependencies] -> [dependencies] (Invariant-1 sanctioned: ail-embed is the sole meeting point; the AILang workspace graph still has data-server count 0; zero compiler-surface diff — not even root Cargo.toml this iter). - additive `adapter` module (ail-embed/src/adapter.rs): tick_to_px, MidPriceStream (lazy Iterator<f64> over SymbolChunkIter), fold_symbol; RED-first (E0432 -> GREEN). m5.1 core untouched except `pub mod adapter;`. - swarm_runner [[bin]]: one thread per symbol, each owning its Kernel. The clean compile IS the compile-time per-thread-ctx proof (Ctx: !Send => a shared-ctx swarm is E0277). - tests/swarm.rs: real-data symbol-fan E2E. `symbol_fan_swarm_bit_exact` is GREEN and live — per-symbol kernel (acc,n) bit-exact vs an independent same-order host reference (EURUSD/GER40/XAUUSD, ~4s). The Task 3 finding (Boss independently confirmed by reading runtime/rc.c): the global Σallocs==Σfrees leak measurement is an INSTRUMENTATION race, not a memory bug. No box crosses a thread (Ctx: !Send), the real refcount/free is correct (bit-exact GREEN every run); only the global g_rc_* stat counters (rc.c:90-91, 161,212) are non-atomic BY rc.c's own documented single-threaded design, losing ++s when 3 worker threads hit the null-__ail_tls_ctx host-side path. M5 is AILang's first concurrent consumer; rc.c's header (rc.c:44-49) explicitly defers exactly this atomic-vs-non- atomic decision to "when it acquires concurrency primitives". The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (#[ignore], body preserved VERBATIM — quarantined not weakened; un-ignore = the runtime fix's acceptance criterion) so main stays green and the finding is pinned as a regression marker. The earlier plan/journal claim that embed_swarm_tsan.rs covers this path was wrong and is corrected on the record (that test uses the scalar kernel — zero box allocs — so it never exercised the host-side global-counter path). Escalated as an M5 bounce-back: resolution touches M3-frozen runtime/ and re-frames M5's "zero runtime change" commitment; multiple substantive options, not unilaterally Boss's to pick in frozen-runtime territory. Boss recommendation on the record = Option A (make only the global-fallback g_rc_* counters atomic, as a standalone RED-first runtime micro-iteration; M5 framing amended). Time-shard + friction-harvest remain m5.3. Includes the per-iter journal (with Boss disposition), stats, and the INDEX.md line.
This commit is contained in:
@@ -0,0 +1,73 @@
|
||||
//! Symbol-fan swarm harness (spec §3 symbol-fan / Testing §2). Run
|
||||
//! as a subprocess by `tests/swarm.rs` under `AILANG_RC_STATS=1` so
|
||||
//! the C-runtime atexit `ailang_rc_stats:` line is observable.
|
||||
//!
|
||||
//! argv[1] = data dir (defaults to `data_server::DEFAULT_DATA_PATH`).
|
||||
//! Enumerates up to `N_SYMBOLS` symbols that have tick files
|
||||
//! (requires >= 2 for a meaningful swarm), spawns one thread per
|
||||
//! symbol — each thread owns its `Kernel` (`Ctx: !Send` forces
|
||||
//! one ctx per thread) — folds at most `MAX_TICKS` mids, and prints
|
||||
//! one `RESULT <symbol> <acc_bits:016x> <count>` line per symbol to
|
||||
//! stdout. Exits 0 on success, non-zero (with a message) otherwise.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use ail_embed::adapter::fold_symbol;
|
||||
use data_server::records::DataFormat;
|
||||
use data_server::{DataServer, DEFAULT_DATA_PATH};
|
||||
|
||||
/// Deterministic bounded prefix per symbol (chunk-aligned via
|
||||
/// `Iterator::take`): enough real volume to be a genuine proof,
|
||||
/// bounded so the E2E stays ~seconds. m5.3 owns the friction
|
||||
/// timing; here it is only a runtime bound.
|
||||
const MAX_TICKS: usize = 2_000_000;
|
||||
/// Max threads/symbols fanned. Real data has >= 2 tick symbols
|
||||
/// (EURUSD, XAUUSD); take min(N_SYMBOLS, available).
|
||||
const N_SYMBOLS: usize = 4;
|
||||
|
||||
fn main() {
|
||||
let data_dir = std::env::args()
|
||||
.nth(1)
|
||||
.unwrap_or_else(|| DEFAULT_DATA_PATH.to_string());
|
||||
|
||||
let server = Arc::new(DataServer::new(&data_dir));
|
||||
|
||||
let mut tick_syms: Vec<String> = server
|
||||
.symbols()
|
||||
.into_iter()
|
||||
.filter(|s| {
|
||||
server.file_count(s, DataFormat::Tick).unwrap_or(0) > 0
|
||||
})
|
||||
.map(|s| s.to_string())
|
||||
.collect();
|
||||
tick_syms.truncate(N_SYMBOLS);
|
||||
|
||||
if tick_syms.len() < 2 {
|
||||
eprintln!(
|
||||
"swarm_runner: need >= 2 tick symbols under {data_dir}, found {}",
|
||||
tick_syms.len()
|
||||
);
|
||||
std::process::exit(2);
|
||||
}
|
||||
|
||||
let handles: Vec<_> = tick_syms
|
||||
.into_iter()
|
||||
.map(|sym| {
|
||||
let srv = Arc::clone(&server);
|
||||
// Each thread builds its own Kernel inside the closure.
|
||||
// `Kernel`/`Ctx` are `!Send`, so this is the ONLY way
|
||||
// this compiles — one ctx per thread, enforced by the
|
||||
// type system, not by convention.
|
||||
std::thread::spawn(move || {
|
||||
let (acc, n) = fold_symbol(&srv, &sym, MAX_TICKS)
|
||||
.expect("symbol has a tick stream");
|
||||
(sym, acc, n)
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
|
||||
for h in handles {
|
||||
let (sym, acc, n) = h.join().expect("worker thread panicked");
|
||||
println!("RESULT {sym} {:016x} {n}", acc.to_bits());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user