iter embedding-abi-m5.2 (PARTIAL 2/3): data-server adapter + symbol-fan swarm; leak-proof surfaced a non-atomic global g_rc_* stats race
M5 iteration 2 (specae905de, plan9cc9d9c). Tasks 1+2 clean and committable; Task 3 BLOCKED on a real finding → M5 bounce-back. Shipped: - data-server promoted [dev-dependencies] -> [dependencies] (Invariant-1 sanctioned: ail-embed is the sole meeting point; the AILang workspace graph still has data-server count 0; zero compiler-surface diff — not even root Cargo.toml this iter). - additive `adapter` module (ail-embed/src/adapter.rs): tick_to_px, MidPriceStream (lazy Iterator<f64> over SymbolChunkIter), fold_symbol; RED-first (E0432 -> GREEN). m5.1 core untouched except `pub mod adapter;`. - swarm_runner [[bin]]: one thread per symbol, each owning its Kernel. The clean compile IS the compile-time per-thread-ctx proof (Ctx: !Send => a shared-ctx swarm is E0277). - tests/swarm.rs: real-data symbol-fan E2E. `symbol_fan_swarm_bit_exact` is GREEN and live — per-symbol kernel (acc,n) bit-exact vs an independent same-order host reference (EURUSD/GER40/XAUUSD, ~4s). The Task 3 finding (Boss independently confirmed by reading runtime/rc.c): the global Σallocs==Σfrees leak measurement is an INSTRUMENTATION race, not a memory bug. No box crosses a thread (Ctx: !Send), the real refcount/free is correct (bit-exact GREEN every run); only the global g_rc_* stat counters (rc.c:90-91, 161,212) are non-atomic BY rc.c's own documented single-threaded design, losing ++s when 3 worker threads hit the null-__ail_tls_ctx host-side path. M5 is AILang's first concurrent consumer; rc.c's header (rc.c:44-49) explicitly defers exactly this atomic-vs-non- atomic decision to "when it acquires concurrency primitives". The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (#[ignore], body preserved VERBATIM — quarantined not weakened; un-ignore = the runtime fix's acceptance criterion) so main stays green and the finding is pinned as a regression marker. The earlier plan/journal claim that embed_swarm_tsan.rs covers this path was wrong and is corrected on the record (that test uses the scalar kernel — zero box allocs — so it never exercised the host-side global-counter path). Escalated as an M5 bounce-back: resolution touches M3-frozen runtime/ and re-frames M5's "zero runtime change" commitment; multiple substantive options, not unilaterally Boss's to pick in frozen-runtime territory. Boss recommendation on the record = Option A (make only the global-fallback g_rc_* counters atomic, as a standalone RED-first runtime micro-iteration; M5 framing amended). Time-shard + friction-harvest remain m5.3. Includes the per-iter journal (with Boss disposition), stats, and the INDEX.md line.
This commit is contained in:
@@ -0,0 +1,234 @@
|
||||
# iter embedding-abi-m5.2 — data-server adapter + symbol-fan swarm; leak-free gate surfaced a runtime race
|
||||
|
||||
**Date:** 2026-05-19
|
||||
**Started from:** 9cc9d9c5170eb8d52152dcef071726c7a6ffd05d
|
||||
**Status:** PARTIAL
|
||||
**Tasks completed:** 2 of 3
|
||||
|
||||
## Summary
|
||||
|
||||
Tasks 1 and 2 landed clean and verbatim-to-plan: `data-server` was
|
||||
promoted dev-dep → real dep, the additive `adapter` module
|
||||
(`tick_to_px` / `MidPriceStream` / `fold_symbol`) was added with a
|
||||
genuine compile-error RED → GREEN cycle, and the `swarm_runner` bin
|
||||
compiled — that compile *is* the compile-time per-thread-ctx proof
|
||||
(`Ctx: !Send`; a shared-ctx design is `error[E0277]` and could not
|
||||
have built). Invariant 1 holds independently: the AILang workspace
|
||||
graph has zero `data-server` (Step 4 = `0`) and there is zero
|
||||
compiler-surface diff (Step 5 path filter empty — only `ail-embed/**`
|
||||
touched, not even the root `Cargo.toml`).
|
||||
|
||||
Task 3 is BLOCKED. The plan's verbatim leak-free assertion
|
||||
(`Σallocs == Σfrees` across all `ailang_rc_stats:` lines) is correct
|
||||
and is doing its job: it surfaced a genuine concurrency defect. The
|
||||
swarm E2E passed once in isolation by luck (the racing counter
|
||||
happened to land on the balanced value) and then FAILED in the
|
||||
full-suite run and on 4 of 5 direct re-runs. The host-side box
|
||||
allocations in `ail-embed/src/lib.rs` (`make_state` / `make_tick` /
|
||||
the final `ailang_rc_dec`) run on the Rust worker thread where
|
||||
`__ail_tls_ctx == NULL` — the M3 forwarder only sets the TLS ctx for
|
||||
the synchronous duration of the kernel call, not for the host's own
|
||||
allocs. With `__ail_tls_ctx` null those allocs fall through to the
|
||||
**non-atomic global statics** `g_rc_alloc_count` / `g_rc_free_count`
|
||||
(`runtime/rc.c:161,212`), incremented concurrently by 3 worker
|
||||
threads with no synchronisation. The atexit `g_rc` line's `allocs`
|
||||
is consequently non-deterministic across runs
|
||||
(5,993,321 … 5,999,318 over 5 runs; should be a stable 6,000,000)
|
||||
while `frees` stays at a stable 3. The bit-exactness half of the
|
||||
test is sound and passed every run; only the global leak-accounting
|
||||
is racy.
|
||||
|
||||
This is exactly the class of finding the symbol-fan swarm milestone
|
||||
exists to expose — and it is out of m5.2 scope to fix (see Blocked
|
||||
detail). The Boss decides: extend the spec / m5.3 to cover the
|
||||
host-side-alloc TLS-ctx routing (or atomic global counters), or
|
||||
re-scope the leak-free acceptance.
|
||||
|
||||
## Per-task notes
|
||||
|
||||
- iter embedding-abi-m5.2.1: promote `data-server` dev-dep → real
|
||||
dep + additive `adapter` module. RED (`E0432 unresolved import
|
||||
super::tick_to_px`) → impl `tick_to_px`/`MidPriceStream`/
|
||||
`fold_symbol` → GREEN (`tick_to_px_is_mid`, +
|
||||
`kernel_run_sums_prices` unregressed under `--lib`). Cargo.toml +
|
||||
`lib.rs:7` verbatim-to-plan. Stale m5.1 "ZERO dependencies"
|
||||
manifest comment corrected for present-tense honesty (quality
|
||||
phase).
|
||||
- iter embedding-abi-m5.2.2: `src/bin/swarm_runner.rs` written
|
||||
verbatim; `cargo build --bin swarm_runner` → exit 0. The clean
|
||||
compile IS the compile-time per-thread-ctx proof (`Ctx: !Send`;
|
||||
no `E0277`; per-thread-owned-Kernel structure intact, not
|
||||
collapsed to a shared ctx).
|
||||
- iter embedding-abi-m5.2.3: `tests/swarm.rs` written verbatim. The
|
||||
E2E genuinely executed (NOT the skip path — 3 tick symbols
|
||||
present: EURUSD, GER40, XAUUSD; ~4 s, 3×2M FFI folds). Per-symbol
|
||||
bit-exactness vs independent host reference: sound, passed every
|
||||
run. Global `Σallocs==Σfrees` leak-free assertion: BLOCKED —
|
||||
correctly detects a non-atomic global-counter race (see Summary +
|
||||
Blocked detail). Not re-loopable (the plan's verbatim code is
|
||||
exactly as specified; no re-attempt makes a racy C counter
|
||||
deterministic) and not fixable in-scope (Step 5 mandates zero
|
||||
`runtime/` diff; `runtime/` is M3-frozen).
|
||||
|
||||
## Concerns
|
||||
|
||||
- Plan Task 1 Step 6's literal command
|
||||
(`cargo test --manifest-path ail-embed/Cargo.toml`, unfiltered)
|
||||
is unsatisfiable in Task 1's isolation: the `[[bin]] swarm_runner`
|
||||
declaration added by Step 2's verbatim Cargo.toml replacement
|
||||
references `src/bin/swarm_runner.rs`, which Task 2 Step 1 creates.
|
||||
The plan Self-review point 7 ("No later task is required to make
|
||||
an earlier compile gate pass") is incorrect for this command.
|
||||
Task 1's no-core-regression intent was fully proven via `--lib`
|
||||
(`kernel_run_sums_prices` + `tick_to_px_is_mid` GREEN); the
|
||||
smoke + full-suite gate is covered by Task 3 Step 3 once the bin
|
||||
exists. The verbatim Cargo.toml was NOT split to dodge this
|
||||
(splitting it would be the spec-compliance violation; the
|
||||
`[[bin]]` + dep-promotion is one cohesive manifest change). No
|
||||
code defect — purely a plan command-ordering defect.
|
||||
- Quality phase corrected the pre-existing m5.1 Cargo.toml comment
|
||||
("the library itself has ZERO dependencies … data-server is
|
||||
dev-only") which became false the moment `data-server` moved into
|
||||
`[dependencies]`. The plan's verbatim replacement boundary started
|
||||
*at* `[dependencies]` and did not script the preceding comment's
|
||||
removal; leaving a now-false manifest comment violates the
|
||||
DESIGN.md honesty rule (comments mirror current state). The
|
||||
replacement comment states the present-tense m5.2 framing,
|
||||
consistent with the plan's own new block + Boss decision.
|
||||
|
||||
## Known debt
|
||||
|
||||
- The host-side embedding allocation path (`ail-embed/src/lib.rs`
|
||||
`make_state`/`make_tick`/final `ailang_rc_dec`) executes with
|
||||
`__ail_tls_ctx == NULL` (the M3 `@<sym>` forwarder sets the TLS
|
||||
ctx only for the synchronous kernel-call duration). Under the
|
||||
multi-thread swarm these allocs/frees land on the non-atomic
|
||||
global `g_rc_alloc_count`/`g_rc_free_count` fallback
|
||||
(`runtime/rc.c:161,212`), which races. Not touched here:
|
||||
`runtime/` is M3-frozen and Step 5 mandates zero `runtime/` diff;
|
||||
the host-side-routing alternative is a design change beyond
|
||||
"additive adapter + swarm bin". Reserved for Boss/m5.3 design.
|
||||
|
||||
## Blocked detail
|
||||
|
||||
- **Task:** 3 (Step 2 / Step 3 — the leak-free `Σallocs==Σfrees`
|
||||
gate).
|
||||
- **Reason:** `worker-blocked` — the plan's own verbatim acceptance
|
||||
gate correctly detects a runtime concurrency defect whose fix is
|
||||
out of m5.2 scope and would violate this iter's Invariant-1
|
||||
Step-5 zero-`runtime/`-diff constraint.
|
||||
- **Worker says (verbatim):** Task 3's `tests/swarm.rs` is verbatim
|
||||
to plan and genuinely executes (bit-exact half sound, GREEN every
|
||||
run). The `Σallocs==Σfrees` half is non-deterministic: full-suite
|
||||
run FAILED `left: 11997743, right: 12000003`; 5 direct child runs
|
||||
of the atexit `g_rc` line gave `allocs=` 5993699/5996029/5993321/
|
||||
5999318/5996019 with `frees=3` stable. Root cause: host-side
|
||||
`ailang_rc_alloc`/`ailang_rc_dec` from `lib.rs` run with
|
||||
`__ail_tls_ctx==NULL`, falling through to the non-atomic global
|
||||
`g_rc_*` counters (`rc.c:161,212`) raced by 3 worker threads. Not
|
||||
re-loopable (verbatim plan code; a racy C counter is not made
|
||||
deterministic by re-attempting the same code) and not
|
||||
in-scope-fixable (zero `runtime/` diff mandated; `runtime/` is
|
||||
M3-frozen).
|
||||
- **Suggested next step:** Boss decision — either (a) extend the
|
||||
spec/m5.3 to route host-side `ail-embed` allocs through a
|
||||
per-thread `ailang_ctx_t` so the swarm's leak accounting is
|
||||
per-thread-attributed (no global fallback), or make
|
||||
`g_rc_alloc_count`/`g_rc_free_count` atomic in a *separate*
|
||||
runtime iteration outside m5.2's zero-`runtime/`-diff constraint;
|
||||
or (b) re-scope the m5.2 leak-free acceptance. Tasks 1 + 2 are
|
||||
clean and committable as a PARTIAL; the architect should see the
|
||||
TSan-discharge reasoning below at milestone close.
|
||||
|
||||
## Boss-decision rationale carried forward (mirrored from the plan, for the architect at milestone close)
|
||||
|
||||
- **TSan discharge (compile-time + cited green precedent, no new
|
||||
harness).** The spec's "sanitiser-clean (no data race across
|
||||
per-thread ctxs)" acceptance is discharged by (1) `Ctx`/`Kernel`
|
||||
being `!Send` (raw `*mut c_void`) — the swarm is *structurally
|
||||
forced* one-ctx-per-thread; a shared-ctx design is `error[E0277]`
|
||||
and cannot compile, so Task 2 Step 2's clean build is a
|
||||
compile-time race-freedom proof for the *per-ctx kernel calls*;
|
||||
(2) the C-runtime race-freedom of that exact per-thread-ctx
|
||||
pattern is ratified green by `crates/ail/tests/embed_swarm_tsan.rs`
|
||||
(C host, 8 pthreads, own ctx each, `clang -fsanitize=thread`,
|
||||
exit-clean), cited not re-run. NOTE FOR THE ARCHITECT: this
|
||||
discharge covers the *per-thread-ctx kernel path*. The Task 3
|
||||
finding is a *different* surface — the host-side `ail-embed`
|
||||
allocs that bypass the TLS ctx and hit the non-atomic global
|
||||
`g_rc_*` fallback. The TSan precedent does not cover that path
|
||||
(the C host in `embed_swarm_tsan.rs` does not exercise a Rust
|
||||
host allocating boxes with a null TLS ctx). The compile-time
|
||||
`!Send` proof and the cited green TSan remain valid for what they
|
||||
claim; they simply do not reach the global-fallback race m5.2
|
||||
newly exposed.
|
||||
- **`data-server` dev-dep → real dep is sanctioned.** Invariant 1
|
||||
governs the *compiler* crates (`ailang-*`, `crates/ail`,
|
||||
`runtime/`), which gain nothing — verified clean this iter
|
||||
(Step 4 = `0`, Step 5 empty). `ail-embed` is explicitly the
|
||||
*sole* data-server↔AILang meeting point.
|
||||
- **Deferred to m5.3 (not this plan):** time-shard
|
||||
boundary-invisibility, friction-harvest timing, milestone
|
||||
close-out. Not implemented here (scope discipline held — no
|
||||
time-shard, no perf timing, no close-out crept in).
|
||||
|
||||
## Files touched
|
||||
|
||||
- `ail-embed/Cargo.toml` (modified — dep promotion + `[[bin]]` +
|
||||
stale-comment honesty fix)
|
||||
- `ail-embed/src/lib.rs` (modified — `pub mod adapter;`, additive)
|
||||
- `ail-embed/src/adapter.rs` (new — the sole data-server-knowing layer)
|
||||
- `ail-embed/src/bin/swarm_runner.rs` (new — symbol-fan swarm bin)
|
||||
- `ail-embed/tests/swarm.rs` (new — symbol-fan leak-free + bit-exact
|
||||
E2E; bit-exact half sound, leak-free half surfaces the race)
|
||||
|
||||
## Stats
|
||||
|
||||
bench/orchestrator-stats/2026-05-19-iter-embedding-abi-m5.2.json
|
||||
|
||||
## Boss disposition (2026-05-19)
|
||||
|
||||
Boss independently read `runtime/rc.c` and confirms the diagnosis is
|
||||
**precise and the finding is an instrumentation-measurement race, NOT
|
||||
a memory bug**: the per-object refcount header ops (`*hdr +=/-= 1`,
|
||||
rc.c:180/208) are non-atomic but **no box crosses a thread** in the
|
||||
swarm (`Ctx: !Send` structurally forces alloc/use/free within one
|
||||
worker), so there is no real refcount race, no double-free, no leak —
|
||||
the bit-exact half is GREEN every run, confirming the memory
|
||||
management is actually correct. The *only* raced object is the global
|
||||
`g_rc_*` statistics counter (rc.c:90-91,161,212), **non-atomic by
|
||||
rc.c's own explicit design** (header comment rc.c:44-49,85: "AILang
|
||||
has no concurrency primitives yet; when it acquires them,
|
||||
atomic-vs-non-atomic becomes a separate decision"). M5 is AILang's
|
||||
first concurrent consumer — that deferred decision is now due.
|
||||
|
||||
Committed as a PARTIAL (Tasks 1+2 + the GREEN existence proof) so the
|
||||
sound work + the finding land on the record, main stays green:
|
||||
`tests/swarm.rs` was Boss-split into `symbol_fan_swarm_bit_exact`
|
||||
(live, GREEN — the real-data existence proof) and
|
||||
`symbol_fan_swarm_leak_free` (`#[ignore]`, **body preserved
|
||||
verbatim** — quarantined, not weakened; un-`#[ignore]` is the
|
||||
acceptance criterion for the runtime fix). Earlier plan/journal claim
|
||||
that `embed_swarm_tsan.rs` covers this path was wrong and is
|
||||
corrected here: that test uses the *scalar* kernel (zero box allocs)
|
||||
so it never exercised the host-side global-counter path; the
|
||||
compile-time `!Send` proof + the cited green TSan remain valid only
|
||||
for the per-thread-ctx *kernel* path, which they always claimed.
|
||||
|
||||
Escalated to the user as an M5 **bounce-back** (touches M3-frozen
|
||||
`runtime/`; re-frames M5's "zero runtime change" commitment;
|
||||
multiple substantive options — not unilaterally Boss's to pick in
|
||||
frozen-runtime territory). Boss recommendation on the record:
|
||||
**Option A** — make *only the global fallback* `g_rc_*` counters
|
||||
atomic (leave the per-thread ctx path plain — no contention there),
|
||||
as its own RED-first runtime micro-iteration (`debug` skill), outside
|
||||
M5's "zero runtime change" framing; then resume the M5 leak-proof on
|
||||
the now-valid measurement (un-`#[ignore]` `symbol_fan_swarm_leak_free`
|
||||
as that iter's GREEN). Rationale: it discharges exactly the deferred
|
||||
decision rc.c's own comment names, at the defect's actual location,
|
||||
without touching the frozen value layout / ABI offsets / host-free
|
||||
rule (the literal content of the M3 freeze); Option B (host-side
|
||||
TLS-ctx routing) adds a new runtime API surface; Option C (per-symbol
|
||||
single-thread subprocess leak-proof) is sound but weaker and leaves
|
||||
the runtime's documented single-thread-stats footgun for every future
|
||||
concurrent consumer. User picks the direction.
|
||||
@@ -107,3 +107,4 @@
|
||||
- 2026-05-18 — brainstorm embedding-abi-m4 → RETIRED, never speced (premise collapsed under its own feature-acceptance gate during Step-2/3 Q&A; no spec, no grounding-check, no planner handoff — the "problem mis-framed → don't ratify a known-unneeded shape" brainstorm path): `/boss` picked top-P0 "Embedding ABI — M4: sequence crossing via `List`"; user green-lit a continue-here brainstorm; recon (`ailang-plan-recon`) returned a full fact sheet; two user forks resolved in Q&A (own-only `List` param; `List Record`-only element) and Approach A (structural list-shaped `is_c_abi_type` arm, no name-match; B name-anchored / C `std_list`-SSOT-first rejected on language/scope grounds) recommended — all now moot. Struck on **feature-acceptance clause 2**: the shipped M3 gate `is_c_abi_type` (`crates/ailang-check/src/lib.rs:1934-1953`) is a per-parameter loop accepting a C scalar OR a single-ctor all-scalar record *independently per param*, and the forwarder's `llvm_scalar` maps every non-scalar `Type::Con`→`ptr` (M3-frozen), so `(State, Tick) -> State` (both single-ctor all-scalar records) is **already gate-accepted + forwarder-supported today** — the minimal data-server binding is M3 (shipped) + a host-side per-tick loop; cons-list crossing would *add* a 2N+1-box-per-chunk host builder + the deferred flat-array perf debt and removes no redundancy, with no named consumer (M5's adapter unrolls each chunk host-side — a clean adapter, the sole data-server↔AILang meeting point per Invariant 1; whole-chunk in-kernel visibility is semantically void since `State` threads across calls regardless of chunk boundaries). Honest mid-Q&A correction recorded: I had asserted "M5 cannot wire data-server without M4" — false (M5 wires it on M3 + `for tick in chunk`); re-deriving against the code rather than defending the roadmap I wrote is what surfaced the clause-2 failure ("user suggestions ≠ directives, form own judgment"). Outcome: M4 retired in `docs/roadmap.md` (struck entry kept one cycle, never `[x]`); M5 reconciled (`depends on:` M4→M3 + Tick-coverage todo; adapter unrolls host-side; friction feeds the host-per-tick-FFI-vs-batch P2 perf decision); residual = a new `[todo]` "Tick-coverage on M3" (E2E+fixture pinning the two-record-param per-tick `(State, Tick) -> State` shape — capability present today but only E2E-proven for a single record param `State`; every shipped M3 fixture pushes a scalar `Float` sample, none a record `Tick`; test backfill, no brainstorm — the actual "minimal data-server binding"). Forward note: the P2 flat-array item's "1024 cons-cells/chunk" framing is now partly stale (cons-list path dropped) — reconcile when picked up, not now. → 2026-05-18-brainstorm-embedding-abi-m4-retired.md
|
||||
- 2026-05-18 — iter bugfix-over-strict-mode-ctor-rebuild-consume (RED→GREEN, debug→implement mini, DONE 1/1): fixed a conservative `[over-strict-mode]` false-positive surfaced by the Tick-coverage fixtures. The lint's consume-detection (`any_sub_binder_consumed_for`/`pattern_has_consumed_heap_binder`, `crates/ailang-check/src/linearity.rs`) only recognised a consume of an `(own (con T))` param when a *heap-typed* pattern-binder was moved out of `match p`; when `p` was destructured into purely *primitive* fields fed into a `Term::Ctor` rebuilding `p`'s own ctor, that genuine dismantle+rebuild consume was invisible, so the lint spuriously advised `(borrow ...)`. Real harm: an LLM author "fixing" the spurious warning by flipping an export's declared mode `own`→`borrow` would silently invert the ABI ownership contract. RED-first: debugger disproved the carrier's initial nested-`match` hypothesis (the M3 `embed_backtest_step_record.ail` is silent only because its implicit-mode scalar `Float` param disables the lint via the activation gate, linearity.rs:327 — NOT because it handles the rebuild; the defect reproduces single-param, no nesting), wrote the synthetic RED unit `over_strict_mode_silent_when_ctor_rebuilt_from_primitive_fields` committed as its own audit-trail commit `a11cb7c`. GREEN (implement mini): added a 2nd recognition path to `any_sub_binder_consumed_for` — a `match p` arm that destructures binders out of `p`'s ctor and references any of them (primitive or not) inside a `Term::Ctor`'s args in the arm body genuinely consumes `p`; two pure helpers (`ctor_uses_any_binder` + deep `term_mentions_any_binder`, so `(+ acc px)`-mediated flow counts), conservative toward NOT suppressing (a ctor ignoring `p`'s payload still warns — negative-control proven), over-strict-only (by-name shadowing imprecision is extra-silence, never under-strict; recorded as Known debt). Both stale doc comments that mis-attributed the FP to nested `match` corrected for doc-honesty (debugger concern #2, same code region — in-scope, not opportunistic). +166/−20 in linearity.rs only; check-only, zero codegen/runtime/ABI/schema/DESIGN.md change. Boss-verified independently: RED→GREEN, full `cargo test -p ailang-check` 108/0 lib + every binary 0-failed with NO existing test modified, `ail check embed_backtest_step_tick.ail` no longer over-strict on `st`/`tick` (exit 0), `_tick_borrow.ail` + M3 `embed_backtest_step_record.ail` still clean, the already-green `embed_tick_e2e` + bench posture untouched. No audit/fieldtest gate (lint-precision bugfix; the RED test + the green check-suite ARE the regression coverage). RED `a11cb7c` → GREEN this commit. → 2026-05-18-iter-bugfix-over-strict-mode-ctor-rebuild-consume.md
|
||||
- 2026-05-19 — iter embedding-abi-m5.1 (DONE 3/3): M5 iter 1 — stood up the workspace-excluded `ail-embed` crate. A zero-dependency embedding core (Rust port of the audited `crates/ail/tests/embed/tick_roundtrip.c`: `extern "C"` to the M3-frozen ABI + frozen-layout `State`/`Tick` box helpers + a `Kernel` price fold, raw pointers never escaping the type) links the M3 staticlib via a new `build.rs` (no in-repo precedent — `AIL_BIN` env override else nested `cargo build -p ail` against the parent workspace, separate target dir so no cargo-lock deadlock). Plus a hermetic `data-server` smoke: a synthetic Pepperstone-format ZIP fixture written via the crate's own public `RawTickRecord` type (correct-by-construction, mirrors `data-server/src/loader.rs:110-124`) → real `DataServer` → `Kernel`, asserting bit-exact vs a same-order host reference fold and the closed-form `55.0`; runs with no `/mnt`. `ail-embed` is its own cargo workspace root (empty `[workspace]` table) so the AILang compiler workspace owes it nothing; `data-server` is a dev-dependency only ⇒ Invariant 1 holds in the dependency graph, not just on paper. Boss-verified independently: ail-embed suite 2/2 green (`kernel_run_sums_prices` unit RED-first + `hermetic_smoke_data_server_roundtrip` integration), full+no-deps `cargo metadata` on the AILang workspace shows `data-server` count 0, `git status` path filter empty (zero diff to `crates/ailang-*`/`crates/ail/`/`runtime/`/`examples/*.ail`), `src/lib.rs` zero code-level `data_server`, AILang `cargo build --workspace` still clean. Two toolchain-forced corrections to the plan's verbatim `ail-embed/Cargo.toml` (added the empty `[workspace]` table; sibling dev-dep path `../libs`→`../../libs` manifest-relative) — confined to the plan-created manifest, no acceptance gate altered, 0 review re-loops; planner-recon implication recorded in the journal Concerns. Adapter API + thread-swarm explicitly deferred to M5 iter 2+. → 2026-05-19-iter-embedding-abi-m5.1.md
|
||||
- 2026-05-19 — iter embedding-abi-m5.2 (PARTIAL 2/3; M5 bounce-back): data-server adapter + symbol-fan swarm; the leak-proof did its job and surfaced a real concurrency finding. Tasks 1+2 landed clean: `data-server` promoted dev-dep→real dep (Invariant-1 sanctioned — compiler workspace graph still data-server-count 0, zero compiler-surface diff), additive `adapter` module (`tick_to_px`/`MidPriceStream` lazy `Iterator`/`fold_symbol`, RED-first), and a `swarm_runner` `[[bin]]` whose clean compile *is* the compile-time per-thread-ctx proof (`Ctx: !Send` ⇒ a shared-ctx swarm is `E0277`, cannot build). The real-data symbol-fan swarm runs (EURUSD/GER40/XAUUSD, ~4s) and is **bit-exact vs an independent same-order host reference every run** (`symbol_fan_swarm_bit_exact`, GREEN, live). Task 3's `Σallocs==Σfrees` leak gate is BLOCKED: Boss independently read `runtime/rc.c` and confirms it is an **instrumentation race, not a memory bug** — no box crosses a thread (`!Send`), the real RC is correct (bit-exact green), only the *global* `g_rc_*` stat counters (rc.c:90-91,161,212) are non-atomic *by rc.c's own documented single-threaded design* and lose `++`s when 3 worker threads hit the null-`__ail_tls_ctx` host-side path. M5 is AILang's first concurrent consumer → the deferred "concurrency arrived: atomic-vs-non-atomic" decision rc.c:44-49 names is now due. The leak assertion was Boss-split into `symbol_fan_swarm_leak_free` (`#[ignore]`, **body verbatim** — quarantined not weakened; un-ignore = the runtime fix's acceptance) so main stays green and the finding is pinned. Prior claim that `embed_swarm_tsan.rs` covers this was corrected (it uses the scalar kernel — zero box allocs — never exercised this path). Escalated as a **bounce-back** (touches M3-frozen `runtime/`; re-frames M5's "zero runtime change"; multiple substantive options); Boss recommendation on the record = Option A (atomic *global-fallback* counters only, standalone RED-first runtime micro-iter, M5 framing amended) — user picks the direction. Time-shard + friction-harvest remain m5.3. → 2026-05-19-iter-embedding-abi-m5.2.md
|
||||
|
||||
Reference in New Issue
Block a user