claude
b048923f1c
Replace the per-call std::thread::scope fan-out in run_indexed with one process-wide rayon pool. run_indexed's callers nest — sweep (grid points), monte_carlo (seeds), and walk_forward (window bounds), where a walk_forward window runs an inner sweep — and each std::thread::scope spawned its own available_parallelism() OS-thread batch, so the two levels multiplied to ~available_parallelism()^2 threads (measured 150-240 concurrent on a 24-core box during a 42-cell campaign, ~10x oversubscription). rayon's pool is process-wide and persistent: a nested par_iter work-steals within the same N workers instead of spawning, so live workers never exceed the pool size. Determinism (C1) is preserved bit-for-bit: run_indexed collects via IndexedParallelIterator in enumeration order (not completion order), and the float aggregation (summarize_r / McAggregate / stitch) runs after the ordered collect, so IEEE-754 non-associativity introduces no variance. The existing _with_threads(1) == _with_threads(8) == public determinism tests stay green, now driving local rayon pools of 1 and N workers via ThreadPool::install. Structural inversion: the public sweep/monte_carlo/walk_forward become the core (calling run_indexed on the ambient pool through a shared assemble_* helper); the _with_threads variants collapse into thin, test-only wrappers (#[cfg(test)]) that run the same assembly inside a local pool. run_one stays Fn + Sync (no + Send): run_indexed borrows it in the map closure and the wrappers borrow it into install — passing it by value would move it into rayon's Map and force F: Send, a bound the public callers do not carry (hence the load-bearing #[allow(clippy::redundant_closure)] on run_indexed). Adds a nested-oversubscription regression test (asserts nested run_indexed never exceeds the pool size — the pre-rayon shape ran POOL^2) and an ignored wall-time benchmark. rayon admitted under the amended-C16 per-case dependency policy; it stays within C1 (parallelism across sims, never within one). This is the shared pool only — the first of three steps toward saturating the cores. The diagnosis found the larger loss is the idle valleys (~42% of the box idle: the sequential cell loop and the single-threaded bootstrap), which the pool alone does not fill. Two follow-ups are filed and recorded on #268: the Registry write path is not concurrency-safe (append_family races on a shared families.jsonl), the prerequisite for parallelizing the C1-disjoint cell loop with a RAM budget bounding concurrently-active instruments (the FileCache retains a symbol's parsed files while any reference is live). Verified: cargo test -p aura-engine (276 passed, the C1 determinism tests green), the new oversubscription test green, the benchmark runs (no overflow), clippy -D warnings clean, cargo test --workspace no failures. closes #268
45 lines
2.4 KiB
TOML
45 lines
2.4 KiB
TOML
[package]
|
|
name = "aura-engine"
|
|
edition.workspace = true
|
|
version.workspace = true
|
|
license.workspace = true
|
|
publish.workspace = true
|
|
|
|
[dependencies]
|
|
aura-core = { path = "../aura-core" }
|
|
# aura-analysis holds the pure trading-domain reductions (R-metrics, the
|
|
# position-event table, the multiple-comparison hurdle math) lifted out of
|
|
# `report` (issue #136). The engine re-exports them via `report::` so callers
|
|
# resolve the moved types through aura-engine unchanged (C18 byte-identity).
|
|
aura-analysis = { path = "../aura-analysis" }
|
|
# serde is admitted under the amended C16 per-case dependency policy (INDEX.md):
|
|
# it gives the run-report types a typed (de)serialization path for the run
|
|
# registry (cycle 0029). serde's output is deterministic (C1-safe).
|
|
serde = { workspace = true }
|
|
# serde_json renders RunReport JSON for both the registry (disk) and stdout
|
|
# (RunReport::to_json) — one shape, no hand-rolled writer (amended C16).
|
|
serde_json = { workspace = true }
|
|
# rayon is admitted under the amended C16 per-case dependency policy (INDEX.md):
|
|
# it backs `run_indexed`'s work-stealing pool for the sweep/campaign executor.
|
|
# It stays within C1 (parallelism across sims, never within one) — each sim
|
|
# still runs its own deterministic, synchronous event loop on one thread; rayon
|
|
# only fans the *disjoint* sims of a family out across threads, replacing
|
|
# `std::thread::scope`. No dep on a workspace precedent (chrono's pattern:
|
|
# direct versioned dep).
|
|
rayon = "1"
|
|
|
|
[dev-dependencies]
|
|
# aura-std is a TEST-ONLY dependency: the engine's own integration tests
|
|
# (r_sma_e2e, random_sweep_e2e, ger40_breakout) drive real standard nodes through
|
|
# a bootstrapped Harness. The engine's library code never names a concrete node — it
|
|
# routes type-erased Scalar records (summarize_r reads the PositionManagement record
|
|
# positionally, by index), so aura-std stays out of [dependencies] and the engine is
|
|
# `-> aura-core` only. The node-naming composite-builders live in `aura-composites`.
|
|
aura-std = { path = "../aura-std" }
|
|
# chrono / chrono-tz build the Berlin-local-wall-clock epoch-ns timestamps the
|
|
# GER40 session-breakout E2E fixture (tests/ger40_breakout.rs) feeds the engine,
|
|
# exactly as `Session`'s own unit test does. Pinned to aura-std's versions (the
|
|
# same DST-aware wall-clock math), test-only — never on the engine's hot path.
|
|
chrono = { version = "0.4", default-features = false, features = ["clock"] }
|
|
chrono-tz = { version = "0.10", default-features = false }
|