b39fd633965071fcf9bd11f337d8a7e905cacc66
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 claude
|
b39fd63396 |
refactor: split the aura-std roster into C28 layer crates
Phase 4 of the Stratification milestone. aura-std held four C28 ladder layers in one roster; this cuts them into layer-aligned, aura-core-only node crates so the import direction is enforced by the crate graph: - aura-std — engine nodes only (arithmetic/logic/rolling + sinks) - aura-market — session, resample - aura-strategy — bias, stops, sizer, cost-model machinery - aura-backtest — sim_broker, position_management - aura-vocabulary — the relocated closed std_vocabulary roster Node modules move verbatim (byte-identical renames); consumers are rewired by import path only. A new structural test (aura-vocabulary/tests/c28_layering.rs) asserts each node crate's [dependencies] stay within its C28-permitted inner set, catching the acyclic-but-outward violation the compiler misses. Behaviour byte-identical: full workspace suite green (1448 tests), no golden edited, clippy -D warnings clean. C28 Status block updated. closes #288 |
||
|
claude
|
b048923f1c |
feat(engine): shared rayon pool for the disjoint-parallel core
Replace the per-call std::thread::scope fan-out in run_indexed with one process-wide rayon pool. run_indexed's callers nest — sweep (grid points), monte_carlo (seeds), and walk_forward (window bounds), where a walk_forward window runs an inner sweep — and each std::thread::scope spawned its own available_parallelism() OS-thread batch, so the two levels multiplied to ~available_parallelism()^2 threads (measured 150-240 concurrent on a 24-core box during a 42-cell campaign, ~10x oversubscription). rayon's pool is process-wide and persistent: a nested par_iter work-steals within the same N workers instead of spawning, so live workers never exceed the pool size. Determinism (C1) is preserved bit-for-bit: run_indexed collects via IndexedParallelIterator in enumeration order (not completion order), and the float aggregation (summarize_r / McAggregate / stitch) runs after the ordered collect, so IEEE-754 non-associativity introduces no variance. The existing _with_threads(1) == _with_threads(8) == public determinism tests stay green, now driving local rayon pools of 1 and N workers via ThreadPool::install. Structural inversion: the public sweep/monte_carlo/walk_forward become the core (calling run_indexed on the ambient pool through a shared assemble_* helper); the _with_threads variants collapse into thin, test-only wrappers (#[cfg(test)]) that run the same assembly inside a local pool. run_one stays Fn + Sync (no + Send): run_indexed borrows it in the map closure and the wrappers borrow it into install — passing it by value would move it into rayon's Map and force F: Send, a bound the public callers do not carry (hence the load-bearing #[allow(clippy::redundant_closure)] on run_indexed). Adds a nested-oversubscription regression test (asserts nested run_indexed never exceeds the pool size — the pre-rayon shape ran POOL^2) and an ignored wall-time benchmark. rayon admitted under the amended-C16 per-case dependency policy; it stays within C1 (parallelism across sims, never within one). This is the shared pool only — the first of three steps toward saturating the cores. The diagnosis found the larger loss is the idle valleys (~42% of the box idle: the sequential cell loop and the single-threaded bootstrap), which the pool alone does not fill. Two follow-ups are filed and recorded on #268: the Registry write path is not concurrency-safe (append_family races on a shared families.jsonl), the prerequisite for parallelizing the C1-disjoint cell loop with a RAM budget bounding concurrently-active instruments (the FileCache retains a symbol's parsed files while any reference is live). Verified: cargo test -p aura-engine (276 passed, the C1 determinism tests green), the new oversubscription test green, the benchmark runs (no overflow), clippy -D warnings clean, cargo test --workspace no failures. closes #268 |
||
 Brummel
|
4de6d5cbad |
rename: retire the stage1-* family for the r-family (r-sma / r-breakout / r-meanrev)
The stage1 ordinal has been dead vocabulary since the C10 reframe dropped the two-stage research model: a 1 structurally implies a 2 that no longer exists. The family is renamed by its live discriminator - the R yardstick - with members named by their signal, uniform with their signal-named siblings (sma, macd, momentum): - selectors: stage1-r -> r-sma, stage1-breakout -> r-breakout, stage1-meanrev -> r-meanrev (old tokens are usage errors, exit 2 - no silent alias) - identifiers: Strategy::RSma/RBreakout/RMeanRev, HarnessKind::RSma, r_sma_*/r_breakout_*/r_meanrev_*, wrap_r, run_signal_r, RGrid, R_SMA_* - persisted identity: the sma_signal composite (param prefix sma_signal.fast.length / .slow.length; fixtures regenerated; content-ids shift - no test pins a literal hash, the registry parses no record names) - e2e test files git-mv'd to r_sma_e2e / r_breakout_e2e / r_meanrev_e2e - dead Stage-1/Stage-2 prose reworded to post-C10 vocabulary across rustdoc, Cargo.tomls, ledger live lines, and the glossary (historical entries stay; fieldtests corpus untouched) - CLAUDE.md invariant 7 rewritten to record the ratified C10 reframe faithfully (the token-swap alone would have laundered the retired gated-currency/realistic-broker design into unmarked live prose); the unbacked account-mode clause dropped Verification: cargo build/test --workspace green (51 targets), clippy -D warnings clean, doc build clean, acceptance grep gate leaves exactly the one resampling-stage false positive (harness.rs), smoke: --harness r-sma runs, --harness stage1-r exits 2 with the new usage line. Decision log: forks and rationale recorded on the issue (reconciliation + implementation-phase comments). closes #174 |
||
|
Brummel
|
94c88eb7e1 |
refactor(0079): extract the trading-analysis leaf into aura-analysis (refs #136)
aura-engine is documented (C16) and named as the reusable, domain-agnostic reactive SoA substrate, yet `report.rs` concentrated the trading-domain analysis layer inside it. This cycle relocates the pure-domain leaf into a new `aura-analysis` crate (deps: aura-core + serde only), erecting the engine/domain seam before the Stage-2 broker milestone deposits more currency/position-event code into the same module. Behaviour-preserving (C1): the full workspace suite stays green unchanged (665 passed), including the C18 on-disk byte-identity goldens (cli_run, the registry legacy-line loads). Moved to aura-analysis (bodies verbatim — no float expression re-associated): RunMetrics, RMetrics (+ its hand-written PartialEq), the private r_col column indices + SQN_CAP, summarize_r, r_metrics_from_rs, derive_position_events, PositionAction (+ From/TryFrom<i64>), PositionEvent, and the deflation stats helpers inv_norm_cdf / expected_max_of_normals. Their unit tests and private helpers (r_row / r_row_full / pm_row) relocate with them — tests live with their subject (the r_col helpers reach the private module directly, which a re-export cannot bridge). Stays in aura-engine for this cycle: the generic trace plumbing (ColumnarTrace, join_on_ts, JoinedRow, f64_field), RunManifest, RunReport, and summarize (it bridges recorded trace columns into RunMetrics, so it is trace-coupled). The mis-placed orchestration types FamilySelection / SelectionMode also stay (their relocation is entangled with the registry-vocabulary fork, deferred). report.rs pub-re-exports the moved symbols, so aura-engine's lib.rs re-export block and every `crate::report::X` / `aura_engine::X` consumer (aura-registry, aura-cli, aura-composites, aura-ingest) compile byte-unchanged; only the new crate, the workspace member list, aura-engine's Cargo.toml dep, and report.rs itself change. Deferred to later cycles (genuine design forks, route through specify when reached): making RunReport generic over a metric type M and re-bounding sweep/mc/walkforward; relocating the aura-registry Metric / metric_cmp / rank vocabulary; relocating FamilySelection / SelectionMode. Fork decisions logged on #136. |
||
|
Brummel
|
1a6eafa056 |
refactor(composites): extract Stage-1 composite-builders into their own crate
Dissolve the aura-engine -> aura-std dependency by moving the vol_stop /
risk_executor composite-builders out of aura-engine into a new aura-composites
crate. This restores the engine's domain-free layering, which the
composites-in-engine placement (cycle 0065) had inverted.
Why: the engine routes type-erased Scalar records and never names a concrete
node -- summarize_r reads the PositionManagement record positionally, by column
index, without linking aura-std (which is exactly why aura-std was a dev-only
dependency before 0065). Parking the node-naming composite-builders inside
aura-engine forced the engine's production code to reference the node library
for the first time (aura-engine -> aura-std), an upside-down layering: the
engine is the foundation the node library builds upon, not the reverse.
The clean home is a dedicated crate that depends on BOTH the engine's
GraphBuilder and the std nodes, leaving each lower crate untouched:
aura-composites -> { aura-engine, aura-std } -> aura-core (acyclic)
aura-engine -> aura-core only (domain-free again)
aura-std -> aura-core only (lean node library)
aura-std reverts to a [dev-dependency] of aura-engine (the engine's own E2E
tests -- stage1_r_e2e, random_sweep_e2e, ger40_breakout -- still drive real
std nodes through a bootstrapped Harness; that test-only edge creates no cycle).
Moved with history (git mv):
- crates/aura-engine/src/composites.rs -> crates/aura-composites/src/lib.rs
- crates/aura-engine/tests/risk_executor.rs -> crates/aura-composites/tests/
- crates/aura-engine/tests/vol_stop_composite.rs -> crates/aura-composites/tests/
Consumers rewired: aura-cli imports risk_executor / StopRule from aura_composites;
the moved tests import the builders from the crate under test. The intra-doc link
to RollMode and the module rationale doc are updated for the new home.
Behaviour-preserving: workspace build clean; clippy --all-targets -D warnings
clean; full suite 513 passed / 0 failed, identical to baseline (the two moved
test files run unchanged under aura-composites); cargo doc --no-deps clean (no
broken intra-doc links).
|
||
|
Brummel
|
a6fc48daa7 |
feat(stage1-r): operate the R layer from the CLI (iter 3)
`aura run --harness stage1-r [--real <SYM>] [--trace <n>]` now runs a Stage-1 R
backtest and prints its signal-quality metrics (the R block) in the RunReport
JSON - one shell call, the research loop's primary verb, ergonomic for Claude
to drive.
What shipped:
- vol_stop + risk_executor promoted from test fixtures to public aura-engine
composite-builders, with a StopRule{Fixed,Vol} axis (C11). aura-std becomes a
normal dependency of aura-engine (the composites are the engine's convenience
layer over the standard nodes - the sibling tier of summarize_r; the graph
stays acyclic: aura-engine -> aura-std -> aura-core). Both fixtures call the
public fns; a Vol-backed RED test pins the new match arm.
- stage1_r_blueprint fans one bias into BOTH a SimBroker (pip) and the
RiskExecutor (R), so one report carries both yardsticks honestly. run_stage1_r
folds summarize (pip) + summarize_r (R, round_trip_cost 0.0 - Stage-1 is
frictionless signal quality) and sets RunMetrics.r = Some(..). An r_equity tap
(cum_realized_r + unrealized_r via LinComb) persists through a separate 3-tap
persist_traces_r and renders via the existing `aura chart --tap r_equity`.
- `aura run --harness <sma|macd|stage1-r>`: a parse_run_args tokenizer replaces
the five `run` literal-slice arms so --harness/--real/--from/--to/--trace
compose in any order; --macd kept as a back-compat alias; --harness sma the
default. A compile-time match over Rust-authored built-ins - not a runtime node
registry, not a DSL (C9/C17): the CLI runs an authored harness, it does not
wire one.
Back-compat: --harness sma/macd leave RunMetrics.r = None, so skip_serializing_if
keeps pip-only and legacy runs.jsonl JSON byte-unchanged; the plain-run tap list
stays ["equity","exposure"].
Refactor beyond the plan (verified behavior-equivalent): the old standalone
parse_real_args was orphaned by the new tokenizer, so it and its now-redundant
unit test were removed and run's --real parsing inlined into parse_run_args. The
--real strictness (empty-symbol, non-i64 ms, repeated-flag, window-requires-real
-> exit 2) is preserved and now pinned by
parse_run_args_rejects_malformed_real_and_repeated_flags (added to restore the
deleted coverage). All five run_real integration tests stay green.
Verification: cargo build/test --workspace green (every crate, 0 failed); clippy
--workspace --all-targets -D warnings clean. The full diff was adversarially
reviewed (run-arg refactor equivalence against the removed function, harness
wiring, C1/C2/C9/C17, the serde back-compat pin) - verdict SOUND; the one flagged
coverage gap is closed here.
Follow-on (noted, not done): the stage1-r manifest reuses the
"sim-optimal(pip_size=...)" broker label; a dedicated "risk-executor" label would
read more honestly for a dual-tap harness that runs a RiskExecutor branch.
closes #129
refs #117 #128
|
||
|
Brummel
|
4187f3bbc1 |
test(aura-engine): GER40 session-breakout e2e — the milestone capstone
Hand-wires all seven new aura-std nodes (Resample, Delay, Gt, Session, EqConst x2, And, Latch) + SimBroker into a raw-index FlatGraph over one synthetic Frankfurt session, pinning the whole vocabulary composed end-to-end: - held = [0,0,1,1,0] over the 5 bar emissions: flat, latched bar3-close (the strict breakout landing on session bar 3) through bar4, exit at bar5-close; - equity = [0,0,0,3,8] pips (SimBroker lagged-exposure integration, pip=1.0); - a no-entry control (bar3 close == bar2 high) pins the strict-> semantic; - two disjoint runs byte-identical (C1). Both spec traps exercised (bar6 rollover closes bar5; bars 1-2 warm Delay[1]). Phase-1 C9 deliverable; build-step 8 (capstone) of milestone 'Strategy node vocabulary I'. closes #91 |
||
|
Brummel
|
f1d0bf00ec |
fix(aura-engine): unify RunReport stdout JSON with the on-disk serde shape
RunReport had two divergent JSON encoders: the registry serialized to disk via serde_json (params as an array-of-pairs, floats as 2.0), while stdout rendered through a hand-rolled to_json (params as an object, whole floats as 2). The same record was a structurally different document on disk than on the wire, and no test pinned their relationship — most visibly, `aura runs list` read a record from disk via serde and reprinted it in a different shape. Retire the hand-rolled writer: RunReport::to_json now delegates to serde_json::to_string(self), so stdout is byte-identical to the runs.jsonl line for the same record. The hand-rolled json_str helper is deleted (the graph_model.rs json_str is a separate symbol, untouched — the graph-model JSON is out of scope, #58/#28). A new pin test, to_json_equals_serde_disk_shape, asserts to_json() == serde_json::to_string(&report) so the two encoders can never silently diverge again. Observable change: stdout `params` is now an array-of-pairs and finite f64 fields carry a fractional part (2 -> 2.0); the nested envelope, field order, window [from,to], and integer-valued seed/exposure_sign_flips are unchanged. The on-disk shape does NOT change — only stdout moves to match disk. serde_json is promoted from a dev-dependency to a normal dependency of aura-engine (to_json is non-test code). Under the amended C16 per-case policy this passes: serde_json is a vetted standard crate already in the workspace (registry, ingest), deterministic (C1-safe, already relied on for the disk path), and is exactly "the vetted standard crate doing what the code would otherwise hand-roll" — it removes the last hand-rolled RunReport JSON writer. Goldens flipped to the serde shape: the engine canonical-form golden, the aura-cli sweep golden (cli_run.rs), and the aura-cli odometer-order sweep golden (main.rs) — the last was not in the plan's inventory; the cargo test --workspace gate surfaced it and it took the identical transformation. Structural CLI asserts and the r1.to_json()==r2.to_json() determinism asserts are shape-agnostic and stay green. Spec docs/specs/0033, plan docs/plans/0033, boss-signed (unanimous 5-lens panel). Two stale prose cross-refs (graph_model.rs:30, docs/design/INDEX.md:729) deferred to cycle-close audit. Gates: cargo build/test --workspace green, clippy --all-targets -D warnings clean. closes #54 |
||
|
Brummel
|
eec876975c |
feat(engine): serde foundation + amend C16 dependency policy (cycle D / iter 1)
Iteration 1 of the run-registry cycle (#33): lay the dependency + serde foundation the registry's typed read-path needs, and amend the contract that forbade it. Contract change (load-bearing — C18/C16). C16's blanket "zero-external- dependency by commitment" + "aura-ingest is the sole external-dependency firewall" framing is struck and replaced by a considered, per-case dependency policy: dependencies are admitted by deliberate review (what a crate pulls in vs. what it buys), with particular scrutiny for anything entering the frozen deploy artifact (C13); standard vetted crates (serde, rayon, ...) pass that review and are used wherever they do the job, including in the bot; hand-rolling what a vetted crate does is the anti-pattern. C16's engine/project split + three-tier node reuse are unchanged. The five source comments that asserted the struck framing (aura-engine/aura-ingest Cargo.toml + aura-ingest/report.rs docs) are rewritten to match; no live source or ledger text still asserts it. Per-case justification for serde (the policy now requires one): it gives the run-report types a typed (de)serialization path — exactly what ranking/compare over stored runs needs, and what the writer-only hand-rolled JSON (C14) could never provide; closes the typed-read-path gap (#17). Its closure is tiny, ubiquitous, heavily audited, and its output deterministic (C1-safe). serde_json is test-only this iteration (dev-dependency); no production serde_json yet. Changes: [workspace.dependencies] serde (derive) + serde_json; serde derives on Timestamp (aura-core) and RunMetrics/RunManifest/RunReport (aura-engine), with serde_json round-trip tests (window renders as a [from,to] integer array via the transparent Timestamp newtype). The hand-rolled to_json writers are untouched (still drive stdout). No aura-registry crate, no SweepPoint change — those are iterations 2 and 3. Verified: cargo test --workspace green (incl. the two new round-trip tests); clippy --workspace --all-targets -D warnings clean; no surviving assertion of the struck zero-dep framing in live source or the ledger. refs #33 |
||
|
Brummel
|
63ea7eb3b1 |
audit: cycle 0011 tidy — record the external-dependency firewall invariant
Architect drift review (f68258b..HEAD) found no code drift — the firewall is
real at the dependency-graph level (aura-core/std/engine/cli carry no external
dep; the chrono/regex/zip tree is confined to aura-ingest), and C3/C7/C1 are
faithfully realized. Four ledger-level items, all resolved here:
- [ledger-drift] The zero-external-dependency commitment — the load-bearing
rationale for making aura-ingest a crate rather than a feature-gate — lived
only in spec 0011. Recorded in the ledger: C16 now states the engine workspace
is zero-external-dependency by commitment, names aura-ingest as the ingestion
edge / external-dependency firewall, and forbids an external dep in any engine
crate other than aura-ingest.
- [ledger-drift] C16's reuse taxonomy did not place aura-ingest. C16 now lists
the non-node engine crates (aura-engine, aura-cli, aura-ingest).
- [ledger-debt] C12's eager-materialization-now / Arc<[T]>-sharing-later choice
was only in the spec. C12 now carries a "Status (cycle 0011)" note recording
the deliberate gap.
- [debt] aura-engine/Cargo.toml carried a stale comment ("data-server enters
when the ingestion task starts") that contradicted the firewall it should
protect. Replaced with the dependency-pure / firewall-in-aura-ingest note.
The External components data-server bullet is updated to reality (pulled in by
aura-ingest; workspace now resolves from a populated cargo cache, not fully
offline).
Regression gate: no-op (commands.regression empty); architect is the sole gate.
Cycle 0011 is drift-clean. (Drift-clean, not a milestone close — the
Walking-skeleton milestone close additionally needs its end-to-end milestone
fieldtest, a separate deliberate act.)
|
||
|
Brummel
|
dd5c3fad96 |
feat(engine): the deterministic single-source sim loop
Cycle 0003: aura-engine's first real content — a Sim that runs a wired DAG of
nodes deterministically, cycle by cycle. First point in the project where
authored nodes execute against data, not just under a hand-fed Ctx.
- `Sim` (aura-engine) — a bootstrapped, frozen root graph: a flat Vec<NodeBox>
(each node owns its input columns, the cycle-0002 shape) + an index edge
table, topologically ordered (Kahn). `bootstrap` sizes every input column from
its node's schema, kind-checks each edge and source target, and rejects
directed cycles — C7's "type check paid once at wiring" generalized to the
whole topology (`BootstrapError::{KindMismatch, BadIndex, Cycle}`).
- `run` — the deterministic loop: per record, forward the source value into its
target slots, evaluate nodes in topo order, capture the observed node's output
at eval time, and forward each `Some` output into its consumers' input columns
(a `None` forwards nothing — the structural seed of sample-and-hold). The loop
destructures `&mut self` into disjoint field borrows and allocates nothing on
the per-cycle path. This is the flat, monomorphized sharpening of RustAst's
reference-counted, interior-mutable observer push graph (C1/C7).
- `Edge` / `Target` (aura-engine) — producer->consumer and source->consumer wiring.
- `Sub` (aura-std) — a 2-input f64-difference node, so the loop is proven on a
real fan-out + join DAG (source -> {SMA(2), SMA(4)} -> Sub), with a determinism
assertion (C1: a second identical run is bit-identical).
Engine library depends only on aura-core; a test-only dev-dependency on aura-std
lets the integration test wire real Sma/Sub nodes. Sim carries a hand-written,
node-opaque `Debug` impl (Box<dyn Node> is not Debug; the impl prints
nodes.len() + the index/topology fields, needed by the bootstrap-rejection tests'
`unwrap_err`).
Deliberately deferred (recorded as decisions): freshness-gated recompute /
sample-and-hold (C5 -> cycle 0004, with the second source that makes it testable);
the explicit monotonic cycle_id counter (its first reader is freshness, 0004);
the Source trait + data-server ingestion + k-way merge (C3/C11); the builder API
(C19); the real sink + run registry (C18/C22).
Gates green: cargo build/test (26: 18 aura-core + 3 aura-std + 5 aura-engine)/
clippy -D warnings clean; surface-purity grep (no dyn-Any / Rc / RefCell) clean.
refs walking-skeleton
|
||
|
Brummel
|
744c2f31a8 |
chore: scaffold aura workspace and wire the skills dev-cycle
Cargo workspace aura-core -> aura-engine -> aura-cli (bin `aura`) plus nodes/ for hot-reloadable cdylib node crates. Crate bodies are intentionally API-free; types arrive from the first spec. Adds the skills profile (.claude/dev-cycle-profile.yml), the project CLAUDE.md with the eight domain invariants, and the design-ledger skeleton. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |