a51789929762257ea51375bf9d552a9854975a6d
43 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
a2959050a4 |
feat(0076): deflate the sweep winner's metric for the number of trials
Adds optimize_deflated beside optimize in aura-registry: the in-sample walk-forward selection now records, on each OOS winner's manifest, how much the winner's metric is inflated by the size of the search it won — a deflated score and (R arm) an overfit probability — without changing which member wins (additive, C23; a regression test pins optimize_deflated's winner == optimize's). - R arm (sqn_normalized/expectancy_r/...): a centred moving-block reality-check. Each member's per-trade R series is mean-subtracted to impose the no-edge null, resampled via the r_bootstrap kernel (extracted to a shared resample_block, byte-identical), and the best-of-K null-max gives overfit_probability = (count(null >= raw)+1)/(n+1) and deflated_score = raw - p95(null). Deterministic given the seed (C1). - total_pips arm: a closed-form expected-max-of-K dispersion floor (inv_norm_cdf + expected_max_of_normals), no probability. The record (FamilySelection on RunManifest) rides the proven serde(default, skip_serializing_if) widening + a compat.rs mirror, so legacy runs.jsonl/families.jsonl lines load unchanged and unstamped sweep/mc/standalone manifests stay byte-identical (C14/C18). The SelectionMode enum reserves a Plateau slot for #145. CLI: walk-forward stamps both arms; runs family <id> rank surfaces a human-readable deflated/overfit line. Quality review (re-dispatched after the implement-loop's quality gate exhausted) caught a real defect: on the R arm with positive resamples but no member carrying trade_rs (a family loaded from disk — trade_rs is serde-skipped), the null was all-NEG_INFINITY, turning deflated_score into +inf. Fixed by collapsing "no usable null" (zero resamples OR no trade_rs) into one degenerate floor (deflated == raw, overfit == 1.0); pinned by optimize_deflated_no_trade_rs_floors_instead_of_infinity. Verified: cargo test --workspace and cargo clippy --workspace --all-targets -D warnings both green. Frictionless Stage-1 R; n_eff advisory and reload-time recompute deferred (spec 0076 Out of scope). closes #144 |
||
|
Brummel
|
2e77ab1dda |
feat(0075): Monte-Carlo R-bootstrap for OOS validation (iter 2)
Add `aura mc --strategy stage1-r [--real <SYM>]`: run the stage1-r
walk-forward, pool its OOS per-trade R series, and report a moving-block
bootstrap E[R] distribution / CI (one `mc_r_bootstrap` line with an `e_r`
quantile block + P(E[R]<=0)). Completes spec 0075's Monte-Carlo half on top
of iter 1's walk-forward R-reporting.
Engine: `r_bootstrap(rs, n_resamples, block_len, seed) -> RBootstrap` — a
deterministic moving-block bootstrap (non-circular, final block truncated;
block_len=1 = i.i.d. trade-shuffle; clamped to [1,n]; empty -> all-zero),
reusing the existing MetricStats/quantile and SplitMix64.
CLI: parse_mc_args/McArgs split — bare/`--name`/`--trace` route to the
unchanged synthetic seed-resweep `run_mc` (byte-for-byte preserved); the R
path accepts ONLY `--strategy stage1-r`, so a bare `--real` stays a usage
error (mc_rejects_real_flag_with_usage_exit_2 preserved, doc-comment
refreshed). run_mc_r_bootstrap reuses walkforward_family + RMetrics.trade_rs.
Corrective fixes over the plan snippets, both verified: the mc usage string
carries a `usage:` token (the new `["mc", rest @ ..]` arm now handles the
`--real` rejection, and the preserved golden asserts `stderr.contains
("usage")`); `#[allow(clippy::large_enum_variant)]` on McArgs keeps the
unboxed Stage1RGrid field layout.
Scope: single `--real <SYM>` per invocation; cross-symbol pooling is a
deferred follow-on (needs a multi-symbol grammar). Verified: cargo build
clean, full `cargo test --workspace` green (609 passed), clippy -D warnings
clean; bare `aura mc`, SMA walkforward, stage1-r single-run, and C18
round-trip goldens all preserved.
refs #139
|
||
|
Brummel
|
f286bb85f7 |
feat(0075): walk-forward strategy-selectable + R-reporting (iter 1)
Make `aura walkforward --strategy stage1-r [--real <SYM>]` roll IS->OOS windows, sweep the stage1-r grid in-sample, pick the winner by an R metric (sqn_normalized), run it out-of-sample, and report per-window + pooled OOS R-metrics. The bare SMA walkforward path is byte-identical (its dispatch arm is verbatim today's body; the pooled `oos_r` block is emitted only when a window carries an `r` block). Engine: - RMetrics gains an in-memory `trade_rs: Vec<f64>` (realised R per closed trade), excluded from serde (`#[serde(skip)]`) and from a hand-written PartialEq, so the C18 wire shape and every existing equality assertion / round-trip stay unchanged. summarize_r now retains the per-trade R vector it used to drop. - New `r_metrics_from_rs(&[f64])` reduces a flat (pooled across-window) R series to RMetrics. Its R-distribution arithmetic is copied verbatim from summarize_r (the byte-pinned floats must not be algebraically refactored); the two copies are guarded in lockstep by a cross-reducer equality test. net_expectancy_r = expectancy_r (exact at the Stage-1 cost=0 invariant); conviction_terciles_r = [0,0,0] (per-trade conviction is not pooled). CLI: - walkforward gains --strategy + the four stage1-r grid flags (reusing parse_csv_list / Stage1RGrid); walkforward_family is strategy-dispatched. - Windowed stage1-r helpers: stage1_r_sweep_over (reduce-mode folded IS sweep, O(trades)/member) and run_oos_r (non-reduce OOS run, for the stitched pip-equity curve), plus stage1_r_space. Frictionless Stage-1 R (costs are Stage-2). Verified: cargo build clean, full `cargo test --workspace` green (SMA walkforward, synthetic mc, stage1_r_single_run_output_golden, and the C18 round-trips all preserved), clippy -D warnings clean. Monte-Carlo R-bootstrap is iter 2. refs #139 |
||
|
Brummel
|
7aaa3e5d5c |
feat(0068): position-event derive — book first-difference table (#115)
Add `derive_position_events(record, instrument_id) -> Vec<PositionEvent>` to aura-engine (report.rs), the broker-independent Stage-2 audit layer of C10: the first difference of the executed book derived from a PositionManagement dense record. A pure post-run reduction, sibling of summarize_r — same positional, type-erased Scalar read of the 14-column record, no in-graph node (the hot path stays domain-free, C14). A reversal emits Close then the opposite open at one event_ts (close first); a window-end open position emits its open with no synthetic Close (the table records actual executed events — unlike summarize_r, which force-closes for the R metric). instrument_id is a caller-supplied scalar (aura-engine depends only on aura-core; it cannot import InstrumentSpec). Two r_col indices added (DIRECTION=4, SIZE=10), the lockstep guard in stage1_r_e2e.rs extended to name-pin `direction`, and an agreement E2E folds both summarize_r and derive_position_events over one recorded ledger (one Close per closed round-trip; every open closed-or-open-at-end; referential integrity). Scope: the derive only. Fixed-fractional / currency / equity-feedback sizing is #116 (it owns the equity->Sizer z^-1 register per C10) — deferring it here avoids a circular dependency. Supersedes the rolled-back 0064 exposure-integral derive (abandoned per #117): this derives from the executed book, never exposure deltas. Verified by the orchestrator: cargo build + clippy --all-targets -D warnings clean; full workspace suite 0 failed; aura-engine lib 214 pass (incl. the 6 derive_* unit tests); stage1_r_e2e 11 pass (incl. the new E2E + extended guard). Fork decisions recorded on #115. closes #115 |
||
|
Brummel
|
1a6eafa056 |
refactor(composites): extract Stage-1 composite-builders into their own crate
Dissolve the aura-engine -> aura-std dependency by moving the vol_stop /
risk_executor composite-builders out of aura-engine into a new aura-composites
crate. This restores the engine's domain-free layering, which the
composites-in-engine placement (cycle 0065) had inverted.
Why: the engine routes type-erased Scalar records and never names a concrete
node -- summarize_r reads the PositionManagement record positionally, by column
index, without linking aura-std (which is exactly why aura-std was a dev-only
dependency before 0065). Parking the node-naming composite-builders inside
aura-engine forced the engine's production code to reference the node library
for the first time (aura-engine -> aura-std), an upside-down layering: the
engine is the foundation the node library builds upon, not the reverse.
The clean home is a dedicated crate that depends on BOTH the engine's
GraphBuilder and the std nodes, leaving each lower crate untouched:
aura-composites -> { aura-engine, aura-std } -> aura-core (acyclic)
aura-engine -> aura-core only (domain-free again)
aura-std -> aura-core only (lean node library)
aura-std reverts to a [dev-dependency] of aura-engine (the engine's own E2E
tests -- stage1_r_e2e, random_sweep_e2e, ger40_breakout -- still drive real
std nodes through a bootstrapped Harness; that test-only edge creates no cycle).
Moved with history (git mv):
- crates/aura-engine/src/composites.rs -> crates/aura-composites/src/lib.rs
- crates/aura-engine/tests/risk_executor.rs -> crates/aura-composites/tests/
- crates/aura-engine/tests/vol_stop_composite.rs -> crates/aura-composites/tests/
Consumers rewired: aura-cli imports risk_executor / StopRule from aura_composites;
the moved tests import the builders from the crate under test. The intra-doc link
to RollMode and the module rationale doc are updated for the new home.
Behaviour-preserving: workspace build clean; clippy --all-targets -D warnings
clean; full suite 513 passed / 0 failed, identical to baseline (the two moved
test files run unchanged under aura-composites); cargo doc --no-deps clean (no
broken intra-doc links).
|
||
|
Brummel
|
a6fc48daa7 |
feat(stage1-r): operate the R layer from the CLI (iter 3)
`aura run --harness stage1-r [--real <SYM>] [--trace <n>]` now runs a Stage-1 R
backtest and prints its signal-quality metrics (the R block) in the RunReport
JSON - one shell call, the research loop's primary verb, ergonomic for Claude
to drive.
What shipped:
- vol_stop + risk_executor promoted from test fixtures to public aura-engine
composite-builders, with a StopRule{Fixed,Vol} axis (C11). aura-std becomes a
normal dependency of aura-engine (the composites are the engine's convenience
layer over the standard nodes - the sibling tier of summarize_r; the graph
stays acyclic: aura-engine -> aura-std -> aura-core). Both fixtures call the
public fns; a Vol-backed RED test pins the new match arm.
- stage1_r_blueprint fans one bias into BOTH a SimBroker (pip) and the
RiskExecutor (R), so one report carries both yardsticks honestly. run_stage1_r
folds summarize (pip) + summarize_r (R, round_trip_cost 0.0 - Stage-1 is
frictionless signal quality) and sets RunMetrics.r = Some(..). An r_equity tap
(cum_realized_r + unrealized_r via LinComb) persists through a separate 3-tap
persist_traces_r and renders via the existing `aura chart --tap r_equity`.
- `aura run --harness <sma|macd|stage1-r>`: a parse_run_args tokenizer replaces
the five `run` literal-slice arms so --harness/--real/--from/--to/--trace
compose in any order; --macd kept as a back-compat alias; --harness sma the
default. A compile-time match over Rust-authored built-ins - not a runtime node
registry, not a DSL (C9/C17): the CLI runs an authored harness, it does not
wire one.
Back-compat: --harness sma/macd leave RunMetrics.r = None, so skip_serializing_if
keeps pip-only and legacy runs.jsonl JSON byte-unchanged; the plain-run tap list
stays ["equity","exposure"].
Refactor beyond the plan (verified behavior-equivalent): the old standalone
parse_real_args was orphaned by the new tokenizer, so it and its now-redundant
unit test were removed and run's --real parsing inlined into parse_run_args. The
--real strictness (empty-symbol, non-i64 ms, repeated-flag, window-requires-real
-> exit 2) is preserved and now pinned by
parse_run_args_rejects_malformed_real_and_repeated_flags (added to restore the
deleted coverage). All five run_real integration tests stay green.
Verification: cargo build/test --workspace green (every crate, 0 failed); clippy
--workspace --all-targets -D warnings clean. The full diff was adversarially
reviewed (run-arg refactor equivalence against the removed function, harness
wiring, C1/C2/C9/C17, the serde back-compat pin) - verdict SOUND; the one flagged
coverage gap is closed here.
Follow-on (noted, not done): the stage1-r manifest reuses the
"sim-optimal(pip_size=...)" broker label; a dedicated "risk-executor" label would
read more honestly for a dual-tap harness that runs a RiskExecutor branch.
closes #129
refs #117 #128
|
||
|
Brummel
|
2c43296c2c |
feat(stage1-r): bias rename + stop-rule + position-management + summarize_r (iter 1)
Iteration 1 of the Stage-1 "R-based signal quality" cycle (spec 0065): a strategy's signal quality, measured in R on its unsized bias stream, feed-forward. New discrete-trade machinery, NOT a SimBroker extension. What lands: - exposure -> bias (refs #126): the Exposure node/type/file/output-field renamed to Bias (computation unchanged: clamp(signal/scale,-1,+1); the SEMANTICS change — the output is the unsized strategy bias, sizing leaves the strategy). Scoped to the semantic core; behaviour-preserving cosmetics are deferred (see below). - stop-rule nodes (refs #119): VolStop = k*EMA(|price - prev_price|) (one fused node; the volatility that defines 1R, close-to-close — true-range ATR deferred, needs OHLC) + FixedStop (constant, the test fixture / fixed-vs-vol structural-axis sibling). - PositionManagement (refs #127): the stateful heart. Latches the entry-cycle stop distance as the FROZEN R-denominator (never re-read), marks/exits no-look-ahead like SimBroker (a position earns from the next cycle; an exit realises against the cycle's close), and emits ONE dense per-cycle R-record (C8). Stop-outs are NOT capped at -1R (a gap through the stop realises R < -1 — the honest loss tail); R is computed size-invariantly (size = (exit-entry)*dir / latched_dist cancels). The trade ledger is the rows where closed_this_cycle; the R-equity is cum_realized + unrealized; a position open at window end is the last row (open=true) — explicit, never silent MtM. - summarize_r + RMetrics (refs #129): the post-run fold (NOT an in-graph node — recorders drain post-run). E[R], win-rate, profit-factor, avg win/loss R, by-trade max R-drawdown, n_open_at_end (window-end force-close). SQN / conviction terciles / net-of-cost / RunMetrics.r are iteration 2. Design adversarially hardened before implementation (12-juror refute panel; decisions on #117). Ratified implementer deviations from the plan snippet, verified by hand: - col-4 `direction` tracks the OPEN position at cycle end (window-end synthesis; names the reopened leg on a reversal), falling back to the closed trade's dir only when flat. summarize_r does not read col 4; the R-metrics are unaffected. - .named("exposure") kept on the 4 previously-unnamed Bias nodes so the auto-derived param-path stays exposure.scale (no manifest/dir-name drift) — the unnamed nodes would otherwise flip to bias.scale. - intra-doc links [`Exposure`] -> [`Bias`] in sim_broker/latch + the sample-model test pin (cosmetic, behaviour-neutral; broken intra-doc links fail cargo doc). - the E2E drives a full bootstrapped Harness (stronger than the plan's direct-node drive — exercises the real cross-crate producer->consumer seam). Deferred (behaviour-preserving, separate cosmetic pass — NOT this iteration): RunMetrics.exposure_sign_flips / Metric::ExposureSignFlips, SimBroker/LongOnly "exposure" input ports, the "exposure" tap/trace names, and the .named("exposure") instance labels. Iteration 2: the Sizer seam, the RiskExecutor composite (+ the Veto documented-seam), summarize_r enrichment, RunMetrics.r, and the CLI/recording surface. Verification (orchestrator-run, not agent-claimed): - cargo build --workspace: clean. - cargo test --workspace: all green, 0 failed (incl. the new bias/stop_rule/ position_management unit tests, the summarize_r arithmetic tests, and the stage1_r_e2e capstone + layout guard). - cargo clippy --workspace --all-targets -- -D warnings: clean (exit 0). - Keystone RED tests pass: no_lookahead_bias_exit_realises_the_held_move, stop_out_is_not_capped_at_minus_one_r, no_gap_stop_is_exactly_minus_one_r, reversal_closes_one_leg_and_reopens, open_at_window_end_is_carried_on_the_last_row. refs #117 #119 #126 #127 #129 |
||
|
Brummel
|
b91c89f964 |
feat(broker-foundation): InstrumentSpec deploy metadata + PositionEvent schema
The realistic-broker milestone's (C10 A-side) two foundation value types — no broker, no derivation yet, fully tested. #113 — InstrumentSpec gains six deploy-grade fields (instrument_id, contract_size, pip_value_per_lot, min_lot, lot_step, quote_currency) and the vetted table is populated for GER40/FRA40/EURUSD/GBPUSD/USDCAD. The struct stays Copy (quote_currency is &'static str); the _ => None refuse-don't-guess arm is the permanent floor. This is the milestone's permanent authored floor (recorded-metadata tier is forward work, #124). #114 — PositionAction { Buy, Sell, Close } (closed enum, serde-encoded as a bare i64 0/1/2 via From/TryFrom) + PositionEvent in aura-engine beside RunMetrics, faithful to the C10 column spec: event_ts, action, position_id, instrument_id, unsigned volume; no open_ts; direction is the action (no signed-volume trick). Re-exported from the crate root. Derived fork decisions (recorded on #113): quote_currency &'static str; action as ordinal i64; stable instrument_id + overridable vetted values. Verified by the orchestrator: cargo test --workspace = 458 passed / 0 failed; clippy --all-targets -D warnings clean. An adversarial 4-lens review (ledger faithfulness, downstream soundness, serde/wire robustness, test honesty) returned 3 SOUND + 1 CONCERN: quote_currency &'static str cannot hold a runtime-sourced currency at the #124 resolver — not a blocker (#115/#116 read only the authored floor), logged on #124 as a deliberate deferred design choice. The implementer's deviation from the plan snippet (vec![..] -> [..]) avoids clippy::useless_vec; semantics identical. Two tester-authored consumer-boundary E2E tests added beyond the inline units: instrument_spec_deploy_metadata.rs (public resolution seam) and position_event_table.rs (persisted bare-int wire shape + reversal table). closes #113 #114 |
||
|
Brummel
|
8bb5256041 |
feat(trace-charts): persist recorder taps to disk (iter 1)
Iteration 1 (persist half) of the visual face — spec 0056 / plan 0055:
tap data from a graph and save it to disk so it can later be charted
(the serve/chart half is iteration 2).
- ColumnarTrace (aura-engine): a drained tap <-> columnar SoA form
({tap, kinds, ts, columns}); values coerced to f64 for plotting with
the base type preserved in `kinds` (C7); to_rows rebuilds uniformly-f64
rows for the serve-side join. Pure (C1), beside join_on_ts/summarize.
- TraceStore (aura-registry): a per-run directory under
runs/traces/<name>/ — one <tap>.json per tap + an index.json (manifest
+ tap order), beside the families.jsonl sibling store. Missing run =
NotFound (no panic); corrupt file = Parse{file}.
- CLI: a shared persist_traces helper threaded into run_sample /
run_macd / run_sample_real via an orthogonal `trace: Option<&str>`;
new arms `aura run [--macd] --trace <name>` and `--trace <name>` on
`run --real`. The default (no --trace) run is byte-for-byte unchanged.
Verified by the orchestrator: `cargo test --workspace` green (incl. the
4 columnar_trace, 3 trace_store, 4 cli_run e2e, and the 4-tuple
parse_real_args tests); `cargo clippy --workspace --all-targets -D
warnings` exit 0. Implementer deviations folded in, all
compiler-prescribed: #[derive(Debug)] on RunTraces (the RED tests
panic-format a Result over it), #[allow(clippy::type_complexity)] on
the parse_real_args 4-tuple (mirrors the crate's sample_harness idiom).
Default-run byte-identity stays pinned by run_prints_json_and_exits_zero.
refs #101
|
||
|
Brummel
|
74324d178f |
feat(aura-engine): join_on_ts — timestamp join for multi-tap sink traces
Ship a generic, spine-anchored join_on_ts helper + JoinedRow in the report layer (sibling of summarize/f64_field), re-exported from lib.rs. It fuses N recorded (Timestamp, Vec<Scalar>) tap streams on the recorded timestamp: exactly one JoinedRow per spine entry, each side looked up by ts as Some(row)/None where it did not fire. Option-per-side semantics keep the engine honest — it reports presence; the consumer interprets absence (the 0.0/-1/false defaults are consumer truths, not engine ones). C8/C18 (post-run reduction over recorded sink output), C3 (no in-graph join), C1 (one row per ts, documented precondition). Two unit tests pin the contract: cardinality-misalignment alignment (the #93 shape — a spine bar a side tap skipped resolves to None, not a zip-misalign) and the spine-anchored drop of a side row at a non-spine ts. RED accepted on the grounding-check record (both symbols were absent pre-cycle, so the tests are RED-by-construction) and verified green here. drain_trace's migration onto this helper + the Recorder doc note follow. refs #93 |
||
|
Brummel
|
91ef69ed7b |
feat(aura-engine): RandomBinder — by-name random param-sweep
Add `RandomBinder`, the by-name sibling to `SweepBinder`, so a random sweep (`RandomSpace`, C12.1) can be built by name against `param_space()` instead of by positional `Vec<ParamRange>` slot order. By-name resolution makes a same-kind transposition (e.g. swapping the I64 ranges for `fast.length` and `slow.length`) structurally impossible — the failure class `RandomSpace::new`'s positional validation passes silently. Direct structural mirror of `SweepBinder`: `Composite::range` opens the binder, `.range(name, ParamRange)` accumulates, `.sweep(count, seed, run)` resolves the named ranges via the shared `resolve_into` (new `resolve_ranges` caller) and runs the disjoint sweep. New `BindError::EmptyRange` mirrors `EmptyAxis`; `ParamRange::is_empty` homes the non-empty invariant the named layer pre-checks so `RandomSpace::new` cannot fail. closes #79 |
||
|
Brummel
|
e17d78f24f |
feat(aura-engine): random param-sweep — RandomSpace + a Space trait + typed ParamRange
Ship the random half of the C12.1 param-sweep axis (grid landed in 0028).
A new `Space` trait (`points`/`param_specs`) generalizes `sweep`/`sweep_with_threads`
from `&GridSpace` to `&S: Space`; `GridSpace`'s trait impl forwards to its existing
inherent methods, so the grid path is behaviour-preserving (C1, every pre-existing
grid sweep test stays green). `RandomSpace` is the second `Space` producer: N seeded
uniform points over per-slot typed `ParamRange { lo, hi }` ranges (I64 inclusive
[lo,hi], F64 half-open [lo,hi)), validated in `new` against the param-space — a
non-numeric slot, a range-kind mismatch, and an empty range are the three new typed
`SweepError` variants, all caught before any run. Sampling reuses the existing
bit-stable `SplitMix64` (promoted to `pub(crate)`) as a code-path-disjoint instance
from the data-edge seed RNG — the #52/#71 World-II source-seam firewall: they share
only the `u64` type, never a path. The sweep-layer signature stays param-only
(`Fn(&[Cell]) -> RunReport`); no `Source`/stream type enters it.
Deviation from the spec's verbatim sampler, accepted as a correctness hardening:
the I64 draw guards the full-width span ([i64::MIN, i64::MAX] computes a span of
2^64 that wraps a u64 to 0) and uses `lo.wrapping_add(draw as i64)` instead of a
plain `+`. This avoids both the `% 0` divide-by-zero at the full domain and the
signed-overflow panic the literal form hits on wide ranges in debug builds, while
preserving uniform-over-[lo,hi] draws and determinism (an extra RED test,
random_points_full_range_i64_does_not_panic, pins it). Modulo bias for spans not
dividing 2^64 remains the spec's documented, accepted simplification (param search
needs no cryptographic uniformity).
Includes a public-API E2E suite (tests/random_sweep_e2e.rs) exercising the feature
exactly as a downstream researcher would — reproducibility at the report boundary,
seed-sensitivity, in-range draws, the typed NonNumericRange gate, a well-formed
empty (count==0) family, and Grid/Random interchangeability through `Space`.
Verified: cargo test --workspace green (incl. 17 new engine tests + 6 E2E);
cargo clippy --workspace --all-targets -- -D warnings clean.
closes #52
|
||
|
Brummel
|
cd3d1ca9ed |
refactor(aura-core): split Scalar into a tag-free Cell + ScalarKind
Motivation
----------
`Scalar` was a tagged enum (I64/F64/Bool/Ts), so every scalar value
physically carried its own kind tag. But the kind is already known from
the schema/port/column the value flows through (C7: the type is a
property of the column, not of the value — the hot path is already
columnar `Column<T>`, and `AnyColumn::get` *reconstructs* the tag from
the column on the way out). The per-value tag was therefore redundant
with the kind the surrounding context already holds.
That redundancy had three costs:
* It baked an implicit `match` (a branch) into every function that read
a Scalar payload — even where the caller statically knew the type.
The tag could never be exploited away.
* Size: a tagged enum is tag + payload = 16 bytes (f64/i64 alignment),
twice the 8 bytes the value needs. A `Column<Scalar>` would be double
the memory and half the cache utilisation.
* It is the shared root of several downstream papercuts we keep hitting
— the lossy f64 manifest field, the `unreachable!` panic on a
non-numeric param, the serde-tag question — all symptoms of "the type
is baked into the value".
Change
------
Introduce `Cell`: a type-erased 64-bit word (`struct Cell(u64)`) that is
not readable without external type context. It is constructed per base
type (`from_i64/from_f64/from_bool/from_ts`) and read only by naming the
type at the call site (`i64()/f64()/bool()/ts()`) — each a branch-free
bit-cast. The hot path resolves the kind once at the boundary (from the
schema) and then reads natively, with no per-value branch. `Cell` knows
nothing of `Scalar` or `ScalarKind`; the dependency is strictly one-way,
and it lives in its own `cell.rs` (more is planned on top of it).
`Scalar` becomes `struct { kind: ScalarKind, cell: Cell }` — the
self-describing form for the dynamic boundaries (builder binding,
serialization, rendering), built on top of `Cell`. Its `as_*` accessors
now `debug_assert` the kind and return the native value (free in
release); calling the wrong accessor is a caller bug, not a checked
`Option`. The variant constructors `Scalar::I64(..)` become associated
fns `Scalar::i64(..)`.
`PartialEq` is hand-written (not derived) to preserve the former enum's
value semantics: kinds must match, then native payloads compare, so f64
keeps IEEE-754 behaviour (`NaN != NaN`, `+0.0 == -0.0`) and a kind
mismatch is never equal even when the raw words coincide. A fixture
(`scalar_eq_is_value_not_bitwise`) pins exactly the cases where bit- and
value-equality diverge, so it can't silently regress. `Cell`'s own
`Eq`/`Hash` stay bitwise — correct for a raw word.
The change is behaviour-preserving: Scalar's observable behaviour is
identical to the pre-Cell enum (the value-equality fixture proves it);
only the internal representation changed. The ~440 call sites across the
workspace are a mechanical constructor rename plus ~12 destructuring
sites (match-arms / `let`-patterns) rewritten to `kind()` + `as_*`.
Verified: cargo build --workspace --all-targets, cargo clippy --workspace
--all-targets -- -D warnings, cargo test --workspace — all green.
|
||
|
Brummel
|
1d9555c468 |
feat(aura-engine,aura-registry): producer-supplied window + registry lineage (0045 iter 1)
Iteration 1 of spec 0045 — the engine + registry core for storing orchestration families as linked records. aura-engine: add Source::bounds() — the inclusive (from, to) data extent a producer will stream, known without materialization (#71 firewall) — and a free window_of() that folds the union extent across a run's sources. This is the producer-supplied replacement for the prices.first()/.last() Vec scans at the CLI call sites (those migrate in iteration 2); a lazy producer now reports its window so stored lineage stays byte-identical whether a source is eager or streamed. aura-ingest: M1FieldSource stores its requested [from_ms, to_ms] window at open and reports it via bounds() (normalized to epoch-ns; None when open-ended — the archive-extent query is a deferred non-goal). aura-registry: new lineage module — FamilyKind / FamilyRunRecord (a RunReport stamped with family_id + kind + ordinal) / Family, plus Registry::append_family (assigns family_id = "{name}-{counter}" via a per-name generate-and-check counter, writes members to a sibling families.jsonl) and load_family_members; group_families re-derives the families (the round-trip), and the three *_member_reports extractors pull the per-kind member reports. The flat runs.jsonl store and its append/load/rank_by/optimize API are byte-for-byte unchanged (a test pins the two stores' disjointness). C9 preserved: aura-engine gains no registry dependency. New dep: serde derive on aura-registry (per-case policy, same basis as serde_json already is). Verification: cargo build --workspace; cargo test --workspace (228 green, incl. 3 new engine bounds/window_of, 1 data-gated ingest bounds run against real archive data, 5 new registry lineage round-trip/counter/disjointness tests); cargo clippy --workspace --all-targets -D warnings clean. The CLI surface (aura mc, family-aware aura runs, the window_of migration) is iteration 2. refs #70 |
||
|
Brummel
|
4764656062 |
feat(aura-engine): walk-forward family — WindowRoller + walk_forward (C12 axis 3)
The third C12 orchestration axis: walk-forward varies the data window. A
WindowRoller is a pure iterator of WindowBounds { is, oos } — bounds only, never
tick data (#71 firewall). walk_forward runs a caller closure per split disjointly
over the shared run_indexed core (C1), and stitches the OOS pip-equity segments
into one continuous curve (each segment offset by the running sum of prior
segments' finals; an empty segment contributes 0.0, mirroring summarize). C2
no-look-ahead is structural: the roller only ever emits oos.0 == is.1 + 1, pinned
by a zero-tick bounds test. The in-sample optimize (axis 2) is closure-supplied,
not called by the engine: aura-engine has no aura-registry dependency (C9), and
C12 forbids baking search policy into the primitive — the CLI bridges both crates.
Param-stability is on-demand, not stored (the R2 decision, recorded with
provenance on #69): WalkForwardResult stores only the raw per-window outcomes +
the stitched curve, and param_stability(&result) -> Vec<MetricStats> is a public
reduction over the retained per-window chosen params. A stored summary would be
recomputable from the windows (redundant) and would force one statistic canonical
when "stability" admits several; this mirrors SweepFamily (raw points + on-demand
optimize), not McFamily (stored aggregate). MetricStats::from_values is extracted
from McAggregate::from_draws (behaviour-preserving — the 7 mc tests stay green)
and gains serde so the CLI summary renders it and #70 lineage can persist it.
`aura walkforward` runs a built-in rolling walk-forward over a synthetic windowed
source: per window it sweeps the built-in grid in-sample, optimizes by total_pips,
runs the chosen params out-of-sample, persists each OOS RunReport (C18), and prints
a stitched summary line.
Two plan deviations, both compiler-forced and consistent with siblings: added
#[derive(Debug)] to WindowRoller (the RED tests' unwrap_err needs Ok: Debug; the
sibling types already derive Debug); removed a now-redundant test-module
SyntheticSpec import after Step 2 hoisted it to the top-level use.
Verification: cargo test --workspace green (aura-engine 152 incl. walkforward 9 +
mc 9, aura-cli 14); clippy --workspace --all-targets -D warnings exit 0; cargo doc
--workspace --no-deps clean.
refs #69
|
||
|
Brummel
|
61a8436c2c |
audit: cycle 0043 — drift-clean (monte-carlo family)
Architect drift review over b920362..HEAD (the 0043 monte-carlo cycle). Cycle is sound: C12 axis 4 faithfully realized (monte_carlo varies the seed, the 0042 seeded source genuinely perturbs each run, so distinct_seeds_produce_distinct_draws is a real property); C1 honoured (one run_indexed core, output sorted by job index = input order independent of completion, 1-vs-8-thread determinism pinned, parallelism across draws never within); the run_indexed extraction is a genuine behaviour-preserving refactor (C11/C23 — the three pre-existing sweep tests stay green, and both family executors are thin adapters over the single core; the points.len() -> n.max(1) clamp is equivalent since GridSpace rejects empty axes); the #71 eager-agnostic firewall is respected (API takes seeds + a per-draw closure, no materialized stream Vec crosses the boundary). One medium doc-lag found and fixed here: aura-engine/src/lib.rs module header listed Monte-Carlo (and, already stale, the grid sweep) under "Still to come"; both orchestration axes ship now, so the header records the grid axis (sweep / SweepFamily) and the seed axis (monte_carlo / McFamily) as delivered, with walk-forward, random param-sweep, registry lineage, and the C10 position-event / broker layer remaining future. Regression gate (test suite — no dedicated regression scripts in project facts): cargo test --workspace green (incl. 7 new mc::tests + the sweep guard tests); clippy --workspace --all-targets -D warnings clean; cargo doc -p aura-engine clean (the new intra-doc links resolve). closes #68 |
||
|
Brummel
|
e2c550f2f9 |
feat(aura-engine): monte_carlo family — McFamily over a seed set (C12 axis 4)
`monte_carlo(base_point, seeds, run_one) -> McFamily`, the Monte-Carlo
orchestration family: a fixed base point run over a seed set, each seed a
disjoint C1 realization. C12 axis 4 — Monte-Carlo IS a sweep over seeds, so it
reuses the disjoint-parallel executor rather than forking a run loop.
The reuse is realized by extracting the index-parallel core out of
`sweep_with_threads` into a `pub(crate) run_indexed<T>` (generic over the
per-job result, over a 0..n job range); `sweep_with_threads` becomes a thin
adapter over it. Behaviour-preserving (C1/C11/C23): the three pre-existing sweep
tests stay green — they ARE the proof the refactor preserved behaviour. A small
incidental win: the adapter consumes `points` via zip, dropping the old
per-point clone.
mc.rs adds McDraw{seed,report} / McFamily{draws,aggregate} (draws in seed-INPUT
order, independent of thread completion) / McAggregate (mean + p5/p25/p50/p75/p95
of all three run metrics — the V1 decision: covers every metric, not a "chosen"
one) / MetricStats, a private type-7 linear-interpolation `quantile`, and
`McAggregate::from_draws` (a pure post-run reduction recomputable from the
retained raw draws). Eager-agnostic firewall (#71): the API takes seeds + a
per-draw closure `Fn(u64,&[Scalar])->RunReport`, never a materialized stream Vec;
seed->Source construction stays a closure-body concern. An empty seed set is a
debug-asserted precondition, preserving the no-Result `-> McFamily` signature.
7 RED-first mc::tests pin: one draw per seed in input order; determinism across
1 vs 8 threads (C1); draw == an independent seeded run; distinct seeds perturb
the family; aggregate == from_draws(draws); type-7 quantile/mean math on a known
fixture; quantile endpoints + singleton. Workspace green, clippy -D warnings
clean, cargo doc clean.
refs #68
|
||
|
Brummel
|
b9203624d3 |
audit: cycle 0042 — drift-clean (seed-as-input)
Architect drift review over 682e459..HEAD (the seed-as-input cycle plus the prior unaudited `aura run --real` work). Cycle is sound: Fork B respected (seed forks at the data-generation edge into the source and the manifest; the engine loop sees only a Source — C12/C1/C3 intact), the contract names Source never Vec (precondition for #68/#52 delivered as specced), and the four seed-free callers thread 0 in lockstep. One medium doc-lag item found and fixed here: aura-engine/src/lib.rs module-doc listed the seed axis under "still to come"; seed-as-input shipped this cycle and RunManifest.seed is now live, so the headline doc now records SyntheticSpec as a delivered seeded Source producer (params + orchestration families remain future). One low carry-on note (not drift): run_sample_seeded exists only under #[cfg(test)] — intended this cycle (no --seed CLI flag per spec non-goals); the only non-test seed value remains 0. A scaling marker for #68, not debt. Regression gate (test suite — no dedicated regression scripts in project facts): cargo test --workspace green; clippy --workspace --all-targets -D warnings clean; cargo doc -p aura-engine clean (the new [`SyntheticSpec`] intra-doc link resolves). |
||
|
Brummel
|
c9f0e438e1 |
feat(aura-engine): seeded source + live RunManifest.seed (C12 seed-as-input)
A seeded source whose stream is fully determined by a u64 seed, making RunManifest.seed a live captured input instead of the dead 0 it was. This is C12's reconciliation of stochastic runs with C1: a run is bit-identical for a fixed seed (the seed is a captured input, not hidden nondeterminism), and a different seed gives a different-but-reproducible run. Precondition for the Monte-Carlo family (#68) and random param-sweep (#52). Engine (aura-engine): - SplitMix64: a ~10-line private, dependency-free PRNG. Chosen over a rand-family crate for guaranteed bit-stability across toolchains/crate versions — the whole value of seed-as-input is reproducibility, and a crate whose algorithm is not version-stable would silently break a recorded seed on a dep bump. No new direct dep. - SyntheticSpec + source(seed) -> impl Source: the Fn(u64) -> impl Source contract (Fork B — seed at the data-generation edge, outside the engine graph). Returns a Source, never a Vec, so the MC family can re-seed N times without materializing N streams. First cut collects to a VecSource internally; the signature does not name Vec, so a lazy generator is a drop-in replacement. Re-exported from lib.rs beside VecSource. CLI (aura-cli): - sim_optimal_manifest gains a seed: u64 param recorded into the manifest (the single Fork B capture site). All four existing seed-free callers (run_sample, run_sample_real, the sweep grid-report, run_macd) pass 0 unchanged — their determinism tests stay green. - A test-only run_sample_seeded + SeededTrace exercise a live seed and expose the drained sink trace, so the three e2e tests assert bit-identity at the trace level (strictly stronger than the folded 3-field metrics). Tests (RED-first): 2 engine producer tests (same-seed identical / different-seed differs) + 3 cli e2e tests (bit-identical trace, different-seed metrics differ, seed recorded in manifest). Full workspace green; clippy -D warnings clean. Errata vs plan 0042 (two compile-forced deviations, neither alters the spec contract): - source returns `impl Source + use<>`: under edition 2024 a bare `impl Source` captures the &self lifetime, which blocks boxing as Box<dyn Source + 'static> for run() (E0597). The source owns its data, so use<> (captures nothing) is correct. - SyntheticSpec is imported inside `#[cfg(test)] mod tests` in main.rs: it is used only by the seeded test vehicle, so a top-level import is unused in the non-test build under clippy -D warnings. closes #66 |
||
|
Brummel
|
eec129ee51 |
feat(aura-engine): Source ingestion seam — run() takes Vec<Box<dyn Source>>
The engine's ingestion input was the only type encoding an eager-dataflow
assumption: `Harness::run(Vec<Vec<(Timestamp, Scalar)>>)` — one fully
materialized stream per source. At realistic research scale (20y, 3 tick
streams, ~7.9e9 ticks) that layout is ~190 GB resident, non-residable. Yet
the merge loop never needs the whole stream: it only ever peeks the head
timestamp of each live source and pops one record. So the eager Vec is
gratuitous.
This lands the first half of the Source seam (plan Tasks 1-2):
- A `Source` trait (`peek(&self) -> Option<Timestamp>`,
`next(&mut) -> Option<(Timestamp, Scalar)>`), object-safe so the merge
holds `Vec<Box<dyn Source>>`. The peek/next split is the producer contract
the k-way merge drives.
- `VecSource`, an adapter over the old `Vec<(Timestamp, Scalar)>` — the
cycle-0011 eager shape, named. It IS the old behaviour.
- `Harness::run` re-typed to `Vec<Box<dyn Source>>`; the merge loop's index
arithmetic becomes peek/next. C4 tie-by-source-index is preserved (the
strictly-`<` replace + source-order scan is byte-for-byte the old
semantics). The forward+eval body is unchanged.
- All 53 call sites threaded VecSource-wrapped in the same change, so the
workspace compiles atomically and run output is byte-identical (C1).
Behaviour-preserving is the load-bearing guarantee: the full suite is green
and unchanged (the two-run C1 determinism pairs — real_bars r1/r2, blueprint
sweep-equality, harness h/h2 — stay byte-identical); aura-engine unit tests
129 -> 132 (+3 VecSource unit tests). No residency change yet: VecSource holds
the same Vec as before. The lazy data-server-backed streaming source and its
end-to-end residency proof are the second half (plan Tasks 3-4).
refs #71
|
||
|
Brummel
|
a6a314bb98 |
feat(aura-engine): name-based blueprint wiring via GraphBuilder
Add an additive, fluent GraphBuilder authoring surface that wires a blueprint's topology by typed node handles and port/field names instead of raw positional indices. Node references are Copy NodeHandle values (a node reference cannot be mistyped); only port/field names are strings, resolved against each node's cached NodeSchema by exactly-one-match at a single fallible build() -> Result<Composite, BuildError> — the Binder posture, one level up. build() lowers to the unchanged Composite::new, so the compilat stays index-wired (C23): names resolve at the authoring boundary and never reach FlatGraph. A new From<Composite> for BlueprintNode lets add() accept nested composites. Verified: builder-authored sma_cross is structurally equal to the hand-wired fixture, and the full builder-authored harness lowers to a byte-identical FlatGraph (equal edges + sources). The five BuildError variants (Unknown/Ambiguous In/Out, BadHandle) are covered, and the SimBroker exposure/price legs are now addressable by name (the #21 legibility win). The structural close of the exposure/price swap (#21) is deliberately out of scope — this builder makes the swap legible, not impossible — tracked as #65. Two plan-test corrections applied during implementation (both in-scope, no behaviour change): the harness parity test uses compile_with_params with a topology-invariant param vector on both sides (the harness declares three params, so a no-param compile() would trip ParamArity), and the error asserts use .err()/Some(...) rather than unwrap_err() (Composite is not Debug). aura-engine 123 tests green (+9); full workspace green; clippy --all-targets -D warnings clean. Existing index-form Composite::new sites and tests untouched (coexistence). closes #64 |
||
|
Brummel
|
1f01badac7 |
refactor(aura-engine): share the SMA-cross harness test fixtures
Collapse the three verbatim-duplicated #[cfg(test)] fixtures — synthetic_prices, sma_cross, composite_sma_cross_harness — that lived in both blueprint.rs's and sweep.rs's test modules into one shared crate-root `#[cfg(test)] mod test_fixtures` (pub(crate) free fns). The two consumers now import from `crate::test_fixtures`; sweep.rs's test-mod `use` block sheds the imports that were only needed by the moved fns. Why: the topology had to move in lockstep across the two copies but no test pinned them equal, so an edit to one silently diverged the other (#53). One definition removes the hazard structurally — better than a pin-them-equal test, which loses its premise once there is a single definition. Test-only; no production code, public API, or runtime path changes. C16 untouched (a #[cfg(test)] module ships in no artifact). The CLI sample is no longer a third peer (cycle 0036 enriched it into a trend+momentum blend), so this is an engine-internal two-copy dedup, not the three-way the issue originally named. graph_model.rs keeps its own intentionally-minimal golden copy, out of scope. Plan glitch fixed in-flight: the plan's literal import list named sma_cross, but neither consumer calls it directly (only transitively via composite_sma_cross_harness), so importing it tripped the plan's own Step-7 clippy -D warnings gate. Trimmed to {composite_sma_cross_harness, synthetic_prices} — within the task's "drop now-unused imports" scope. Verified: cargo test --workspace green (aura-engine 114 passed, count unchanged — behaviour-preserving); the two load-bearing fixture consumers composite_sma_cross_runs_bit_identical_to_hand_wired (C1) and sweep_equals_n_independent_runs (C11/C12) both green; cargo clippy --workspace --all-targets -- -D warnings clean. closes #53 |
||
|
Brummel
|
1b7e4ad169 |
feat(aura-engine): param_space() injectivity as a compile invariant
A non-injective param_space() — two slots under one path-qualified name — is the by-name knob address space (C12/C19) being broken: no binding can select one slot without the other. This cycle makes it a structural compile error. One check, check_param_namespace_injective over the param_space() names, replaces the whole fan-in machinery (signature_of, leaf_has_param, check_fan_in_distinguishability, check_composite_fan_in) and runs from compile_with_params AND from both binders before resolve/resolve_axes — so the canonical by-name author sees the structural cure (CompileError::DuplicateParamPath, carrying the path and pointing at .named(...)) instead of the AmbiguousKnob symptom (#59). With an injective space guaranteed before resolve, no bound name can multi-match, so BindError::AmbiguousKnob is dead and removed (both resolver arms -> unreachable!). param_space() stays infallible; the invariant lives in the compile step (C23). The check is strictly the by-name-addressability property. The old fan-in predicate (equal signature AND >=1 param) also rejected two collisions that are NOT param_space duplicates — an asymmetric param/paramless collision and a role-vs-leg collision — which guarded render identity, dead since the renderer was retired in 0026. Both are intentionally dropped; a future node-/wiring-name check, if wanted, is decoupled from injectivity. The new invariant correctly rejected a pre-existing fixture (multi_output_composite's two unnamed Sma legs); cured in place with the .named(fast/slow) the invariant mandates. A new E2E test drives the cured cross through the by-name binder end-to-end (resolve + bootstrap + run to a populated exposure trace). C9/C12/C19/C23 ledger notes amended. Verified: cargo build/test --workspace green (0 failed), clippy --workspace --all-targets -D warnings clean. closes #59 |
||
|
Brummel
|
ffed8cc612 |
feat(aura-core,aura-engine,aura-cli): node-instance naming retires ParamAlias
Every blueprint node now carries a name. A primitive builder gains an optional
instance name (Sma::builder().named("fast")); when omitted it defaults to the
node's type label, ASCII-lowercased verbatim ("SMA"->"sma", "SimBroker"->
"simbroker"). param_space() is now uniformly <node>.<param> at every level
including the root (sma_cross.fast.length, exposure.scale). Fan-in
distinguishability (C9) and signature_of re-key onto the node name: two same-type
siblings both defaulting to "sma" collide and IndistinguishableFanIn fires, so
one act -- naming the legs "fast"/"slow" -- fixes both the naming collision and
the fan-in check. The index-addressed ParamAlias overlay, Composite.params, the
Composite::new 5th argument, aliases_on, and check_alias_indices are removed
(Role.name and OutField.name untouched). Ledger C9 and C23 amended.
This is why the change is correct, not merely convenient: blueprints are
value-empty (C11), so the thing that would otherwise distinguish two SMAs --
their length -- is not present at construction; identity must come from a name,
not a deferred value. Closes the #56 friction surfaced by the param-space &
sweep milestone fieldtest (a freshly-authored 2-SMA cross was unbindable and
would not bootstrap without two hand-counted ParamAlias overlays).
Two plan defects were corrected during implementation and verified:
- the three new fan-in/path tests authored sma_cross at root level, which would
qualify to "fast.length" (root prefix is empty) and is not the nested case the
fan-in check inspects; nesting sma_cross under a root (a shared
sma_cross_under_root helper) restores the asserted "sma_cross.fast.length" and
IndistinguishableFanIn{node:2};
- three cycle-0030 named-binder tests bound the real harness by the old names
(sma_cross.fast / scale) and were migrated to the new path strings, surfaced by
the workspace test gate.
Verified by the orchestrator: cargo build/test --workspace green (198 tests,
0 red), clippy --all-targets -D warnings clean. model_to_json emits the type
label + factory param names (node-name-independent), so sample-model.json and the
graph_model golden are byte-identical (untouched). NodeSchema gained a Default
derive (the builder default-name test constructs an empty schema). fieldtests/
are frozen non-workspace records, not migrated.
closes #56
|
||
|
Brummel
|
937257e368 |
feat(aura-engine,aura-cli): named param binding — sweep axes .axis()/.sweep() (0030 iter 2)
Bind a sweep's axes by name instead of by a positional GridSpace:
bp.axis("sma_cross.fast", [2, 3]).axis("sma_cross.slow", [4, 5]).axis("scale", [0.5]).sweep(run)
The sweep half of spec 0030, completing the named-binding feature (#35). A pure
authoring layer over the existing GridSpace::new / sweep primitives; engine core
untouched (C1/C12/C19/C23).
This iteration:
- resolve_axes(): shares resolve's name->slot mapping (iter 1), adds the EmptyAxis
check (the variant defined in iter 1 is now first constructed here) and a
per-element axis kind-check (every element of each axis, first offending element
in axis order). Its validation is a strict superset of GridSpace::new's
(arity / non-empty / per-element kind), so SweepBinder::sweep's downstream
GridSpace::new(...).expect(...) is infallible by construction — it can never
panic on author input.
- Composite::axis -> SweepBinder -> SweepBinder::sweep (Result<SweepFamily, BindError>).
- The CLI sample sweep converted to the named form (behaviour-preserving).
Plan correction (verified by the orchestrator): the iteration plan said only the
GridSpace import was orphaned by the CLI conversion, but replacing the free
sweep(&grid, ..) call with the .sweep(..) method also orphans the free `sweep`
import — both were dropped from aura-cli to satisfy clippy -D warnings. The Scalar
import (from aura_core) stays used and was untouched.
Verified (run by the orchestrator): 6 new aura-engine tests green —
resolve_axes round-trip, EmptyAxis, MissingKnob, the per-element KindMismatch on a
mixed axis (the superset/no-panic guarantee), a builder no-panic wrong-kind test,
and a GridSpace .points() parity test; the two sweep JSON goldens
(sweep_report_renders_four_points_in_odometer_order + the cli_run integration
golden) green unchanged (name/value/order-preserving conversion); full
`cargo test --workspace` green and `cargo clippy --workspace --all-targets
-- -D warnings` clean.
refs #35
|
||
|
Brummel
|
64b19ab3d5 |
feat(aura-engine,aura-cli): named param binding — single-run .with()/.bootstrap() (0030 iter 1)
Bind a blueprint's open knobs by name for a single run instead of by a positional,
Scalar-wrapped Vec in param_space() order:
bp.with("sma_cross.fast", 2).with("sma_cross.slow", 4).with("scale", 0.5).bootstrap()
A pure authoring layer over the existing Composite::param_space / bootstrap_with_params
primitives; the engine run loop and the construction core are untouched (C1/C12/C19/C23
preserved). Raw literals lower via Into<Scalar> (the literal fixes the variant: 2->I64,
0.5->F64, pinned by
|
||
|
Brummel
|
a665a966a7 |
feat(aura-engine): param-sweep grid primitive (the World, cycle C / iter 1)
Turn one value-empty blueprint into a family of disjoint instances over a
declared grid, run them in parallel, and collect per-point metrics. This is
C12.1's grid axis — the first "N instead of 1": #30/#31 made a blueprint
param-generic and bindable; this reaps it.
New module crates/aura-engine/src/sweep.rs, three parts:
- GridSpace: a validated cartesian grid over a blueprint's param_space() —
one discrete value-list per slot, kind/arity/non-empty checked at
construction (SweepError::{Arity, KindMismatch, EmptyAxis}, the typed gate
before any run). points() enumerates in odometer order (last axis fastest),
deterministic.
- sweep(): closure-driven. The engine owns enumeration + disjoint execution
+ collection; the author owns run_one: Fn(&[Scalar]) -> RunMetrics (build
fresh -> bootstrap_with_params -> run -> drain -> summarize). Metrics
reduction is harness-specific sink glue the engine cannot generically own
(C8/C18), and a fresh per-point build resolves #31's two structural facts:
bootstrap_with_params consumes the blueprint, and a Recorder bakes its
channel in — so each point gets its own drainable instance.
- SweepFamily / SweepPoint: ordered, self-describing (params carry the point
coordinate), in enumeration order independent of thread completion.
Execution is std::thread::scope over available_parallelism() workers pulling
point indices from a shared atomic cursor (work-stealing balances uneven
per-point cost); results are tagged by index and sorted after join, so the
family is bit-identical across thread counts and repeated runs (C1: disjoint,
lock-free; order = enumeration, not completion). No external dependency —
std only, never rayon (C16: the engine workspace stays zero-dep for deploy
purity). A private sweep_with_threads(nthreads) lets the tests pin
determinism at 1 vs N workers while the public sweep() derives the count.
Out of scope this iteration: the `aura sweep` CLI demonstrator (cycle 0028
iter 2); random enumeration, per-point manifest assembly (#33), value-domain
validation (C20) are deferred per the spec.
One deviation from the plan's verbatim code: GridSpace derives Debug
(load-bearing — the three .unwrap_err() fault tests need the Ok type to be
Debug to compile).
Verified: 8 new sweep:: tests green (enumeration/odometer order, the three
typed faults, sweep == N independent runs, determinism across thread counts,
distinct-points-distinct-metrics); cargo test --workspace green (92 engine
lib tests, no regression); cargo clippy --workspace --all-targets
-- -D warnings clean.
refs #32
|
||
|
Brummel
|
288f58953e |
feat(aura-engine): graph model serializer (0026 it1)
Iteration 1 of the graph render redesign: a read-only model_to_json(&Composite) that serializes the harness root + every distinct composite type into the canonical, deterministic JSON model the browser viewer will consume. New crates/aura-engine/src/graph_model.rs; lib.rs re-exports model_to_json. - House style: hand-rolled JSON like RunReport::to_json, NO serde (C14). - Model shape (faithful to the types, resolving the spec's prototype-flavoured example): node keys are indices (C23, the blueprint has no unique names); input ports carry kind+firing only, no name (PortSpec has none — acceptance criterion 3); a vec![] output is the C8 sink; bound root source-roles are minted as synthetic src_<role> source nodes; composite boundaries use @role (input) and #N (output) endpoints; distinct composites collected once. - Improvement over the plan: a composite's input kind is read from the actually fed interior slot (matching blueprint::derive_signature) rather than a hardcoded f64 — invents nothing. - Read-only (C9): &Composite -> String, no eval/compile/bootstrap on the path. - 10 new tests green: byte golden on the sample harness, determinism, structural mis-wire differs (param swaps are invisible to the pre-compile model — the property is carried by a structural re-target), read-only witness, sink / two-input / distinct-once structural assertions. Workspace green, clippy clean. aura graph still renders via ascii-dag (untouched); iteration 2 ports the viewer JS, vendors the WASM-Graphviz asset, emits the self-contained HTML, and retires ascii-dag. |
||
|
Brummel
|
1b3909316e |
feat(aura): node signature lives in the blueprint; collapse Blueprint into Composite
Consolidate the node data structure so every node's signature (NodeSchema:
inputs/output/params) is declared once and exists in the blueprint pre-build,
and dissolve the special "root graph" type. Behaviour-preserving (C1).
Signature vs sizing
- NodeSchema is now the static signature only: InputSpec -> PortSpec{kind,firing},
with lookback removed. The signature is fully static per blueprint (input
kinds/firing, output fields, params); LinComb's variable arity is a builder arg,
not an injected param.
- The one param-dependent quantity, an input's buffer lookback (e.g. Sma's window =
its injected length), moves out of the signature to Node::lookbacks() -> Vec<usize>,
read only by bootstrap for sizing. Node::schema() is removed.
- LeafFactory -> PrimitiveBuilder, which carries the full NodeSchema. The built node
no longer re-declares it: closes the params-declared-twice drift (#36, the 8
per-node factory_params_match_built_node_schema lockstep tests are deleted — their
subject is now structurally impossible) and a value-empty recipe exposes its full
I/O interface pre-build (#43).
Root is just a bound composite
- struct Blueprint is deleted; its compile/bootstrap/param_space methods move onto
Composite. Role gains source: Option<ScalarKind> (None = open interior port,
Some = bound ingestion feed). A composite is runnable iff every root role is bound;
the "main graph" is no longer a category, only the fully-source-bound composite.
New error CompileError::UnboundRootRole for an open root role.
- BlueprintNode::signature() answers uniformly for both arms: Primitive returns the
builder's declared schema, Composite derives it from the interior (role kinds in,
OutField kinds out, aggregated params), pre-build, no build.
compile -> FlatGraph -> bootstrap
- compile validates structure pre-build via signature() (validate_wiring: range +
kind, returning the same variants as before, so an edge kind fault is now caught
before any build closure fires) and emits FlatGraph{nodes,signatures,sources,edges}.
- bootstrap consumes the FlatGraph: kinds/firing/output from the carried signatures,
buffer depth from node.lookbacks(). SourceSpec survives as the flat descriptor.
Renames: BlueprintNode::Leaf -> Primitive, LeafFactory -> PrimitiveBuilder.
Render (aura-cli/src/graph.rs) is migrated compile-only: it takes &Composite, maps
bound roles to the same source-entry shape, so both render goldens reproduce
byte-identical output (no re-capture needed). Render-fidelity tuning is the next cycle.
Verification (orchestrator-run, not agent-reported): cargo build --workspace green;
cargo test --workspace 150 passed / 0 failed; cargo clippy --workspace --all-targets
-D warnings clean. All pinned determinism/run-output tests pass with values unchanged;
no behavioural assertion was altered to go green. 5 new tests assert the signature is
pre-build and uniform, that compile rejects a kind mismatch without building (via a
panicking builder), UnboundRootRole, and lookbacks()/signature arity agreement.
Deferred to cycle-close audit (per plan): docs/design/INDEX.md and some aura-std
module docs still name the old Node::schema()/LeafFactory/BlueprintNode::Leaf/
Blueprint::param_space contracts; prose reconciliation is the architect's at audit.
closes #43 #36
|
||
|
Brummel
|
69d20949c0 |
tidy(aura-engine): single-own alias-validity + shared alias-count anchor
Collapse the two correctness-neutral debt items the 0021 fan-in pre-pass left behind (architect drift review at cycle close). Alias-validity: the `BadInteriorIndex` check was raised in two places — the structural pre-pass `check_composite_fan_in` and `inline_composite`. Since `compile_with_params` runs the pre-pass over every composite before any lowering, `inline_composite`'s copy was dead for that error path. The check now lives in one helper `check_alias_indices`, called solely by the pre-pass; `inline_composite` drops its copy (and the now-unused `param_aliases` binding — the overlay is a pure naming layer, unused in lowering, which is driven by the injected scalar vector). Alias-count: the per-node `a.node == node` predicate was re-derived in three spots (signature base, the unaliased-param test, the CLI render base-length). All three now route through one exported anchor `aliases_on`, so an alias-semantics change touches one site. Behaviour-preserving: the full workspace suite is the guard (15+6 cli, 25 core, 69 engine, 6+1+1 ingest/misc, 29 std — 0 failed), `out_of_range_param_alias_rejected` still green (now raised by the pre-pass), clippy -D warnings clean. No ledger change (the C9 refinement landed in 0021). closes #45 |
||
|
Brummel
|
3f4d756ed2 |
feat(aura): fan-in input distinguishability — construction constraint + source-derived render
A fan-in node (>1 input) whose colliding sources hide an unnamed configuration axis is now illegal at construction, and the definition view renders each fan-in input as a source-derived recursive-signature identifier instead of the positional, meaningless #A/#B. The root defect was sma_cross: two Sma into a Sub with unaliased length params, rendering sma_cross() -> (cross) with two indistinguishable [SMA] and [Sub(#A,#B)] — the author meant fast vs slow but the identity hid it. Now it renders sma_cross(fast:i64, slow:i64) with [SMA(fast)]/[SMA(slow)] and [Sub(#Sf,#Ss)]. Shape (single source of truth for "signature" shared by engine + CLI): - aura-engine signature_of(): a node's recursive authoring identity — type initial + alias initials + each wired input's signature (recursing into interior leaves, stopping at named roles/composites at their initial). - aura-cli leaf_label/fan_in_identifiers: shortest sibling-unique prefix of a source's signature, never below its base (type + alias initials); a role-fed slot renders its name verbatim (#price); equal-signature interchangeable inputs keep the positional letter. - aura-engine IndistinguishableFanIn: a signature collision is a fault only when a colliding source carries an unaliased param (the unnamed axis); param-less interchangeable sources (fan_composite's Pass/Pass) stay legal. Param-aware criterion (refined during design): keeps the blast radius to the param-bearing alias-less fixtures (sma_cross CLI + engine, the nested fast_slow); fan_composite and the hand-wired flat fixtures stay green. Placement decision (deviates from the plan): the constraint runs as a structural pre-pass (check_fan_in_distinguishability) at the head of compile_with_params, BEFORE the param-arity gate — not inside inline_composite as the plan drafted. Reason: the no-param compile() of a param-bearing composite would hit ParamArity first and preempt the fault; the spec mandates a structural check needing no param values, so the pre-pass is the spec-aligned home. Alias-validity ordering (BadInteriorIndex before the fan-in fault) is preserved at the pre-pass head. C23 untouched: the check is construction-phase; the compilat stays name-free (compiled_view_golden byte-stable). Ledger C9 carries the refinement. Known debt (non-gating): the alias-index validity check now exists in both inline_composite and the pre-pass (the pre-pass reaches every composite first, making inline_composite's copy effectively dead). Left for a separate tidy — a 5-line, correctness-neutral removal with its own test surface. closes #44 |
||
|
Brummel
|
41cbb5506f |
feat(aura-engine): name the composite boundary — input roles + param aliases
Brings the other two composite-boundary edge-kinds to the named-projection shape #40 gave outputs. input_roles changes from a bare Vec<Vec<Target>> to Vec<Role { name, targets }> (rendered [in:<name>]); a composite gains params: Vec<ParamAlias { name, node, slot }> that relabels an interior leaf param slot's surface name in param_space() (rendered [param:<name>]). Param aliasing is a PURE NAMING OVERLAY, not curation (the load-bearing decision, per spec 0019): every interior param slot stays in param_space() and sweepable; the alias only relabels in place, never reorders or hides. Proven empirically — the MACD run is byte-identical (total_pips 0.1637945563898923, 3 sign flips), only the param labels improved: param_space() now surfaces [macd.fast, macd.slow, macd.signal] instead of three indistinguishable macd.length, and the manifest reads ema_fast/ ema_slow/ema_signal. C23 honoured: role/param/output names are non-load-bearing debug symbols dropped at lowering; identity is positional (role index, param slot, output field). compiled_view_golden is byte-identical (verified: the golden region is untouched in the diff). An out-of-range alias (missing/ non-leaf node or slot past the leaf's param count) is rejected at compile_with_params as BadInteriorIndex, mirroring the output range-check (no new variant). Orthogonal to #36 — purely additive at the composite level. Aliasing is demonstrated on the CLI MACD site only (the spec's worked example + a new E2E test macd_param_space_surfaces_the_three_named_aliases); sma_cross, the engine test fixtures, and the construction-layer fieldtests get the forced role-name + empty params, so the param_space C23 anchor goldens (param_space_mirrors_compiled_flat_node_param_order + siblings) stay byte-identical. Verification (orchestrator-run, not trusted from the agent report): cargo build/test/clippy --workspace -D warnings all green (engine 66, cli 12); the separate-workspace construction-layer fieldtest crate builds (guards the #42 latent-drift recurrence); compiled_view_golden + MACD determinism unchanged. Two faithful repairs to the plan's literal test/code bodies, no semantic change: the out-of-range test uses .err()/Some(BadInteriorIndex) (the Ok arm Vec<Box<dyn Node>> is not Debug, so .unwrap_err() would not compile), and the inline_composite destructure binds params as `param_aliases` to avoid shadowing the injected `params: &[Scalar]` arg. closes #41 |
||
|
Brummel
|
784e6c917a |
feat(aura-engine): composite output is a named multi-field record
Replace a composite's single output port (OutPort { node, field }) with
an ordered, named record (Vec<OutField { node, field, name }>). A
composite can now re-export K interior fields as one output; a consumer
selects one via the existing Edge::from_field — the same field-wise
selector that already works for multi-output leaf producers (OHLCV).
Why: multi-line indicators (MACD, Bollinger, Stochastic, Ichimoku) could
not be authored as a composition and re-exported as a unit — only one
line escaped the boundary. The MACD PoC was forced to expose only the
histogram. This lifts the three `field == 0` / `from_field == 0` caps at
the composite boundary (inline_composite nested arm, rewrite_edge
composite arm), range-checking the index instead.
Boundary completion, not an invariant change:
- C8/C7/C4 untouched — one port, one row per eval, K base columns (a
3-line MACD is identical in kind to OHLCV).
- C23 honoured — re-export names live at the blueprint boundary only and
are dropped in the compilat (raw index wiring). compiled_view_golden
stayed byte-identical (the regression guard): no name leaked into the
flat graph.
The MACD PoC now re-exports macd / signal / histogram as a record; the
strategy still trades the histogram, selected via from_field: 2. The
render shows K [out:<name>] markers per multi-output composite (input
roles stay [in:<index>] — naming them is the dependent #41).
Output naming ships with the capability (OutField is born name-ready) so
#41 — composite param aliasing + input-role naming — does not reopen the
type.
Verification: cargo build/test/clippy --workspace -- -D warnings all
green; aura-engine 62 tests (+3 capability, +1 through-run E2E), aura-cli
unchanged-count green; compiled_view_golden byte-identical.
Known debt (out of scope, pre-existing): the non-workspace construction-
layer fieldtests (fieldtests/milestone-construction-layer/mc_1..mc_4)
carry their OutField sweep but still do not compile standalone — they use
the Sma::new / BlueprintNode::from(Sma) API retired in the cycle-0016
value-empty migration, never propagated to these fixtures. Tracked as a
follow-up.
closes #40
|
||
|
Brummel
|
931109df58 |
feat(aura-core,aura-std,aura-engine): declare node tunable params (#30)
Cycle A of milestone 'The World — parameter-space & sweep'. A node now declares its tunable parameters in its C8 schema, and a blueprint aggregates them into one flat, inspectable param-space — the root that unlocks the C12 orchestration axes (#31 bind, #32 sweep), filling the gap C8/C23 name 'deliberately not in the schema yet'. - aura-core: ParamSpec { name: String, kind: ScalarKind } as a third schema- declaration type; NodeSchema gains a third field 'params'. name is String (not &'static like FieldSpec) because a vector knob carries a runtime index and aggregation prefixes the composite path. - aura-std: Sma declares [length:I64], Exposure [scale:F64], LinComb expands to N flat [weights[i]:F64] (N = its input arity, topology-fixed per C19); Sub/Add/ SimBroker/Recorder declare none (pip_size is metadata, C10/C15; Recorder is wiring). - aura-engine: Blueprint::param_space() walks the graph-as-data depth-first in lower_items order, path-qualifying via the already-public Composite::name() — a read-only projection (C9). compile/inline_composite/lower_items are untouched, so the compilat stays bit-identical (composite_sma_cross_runs_bit_identical_to_hand_wired and the golden render tests stay green). Design: param identity is positional (slot after the deterministic inline order, C23 'by index not name'); the path-qualified name is a non-load-bearing debug symbol — same-type siblings in one composite share a name, uniqueness is at the slot. Flat over a structured arity-bearing ParamSpec because flattening is unavoidable (a sweep enumerates a flat point-space) and structure-in-the-runtime is the nested-composite reading C23 rejects (see spec 0015 for the full rationale). Scope is declaration + aggregation + inspection only; binding (#31), sweep enumeration (#32), search-range, validity-constraint, and default-range/slider are deferred. An E2E test pins the load-bearing C23/#31 invariant: param_space() slot order mirrors compile()'s flat-node param order, kind-by-slot, on the SMA-cross harness. fieldtests/ (excluded crates) left as frozen snapshots. Verified: cargo build/test --workspace green (64 tests), clippy --all-targets -D warnings clean; compile/inline diff empty. closes #30 |
||
|
Brummel
|
561482c422 |
tidy(aura-engine): re-export scalar vocab (#29) + drop duplicate Recorder fixtures (#14)
Two behaviour-preserving tidies from the post-0013 fieldtest/backlog. #29: aura-engine re-exports Firing/Scalar/ScalarKind/Timestamp at the crate root. A Blueprint builder needs ScalarKind for SourceSpec.kind and Scalar/ Firing/Timestamp for sources & Recorder columns, but aura-engine previously re-exported only the wiring types — forcing a second aura-core import for the vocabulary its own public structs (SourceSpec.kind, ...) demand. The fieldtest flagged this paper cut (mc_4). One import surface now. Guarded by a new reexport_tests module. #14: the two near-identical #[cfg(test)] Recorder fixtures in report.rs and harness.rs are deleted in favour of the shipped aura-std::Recorder (already a dev-dependency). Constructor signature, schema, and try_iter() drain are identical, so no call site changed — only the struct/impl deletion plus adding Recorder to each file's existing aura_std import. report.rs additionally drops four imports (Ctx/InputSpec/Node/NodeSchema) that only the deleted fixture used. The separate TapForward fixture in harness.rs is untouched. Verified: cargo build/test/clippy --workspace all green (49 aura-engine lib tests incl. the new one); bit-identical, golden-snapshot, SMA-disambiguation, and mixed-kind recording tests all stay green — behaviour preserved. closes #29 closes #14 |
||
|
Brummel
|
a4fb5d7182 |
feat(aura-engine): blueprint construction layer with composite inlining
Add the construction layer (C9/C19/C23): a named, param-generic graph-as-data (`Blueprint`) that *compiles* to the flat, type-erased instance the run loop already runs. The unit of reuse is the `Composite` — a nestable sub-graph fragment exposing one output port (C8) and named input roles — which `Blueprint::compile()` **inlines** by raw-index lowering into the flat `(nodes, sources, edges)` the unchanged `Harness::bootstrap` consumes. Design (settled in spec 0012, ledger C9/C19/C23): a composite is an authoring-level node, NOT a runtime `Box<dyn Node>` sub-engine. The chosen inlining approach keeps the sacrosanct deterministic run loop untouched and leaves the composite interior fully inspectable for the deferred cross-graph optimiser; the rejected "runtime sub-engine" reading would have put throwaway complexity into the run loop and kept the interior opaque. The compilat is wired by raw index, not by name; names survive only as non-load-bearing debug symbols (as `FieldSpec.name` already is). Lowering is recursive index rewriting: interior items append at an offset, interior edges rewrite by that offset, an edge into a composite fans through its input roles (one blueprint edge -> several flat edges), an edge out resolves to the interior output port, and nesting recurses inside-out. Construction-phase faults are caught as a typed `CompileError` (BadInteriorIndex / RoleKindMismatch / OutputPortOutOfRange); the lowered flat compilat is validated by bootstrap's existing kind- and Kahn-cycle-check (wrapped as `CompileError::Bootstrap`), with no re-implementation. Non-goals (deferred per C23/C16): no optimisation pass (CSE/DCE, sweep-invariant hoisting), no external optimisation crate, no named-handle ergonomic wiring, no `aura graph` render (#13). Verification: 48 engine tests green (8 blueprint: derived-schema, single + nested composite inlining, the three CompileError paths, bootstrap-error wrap, and the headline `composite_sma_cross_runs_bit_identical_to_hand_wired` C1 demonstrator — the SMA-cross composite lowers to a flat graph byte-identical to the hand-wired sample harness, producing bit-for-bit identical equity + exposure traces); `cargo clippy --workspace --all-targets -D warnings` clean. `Node`, `Harness::bootstrap`, the run loop, and `Edge`/`Target`/`SourceSpec` are unchanged. closes #12 |
||
|
Brummel
|
80d7bfbbf0 |
feat(aura-engine): run metrics + manifest report surface
Cycle 0009 (Walking skeleton milestone). A run's two C18 "from day one"
artefacts — a reproducible manifest and summary metrics — land as a new,
pure-additive `report` module in aura-engine:
- summarize(equity, exposure) -> RunMetrics: a post-run pure reduction over a
run's recorded pip-equity + exposure streams. total_pips (last cumulative
value), max_drawdown (worst running-peak-to-trough), exposure_sign_flips (a
turnover proxy: adjacent normalized-sign changes, flat distinct from
long/short via a three-way sign0). Total and pure: empty -> zeros, identical
inputs -> identical metrics (C1/C12).
- RunManifest: the reproducible descriptor (commit, params as name->value
pairs, data-window, seed, broker label). Caller-supplied — the engine cannot
introspect a git commit or seed. params is the honest precursor to the
deferred typed param-space (node.rs; 0008 audit).
- RunReport { manifest, metrics } + to_json(): canonical, hand-rolled JSON for
the structured C14 face.
- f64_field(rows, field): bridges a recording sink's Vec<Scalar> rows to
summarize; panics on a kind/width mismatch (a wiring bug), like the engine's
other "checked at wiring" violations.
Why post-run, not a node: a node emits one record per eval with no terminal
eval (C8), so an end-of-run reduction has no home as a node; the World drains
its cycle-0006 recording sinks after Harness::run and folds them here. Matches
C18's "store manifests + metrics, re-derive results on demand".
Why hand-rolled JSON, not serde: the workspace is deliberately zero-dependency
and the schema is tiny/closed/flat; serde is reconsidered when the run-registry
milestone grows the schema. (User-approved at spec review.)
Scope: the surface + a determinism end-to-end test reusing the cycle-0007
two-sink harness; the `aura run` subcommand that prints the report is #8. No
engine/Harness/node-contract change; workspace stays zero-dependency.
Verified: cargo test --workspace green (aura-engine 40, incl. 10 new report
tests); clippy --all-targets -D warnings clean; RUSTDOCFLAGS=-D warnings cargo
doc clean.
closes #6
|
||
|
Brummel
|
f37b2a35d3 |
feat(engine): firing policies A/B + the k-way ingestion merge
Cycle 0004: the reactive-semantics layer. The engine now merges multiple
timestamped sources into one chronological cycle stream (C3) and gates each
node's re-evaluation by a per-input firing policy (C5/C6) — both rails shipping
together, as the user required ("must sit right from the start").
- `Firing` (aura-core) — a per-input enum on `InputSpec`, where C8 places firing:
`Any` (mode A, fire-on-any-fresh + hold / as-of join) and `Barrier(u8)` (mode B,
all-fresh barrier / synchronizing join). Default-free; Sma/Sub declare
`Firing::Any`, so under a single source ticking every cycle their behavior is
unchanged (backward compatible — the 0003 fan-out/join DAG still emits
[None,None,None,2,2,2]).
- `SourceSpec` + the k-way merge (aura-engine) — `run` takes one
`(Timestamp, Scalar)` stream per source and merges them by (timestamp, source
index) (C4 ties by declaration order). One record = one cycle. This is the
permanent ingestion core; the Source trait + data-server IO (C3/C11) are a
later orthogonal cycle (the user scoped this cycle to the reactive semantics).
- Two read clocks, both load-bearing (resolves the 0003 audit's "0004 owns
cycle_id" with no write-only field): each push stamps the input slot with
`fresh_at` (the cycle_id — freshness epoch, C5: fresh iff fresh_at == cycle_id)
and `last_ts` (the data timestamp — barrier token, C6: a barrier group is
complete iff all members carry last_ts == T). The `fires()` predicate ORs the
groups; the ">=1 member fresh this cycle" clause is the once-per-timestamp
guard. Sample-and-hold falls out of the push model: a non-firing node pushes
nothing, so consumers keep seeing the held value via window[0].
Key design call, studied against RustAst (src/ast/rtl/streams/): the barrier
token is the data *timestamp*, not cycle_id. RustAst's
last_cycle_per_input.all(== cycle_id) synchronizes one source's fan-out but
cannot synchronize independent sources — under C4 four same-timestamp sources
are four different cycle_ids, so the cycle_id barrier never fires for C6's own
OHLC example. Substituting timestamp is the minimal faithful generalization
(fires both the diamond rejoin and the multi-source bar). Mode A and the k-way
merge are both new (RustAst had neither); RustAst's total_count push counter is
aura's already-built Column::run_count.
Both rails proven on identical sources, firing-only difference:
- mode A (AsOfSum): [None,None,120,130,140,240] — holds the slow input;
- mode B (BarrierSum): [None,None,120,None,None,240] — waits for timestamp
coincidence;
- mixed A+B (MixedSum): the OR-combine fires both when the barrier pair completes
(holding the as-of input) and when the as-of input ticks (holding the pair).
Plus determinism (C1, bit-identical re-run of each rail) and the three bootstrap
rejections re-expressed on SourceSpec.
Gates green (re-run by the orchestrator, not just the agent): cargo build/test
(30: aura-core 19 + aura-std 3 + aura-engine 8) / clippy --workspace
--all-targets -D warnings clean; surface-purity grep (no dyn-Any / Rc / RefCell)
clean — the harness doc phrases RustAst's model without the literal tokens to
avoid self-match.
refs walking-skeleton
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Brummel
|
02fe89176f |
audit: cycle 0003 tidy — rename Sim -> Harness (glossary alignment)
Architect drift review over 71c8f6e..dd5c3fa (deterministic-sim-loop). Invariants hold: C1 (single-threaded, bit-identical re-run, alloc-free per-cycle ring path), C7 (AnyColumn type-erased edges, Box<dyn Node> is the node object not a dyn-Any payload, kinds checked once at bootstrap), C8/C9 (DAG wired, Kahn rejects cycles, None forwards nothing = sample-and-hold seed). Column::run_count already exists, so the C5 freshness primitive lives at the substrate — the deferral is honest, not a hole. Resolved [high]: the runtime type was named `Sim`, colliding with the glossary's load-bearing distinction — `sim` is the disjoint *executable unit / parallelism* framing, whereas the thing built here (a closed root graph of source + nodes + observe under a clock) is by C20 / the glossary a **harness**. Renamed `Sim` -> `Harness`, `sim.rs` -> `harness.rs`, and the module/exports/doc to match, before any other code takes a dependency on the wrong term. Gates re-run green (26 tests, clippy -D warnings, purity grep). Resolved [med] (glossary gap): added an `edge` entry anchoring the new wiring vocabulary (Edge / source target) to the glossary — record-reality per the boss glossary-write authority. Carry-on with forward notes (next-cycle owners, not fixes now): - [med] C4 monotonic cycle_id has no realization in the runtime type yet (the cycle clock is the record iteration; the explicit counter's first reader is freshness). Cycle 0004 owns materializing it together with C5. - [low] Sub declares lookback 1; once depth>1 join nodes exist, add a test guarding C8's "a node never reads beyond its declared lookback". Regression gate: profile regression list empty — architect is the sole gate, now clean. Next direction: freshness-gated recompute + a second source (C5/C3). |
||
|
Brummel
|
dd5c3fad96 |
feat(engine): the deterministic single-source sim loop
Cycle 0003: aura-engine's first real content — a Sim that runs a wired DAG of
nodes deterministically, cycle by cycle. First point in the project where
authored nodes execute against data, not just under a hand-fed Ctx.
- `Sim` (aura-engine) — a bootstrapped, frozen root graph: a flat Vec<NodeBox>
(each node owns its input columns, the cycle-0002 shape) + an index edge
table, topologically ordered (Kahn). `bootstrap` sizes every input column from
its node's schema, kind-checks each edge and source target, and rejects
directed cycles — C7's "type check paid once at wiring" generalized to the
whole topology (`BootstrapError::{KindMismatch, BadIndex, Cycle}`).
- `run` — the deterministic loop: per record, forward the source value into its
target slots, evaluate nodes in topo order, capture the observed node's output
at eval time, and forward each `Some` output into its consumers' input columns
(a `None` forwards nothing — the structural seed of sample-and-hold). The loop
destructures `&mut self` into disjoint field borrows and allocates nothing on
the per-cycle path. This is the flat, monomorphized sharpening of RustAst's
reference-counted, interior-mutable observer push graph (C1/C7).
- `Edge` / `Target` (aura-engine) — producer->consumer and source->consumer wiring.
- `Sub` (aura-std) — a 2-input f64-difference node, so the loop is proven on a
real fan-out + join DAG (source -> {SMA(2), SMA(4)} -> Sub), with a determinism
assertion (C1: a second identical run is bit-identical).
Engine library depends only on aura-core; a test-only dev-dependency on aura-std
lets the integration test wire real Sma/Sub nodes. Sim carries a hand-written,
node-opaque `Debug` impl (Box<dyn Node> is not Debug; the impl prints
nodes.len() + the index/topology fields, needed by the bootstrap-rejection tests'
`unwrap_err`).
Deliberately deferred (recorded as decisions): freshness-gated recompute /
sample-and-hold (C5 -> cycle 0004, with the second source that makes it testable);
the explicit monotonic cycle_id counter (its first reader is freshness, 0004);
the Source trait + data-server ingestion + k-way merge (C3/C11); the builder API
(C19); the real sink + run registry (C18/C22).
Gates green: cargo build/test (26: 18 aura-core + 3 aura-std + 5 aura-engine)/
clippy -D warnings clean; surface-purity grep (no dyn-Any / Rc / RefCell) clean.
refs walking-skeleton
|
||
|
Brummel
|
1467fcd30f |
docs: brokers are consumer nodes (C10), several attachable for comparable curves
Correct the broker mechanism: a broker is an ordinary downstream consumer node (C8/C9), not an external plugin/subsystem. It consumes the position-event stream + price streams and emits an equity stream; several brokers (e.g. sim-optimal pip + realistic currency) can be attached to the same position table at once, yielding directly comparable equity curves. Updates C10, CLAUDE.md invariant 7, aura-engine/aura-std crate docs, and the day-in-the-life doc. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Brummel
|
d8ccdcd806 |
docs: fix three drifts found in the session audit
C8: a node is producer/consumer/both — at most one output, pure consumers (sinks: chart/equity/logger) have none; records the node-role taxonomy the substrate requires (was wrongly 'exactly one output', contradicting pure consumers and C14). C3: merge by timestamp, normalizing source-native units (data-server Unix-ms) to the canonical epoch-ns of C7. aura-engine lib doc: broker is a downstream plugin over the position table (sim-optimal pip / realistic currency), not an in-strategy 'broker/portfolio node'. Plus a provenance line-wrap nit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Brummel
|
744c2f31a8 |
chore: scaffold aura workspace and wire the skills dev-cycle
Cargo workspace aura-core -> aura-engine -> aura-cli (bin `aura`) plus nodes/ for hot-reloadable cdylib node crates. Crate bodies are intentionally API-free; types arrive from the first spec. Adds the skills profile (.claude/dev-cycle-profile.yml), the project CLAUDE.md with the eight domain invariants, and the design-ledger skeleton. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |