be8d267019b9fcdc92ecaa11270b4c765e40f9af
116 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
be8d267019 |
docs(aura-engine): SweepError summary names RandomSpace::new too
The SweepError type-level rustdoc still read "A structural fault constructing a `GridSpace`" though the enum now also gates `RandomSpace::new` (the three random-only variants are each documented individually, but the type summary was stale). Broaden it to name both spaces and group the grid vs. random faults. Doc-only, behaviour-preserving — the same low-grade doc-debt class the 0049 audit fixed inline for the module doc. Surfaced by the cycle-0049 fieldtest. refs #52 |
||
|
Brummel
|
3fca7810d0 |
audit: cycle 0049 — drift-clean (random param-sweep)
Architect drift review over 3de00e2..HEAD (the 0049 spec/plan/feat plus the two intervening refactors that had not been audited: aura-ingest M1 transpose |
||
|
Brummel
|
e17d78f24f |
feat(aura-engine): random param-sweep — RandomSpace + a Space trait + typed ParamRange
Ship the random half of the C12.1 param-sweep axis (grid landed in 0028).
A new `Space` trait (`points`/`param_specs`) generalizes `sweep`/`sweep_with_threads`
from `&GridSpace` to `&S: Space`; `GridSpace`'s trait impl forwards to its existing
inherent methods, so the grid path is behaviour-preserving (C1, every pre-existing
grid sweep test stays green). `RandomSpace` is the second `Space` producer: N seeded
uniform points over per-slot typed `ParamRange { lo, hi }` ranges (I64 inclusive
[lo,hi], F64 half-open [lo,hi)), validated in `new` against the param-space — a
non-numeric slot, a range-kind mismatch, and an empty range are the three new typed
`SweepError` variants, all caught before any run. Sampling reuses the existing
bit-stable `SplitMix64` (promoted to `pub(crate)`) as a code-path-disjoint instance
from the data-edge seed RNG — the #52/#71 World-II source-seam firewall: they share
only the `u64` type, never a path. The sweep-layer signature stays param-only
(`Fn(&[Cell]) -> RunReport`); no `Source`/stream type enters it.
Deviation from the spec's verbatim sampler, accepted as a correctness hardening:
the I64 draw guards the full-width span ([i64::MIN, i64::MAX] computes a span of
2^64 that wraps a u64 to 0) and uses `lo.wrapping_add(draw as i64)` instead of a
plain `+`. This avoids both the `% 0` divide-by-zero at the full domain and the
signed-overflow panic the literal form hits on wide ranges in debug builds, while
preserving uniform-over-[lo,hi] draws and determinism (an extra RED test,
random_points_full_range_i64_does_not_panic, pins it). Modulo bias for spans not
dividing 2^64 remains the spec's documented, accepted simplification (param search
needs no cryptographic uniformity).
Includes a public-API E2E suite (tests/random_sweep_e2e.rs) exercising the feature
exactly as a downstream researcher would — reproducibility at the report boundary,
seed-sensitivity, in-range draws, the typed NonNumericRange gate, a well-formed
empty (count==0) family, and Grid/Random interchangeability through `Space`.
Verified: cargo test --workspace green (incl. 17 new engine tests + 6 E2E);
cargo clippy --workspace --all-targets -- -D warnings clean.
closes #52
|
||
|
Brummel
|
67c1f51cfe |
perf(aura-std): O(1) Kahan-compensated running-sum SMA (was O(length) re-sum)
Sma::eval re-summed the whole window every tick — O(length)/tick, multiplied across millions of bars and every sweep point. Replace with the industry-standard incremental running sum (ta-lib shape): the node owns the window ring and keeps a running sum, adding the new sample and subtracting the evicted one each cycle, so the input column drops to depth 1 (lookbacks() = [1], like Ema). The running sum carries Kahan/Neumaier compensation (the fix pandas' rolling mean adopted) so it does not drift over long runs; Ema needs none — its recurrence is contractive. Determinism (C1) holds: same input -> same run, reproducibly. The float output differs from the full re-sum in the last ULPs (a new, deterministic baseline); the suite needed only one arity-assumption update (Sma lookback 3 -> 1), no equity golden changed. closes #39 |
||
|
Brummel
|
6390093f93 |
refactor(aura-ingest): chunk-direct M1 transpose, drop the AoS intermediate
load_m1_window collected every bar into an intermediate Vec<M1Parsed> (a full AoS materialization, un-presized so it grew by reallocation) before transpose_m1 copied it again into the SoA columns — ~2x peak load memory plus realloc churn. Transpose each chunk straight into the columns instead, via a new shared M1Columns::extend_from_bars (reserves per chunk for amortized growth). One fewer full copy, halved peak. Behaviour-preserving: the resulting M1Columns is byte-identical and transpose_m1's / load_m1_window's signatures are unchanged. Pinned by a new parity test that needs no live DataServer. closes #78 |
||
|
Brummel
|
9febdb9623 |
refactor(aura-engine): walk-forward param plane — space on family, tag-free chosen_params, schema-checked param_stability
Make the walk-forward param plane C7-clean and symmetric with the sweep param plane. WalkForwardResult gains `space: Vec<ParamSpec>` (the kinds, once, mirroring SweepFamily.space) and WindowRun.chosen_params changes Vec<Scalar> -> Vec<Cell> (the tag-free coordinate the inner sweep already produces). param_stability now resolves its numeric coercion once per slot against the schema (result.space) into a Vec<fn(Cell)->f64>, then runs a type-blind reduction loop — replacing the old per-window-value scalar_as_f64 with its per-value unreachable!. scalar_as_f64 is deleted.
This overturns cycle 0047's "Fork A" (chosen_params stays Vec<Scalar>), which was justified by Scalar's new serializability. That defense was moot: WalkForwardResult and WindowRun carry no serde derives — they are never serialized; only the embedded oos_report (a RunReport, typed via the 0047 lift) is. So chosen_params is a pure in-memory substrate for param_stability, and Vec<Scalar> made it C7-impure (carrying the kind N×M times at the value, where the kind is per-slot constant). The substantive case — kind at the schema, SweepFamily symmetry, per-value unreachable! eliminated — wins, the same argument-form that retired the {kind,cell} Scalar struct in 0047.
The seam ([A]): walk_forward / walk_forward_with_threads take `space: Vec<ParamSpec>` by value, a sibling of the run-window closure; the Fn(WindowBounds)->WindowRun+Sync bound is unchanged and run_indexed is untouched (space is never captured by the Sync closure, moved onto the result at the end). A debug_assert in walk_forward_with_threads guards the arity coupling (every window's chosen_params.len() == space.len(), guaranteed by the same blueprint). walkforward_family computes the space once before the roll and passes the tag-free winner (chosen_params: best.params) directly, dropping the from_cell+zip reconstruction.
[D]: a Bool param slot coerces to 0/1 (mean = fraction "on"), keeping the coercer total over the knob kinds; a Timestamp slot is structurally impossible (C20) so the schema pass carries one unreachable! (once per slot, never per value). param_stability keeps an `if result.windows.is_empty() { return Vec::new(); }` guard to preserve its documented empty-on-no-windows contract. WalkForwardResult and SweepFamily share the (space + tag-free Cell points + zip_params) idiom but stay distinct types — different axis (Sweep enumerates input coordinates; WF rolls time and chosen_params is the optimizer's output), and WF additionally carries bounds + oos_equity + stitched_oos_equity.
Behaviour-preserving (C1): the three migrated walk-forward tests stay green with identical MetricStats (param_stability_reduces_chosen_params_per_slot still pins mean 2.5 / p50 2.5 / mean 1.5); the i64->f64 and f64 coercions are value-identical to the deleted scalar_as_f64. The one genuinely new behaviour (Bool 0/1) is unexercised by built-in grids and pinned by the new param_stability_reduces_bool_slot_to_fraction_true (mean 0.75, 3-of-4 true).
Gates: build --workspace --all-targets, test --workspace, clippy --workspace --all-targets -D warnings, doc --workspace --no-deps — all clean.
closes #75
|
||
|
Brummel
|
86746e3d5d |
refactor(aura-core): Scalar as a native tagged enum, disjoint from Cell; typed RunManifest.params
Scalar was `struct { kind: ScalarKind, cell: Cell }` — "a Cell wearing a kind hat." The recorded reason for that shape was migration ease, which is not a design rationale (CLAUDE.md: rationale != effort), so the struct had no substantive defense. Redefine it as the native tagged union it conceptually is:
enum Scalar { I64(i64), F64(f64), Bool(bool), Timestamp(Timestamp) }
This is substantively better on four axes: kind/bits skew becomes unrepresentable (illegal states gone); accessors panic loudly on the wrong variant instead of a release-mode silent bit reinterpret; PartialEq derives the documented IEEE-754 / cross-kind value semantics (the hand-roll, needed only because the struct inherited Cell's bitwise compare, is gone); and serde is a plain derive emitting the externally-tagged wire form ({"I64":10}/{"F64":2.5}) — the private ScalarRepr shadow enum that motivated this was never needed. It is also C7-honest: erased-on-the-hot-path (Cell) and self-describing-at-the-edge (Scalar) are two disjoint types bridged by explicit conversion.
The whole public API is preserved (Scalar's fields were private), so call sites do not churn: the enum change is contained to scalar.rs, where cell() now encodes and from_cell() decodes per kind. Cell and ScalarKind are untouched.
With Scalar serializable, lift RunManifest.params from Vec<(String, f64)> to Vec<(String, Scalar)>: the param's kind (an i64 length vs an f64 scale) now survives into the C18 record (runs.jsonl) and the CLI JSON instead of collapsing to f64. scalar_as_param_f64 is deleted; sim_optimal_manifest passes typed params through. This is a deliberate wire-shape change — params now render as tagged scalars; the JSON-asserting tests are updated to the new shape on purpose.
Hand-authored manifest params across the CLI's single-run/mc/macd sites use honest kinds (lengths -> i64, scales -> f64) so they match the sweep path (which already derives correct kinds via zip_params); their in-binary JSON assertions are re-tagged accordingly.
Walk-forward fork resolves with no code change: WindowRun.chosen_params stays Vec<Scalar> (the serializable record carrier), reduced to f64 only at the param_stability statistic boundary (scalar_as_f64 retained). Glossary 'cell' entry updated to describe Scalar as the disjoint tagged union, not 'a cell plus its kind tag'.
Gates: build --all-targets, test --workspace (incl. new scalar_serde_round_trips), clippy -D warnings, doc --no-deps — all clean.
|
||
|
Brummel
|
43716be10e |
refactor(aura-engine): Cell into the construction path — param-plane base/frontend split
Cell becomes the carrier of the construction path; Scalar narrows to the author/render boundaries. The validated/enumerated param point carries no redundant kind (it lives once, in the declared param-space); at the author edge the kind is a checksum — two independent sources, the typed value vs. the slot — so the self-describing Scalar stays there. The name->slot binding is dynamic (C23), so the check is necessarily a runtime one and the value must self-describe for it. Base/frontend split (the AnyColumn push/push_cell pattern one level up): compile_with_cells / bootstrap_with_cells are the kind-check-free base; compile_with_params / bootstrap_with_params are the frontend that adds only the per-value kind checksum, strips to cells (new Scalar::cell(), the partner of Scalar::from_cell), and delegates. lower_items loses its per-primitive kind-check; PrimitiveBuilder::build and the std node builders (sma/ema/exposure/lincomb) read cells. Boundary: construction -> Cell (PrimitiveBuilder::build, lower_items, GridSpace.axes/points, SweepPoint.params, the sweep closure). Author edges stay Scalar (GridSpace::new, bind, compile_with_params/bootstrap_with_params). walkforward chosen_params stays a self-describing Scalar report record (Option A — WalkForwardResult carries no space); the cell winner is reconstructed once at the WindowRun site via from_cell. injective-check moved ahead of the arity-check in the frontend to preserve the pre-split error order (DuplicateParamPath before ParamArity). The lossy i64->f64 param projection (scalar_as_f64 / scalar_as_param_f64) is deliberately untouched — a separate follow-up. Behaviour-preserving (C1): build --all-targets / test / clippy -D warnings / doc all clean across the workspace. |
||
|
Brummel
|
82635fa259 |
refactor: Cell becomes the hot-path value carrier (C7/C8 realization)
Motivation ---------- The C7 realization note ( |
||
|
Brummel
|
cd3d1ca9ed |
refactor(aura-core): split Scalar into a tag-free Cell + ScalarKind
Motivation
----------
`Scalar` was a tagged enum (I64/F64/Bool/Ts), so every scalar value
physically carried its own kind tag. But the kind is already known from
the schema/port/column the value flows through (C7: the type is a
property of the column, not of the value — the hot path is already
columnar `Column<T>`, and `AnyColumn::get` *reconstructs* the tag from
the column on the way out). The per-value tag was therefore redundant
with the kind the surrounding context already holds.
That redundancy had three costs:
* It baked an implicit `match` (a branch) into every function that read
a Scalar payload — even where the caller statically knew the type.
The tag could never be exploited away.
* Size: a tagged enum is tag + payload = 16 bytes (f64/i64 alignment),
twice the 8 bytes the value needs. A `Column<Scalar>` would be double
the memory and half the cache utilisation.
* It is the shared root of several downstream papercuts we keep hitting
— the lossy f64 manifest field, the `unreachable!` panic on a
non-numeric param, the serde-tag question — all symptoms of "the type
is baked into the value".
Change
------
Introduce `Cell`: a type-erased 64-bit word (`struct Cell(u64)`) that is
not readable without external type context. It is constructed per base
type (`from_i64/from_f64/from_bool/from_ts`) and read only by naming the
type at the call site (`i64()/f64()/bool()/ts()`) — each a branch-free
bit-cast. The hot path resolves the kind once at the boundary (from the
schema) and then reads natively, with no per-value branch. `Cell` knows
nothing of `Scalar` or `ScalarKind`; the dependency is strictly one-way,
and it lives in its own `cell.rs` (more is planned on top of it).
`Scalar` becomes `struct { kind: ScalarKind, cell: Cell }` — the
self-describing form for the dynamic boundaries (builder binding,
serialization, rendering), built on top of `Cell`. Its `as_*` accessors
now `debug_assert` the kind and return the native value (free in
release); calling the wrong accessor is a caller bug, not a checked
`Option`. The variant constructors `Scalar::I64(..)` become associated
fns `Scalar::i64(..)`.
`PartialEq` is hand-written (not derived) to preserve the former enum's
value semantics: kinds must match, then native payloads compare, so f64
keeps IEEE-754 behaviour (`NaN != NaN`, `+0.0 == -0.0`) and a kind
mismatch is never equal even when the raw words coincide. A fixture
(`scalar_eq_is_value_not_bitwise`) pins exactly the cases where bit- and
value-equality diverge, so it can't silently regress. `Cell`'s own
`Eq`/`Hash` stay bitwise — correct for a raw word.
The change is behaviour-preserving: Scalar's observable behaviour is
identical to the pre-Cell enum (the value-equality fixture proves it);
only the internal representation changed. The ~440 call sites across the
workspace are a mechanical constructor rename plus ~12 destructuring
sites (match-arms / `let`-patterns) rewritten to `kind()` + `as_*`.
Verified: cargo build --workspace --all-targets, cargo clippy --workspace
--all-targets -- -D warnings, cargo test --workspace — all green.
|
||
|
Brummel
|
fb8cabf13f |
feat(aura-engine): sweep named-binding — zip_params + SweepFamily.named_params (C12 #57)
Thread a derived named view of a sweep point so consumers stop hand-zipping param_space() names onto the bare &[Scalar]. The free function zip_params(space, point) -> Vec<(String, Scalar)> in aura-core is the one shared projection; GridSpace retains the ParamSpec list it already receives in new() (it was validated then discarded); SweepFamily carries it (stamped once in sweep_with_threads) and exposes named_params(i). The three CLI hand-zip sites collapse to one zip_params call each. The run-closure signature `Fn(&[Scalar]) -> RunReport` is byte-unchanged — the named view is a derived projection (C23: slot is identity, name is derived), not a closure-currency change — so #52's RandomSpace plugs into the same sweep() execution layer with zero signature reconciliation (#71 firewall held). sim_optimal_manifest now takes typed Vec<(String, Scalar)> and does the lossy f64 collapse internally (one place; the manifest's f64-precursor field owns its own lossiness — the typed-manifest upgrade stays the deferred typed-param-space item, a non-goal here). All eight call sites migrated: three sweep closures pass zip_params, five hand-listing callers (run_sample, run_sample_real, mc_family, run_macd, run_sample_seeded) pass Scalar::F64 literals. Behaviour-preserving: the collapse output is identical to the old per-site zip, so aura sweep / walkforward / run output is byte-identical; SweepFamily.space threaded into the aura-registry optimize test literal. Verified by the orchestrator: cargo test --workspace green (4 new tests: zip_params x2, sweep_family_carries_param_space, family_named_params_round_trips), cargo clippy --workspace --all-targets -D warnings clean. (rust-analyzer emitted stale mid-edit diagnostics; the real cargo build/test is clean.) refs #57 |
||
|
Brummel
|
c2756732d4 |
refactor(aura-registry): split FamilyRunRecord family_id into {family, run} + derived id
Behaviour-preserving normalisation of the lineage record. The fused
`family_id: String` ("{name}-{counter}") is split into its two factors —
`family: String` + `run: usize` — and the user-facing handle becomes a derived
`family_id()` method ("{family}-{run}"), never stored or parsed.
Why: the fused field forced the counter logic to generate-and-check candidate
strings (to stay robust to '-' inside a name) rather than read it; with `run` a
plain int, `next_run` is a clean numeric max+1 over the field. The split is more
normalised (one fact per field) and the only property the fused token bought — a
single paste-able CLI handle — is recovered by deriving it. Storage and all
user-visible output stay byte-identical: append_family still returns
"{name}-{run}", Family.id is still "{name}-{run}", and the CLI prints the same
"family_id" lines (the json! key + the returned String, both unchanged).
Internal-only: FamilyRunRecord and group_families (now keyed on the (family,
run) tuple, first-seen order preserved) change; lib.rs re-exports, the
extractors, and all of aura-cli are untouched (everything downstream goes
through Family.id + append_family's returned String). The on-disk
families.jsonl key changes ({"family","run"} vs {"family_id"}) — a gitignored
runtime artifact with no committed fixture, and tests round-trip within a temp
dir, so no migration is owed. Doc reconciliation: the lineage/lib module headers
and the C18 ledger realization note now describe the split shape.
Verification: a workflow ran the refactor in an implementer agent, then three
adversarial lenses (behaviour-preservation, new-logic correctness,
completeness/doc-lag) tried to refute it — the first two found nothing, the
third flagged the four doc-lag sites now fixed here. Self-run gates:
cargo test --workspace green (registry 11, incl. a strengthened round-trip test
pinning the split fields + derived handle), cargo clippy --workspace
--all-targets -D warnings clean, cargo doc clean.
|
||
|
Brummel
|
bcd1072ca8 |
feat(aura-cli): aura mc + family-aware runs + window_of migration (0045 iter 2)
Iteration 2 of spec 0045 — the CLI surface for orchestration families. window_of migration: every manifest-window scan (run_sample, sweep_family, walkforward_family, sweep_over, run_oos, run_macd) now reads the window from Source::bounds() via window_of(&sources) instead of prices.first()/.last(). Behaviour-preserving (the walk-forward roller boundaries are bar-aligned, so the producer-supplied window equals the old first/last) — pinned by the unchanged run_sample (1,7) and walk-forward determinism tests. aura mc: new built-in Monte-Carlo family on the CLI (monte_carlo over a built-in seed set + the sample harness re-seeded per draw), persisted via append_family and rendered as one member line per seed (carrying the family_id) plus an mc_aggregate line. McAggregate is not Serialize, so the aggregate line is built from its three MetricStats blocks. family-aware runs: aura sweep / walkforward / mc now persist to the family store via append_family with an optional --name (per-kind default), printing the assigned family_id per member. aura runs families lists family headers (id, kind, member count); aura runs family <id> [rank <metric>] lists/ranks one family's members as a unit. An unknown family id is an empty family (exit 0); an unknown metric is a usage error (exit 2). Two pre-existing cli_run.rs E2E tests were adapted to the new family contract (a required consequence of sweep/walkforward switching from the flat store to the family store — a plan gap caught at implement time): the sweep-output test now asserts the family-wrapper, and the runs list/rank flow became the family list / per-family-rank flow (which also exercises the per-name counter sweep-0/sweep-1 end-to-end). A new E2E test drives aura mc through the binary. Known cosmetic detail: the family-wrapped sweep/mc/walkforward stdout nests the report via serde_json::json! (to_value -> alphabetical keys, leading "broker"), while runs family / runs list print via RunReport::to_json() (declaration order, leading "commit"). Both valid JSON; key order is not load-bearing and the stored families.jsonl uses to_string (declaration order), so the round-trip is unaffected. Verification: cargo test --workspace green; cargo clippy --workspace --all-targets -D warnings clean. refs #70 |
||
|
Brummel
|
1d9555c468 |
feat(aura-engine,aura-registry): producer-supplied window + registry lineage (0045 iter 1)
Iteration 1 of spec 0045 — the engine + registry core for storing orchestration families as linked records. aura-engine: add Source::bounds() — the inclusive (from, to) data extent a producer will stream, known without materialization (#71 firewall) — and a free window_of() that folds the union extent across a run's sources. This is the producer-supplied replacement for the prices.first()/.last() Vec scans at the CLI call sites (those migrate in iteration 2); a lazy producer now reports its window so stored lineage stays byte-identical whether a source is eager or streamed. aura-ingest: M1FieldSource stores its requested [from_ms, to_ms] window at open and reports it via bounds() (normalized to epoch-ns; None when open-ended — the archive-extent query is a deferred non-goal). aura-registry: new lineage module — FamilyKind / FamilyRunRecord (a RunReport stamped with family_id + kind + ordinal) / Family, plus Registry::append_family (assigns family_id = "{name}-{counter}" via a per-name generate-and-check counter, writes members to a sibling families.jsonl) and load_family_members; group_families re-derives the families (the round-trip), and the three *_member_reports extractors pull the per-kind member reports. The flat runs.jsonl store and its append/load/rank_by/optimize API are byte-for-byte unchanged (a test pins the two stores' disjointness). C9 preserved: aura-engine gains no registry dependency. New dep: serde derive on aura-registry (per-case policy, same basis as serde_json already is). Verification: cargo build --workspace; cargo test --workspace (228 green, incl. 3 new engine bounds/window_of, 1 data-gated ingest bounds run against real archive data, 5 new registry lineage round-trip/counter/disjointness tests); cargo clippy --workspace --all-targets -D warnings clean. The CLI surface (aura mc, family-aware aura runs, the window_of migration) is iteration 2. refs #70 |
||
|
Brummel
|
4764656062 |
feat(aura-engine): walk-forward family — WindowRoller + walk_forward (C12 axis 3)
The third C12 orchestration axis: walk-forward varies the data window. A
WindowRoller is a pure iterator of WindowBounds { is, oos } — bounds only, never
tick data (#71 firewall). walk_forward runs a caller closure per split disjointly
over the shared run_indexed core (C1), and stitches the OOS pip-equity segments
into one continuous curve (each segment offset by the running sum of prior
segments' finals; an empty segment contributes 0.0, mirroring summarize). C2
no-look-ahead is structural: the roller only ever emits oos.0 == is.1 + 1, pinned
by a zero-tick bounds test. The in-sample optimize (axis 2) is closure-supplied,
not called by the engine: aura-engine has no aura-registry dependency (C9), and
C12 forbids baking search policy into the primitive — the CLI bridges both crates.
Param-stability is on-demand, not stored (the R2 decision, recorded with
provenance on #69): WalkForwardResult stores only the raw per-window outcomes +
the stitched curve, and param_stability(&result) -> Vec<MetricStats> is a public
reduction over the retained per-window chosen params. A stored summary would be
recomputable from the windows (redundant) and would force one statistic canonical
when "stability" admits several; this mirrors SweepFamily (raw points + on-demand
optimize), not McFamily (stored aggregate). MetricStats::from_values is extracted
from McAggregate::from_draws (behaviour-preserving — the 7 mc tests stay green)
and gains serde so the CLI summary renders it and #70 lineage can persist it.
`aura walkforward` runs a built-in rolling walk-forward over a synthetic windowed
source: per window it sweeps the built-in grid in-sample, optimizes by total_pips,
runs the chosen params out-of-sample, persists each OOS RunReport (C18), and prints
a stitched summary line.
Two plan deviations, both compiler-forced and consistent with siblings: added
#[derive(Debug)] to WindowRoller (the RED tests' unwrap_err needs Ok: Debug; the
sibling types already derive Debug); removed a now-redundant test-module
SyntheticSpec import after Step 2 hoisted it to the top-level use.
Verification: cargo test --workspace green (aura-engine 152 incl. walkforward 9 +
mc 9, aura-cli 14); clippy --workspace --all-targets -D warnings exit 0; cargo doc
--workspace --no-deps clean.
refs #69
|
||
|
Brummel
|
61a8436c2c |
audit: cycle 0043 — drift-clean (monte-carlo family)
Architect drift review over b920362..HEAD (the 0043 monte-carlo cycle). Cycle is sound: C12 axis 4 faithfully realized (monte_carlo varies the seed, the 0042 seeded source genuinely perturbs each run, so distinct_seeds_produce_distinct_draws is a real property); C1 honoured (one run_indexed core, output sorted by job index = input order independent of completion, 1-vs-8-thread determinism pinned, parallelism across draws never within); the run_indexed extraction is a genuine behaviour-preserving refactor (C11/C23 — the three pre-existing sweep tests stay green, and both family executors are thin adapters over the single core; the points.len() -> n.max(1) clamp is equivalent since GridSpace rejects empty axes); the #71 eager-agnostic firewall is respected (API takes seeds + a per-draw closure, no materialized stream Vec crosses the boundary). One medium doc-lag found and fixed here: aura-engine/src/lib.rs module header listed Monte-Carlo (and, already stale, the grid sweep) under "Still to come"; both orchestration axes ship now, so the header records the grid axis (sweep / SweepFamily) and the seed axis (monte_carlo / McFamily) as delivered, with walk-forward, random param-sweep, registry lineage, and the C10 position-event / broker layer remaining future. Regression gate (test suite — no dedicated regression scripts in project facts): cargo test --workspace green (incl. 7 new mc::tests + the sweep guard tests); clippy --workspace --all-targets -D warnings clean; cargo doc -p aura-engine clean (the new intra-doc links resolve). closes #68 |
||
|
Brummel
|
e2c550f2f9 |
feat(aura-engine): monte_carlo family — McFamily over a seed set (C12 axis 4)
`monte_carlo(base_point, seeds, run_one) -> McFamily`, the Monte-Carlo
orchestration family: a fixed base point run over a seed set, each seed a
disjoint C1 realization. C12 axis 4 — Monte-Carlo IS a sweep over seeds, so it
reuses the disjoint-parallel executor rather than forking a run loop.
The reuse is realized by extracting the index-parallel core out of
`sweep_with_threads` into a `pub(crate) run_indexed<T>` (generic over the
per-job result, over a 0..n job range); `sweep_with_threads` becomes a thin
adapter over it. Behaviour-preserving (C1/C11/C23): the three pre-existing sweep
tests stay green — they ARE the proof the refactor preserved behaviour. A small
incidental win: the adapter consumes `points` via zip, dropping the old
per-point clone.
mc.rs adds McDraw{seed,report} / McFamily{draws,aggregate} (draws in seed-INPUT
order, independent of thread completion) / McAggregate (mean + p5/p25/p50/p75/p95
of all three run metrics — the V1 decision: covers every metric, not a "chosen"
one) / MetricStats, a private type-7 linear-interpolation `quantile`, and
`McAggregate::from_draws` (a pure post-run reduction recomputable from the
retained raw draws). Eager-agnostic firewall (#71): the API takes seeds + a
per-draw closure `Fn(u64,&[Scalar])->RunReport`, never a materialized stream Vec;
seed->Source construction stays a closure-body concern. An empty seed set is a
debug-asserted precondition, preserving the no-Result `-> McFamily` signature.
7 RED-first mc::tests pin: one draw per seed in input order; determinism across
1 vs 8 threads (C1); draw == an independent seeded run; distinct seeds perturb
the family; aggregate == from_draws(draws); type-7 quantile/mean math on a known
fixture; quantile endpoints + singleton. Workspace green, clippy -D warnings
clean, cargo doc clean.
refs #68
|
||
|
Brummel
|
b9203624d3 |
audit: cycle 0042 — drift-clean (seed-as-input)
Architect drift review over 682e459..HEAD (the seed-as-input cycle plus the prior unaudited `aura run --real` work). Cycle is sound: Fork B respected (seed forks at the data-generation edge into the source and the manifest; the engine loop sees only a Source — C12/C1/C3 intact), the contract names Source never Vec (precondition for #68/#52 delivered as specced), and the four seed-free callers thread 0 in lockstep. One medium doc-lag item found and fixed here: aura-engine/src/lib.rs module-doc listed the seed axis under "still to come"; seed-as-input shipped this cycle and RunManifest.seed is now live, so the headline doc now records SyntheticSpec as a delivered seeded Source producer (params + orchestration families remain future). One low carry-on note (not drift): run_sample_seeded exists only under #[cfg(test)] — intended this cycle (no --seed CLI flag per spec non-goals); the only non-test seed value remains 0. A scaling marker for #68, not debt. Regression gate (test suite — no dedicated regression scripts in project facts): cargo test --workspace green; clippy --workspace --all-targets -D warnings clean; cargo doc -p aura-engine clean (the new [`SyntheticSpec`] intra-doc link resolves). |
||
|
Brummel
|
c9f0e438e1 |
feat(aura-engine): seeded source + live RunManifest.seed (C12 seed-as-input)
A seeded source whose stream is fully determined by a u64 seed, making RunManifest.seed a live captured input instead of the dead 0 it was. This is C12's reconciliation of stochastic runs with C1: a run is bit-identical for a fixed seed (the seed is a captured input, not hidden nondeterminism), and a different seed gives a different-but-reproducible run. Precondition for the Monte-Carlo family (#68) and random param-sweep (#52). Engine (aura-engine): - SplitMix64: a ~10-line private, dependency-free PRNG. Chosen over a rand-family crate for guaranteed bit-stability across toolchains/crate versions — the whole value of seed-as-input is reproducibility, and a crate whose algorithm is not version-stable would silently break a recorded seed on a dep bump. No new direct dep. - SyntheticSpec + source(seed) -> impl Source: the Fn(u64) -> impl Source contract (Fork B — seed at the data-generation edge, outside the engine graph). Returns a Source, never a Vec, so the MC family can re-seed N times without materializing N streams. First cut collects to a VecSource internally; the signature does not name Vec, so a lazy generator is a drop-in replacement. Re-exported from lib.rs beside VecSource. CLI (aura-cli): - sim_optimal_manifest gains a seed: u64 param recorded into the manifest (the single Fork B capture site). All four existing seed-free callers (run_sample, run_sample_real, the sweep grid-report, run_macd) pass 0 unchanged — their determinism tests stay green. - A test-only run_sample_seeded + SeededTrace exercise a live seed and expose the drained sink trace, so the three e2e tests assert bit-identity at the trace level (strictly stronger than the folded 3-field metrics). Tests (RED-first): 2 engine producer tests (same-seed identical / different-seed differs) + 3 cli e2e tests (bit-identical trace, different-seed metrics differ, seed recorded in manifest). Full workspace green; clippy -D warnings clean. Errata vs plan 0042 (two compile-forced deviations, neither alters the spec contract): - source returns `impl Source + use<>`: under edition 2024 a bare `impl Source` captures the &self lifetime, which blocks boxing as Box<dyn Source + 'static> for run() (E0597). The source owns its data, so use<> (captures nothing) is correct. - SyntheticSpec is imported inside `#[cfg(test)] mod tests` in main.rs: it is used only by the seeded test vehicle, so a top-level import is unused in the non-test build under clippy -D warnings. closes #66 |
||
|
Brummel
|
0a2e2022a5 |
docs(aura-cli): note the --real real-data path in the module header
The header described `aura run` as synthetic-only; --real now also streams real M1 bars through the #71 Source seam. Keep the entry-point doc honest. |
||
|
Brummel
|
0ce843bada |
feat(aura-cli): aura run --real <SYMBOL> backtests the sample on real M1 bars
The first real-data backtest from the CLI: `aura run --real <SYMBOL> [--from <ms>] [--to <ms>]` runs the existing built-in sample signal-quality harness over real M1 *close* bars instead of synthetic prices, printing the same RunReport JSON as `aura run`. Bars are streamed lazily through aura_ingest::M1FieldSource (a Box<dyn Source>) — dogfooding the #71 Source seam, O(one chunk) resident, never eager. aura-ingest + data-server enter aura-cli's deps (data-server git line mirrored from aura-ingest). M1 only this cycle, deliberately. The TickSource is deferred: raw ticks are the wrong input for the bar-semantic SMA sample (an SMA over ticks is microstructure noise until a resampler node exists, C2), and tick's real driver is the realistic broker's bid/ask execution (C10) — it should arrive with that consumer, not speculatively here. Tick data is also local to only EURUSD/XAUUSD/GER40, whereas M1 covers the test symbol AAPL.US. Shape: run_sample_real builds sample_harness(), guards has_symbol / M1FieldSource::open (unknown symbol or empty window -> 'aura: no local data' + exit 2, never a panic), and reuses sim_optimal_manifest + the run_sample fold. parse_real_args is a pure, unit-tested grammar (mandatory symbol, then --from/--to in any order; bad/missing/duplicate flag -> Err). Data path is data_server::DEFAULT_DATA_PATH (no --data-path flag this cycle). Known simplification: the manifest window is derived by draining a *separate* single-pass probe source for first/last ts, so an unbounded window streams the archive twice (once to bound, once to run). Acceptable and O(1)-resident for now; a future Harness::run could surface the first/last cycle ts to drop the probe pass. Verified: cargo build/test/clippy --workspace all green; the gated run_sample_real_streams_real_close_bars_deterministically RAN (not skipped) over AAPL.US 2006-08; manual CLI smoke of the success + two error paths. |
||
|
Brummel
|
682e459554 |
audit: cycle 0041 — drift-clean (Source ingestion seam)
Architect drift review over 8b330e3..HEAD (the Source-seam cycle #71 plus the interim #67 optimize work and refactors). No semantic or contract drift: the seam faithfully preserves C3 (ms→epoch-ns at the one ingestion boundary), C4 (tie-by-source-index, byte-for-byte), and C7 (field-per-source SoA); the full suite is byte-identical green and the gated streaming tests pass on real data. Regression gate: `cargo test --workspace` green (the project declares no dedicated regression script; the suite is the gate). Tidied stale prose the cycle left behind (no behaviour change): - docs/design/INDEX.md (C12): replaced the cycle-0011 "deliberate eager gap" status note with a cycle-0041 realization note — the producer `Source` seam + streaming `M1FieldSource` now deliver per-source O(one-chunk) streaming; precisely scopes what remains open (cross-*sim* Arc<[T]> window sharing, still unbuilt until the orchestration families #66/#68/#69 consume it). - aura-core/src/lib.rs: dropped the now-shipped "Source trait + data-server ingestion" from the module doc's "still to come" list (aura-engine's twin line was fixed in the cycle; aura-core's was missed). - aura-ingest/src/lib.rs: module header now documents both coexisting paths (eager load_m1_window/M1Columns vs lazy streaming M1FieldSource). - aura-registry/src/lib.rs: rank_by/optimize doc comments linked to the private `metric_cmp` (a public→private intra-doc-link warning from the #67 commit); demoted to plain code spans, so `cargo doc --workspace` is now warning-clean. |
||
|
Brummel
|
8158e97dc6 |
feat(aura-ingest): streaming M1FieldSource — lazy data-server window through the seam
Completes the Source ingestion seam (plan Tasks 3-4): a real, lazily-streamed
data-server source proving the producer seam against the actual data source,
and closing the cycle-0011 deliberate eager-materialization gap.
- M1Field + a free decode(field, bar) — a pure projection of one M1 bar into
(epoch-ns ts, Scalar), reusing unix_ms_to_epoch_ns so ms->ns stays the one
C3 normalization point. Pure ⇒ hermetically unit-tested without an iterator.
- M1FieldSource: a streaming Source over a data-server M1 window. Holds one
Arc<[M1Parsed]> chunk (a zero-copy clone of the cache's chunk) + a cursor +
a pre-decoded head (so peek(&self) is cheap over a &mut next_chunk refill);
constructs each Scalar per-pull, refills when the chunk drains. Resident
footprint is O(one chunk), independent of window length.
- aura-engine moved [dev-dependencies] -> [dependencies] (M1FieldSource is a
library item implementing aura_engine::Source, not a test-only type).
- A gated streaming integration test (skips where local data is absent):
* end-to-end backtest through the seam to a RunReport, C1-deterministic
(two runs bit-identical);
* residency predicate — driving the source the way run() does (peek/next)
over the *unbounded* archive (~1.77M bars locally), peak resident_records
stays <= one data-server chunk (CHUNK_SIZE) while total >> CHUNK_SIZE:
O(one chunk), NOT O(window). A O(window) source would peak at `total`;
* two-field sharing (close + volume over one window, same ts axis).
The eager load_m1_window / close_stream path is kept (still valid for bounded
loads); the gap is closed by a streaming path now existing, not by deleting the
eager one. The residency test drives the full archive rather than the plan's
narrow 2006-08 window, which holds only ~21 bars locally — too few for the
> CHUNK_SIZE non-vacuity precondition; the unbounded archive is the stronger
"independent of window length" proof.
Verified: cargo build/test/clippy --workspace clean; the 3 gated tests ran
against real local data and passed; aura-ingest/aura-engine docs clean (two
pre-existing aura-registry doc-link warnings are from an earlier commit, not
this cycle).
closes #71
|
||
|
Brummel
|
eec129ee51 |
feat(aura-engine): Source ingestion seam — run() takes Vec<Box<dyn Source>>
The engine's ingestion input was the only type encoding an eager-dataflow
assumption: `Harness::run(Vec<Vec<(Timestamp, Scalar)>>)` — one fully
materialized stream per source. At realistic research scale (20y, 3 tick
streams, ~7.9e9 ticks) that layout is ~190 GB resident, non-residable. Yet
the merge loop never needs the whole stream: it only ever peeks the head
timestamp of each live source and pops one record. So the eager Vec is
gratuitous.
This lands the first half of the Source seam (plan Tasks 1-2):
- A `Source` trait (`peek(&self) -> Option<Timestamp>`,
`next(&mut) -> Option<(Timestamp, Scalar)>`), object-safe so the merge
holds `Vec<Box<dyn Source>>`. The peek/next split is the producer contract
the k-way merge drives.
- `VecSource`, an adapter over the old `Vec<(Timestamp, Scalar)>` — the
cycle-0011 eager shape, named. It IS the old behaviour.
- `Harness::run` re-typed to `Vec<Box<dyn Source>>`; the merge loop's index
arithmetic becomes peek/next. C4 tie-by-source-index is preserved (the
strictly-`<` replace + source-order scan is byte-for-byte the old
semantics). The forward+eval body is unchanged.
- All 53 call sites threaded VecSource-wrapped in the same change, so the
workspace compiles atomically and run output is byte-identical (C1).
Behaviour-preserving is the load-bearing guarantee: the full suite is green
and unchanged (the two-run C1 determinism pairs — real_bars r1/r2, blueprint
sweep-equality, harness h/h2 — stay byte-identical); aura-engine unit tests
129 -> 132 (+3 VecSource unit tests). No residency change yet: VecSource holds
the same Vec as before. The lazy data-server-backed streaming source and its
end-to-end residency proof are the second half (plan Tasks 3-4).
refs #71
|
||
|
Brummel
|
2b56e090a9 |
feat(aura-registry): optimize — argmax over a SweepFamily by a named metric
Ships C12 orchestration axis 2 (optimize): pick the single best point of a sweep family by a named RunMetrics field. The GREEN half of the RED-first iteration whose executable-spec landed in the previous commit. optimize(&SweepFamily, metric) -> Result<SweepPoint, RegistryError> returns the whole winning SweepPoint — its params coordinate AND its RunReport — because the caller needs the params that won, not just the metric. "Best" is fixed per metric (total_pips higher-is-better; max_drawdown / exposure_sign_flips lower-is-better); ties resolve to the earliest enumeration-order point (the family is in odometer order, and only a strictly-better later point displaces the incumbent). Single source of truth for "best", as #67 required ("reuse rank_by; do not fork"): rank_by's per-metric direction is extracted into a private `metric_cmp` helper that resolves the boundary metric *name* to a closed `enum Metric` once and returns one comparator closure. Both rank_by (sort by it) and optimize (argmax via `reduce`, strictly-better-displaces) now call it, so the per-metric direction lives in exactly one match. rank_by is behaviour-preserving — its existing tests pass unchanged. The sweep executor is untouched. No caller-supplied `direction`: one definition of best per metric, consistent with the shipped rank_by invariant (a caller direction would admit incoherent asks like the worst max_drawdown). An unknown metric is UnknownMetric. A SweepFamily is non-empty by construction (EmptyAxis is rejected upstream), so the argmax always yields a winner. CLI surface: the "pick the best" view is already served by `aura runs rank <metric>`, whose first line is the optimum over the persisted registry; optimize() is the new *programmatic* axis-2 entry, for World programs that hold a live SweepFamily and need the winning params back in-process — which the registry/RunReport path cannot give. No redundant CLI command added. closes #67 |
||
|
Brummel
|
bdc383eb8b |
test(aura-registry): RED executable-spec for optimize argmax — refs #67
The RED half of a RED-first iteration for the optimize/argmax orchestration axis (C12 axis 2). Pins the headline behaviour before the feature exists: optimize(&SweepFamily, metric) returns the single winning SweepPoint — its params coordinate AND its RunReport — chosen by the metric's fixed per-metric sense of best (reusing rank_by's direction semantics), with ties resolving to the earliest enumeration-order point. No caller-supplied direction: that would contradict the shipped rank_by invariant that "best" is fixed by each metric's meaning, and would admit incoherent asks (e.g. the worst max_drawdown). One definition of best per metric. RED for the right reason — optimize is absent (E0425), the sole error; cargo build --workspace stays green (test-only module). GREEN lands in the following commit, which closes #67. |
||
|
Brummel
|
d8e0fb70c8 |
refactor(aura-cli): centralize the sim-optimal RunManifest construction
Three RunReport builders hand-rolled the same RunManifest with identical commit/seed/broker fields (only params/window varied), drifting independently. Extract sim_optimal_manifest(params, window) — one source for the broker label, so #22's per-asset pip work has a single edit point. |
||
|
Brummel
|
6021ce6aeb |
docs(aura-engine): drop two stale aura-cli cross-references in graph_model
kind_str and collect_distinct_composites no longer exist in aura-cli (it consumes model_to_json directly), so the "mirrors aura-cli's ..." notes pointed at nothing. Keep the substantive rationale (Debug is PascalCase). |
||
|
Brummel
|
941503ed3d |
refactor(aura-engine): fold resolve/resolve_axes into one generic resolve_into
resolve (single Scalar per slot) and resolve_axes (a Vec<Scalar> axis per slot) were the same two-phase named-binding resolution against param_space(), duplicated almost verbatim — the live instance of the "two raise-sites for one invariant" hazard (cf. #45/#53). A change to binding precedence or error ordering had to touch both in lockstep. Collapse the shared skeleton into resolve_into<T>, parameterized by two closures: claim_ok (phase-1 claim-time gate, axis-only EmptyAxis, before the duplicate check) and kind_ok (phase-2 per-slot kind gate). The error total order now lives in exactly one place. Behaviour-preserving: same error for same input, all 231 tests green (the resolve/bind precedence tests pin it). |
||
|
Brummel
|
a8bd52eaff |
refactor(aura-core): own FieldSpec.name to drop the derive_signature heap leak
FieldSpec.name was &'static str, forcing derive_signature to Box::leak every re-exported composite OutField name. Because compile_with_params reruns derive_signature per sweep point, an N-point sweep over an M-output-field composite leaked N×M boundary names unbounded — a scaling wall the moment the World/sweep layer compiles families per point. Pull FieldSpec.name to String, mirroring the already-owned PortSpec.name and ParamSpec.name: a composite re-exports interior fields under runtime boundary names, so ownership is the correct model, not a workaround. FieldSpec loses Copy; the only fallout is one borrow at the render site. leak_name/Box::leak deleted. Behaviour-preserving: signature contents unchanged, all 231 tests green. |
||
|
Brummel
|
21c1621bd0 |
docs,engine: drop coined "compilat", use FlatGraph / "flat graph"
"Compilat" (German "Kompilat") was a coined noun for the product of the
bootstrap compilation — neither English nor a natural fit. The runtime
artifact already has a code identifier for exactly this thing: the
`FlatGraph` struct (harness.rs). Replace the coinage with that identifier:
- prose mentions -> "flat graph" (mirrors the type, reads plainly)
- definitional anchors -> `FlatGraph` (C11, C23, the running-graph line)
- `render_compilat` -> `render_flat_graph` (historical render symbol;
keeps the `render_blueprint` / `render_flat_graph`
source-vs-product pairing)
- `intra-compilat` -> `intra-graph`
- `from_compilat` (test) -> `from_flat`
"compilation" / "re-compilation" / "bootstrap-as-compilation" (the process,
ordinary English) are deliberately left untouched. Behaviour-preserving:
only comments, design ledger, specs/plans, one test-local variable and its
assert messages change. Full workspace test suite green; clippy clean.
|
||
|
Brummel
|
227d004c9d |
feat(aura-engine): reject unwired or double-wired input ports
A graph that leaves an interior input slot unconnected, or wires one slot from more than one producer, is now a compile-time error instead of a silently-wrong run. Until now an unwired interior slot was accepted and bootstrapped to an empty column (harness.rs:137-148), so a forgotten connection ran a deterministic but wrong backtest; two producers into one slot — ill-formed, since a slot holds one column — was likewise uncompiled-against. A single name-free check, `check_ports_connected`, is added to `validate_wiring` after the existing index/kind checks and before the nested-composite recursion. It counts coverage of each (interior-node, slot) uniformly across interior edges and role targets, and rejects zero coverage (new `CompileError::UnconnectedPort`) or >1 (new `CompileError::DoubleWiredPort`). Because `validate_wiring` is called once from `compile_with_params` and recurses, both the raw `Composite::new` path and the `GraphBuilder::build()` path inherit it at every nesting level with no builder-side code (mirrors `check_param_namespace_injective`, the param-side sibling). A composite's own open input roles (source: None) are coverage providers — the wired-by-enclosing boundary, the root already guarded by UnboundRootRole — not consumers. Index-based and name-free: nothing reaches the compilat, so C23 holds. Supporting fix: the check computes `signature()` on every interior node before the recursion validates that node's interior, so `derive_signature` and `interior_slot_kind` are made bounds-total (a placeholder kind for an out-of-range OutField/target, never a panic; the real fault is still reported by validate_wiring's guarded OutputPortOutOfRange/BadInteriorIndex). This latent panic was exposed by the new check, not introduced by it. Test maintenance (blast radius enumerated empirically, not hand-traced): four existing tests relied incidentally on under-wiring being accepted — three negative tests get a covering root role so their intended deep fault still surfaces via the recursion, and one param-order nesting test is fully wired (param order unchanged). Six new tests: five raw-surface (unwired, double-wire via edge+role and edge+edge, nested, open-role boundary) and one builder-surface forgotten-leg (proving the GraphBuilder inherits the check). Narrowed scope of #65: the original "promote names to load-bearing wiring keys / close the exposure-price swap structurally" goal was dropped — #64 already moved authoring to handles + visible port names, so the residual same-kind swap is a valid-but-wrong name choice no structural check catches, and a structural fix would push domain semantics into the domain-free engine (C10/C7). This ships the name-free half that stands on its own: wiring completeness, which catches forgotten and doubled connections. Verified: cargo test --workspace 231 passed / 0 failed; cargo clippy --workspace --all-targets -- -D warnings clean. closes #65 |
||
|
Brummel
|
b6174cff80 |
refactor(aura-engine): rename NodeHandle::in_/out to input/output
The trailing underscore on in_() was a keyword-escape scar (`in` is reserved).
Rename the NodeHandle port/field accessors to input()/output(), which also
aligns the authoring surface with the schema's own nomenclature
(NodeSchema.inputs/output, "input port"/"output field"): node.input("lhs")
now reads in the same vocabulary the schema uses, instead of a second one.
Mechanical, behaviour-preserving: the two method definitions plus every call
site in builder.rs and the aura-cli sample blueprints. Full workspace green;
clippy --all-targets -D warnings clean.
|
||
|
Brummel
|
50105c1957 |
refactor(aura-cli): author the sample blueprints via GraphBuilder
Adopt the name-based GraphBuilder (cycle 0039) in the CLI sample blueprints, which previously hand-wired raw positional Edge/Role/OutField via Composite::new. sma_cross, macd, signals, sample_blueprint_with_sinks, and macd_strategy_blueprint now read as typed-handle authoring: nodes are added for NodeHandles, ports/fields are wired by name (series/lhs/rhs/value, exposure/price, term[i], col[0], the macd histogram/signal/macd outputs), and build() lowers to the same index-wired Composite. This is what #64 was for — the builder was previously exercised only in its own tests; the showcase sample now dogfoods it. Behaviour-preserving: node order, instance names (.named), bound params (blend.weights[2]), and wiring are byte-identical, so param_space, the graph render, the single run, and the sweep are unchanged — guarded green by the existing aura-cli sample/sweep/param_space/render tests and the engine suite (full workspace 0 failed; clippy --all-targets -D warnings clean). sample_harness is deliberately left as a direct FlatGraph construction (baked Sma::new(2) nodes, the single-run path) — it builds the compilat directly, not a value-empty Composite, so it is not a GraphBuilder target. |
||
|
Brummel
|
a6a314bb98 |
feat(aura-engine): name-based blueprint wiring via GraphBuilder
Add an additive, fluent GraphBuilder authoring surface that wires a blueprint's topology by typed node handles and port/field names instead of raw positional indices. Node references are Copy NodeHandle values (a node reference cannot be mistyped); only port/field names are strings, resolved against each node's cached NodeSchema by exactly-one-match at a single fallible build() -> Result<Composite, BuildError> — the Binder posture, one level up. build() lowers to the unchanged Composite::new, so the compilat stays index-wired (C23): names resolve at the authoring boundary and never reach FlatGraph. A new From<Composite> for BlueprintNode lets add() accept nested composites. Verified: builder-authored sma_cross is structurally equal to the hand-wired fixture, and the full builder-authored harness lowers to a byte-identical FlatGraph (equal edges + sources). The five BuildError variants (Unknown/Ambiguous In/Out, BadHandle) are covered, and the SimBroker exposure/price legs are now addressable by name (the #21 legibility win). The structural close of the exposure/price swap (#21) is deliberately out of scope — this builder makes the swap legible, not impossible — tracked as #65. Two plan-test corrections applied during implementation (both in-scope, no behaviour change): the harness parity test uses compile_with_params with a topology-invariant param vector on both sides (the harness declares three params, so a no-param compile() would trip ParamArity), and the error asserts use .err()/Some(...) rather than unwrap_err() (Composite is not Debug). aura-engine 123 tests green (+9); full workspace green; clippy --all-targets -D warnings clean. Existing index-form Composite::new sites and tests untouched (coexistence). closes #64 |
||
|
Brummel
|
1f01badac7 |
refactor(aura-engine): share the SMA-cross harness test fixtures
Collapse the three verbatim-duplicated #[cfg(test)] fixtures — synthetic_prices, sma_cross, composite_sma_cross_harness — that lived in both blueprint.rs's and sweep.rs's test modules into one shared crate-root `#[cfg(test)] mod test_fixtures` (pub(crate) free fns). The two consumers now import from `crate::test_fixtures`; sweep.rs's test-mod `use` block sheds the imports that were only needed by the moved fns. Why: the topology had to move in lockstep across the two copies but no test pinned them equal, so an edit to one silently diverged the other (#53). One definition removes the hazard structurally — better than a pin-them-equal test, which loses its premise once there is a single definition. Test-only; no production code, public API, or runtime path changes. C16 untouched (a #[cfg(test)] module ships in no artifact). The CLI sample is no longer a third peer (cycle 0036 enriched it into a trend+momentum blend), so this is an engine-internal two-copy dedup, not the three-way the issue originally named. graph_model.rs keeps its own intentionally-minimal golden copy, out of scope. Plan glitch fixed in-flight: the plan's literal import list named sma_cross, but neither consumer calls it directly (only transitively via composite_sma_cross_harness), so importing it tripped the plan's own Step-7 clippy -D warnings gate. Trimmed to {composite_sma_cross_harness, synthetic_prices} — within the task's "drop now-unused imports" scope. Verified: cargo test --workspace green (aura-engine 114 passed, count unchanged — behaviour-preserving); the two load-bearing fixture consumers composite_sma_cross_runs_bit_identical_to_hand_wired (C1) and sweep_equals_n_independent_runs (C11/C12) both green; cargo clippy --workspace --all-targets -- -D warnings clean. closes #53 |
||
|
Brummel
|
47e0e605e1 |
feat(aura): surface bind-bound params in the graph model + viewer signature
A node built with `.bind(slot, value)` fixes a declared param to a structural
constant: the slot correctly leaves the tunable surface (param_space / sweep axes),
but it also vanished from the RENDERED signature, because prim_record emitted only
schema.params. The cycle-0036 `blend` (a LinComb(3) with weights[2] bound to 0.5)
rendered `LinComb[weights[0], weights[1]]` — a 2-arity signature over a 3-port box,
with the fixed 0.5 invisible. The signature misrepresented the node.
bind now ANNOTATES instead of erasing. All slots are shown; the bound one renders
`name=value`, dimmed. The blend renders:
blend: LinComb[weights[0], weights[1], weights[2]=0.5]
Structurally a twin of cycle 0035's instance-name thread (commit
|
||
|
Brummel
|
d069484559 |
audit: cycle 0036 — refresh stale sample cross-refs; drift-clean otherwise
Close-audit for cycle 0036 (commit range 560c2d0..94af4c7). Architect drift review against the design ledger + CLAUDE.md: no contract drift. No regression scripts are configured (the architect is the gate); build + test + clippy verified green. cycle 0036 tidy (clean): - C8/C9/C10 preserved: the enriched sample is multiply-nested composites over shipped primitives — multi-output is the macd composite re-exporting columns (not a >1-record primitive); brokers stay downstream nodes. - C12/C19 preserved: the eight sweep axes are a bijection with the injective, path-qualified param_space under nesting; the bound blend.weights[2] is correctly absent from both the axes and the surface. The re-captured render golden, the two in-crate pins, and the cli_run E2E pin all carry the identical eight-tuple in lockstep. - C14 preserved: the re-captured sample-model.json is the deterministic model of the enriched sample; the engine's own minimal model_golden (sample_root) is independent and byte-unchanged. - C23 preserved: bind resolves the param name to a fixed position; the fixed blend weight is a structural constant (deform-not-tune), dropped from the tuning surface, and the compilat is unchanged. Two stale cross-references found and refreshed (doc debt, no behaviour change): - crates/aura-engine/src/graph_model.rs: sample_root's "Mirrors build_sample() (...:161-190)" was doubly stale — build_sample moved and the CLI sample is now the richer signals graph; re-grounded to state sample_root is a deliberately minimal, independent serializer fixture. - crates/aura-cli/src/main.rs: sample_blueprint_with_sinks' "single source of the sample topology" + "SMA lengths + exposure scale" overclaimed post-enrichment; re-grounded to the root harness whose signal is the nested signals composite, with the eight free params. |
||
|
Brummel
|
94af4c788c |
feat(aura-cli): enrich the graph sample into a trend+momentum blend showcase
The built-in sample that `aura graph` renders and `aura sweep` runs is now a
"trend + momentum, weighted blend" strategy that exercises four authoring/viewer
capabilities the single-level sample left dark:
- multiply-nested composites: root -> signals -> {trend = sma_cross, momentum = macd};
- a multi-param node inside a composite: blend = LinComb(3) (weights[0]/[1] shown);
- a multi-output node: momentum (the macd composite) re-exports macd/signal/histogram,
two of which the blend consumes;
- bind(): blend.weights[2] is fixed as a structural constant, so it drops out of the
sweepable param surface (and renders absent).
The new `signals` composite reuses the existing sma_cross/macd builders unchanged; the
sweep re-paths its axes to the eight free params (still a 4-point grid) and runs an
18-tick warm-up stream (showcase_prices) so the MACD EMA-of-EMA chain and the all-Any
LinComb join warm up. The render fixture (sample-model.json) is re-captured, and a new
headless guard (viewer_nested_depth) pins that the viewer renders the nesting to depth 2
with valid Graphviz ids. Pure authoring over already-shipped primitives — no engine or
graph-viewer.js change.
The flat run_sample, run_macd/macd(), the inline model_golden test, and the viewer are
untouched. A third sweep-param pin in tests/cli_run.rs (the aura sweep E2E) was re-pathed
in lockstep with its in-crate twin.
closes #62
|
||
|
Brummel
|
0ae8320d14 |
feat(aura): surface explicit node instance name in the graph viewer
A leaf primitive built with `.named("fast")` now renders its `aura graph` viewer
box head as `fast: SMA[length]` — the instance name as a `:` declaration prefix.
An unnamed leaf keeps the bare `SMA[length]`.
Engine half: a new raw `PrimitiveBuilder::instance_name() -> Option<&str>` (the
explicit name, default not resolved) feeds a conditional leading `"name"` field
in prim_record, present only for an explicitly-named node; both golden twins
(inline model_golden + sample-model.json) re-captured. Viewer half: adaptNodes
carries `name`, the genDot leaf-emit forwards it, and cellLabel prepends the
`name: ` prefix when present (composites stay bare). `named()`'s non-empty
debug_assert is unchanged, its doc re-grounded to the now load-bearing Some/None
invariant (knob-address segment + prefix switch).
The name is a render/model-only debug symbol — dropped at lowering (C23), and the
model stays deterministic (C14). Parameter ganging (one knob, several nodes) was
considered and spun off as an explicit composite-shared-param idea (#61), not via
name collision (param_space is injective, C12/C19).
closes #58
|
||
|
Brummel
|
a051571d79 |
feat(aura-cli): render leaf param signature as TYPE[...] in the graph viewer
The viewer box head built the param signature in round parens — SMA(length) — which reads as a function call. Switch the brackets to square — SMA[length] — so the head reads as parametrisation, not invocation. cellLabel render-only change in graph-viewer.js; the JSON model surface and both goldens are untouched. |
||
|
Brummel
|
7d587d0c4e |
feat(aura-core): PrimitiveBuilder::bind — fix a node param as a structural constant
Add `PrimitiveBuilder::bind(slot, value)`: bind a declared param to a structural constant, removed from param_space entirely rather than pinned in the injected vector. This closes the gap #55 names — param-bearing nodes (Sma/Exposure/LinComb) had no constant path, only the no-sweep-mode SimBroker precedent. The motivating case is "SMA2-entry": the 2 is bound to the two-candle construction, so sweeping it would enumerate deformed strategies as valid family members. Approach (Option B, builder-level — the originally-planned direction): bind shrinks the builder's declared param surface (schema.params) and wraps its build closure to re-splice the captured constant at its original positional slot. The construction layer (collect_params / lower_items / param_space / compile_with_params) is byte-unchanged: both dock sites already key off builder.params(), so the shrink propagates for free. Addressing is by param name (the by-name authoring address space, C23/0032); the compilat stays wired by raw index — names never reach it. bind enforces "exactly one open param matches" itself (collect-all-then-reject, panic on zero / on duplicate), mirroring the resolve binder's posture, because a node's own schema.params is not covered by check_param_namespace_injective. Chained binds reconstruct the correct positional vector because each layer computes its slot index relative to the param list it sees (no global position table). Considered and rejected: a per-node builder_const(N) constructor (explodes combinatorially over which subset is constant, not dynamic); a ParamBinding overlay struct on Composite (would need new bookkeeping the builder-level form avoids). Verified: cargo build/clippy/test --workspace all green; new tests cover slot removal, positional reconstruction (chained + partial, via eval), the three panic paths, and the composite param_space projection; blueprint.rs construction layer confirmed byte-unchanged. Deferred (out of scope, follow-up issue): exporting a named frozen strategy as a blueprint-as-values collection — the issue's third fork, needs its own brainstorm. closes #55 |
||
|
Brummel
|
52e0214adb |
audit: cycle 0033 (#54) — refresh stale JSON-writer cross-refs after to_json→serde
Architect drift review over 065cf1d..HEAD. Code + contracts hold; the only drift was documentation cross-references made stale by retiring RunReport's hand-rolled to_json. Contracts (all hold): - C14: to_json still emits canonical, flat, deterministic JSON — now serde-produced. model_to_json (the graph-model JSON) stays hand-rolled and was correctly untouched. - C16: serde_json dev-dep → normal-dep in aura-engine is squarely sanctioned by the amended per-case policy (INDEX.md:576-583) — a vetted standard crate used where it does the job (incl. the frozen bot), removing a hand-rolled writer. No gap. - C1: report types have no HashMap; serde is declaration-order/deterministic; the r1.to_json()==r2.to_json() asserts and the new to_json_equals_serde_disk_shape pin stay green. Regression: none configured (profile regression: []) — no-op; architect is the gate. Resolution — fix (3 prose cross-refs, fixed inline this commit): - graph_model.rs:5 (module doc) — claimed the model JSON is in the "RunReport::to_json house style (no serde)"; to_json is serde now. Reframed: the model JSON is hand-rolled (no serde), and is the engine's last hand-rolled JSON writer since 0033. - graph_model.rs:30 (json_str doc) — cross-referenced RunReport::json_str, deleted this cycle. Reworded to describe the escape set directly (graph_model's own json_str stays). - docs/design/INDEX.md:729 — "golden-tested like RunReport::to_json" framed the hand-rolled model JSON by a now-serde to_json. Reframed to name model_to_json as the surviving hand-rolled writer. Cycle 0033 drift-clean after this tidy. Not a milestone close (the param-sweep milestone fieldtest is a separate deliberate act). Gates: clippy --workspace --all-targets -D warnings clean (doc-only edits, compile-inert). refs #54 |
||
|
Brummel
|
f1d0bf00ec |
fix(aura-engine): unify RunReport stdout JSON with the on-disk serde shape
RunReport had two divergent JSON encoders: the registry serialized to disk via serde_json (params as an array-of-pairs, floats as 2.0), while stdout rendered through a hand-rolled to_json (params as an object, whole floats as 2). The same record was a structurally different document on disk than on the wire, and no test pinned their relationship — most visibly, `aura runs list` read a record from disk via serde and reprinted it in a different shape. Retire the hand-rolled writer: RunReport::to_json now delegates to serde_json::to_string(self), so stdout is byte-identical to the runs.jsonl line for the same record. The hand-rolled json_str helper is deleted (the graph_model.rs json_str is a separate symbol, untouched — the graph-model JSON is out of scope, #58/#28). A new pin test, to_json_equals_serde_disk_shape, asserts to_json() == serde_json::to_string(&report) so the two encoders can never silently diverge again. Observable change: stdout `params` is now an array-of-pairs and finite f64 fields carry a fractional part (2 -> 2.0); the nested envelope, field order, window [from,to], and integer-valued seed/exposure_sign_flips are unchanged. The on-disk shape does NOT change — only stdout moves to match disk. serde_json is promoted from a dev-dependency to a normal dependency of aura-engine (to_json is non-test code). Under the amended C16 per-case policy this passes: serde_json is a vetted standard crate already in the workspace (registry, ingest), deterministic (C1-safe, already relied on for the disk path), and is exactly "the vetted standard crate doing what the code would otherwise hand-roll" — it removes the last hand-rolled RunReport JSON writer. Goldens flipped to the serde shape: the engine canonical-form golden, the aura-cli sweep golden (cli_run.rs), and the aura-cli odometer-order sweep golden (main.rs) — the last was not in the plan's inventory; the cargo test --workspace gate surfaced it and it took the identical transformation. Structural CLI asserts and the r1.to_json()==r2.to_json() determinism asserts are shape-agnostic and stay green. Spec docs/specs/0033, plan docs/plans/0033, boss-signed (unanimous 5-lens panel). Two stale prose cross-refs (graph_model.rs:30, docs/design/INDEX.md:729) deferred to cycle-close audit. Gates: cargo build/test --workspace green, clippy --all-targets -D warnings clean. closes #54 |
||
|
Brummel
|
1b7e4ad169 |
feat(aura-engine): param_space() injectivity as a compile invariant
A non-injective param_space() — two slots under one path-qualified name — is the by-name knob address space (C12/C19) being broken: no binding can select one slot without the other. This cycle makes it a structural compile error. One check, check_param_namespace_injective over the param_space() names, replaces the whole fan-in machinery (signature_of, leaf_has_param, check_fan_in_distinguishability, check_composite_fan_in) and runs from compile_with_params AND from both binders before resolve/resolve_axes — so the canonical by-name author sees the structural cure (CompileError::DuplicateParamPath, carrying the path and pointing at .named(...)) instead of the AmbiguousKnob symptom (#59). With an injective space guaranteed before resolve, no bound name can multi-match, so BindError::AmbiguousKnob is dead and removed (both resolver arms -> unreachable!). param_space() stays infallible; the invariant lives in the compile step (C23). The check is strictly the by-name-addressability property. The old fan-in predicate (equal signature AND >=1 param) also rejected two collisions that are NOT param_space duplicates — an asymmetric param/paramless collision and a role-vs-leg collision — which guarded render identity, dead since the renderer was retired in 0026. Both are intentionally dropped; a future node-/wiring-name check, if wanted, is decoupled from injectivity. The new invariant correctly rejected a pre-existing fixture (multi_output_composite's two unnamed Sma legs); cured in place with the .named(fast/slow) the invariant mandates. A new E2E test drives the cured cross through the by-name binder end-to-end (resolve + bootstrap + run to a populated exposure trace). C9/C12/C19/C23 ledger notes amended. Verified: cargo build/test --workspace green (0 failed), clippy --workspace --all-targets -D warnings clean. closes #59 |
||
|
Brummel
|
ffed8cc612 |
feat(aura-core,aura-engine,aura-cli): node-instance naming retires ParamAlias
Every blueprint node now carries a name. A primitive builder gains an optional
instance name (Sma::builder().named("fast")); when omitted it defaults to the
node's type label, ASCII-lowercased verbatim ("SMA"->"sma", "SimBroker"->
"simbroker"). param_space() is now uniformly <node>.<param> at every level
including the root (sma_cross.fast.length, exposure.scale). Fan-in
distinguishability (C9) and signature_of re-key onto the node name: two same-type
siblings both defaulting to "sma" collide and IndistinguishableFanIn fires, so
one act -- naming the legs "fast"/"slow" -- fixes both the naming collision and
the fan-in check. The index-addressed ParamAlias overlay, Composite.params, the
Composite::new 5th argument, aliases_on, and check_alias_indices are removed
(Role.name and OutField.name untouched). Ledger C9 and C23 amended.
This is why the change is correct, not merely convenient: blueprints are
value-empty (C11), so the thing that would otherwise distinguish two SMAs --
their length -- is not present at construction; identity must come from a name,
not a deferred value. Closes the #56 friction surfaced by the param-space &
sweep milestone fieldtest (a freshly-authored 2-SMA cross was unbindable and
would not bootstrap without two hand-counted ParamAlias overlays).
Two plan defects were corrected during implementation and verified:
- the three new fan-in/path tests authored sma_cross at root level, which would
qualify to "fast.length" (root prefix is empty) and is not the nested case the
fan-in check inspects; nesting sma_cross under a root (a shared
sma_cross_under_root helper) restores the asserted "sma_cross.fast.length" and
IndistinguishableFanIn{node:2};
- three cycle-0030 named-binder tests bound the real harness by the old names
(sma_cross.fast / scale) and were migrated to the new path strings, surfaced by
the workspace test gate.
Verified by the orchestrator: cargo build/test --workspace green (198 tests,
0 red), clippy --all-targets -D warnings clean. model_to_json emits the type
label + factory param names (node-name-independent), so sample-model.json and the
graph_model golden are byte-identical (untouched). NodeSchema gained a Default
derive (the builder default-name test constructs an empty schema). fieldtests/
are frozen non-workspace records, not migrated.
closes #56
|
||
|
Brummel
|
acaa6b0df8 |
audit: cycle 0030 (#35) — drift-clean; named param binding
cycle 0030 tidy (clean). The architect drift review against docs/design/INDEX.md
and CLAUDE.md found the cycle drift-clean: every load-bearing contract the named
binding layer touches holds.
- C1 / C12 / C19 / C23: a pure authoring layer — sweep.rs, the lowering path,
collect_params, and param_space() are unchanged; with/axis/Binder/SweepBinder/
resolve/resolve_axes all terminate into the existing bootstrap_with_params /
GridSpace::new / sweep. No new state, no concurrency, no path into the run loop.
Equivalence pinned by named_binder_runs_bit_identical_to_positional and
named_axes_grid_parity_with_positional. The match key is the exact param_space()
name (p.name == name), read-only.
- C7: pure-equality kind check (v.kind() != p.kind), no coercion.
- C16 / C10: std-only (the
|
||
|
Brummel
|
937257e368 |
feat(aura-engine,aura-cli): named param binding — sweep axes .axis()/.sweep() (0030 iter 2)
Bind a sweep's axes by name instead of by a positional GridSpace:
bp.axis("sma_cross.fast", [2, 3]).axis("sma_cross.slow", [4, 5]).axis("scale", [0.5]).sweep(run)
The sweep half of spec 0030, completing the named-binding feature (#35). A pure
authoring layer over the existing GridSpace::new / sweep primitives; engine core
untouched (C1/C12/C19/C23).
This iteration:
- resolve_axes(): shares resolve's name->slot mapping (iter 1), adds the EmptyAxis
check (the variant defined in iter 1 is now first constructed here) and a
per-element axis kind-check (every element of each axis, first offending element
in axis order). Its validation is a strict superset of GridSpace::new's
(arity / non-empty / per-element kind), so SweepBinder::sweep's downstream
GridSpace::new(...).expect(...) is infallible by construction — it can never
panic on author input.
- Composite::axis -> SweepBinder -> SweepBinder::sweep (Result<SweepFamily, BindError>).
- The CLI sample sweep converted to the named form (behaviour-preserving).
Plan correction (verified by the orchestrator): the iteration plan said only the
GridSpace import was orphaned by the CLI conversion, but replacing the free
sweep(&grid, ..) call with the .sweep(..) method also orphans the free `sweep`
import — both were dropped from aura-cli to satisfy clippy -D warnings. The Scalar
import (from aura_core) stays used and was untouched.
Verified (run by the orchestrator): 6 new aura-engine tests green —
resolve_axes round-trip, EmptyAxis, MissingKnob, the per-element KindMismatch on a
mixed axis (the superset/no-panic guarantee), a builder no-panic wrong-kind test,
and a GridSpace .points() parity test; the two sweep JSON goldens
(sweep_report_renders_four_points_in_odometer_order + the cli_run integration
golden) green unchanged (name/value/order-preserving conversion); full
`cargo test --workspace` green and `cargo clippy --workspace --all-targets
-- -D warnings` clean.
refs #35
|
||
|
Brummel
|
64b19ab3d5 |
feat(aura-engine,aura-cli): named param binding — single-run .with()/.bootstrap() (0030 iter 1)
Bind a blueprint's open knobs by name for a single run instead of by a positional,
Scalar-wrapped Vec in param_space() order:
bp.with("sma_cross.fast", 2).with("sma_cross.slow", 4).with("scale", 0.5).bootstrap()
A pure authoring layer over the existing Composite::param_space / bootstrap_with_params
primitives; the engine run loop and the construction core are untouched (C1/C12/C19/C23
preserved). Raw literals lower via Into<Scalar> (the literal fixes the variant: 2->I64,
0.5->F64, pinned by
|
||
|
Brummel
|
e97906a0e1 |
test(aura-core): pin bare-literal Into<Scalar> kind inference
Authoring layers that lower raw literals via `impl Into<Scalar>` (the named-param-binding surface, spec 0030 / #35) rely on a bare integer literal inferring as i64 and a bare float as f64 — because i64/f64/bool are the only `From<…> for Scalar` impls, each literal class resolves to exactly one. This was true but untested: `from_widenings` uses suffixed literals (`7i64`), so a regression in the inference path would survive it. Behaviour-preserving pin of existing aura-core behaviour. Closes the one grounding gap the grounding-check flagged on spec 0030; also guards the property independently — adding a second integer `From` impl would now break loudly here rather than silently at a `.with("knob", 2)` call site. refs #35 |