4a3ae05db2bc4a5fc0500e351a804aba85547fff
39 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
86841b0740 |
audit(0088): cycle close — drift fixes + C24 ledger refresh; close construction service
Architect drift review of cycle 0088 (#157, the introspectable fail-fast construction service: §A shared gate predicates, §B the GraphSession/Op/replay per-op surface + introspection, §C the aura graph build/introspect CLI). What holds (confirmed against the diff): no-second-validator (C24) — edge_kind_check is the single kind-check site called by both validate_wiring and GraphSession::connect, and validate_wiring / check_param_namespace_injective are reused verbatim in finish(); engine domain-free (invariant 9) — construction takes an injected Fn(&str)->Option<PrimitiveBuilder>, no registry, the closed match + std_vocabulary_types live in aura-std; topology-as-data / no DSL (C24/C17/inv 10) — the engine Op is serde-free, the OpDoc wire DTO is CLI-side, ops are static and non-Turing-complete; replay-equals-builder byte-identity (construction_e2e.rs) protects C9/C19/C23. Resolution: - fix (drift-med): the root-role-boundness gate was the one holistic check finish() re-implemented instead of sharing — extracted check_root_roles_bound (blueprint.rs) and called from BOTH compile_with_cells and finish(), so finish() now reuses ALL the holistic gates and the no-second-validator claim is literally true. Behaviour- preserving (same UnboundRootRole variant; unbound_root_role_is_rejected stays green). - fix (drift-med): removed the #![allow(dead_code)] on construction.rs — empirically unnecessary (the pub re-exported surface is reachable; clippy -D warnings green without it) — and its now-inaccurate comment (it claimed removal "when the consumer lands", but the consumer is a separate crate). Also tidied a stale graph_construct.rs module-doc line. - fix (ledger): C24 Status refreshed — #157 moved out of "Remaining" with a cycle-0088 realization note (the construction service: eager/holistic gate split, build-free introspection, emits the #155 blueprint); #159 reframed as paired with #157. - carry-on (debt-med, forward-noted): the std_vocabulary roster is now a third hand-kept copy (match + test list + std_vocabulary_types) and introspect --vocabulary escalates #160's gap to user-facing — noted on #160; still fails safe. - carry-on (scaling-low, forward-noted): the engine Op -> wire OpDoc direction is unguarded for future Op variants — noted on #156 (the format-extension track). No architect-sweep / regression script is declared in the project facts, so the suite is the gate: cargo test --workspace 51 suites green; clippy --all-targets -D warnings clean. Cycle drift-clean. Ephemeral cycle artifacts removed (lifecycle convention): docs/specs/0088-*, docs/plans/0088-*. closes #157 |
||
|
Brummel
|
ea1ca32dbd |
feat(0088): §A shared gate predicates — edge_kind_check, resolution helpers, try_bind
Iteration-1 §A of the construction service (#157): the surface-agnostic shared predicates the eager op-script path and the holistic finalize gates both call — no second validator (C24). - edge_kind_check (blueprint.rs): the per-edge producer/consumer kind check, lifted verbatim out of validate_wiring's edge loop into a pub(crate) fn that validate_wiring now calls — behaviour-preserving (same Bootstrap(KindMismatch) variant; the existing compile-time gate test stays green). - resolve_input_slot / resolve_output_field (builder.rs): the exactly-one-match name→index resolution extracted as pub(crate) fns over (&NodeSchema, &str), so the runtime-String op-script names share GraphBuilder's resolution; GraphBuilder delegates and maps to the unchanged BuildError variants. - try_bind + BindOpError (aura-core node.rs): the fallible Result twin of bind (bind keeps its panic contract and pinned messages verbatim; try_bind is a separate method reporting the same three conditions as values). - check_param_namespace_injective + validate_wiring widened to pub(crate) for the finalize-stage reuse by the upcoming GraphSession::finish (§B). Partial iteration: §B (the GraphSession/Op/replay surface + introspection) and the §A→§B integration land next (plan Tasks 4-8). compile_with_params / check_ports_connected are behaviourally unchanged. Full suite green; clippy clean. Plan correction folded in: Task 3's test originally referenced aura_std::Sma — infeasible inside aura-core (aura-std depends on aura-core; the reverse edge is a dependency cycle). Adapted to a local PrimitiveBuilder probe with identical assertions. refs #157 |
||
|
Brummel
|
d5602ec5ad |
feat(0087): blueprint serialization & loader — close the graph-as-data loop
C24 first cut (#155): a blueprint — the param-generic, named graph-as-data a Rust builder produces — is now a first-class serializable data value with a canonical, versioned format, and the engine has the missing load path (data -> blueprint -> FlatGraph), not only the Rust-builder construction path. What ships: - `blueprint_to_json` / `blueprint_from_json` in a new `aura-engine` blueprint_serde module: a faithful serde DTO projection (the build closures dropped; re-derived on load), top-level `format_version` envelope, canonical compact JSON (struct field order, defaults omitted via skip_serializing_if). - A resolver seam: the loader is generic over an injected `Fn(&str) -> Option<PrimitiveBuilder>`; the concrete closed `match` over the aura-std vocabulary lives in aura-std (`std_vocabulary`), never in the engine (the engine stays domain-free: lib deps aura-core + aura-analysis only, never the node-vocabulary crate). This is C24's compiled-in closed-set referenced by type identity, NOT a dynamic/marketplace node registry (domain invariant 9). - serde derives on the serialized plain types (Edge, Target, OutField, Role, BoundParam, ScalarKind); BoundParam additionally re-exported from aura-core's root (an additive fix the plan's file list missed). Load-bearing scope decisions (derived, logged on #155): - Sinks are excluded from the serialized blueprint — a recording sink captures an mpsc::Sender (runtime identity, not param/topology, C19 value-empty); sink addressing is the C18-registry / #101 concern. The round-trip test attaches identical sinks post-load to both twins. - Construction-arg builders (LinComb(arity), CostSum(n), SimBroker(pip), Session(..)) are out of the round-trippable set: their structural args are a C20 structural-axis concern the param-generic format does not yet encode; an absent type_id fails the load cleanly (UnknownNodeType), never a silent wrong graph. #155's vocabulary is the 22 zero-arg aura-std builders. Additively extensible later (#156). Orchestrator hardening on review: the serializer now emits bound params in ascending original-slot order (mirroring the loader), so the canonical form is bind-order-independent — a precondition for content-addressing (#158). Identity today (every #155 node has <=1 param); holds by construction for future multi-param nodes. Acceptance (all green): a serialized blueprint runs bit-identical (C1) to its Rust-built twin; serializer/loader are inverse (canonical idempotence, incl. a recursive nested composite); unknown-type and unsupported-version fail named, never panicking. cargo test --workspace 748 passed; clippy --all-targets -D warnings clean. closes #155 |
||
|
Brummel
|
0a06afed29 |
feat(0078): cross-instrument generalization as a validation step
Last piece of the Inferential-validation milestone (#146): a new `aura generalize` subcommand grades how consistently a single candidate holds across a set of instruments — the across-instrument axis of the same anti-false-discovery concern as trials-deflation (#144, within one family) and plateau-over-peak (#145, within one instrument's surface). Aggregator/validator, not a third selector (user-ratified): it runs a brought candidate (a single-cell stage1-r grid — --fast/--slow/--stop- length/--stop-k pinned to single values) across an instrument list (--real SYM1,SYM2,...), collects per-instrument R-metrics, and reduces them to a worst-case floor (min over instruments) + a sign-agreement count + the per-instrument breakdown. R-only (C10 — R is the only account/instrument-agnostic unit); the worst-case min is the cross- instrument sibling of PlateauMode::Worst and satisfies the non- domination constraint by construction (a strong instrument cannot lift a min). Rejected: a cross-instrument t-stat mean/std*sqrt(M), which reintroduces the pooled mean the milestone warns against. - RunManifest gains a first-class `instrument` lineage field (serde- widened, C14/C18 — legacy lines load as None, existing manifest bytes unchanged); the compat read-mirror threads it in lockstep. - FamilyKind::CrossInstrument (C12 comparison axis); the M per-instrument runs persist as a CrossInstrument family, each member self-identifying via its stamped instrument. The aggregate is recomputable from the members (McAggregate precedent), printed live. - The generalization reduction lives registry-side beside optimize_ deflated/optimize_plateau (C9), reusing resolve_metric/metric_value/ is_r_metric; check_r_metric is the data-free pre-check the CLI runs before evaluating any instrument. - Additive: existing sweep/walkforward/mc/standalone paths stay byte- identical (C23 — instrument omitted when None via skip_serializing_if). Verified by the orchestrator: cargo test --workspace green (0 failed — the new generalization_*, check_r_metric, parse_generalize_*, and the gated cross-instrument E2E all pass; the E2E ran the real GER40/USDJPY path on this host, exit 0); cargo clippy --workspace --all-targets -D warnings clean. closes #146 |
||
|
Brummel
|
cb32658fd1 |
feat(0077): prefer a robust parameter plateau over the in-sample peak
Add an opt-in plateau selection objective for walk-forward, recorded on the same RunManifest.selection carrier #144 introduced. Default argmax is byte-identical (C23); plateau is reached only via a new --select flag. What landed: - FamilySelection / SelectionMode reshape (report.rs): the selection RULE (mode) and its ANNOTATION are orthogonal — deflation fields become Option (present iff Argmax), plus PlateauMean/PlateauWorst variants and neighbourhood_score/n_neighbours (present iff Plateau*). Legacy lines still load: the deflation scalars deserialize from bare values via serde default (C14/C18). compat.rs embeds FamilySelection by value — reshape flows through untouched. - GridSpace::axis_lens() + SweepBinder::sweep_with_lattice (engine): the grid radixes in param_space()/odometer order. sweep() delegates to sweep_with_lattice with the lattice dropped, so every existing .sweep() caller is byte-unchanged. - optimize_plateau + PlateauMode + closed_neighbourhood (aura-registry, C9): each member scores as the mean/worst of its closed mixed-radix grid neighbourhood's metric_value; the winner is the smoothed argmax by the metric's direction (earliest-odometer tie, as optimize). Pure, no RNG (C1); in-sample only (C2). A higher_is_better helper now sources the per-metric direction once, shared by metric_cmp and optimize_plateau. - CLI --select <argmax|plateau:mean|plateau:worst> (default argmax; unknown token exits 2). walkforward_family dispatches via a select_winner helper; sweep_over/stage1_r_sweep_over carry the lattice as Option<Vec<usize>>. runs-family display gains a plateau(<mode>)=<score> over <n> cells line. RandomSpace-refuse: walkforward has no --random producer, so the plateau-without-lattice guard is structural — select_winner returns the refuse on a None lattice (the caller prints the message and exits 2), unit-tested at the helper, not via a --random E2E (decided + recorded on #145 comment 1974, with the lattice-seam fork). Tests: plateau-picks-plateau-not-spike, mean-vs-worst, lower-is-better direction-flip, closed-neighbourhood golden, single-member degeneracy, C1 determinism; CLI select-parse, select_winner refuse; E2E argmax-byte-identical (C23), plateau:mean and plateau:worst provenance. Full workspace suite green; clippy --all-targets -D warnings clean. closes #145 |
||
|
Brummel
|
a2959050a4 |
feat(0076): deflate the sweep winner's metric for the number of trials
Adds optimize_deflated beside optimize in aura-registry: the in-sample walk-forward selection now records, on each OOS winner's manifest, how much the winner's metric is inflated by the size of the search it won — a deflated score and (R arm) an overfit probability — without changing which member wins (additive, C23; a regression test pins optimize_deflated's winner == optimize's). - R arm (sqn_normalized/expectancy_r/...): a centred moving-block reality-check. Each member's per-trade R series is mean-subtracted to impose the no-edge null, resampled via the r_bootstrap kernel (extracted to a shared resample_block, byte-identical), and the best-of-K null-max gives overfit_probability = (count(null >= raw)+1)/(n+1) and deflated_score = raw - p95(null). Deterministic given the seed (C1). - total_pips arm: a closed-form expected-max-of-K dispersion floor (inv_norm_cdf + expected_max_of_normals), no probability. The record (FamilySelection on RunManifest) rides the proven serde(default, skip_serializing_if) widening + a compat.rs mirror, so legacy runs.jsonl/families.jsonl lines load unchanged and unstamped sweep/mc/standalone manifests stay byte-identical (C14/C18). The SelectionMode enum reserves a Plateau slot for #145. CLI: walk-forward stamps both arms; runs family <id> rank surfaces a human-readable deflated/overfit line. Quality review (re-dispatched after the implement-loop's quality gate exhausted) caught a real defect: on the R arm with positive resamples but no member carrying trade_rs (a family loaded from disk — trade_rs is serde-skipped), the null was all-NEG_INFINITY, turning deflated_score into +inf. Fixed by collapsing "no usable null" (zero resamples OR no trade_rs) into one degenerate floor (deflated == raw, overfit == 1.0); pinned by optimize_deflated_no_trade_rs_floors_instead_of_infinity. Verified: cargo test --workspace and cargo clippy --workspace --all-targets -D warnings both green. Frictionless Stage-1 R; n_eff advisory and reload-time recompute deferred (spec 0076 Out of scope). closes #144 |
||
|
Brummel
|
68605b7d88 |
refactor(stage1-r): rename the strategy-output param namespace exposure -> bias
Complete the deferred rename tail from cycle 0065's #126 (which renamed only the typed metric, keeping the param namespace uniformly "exposure" so the single-run rename would not smuggle in a ~30-site sweep-pinned change). Now renamed together as one consistent unit, all driven by the Bias node's instance name: - Bias::builder().named("exposure") -> .named("bias") (the harness builders + the engine fixtures in test_fixtures.rs / blueprint.rs); - the derived knob path exposure.scale -> bias.scale (Composite::param_space derives the knob from the node name) -- the sweep .axis/.range bindings and .with() points (blueprint.rs, main.rs), the walkforward ParamSpecs (walkforward.rs), the member_key dir names, and the JSON byte-pins in cli_run.rs / report.rs / main.rs tests; - the exposure_scale manifest param label -> bias_scale (the single-run manifest builders + the streaming_seam / real_bars fixtures). The rename is surgical: the three targets (named("exposure"), exposure.scale, exposure_scale) are syntactically distinct from the deliberately-kept "exposure" forms, which are untouched -- SimBroker's pre-reframe exposure input slot + prev_exposure + the exposure-integral; the LongOnly / gate / latch ports named "exposure"; and the on-disk "exposure" tap label (the recorded bias series that feeds `chart --tap exposure`, decoupled from the node name via ColumnarTrace::from_rows("exposure", ...)). No on-disk back-compat break: the knob path is a runtime param address, and recorded params in old runs.jsonl / families.jsonl are inert data strings (a pre-existing recorded "exposure.scale" still loads, it is just data). INDEX.md cycle-0065 realization note amended: the param namespace is recorded as the now-completed tail; the SimBroker slot + the on-disk tap label remain the deliberate permanent keeps (#117), and exposure_sign_flips stays a serde alias. Verified: cargo build --workspace --all-targets clean; clippy --all-targets -D warnings clean; full suite 514 passed / 0 failed (source + byte-pins renamed together); live `aura run` emits "bias_scale", `aura sweep` emits "bias.scale". closes #134 refs #126 #117 |
||
|
Brummel
|
2c43296c2c |
feat(stage1-r): bias rename + stop-rule + position-management + summarize_r (iter 1)
Iteration 1 of the Stage-1 "R-based signal quality" cycle (spec 0065): a strategy's signal quality, measured in R on its unsized bias stream, feed-forward. New discrete-trade machinery, NOT a SimBroker extension. What lands: - exposure -> bias (refs #126): the Exposure node/type/file/output-field renamed to Bias (computation unchanged: clamp(signal/scale,-1,+1); the SEMANTICS change — the output is the unsized strategy bias, sizing leaves the strategy). Scoped to the semantic core; behaviour-preserving cosmetics are deferred (see below). - stop-rule nodes (refs #119): VolStop = k*EMA(|price - prev_price|) (one fused node; the volatility that defines 1R, close-to-close — true-range ATR deferred, needs OHLC) + FixedStop (constant, the test fixture / fixed-vs-vol structural-axis sibling). - PositionManagement (refs #127): the stateful heart. Latches the entry-cycle stop distance as the FROZEN R-denominator (never re-read), marks/exits no-look-ahead like SimBroker (a position earns from the next cycle; an exit realises against the cycle's close), and emits ONE dense per-cycle R-record (C8). Stop-outs are NOT capped at -1R (a gap through the stop realises R < -1 — the honest loss tail); R is computed size-invariantly (size = (exit-entry)*dir / latched_dist cancels). The trade ledger is the rows where closed_this_cycle; the R-equity is cum_realized + unrealized; a position open at window end is the last row (open=true) — explicit, never silent MtM. - summarize_r + RMetrics (refs #129): the post-run fold (NOT an in-graph node — recorders drain post-run). E[R], win-rate, profit-factor, avg win/loss R, by-trade max R-drawdown, n_open_at_end (window-end force-close). SQN / conviction terciles / net-of-cost / RunMetrics.r are iteration 2. Design adversarially hardened before implementation (12-juror refute panel; decisions on #117). Ratified implementer deviations from the plan snippet, verified by hand: - col-4 `direction` tracks the OPEN position at cycle end (window-end synthesis; names the reopened leg on a reversal), falling back to the closed trade's dir only when flat. summarize_r does not read col 4; the R-metrics are unaffected. - .named("exposure") kept on the 4 previously-unnamed Bias nodes so the auto-derived param-path stays exposure.scale (no manifest/dir-name drift) — the unnamed nodes would otherwise flip to bias.scale. - intra-doc links [`Exposure`] -> [`Bias`] in sim_broker/latch + the sample-model test pin (cosmetic, behaviour-neutral; broken intra-doc links fail cargo doc). - the E2E drives a full bootstrapped Harness (stronger than the plan's direct-node drive — exercises the real cross-crate producer->consumer seam). Deferred (behaviour-preserving, separate cosmetic pass — NOT this iteration): RunMetrics.exposure_sign_flips / Metric::ExposureSignFlips, SimBroker/LongOnly "exposure" input ports, the "exposure" tap/trace names, and the .named("exposure") instance labels. Iteration 2: the Sizer seam, the RiskExecutor composite (+ the Veto documented-seam), summarize_r enrichment, RunMetrics.r, and the CLI/recording surface. Verification (orchestrator-run, not agent-claimed): - cargo build --workspace: clean. - cargo test --workspace: all green, 0 failed (incl. the new bias/stop_rule/ position_management unit tests, the summarize_r arithmetic tests, and the stage1_r_e2e capstone + layout guard). - cargo clippy --workspace --all-targets -- -D warnings: clean (exit 0). - Keystone RED tests pass: no_lookahead_bias_exit_realises_the_held_move, stop_out_is_not_capped_at_minus_one_r, no_gap_stop_is_exactly_minus_one_r, reversal_closes_one_leg_and_reopens, open_at_window_end_is_carried_on_the_last_row. refs #117 #119 #126 #127 #129 |
||
|
Brummel
|
27f850dc52 |
feat(sweep-key): generic filesystem-portable member key + LongOnly bool node
Generalises the #105 foot-gun: the family-member trace key was derived from two hardcoded axis names (f{fast}s{slow}), collision-free only over the built-in SMA grid. It is now derived from the axes that actually VARY (SweepBinder::varying_axes) and rendered as a filesystem-conformant directory component: - charset restricted to [A-Za-z0-9._-] (valid on Linux/Windows/macOS, also URL-path- and cloud-sync-safe); any other byte sanitised; - case-less values (i64/timestamp decimal, bool true/false, f64 via Display, no scientific notation) so two members of one family never differ only by letter case -> no silent overwrite on case-insensitive filesystems (NTFS/APFS); - length-capped (MAX_KEY=200) with a version-stable FNV-1a fallback for the unbounded-grid edge. Pinned singleton axes are omitted; the SMA sweep's member dirs change from f2s4 to signals.trend.fast.length-2_signals.trend.slow.length-4 (its --trace integration test + the stale prose updated accordingly). Adds the first aura-std node with a bool *param*, LongOnly (a long-only exposure gate: enabled=true clamps short exposure to >=0, false passes through) — the block the momentum demo strategy (next commit) sweeps to prove the key generic over a bool axis. The engine change is one additive pure read accessor; no existing signature changes, trace state stays out of the engine. Self-verified: cargo build --workspace, cargo test --workspace (all green incl. the LongOnly node tests, the member_key worked-examples corpus, the varying_axes accessor test, and the updated SMA sweep-trace integration test), cargo clippy --workspace --all-targets -D warnings. refs #105 |
||
|
Brummel
|
91ef69ed7b |
feat(aura-engine): RandomBinder — by-name random param-sweep
Add `RandomBinder`, the by-name sibling to `SweepBinder`, so a random sweep (`RandomSpace`, C12.1) can be built by name against `param_space()` instead of by positional `Vec<ParamRange>` slot order. By-name resolution makes a same-kind transposition (e.g. swapping the I64 ranges for `fast.length` and `slow.length`) structurally impossible — the failure class `RandomSpace::new`'s positional validation passes silently. Direct structural mirror of `SweepBinder`: `Composite::range` opens the binder, `.range(name, ParamRange)` accumulates, `.sweep(count, seed, run)` resolves the named ranges via the shared `resolve_into` (new `resolve_ranges` caller) and runs the disjoint sweep. New `BindError::EmptyRange` mirrors `EmptyAxis`; `ParamRange::is_empty` homes the non-empty invariant the named layer pre-checks so `RandomSpace::new` cannot fail. closes #79 |
||
|
Brummel
|
67c1f51cfe |
perf(aura-std): O(1) Kahan-compensated running-sum SMA (was O(length) re-sum)
Sma::eval re-summed the whole window every tick — O(length)/tick, multiplied across millions of bars and every sweep point. Replace with the industry-standard incremental running sum (ta-lib shape): the node owns the window ring and keeps a running sum, adding the new sample and subtracting the evicted one each cycle, so the input column drops to depth 1 (lookbacks() = [1], like Ema). The running sum carries Kahan/Neumaier compensation (the fix pandas' rolling mean adopted) so it does not drift over long runs; Ema needs none — its recurrence is contractive. Determinism (C1) holds: same input -> same run, reproducibly. The float output differs from the full re-sum in the last ULPs (a new, deterministic baseline); the suite needed only one arity-assumption update (Sma lookback 3 -> 1), no equity golden changed. closes #39 |
||
|
Brummel
|
43716be10e |
refactor(aura-engine): Cell into the construction path — param-plane base/frontend split
Cell becomes the carrier of the construction path; Scalar narrows to the author/render boundaries. The validated/enumerated param point carries no redundant kind (it lives once, in the declared param-space); at the author edge the kind is a checksum — two independent sources, the typed value vs. the slot — so the self-describing Scalar stays there. The name->slot binding is dynamic (C23), so the check is necessarily a runtime one and the value must self-describe for it. Base/frontend split (the AnyColumn push/push_cell pattern one level up): compile_with_cells / bootstrap_with_cells are the kind-check-free base; compile_with_params / bootstrap_with_params are the frontend that adds only the per-value kind checksum, strips to cells (new Scalar::cell(), the partner of Scalar::from_cell), and delegates. lower_items loses its per-primitive kind-check; PrimitiveBuilder::build and the std node builders (sma/ema/exposure/lincomb) read cells. Boundary: construction -> Cell (PrimitiveBuilder::build, lower_items, GridSpace.axes/points, SweepPoint.params, the sweep closure). Author edges stay Scalar (GridSpace::new, bind, compile_with_params/bootstrap_with_params). walkforward chosen_params stays a self-describing Scalar report record (Option A — WalkForwardResult carries no space); the cell winner is reconstructed once at the WindowRun site via from_cell. injective-check moved ahead of the arity-check in the frontend to preserve the pre-split error order (DuplicateParamPath before ParamArity). The lossy i64->f64 param projection (scalar_as_f64 / scalar_as_param_f64) is deliberately untouched — a separate follow-up. Behaviour-preserving (C1): build --all-targets / test / clippy -D warnings / doc all clean across the workspace. |
||
|
Brummel
|
82635fa259 |
refactor: Cell becomes the hot-path value carrier (C7/C8 realization)
Motivation ---------- The C7 realization note ( |
||
|
Brummel
|
cd3d1ca9ed |
refactor(aura-core): split Scalar into a tag-free Cell + ScalarKind
Motivation
----------
`Scalar` was a tagged enum (I64/F64/Bool/Ts), so every scalar value
physically carried its own kind tag. But the kind is already known from
the schema/port/column the value flows through (C7: the type is a
property of the column, not of the value — the hot path is already
columnar `Column<T>`, and `AnyColumn::get` *reconstructs* the tag from
the column on the way out). The per-value tag was therefore redundant
with the kind the surrounding context already holds.
That redundancy had three costs:
* It baked an implicit `match` (a branch) into every function that read
a Scalar payload — even where the caller statically knew the type.
The tag could never be exploited away.
* Size: a tagged enum is tag + payload = 16 bytes (f64/i64 alignment),
twice the 8 bytes the value needs. A `Column<Scalar>` would be double
the memory and half the cache utilisation.
* It is the shared root of several downstream papercuts we keep hitting
— the lossy f64 manifest field, the `unreachable!` panic on a
non-numeric param, the serde-tag question — all symptoms of "the type
is baked into the value".
Change
------
Introduce `Cell`: a type-erased 64-bit word (`struct Cell(u64)`) that is
not readable without external type context. It is constructed per base
type (`from_i64/from_f64/from_bool/from_ts`) and read only by naming the
type at the call site (`i64()/f64()/bool()/ts()`) — each a branch-free
bit-cast. The hot path resolves the kind once at the boundary (from the
schema) and then reads natively, with no per-value branch. `Cell` knows
nothing of `Scalar` or `ScalarKind`; the dependency is strictly one-way,
and it lives in its own `cell.rs` (more is planned on top of it).
`Scalar` becomes `struct { kind: ScalarKind, cell: Cell }` — the
self-describing form for the dynamic boundaries (builder binding,
serialization, rendering), built on top of `Cell`. Its `as_*` accessors
now `debug_assert` the kind and return the native value (free in
release); calling the wrong accessor is a caller bug, not a checked
`Option`. The variant constructors `Scalar::I64(..)` become associated
fns `Scalar::i64(..)`.
`PartialEq` is hand-written (not derived) to preserve the former enum's
value semantics: kinds must match, then native payloads compare, so f64
keeps IEEE-754 behaviour (`NaN != NaN`, `+0.0 == -0.0`) and a kind
mismatch is never equal even when the raw words coincide. A fixture
(`scalar_eq_is_value_not_bitwise`) pins exactly the cases where bit- and
value-equality diverge, so it can't silently regress. `Cell`'s own
`Eq`/`Hash` stay bitwise — correct for a raw word.
The change is behaviour-preserving: Scalar's observable behaviour is
identical to the pre-Cell enum (the value-equality fixture proves it);
only the internal representation changed. The ~440 call sites across the
workspace are a mechanical constructor rename plus ~12 destructuring
sites (match-arms / `let`-patterns) rewritten to `kind()` + `as_*`.
Verified: cargo build --workspace --all-targets, cargo clippy --workspace
--all-targets -- -D warnings, cargo test --workspace — all green.
|
||
|
Brummel
|
eec129ee51 |
feat(aura-engine): Source ingestion seam — run() takes Vec<Box<dyn Source>>
The engine's ingestion input was the only type encoding an eager-dataflow
assumption: `Harness::run(Vec<Vec<(Timestamp, Scalar)>>)` — one fully
materialized stream per source. At realistic research scale (20y, 3 tick
streams, ~7.9e9 ticks) that layout is ~190 GB resident, non-residable. Yet
the merge loop never needs the whole stream: it only ever peeks the head
timestamp of each live source and pops one record. So the eager Vec is
gratuitous.
This lands the first half of the Source seam (plan Tasks 1-2):
- A `Source` trait (`peek(&self) -> Option<Timestamp>`,
`next(&mut) -> Option<(Timestamp, Scalar)>`), object-safe so the merge
holds `Vec<Box<dyn Source>>`. The peek/next split is the producer contract
the k-way merge drives.
- `VecSource`, an adapter over the old `Vec<(Timestamp, Scalar)>` — the
cycle-0011 eager shape, named. It IS the old behaviour.
- `Harness::run` re-typed to `Vec<Box<dyn Source>>`; the merge loop's index
arithmetic becomes peek/next. C4 tie-by-source-index is preserved (the
strictly-`<` replace + source-order scan is byte-for-byte the old
semantics). The forward+eval body is unchanged.
- All 53 call sites threaded VecSource-wrapped in the same change, so the
workspace compiles atomically and run output is byte-identical (C1).
Behaviour-preserving is the load-bearing guarantee: the full suite is green
and unchanged (the two-run C1 determinism pairs — real_bars r1/r2, blueprint
sweep-equality, harness h/h2 — stay byte-identical); aura-engine unit tests
129 -> 132 (+3 VecSource unit tests). No residency change yet: VecSource holds
the same Vec as before. The lazy data-server-backed streaming source and its
end-to-end residency proof are the second half (plan Tasks 3-4).
refs #71
|
||
|
Brummel
|
941503ed3d |
refactor(aura-engine): fold resolve/resolve_axes into one generic resolve_into
resolve (single Scalar per slot) and resolve_axes (a Vec<Scalar> axis per slot) were the same two-phase named-binding resolution against param_space(), duplicated almost verbatim — the live instance of the "two raise-sites for one invariant" hazard (cf. #45/#53). A change to binding precedence or error ordering had to touch both in lockstep. Collapse the shared skeleton into resolve_into<T>, parameterized by two closures: claim_ok (phase-1 claim-time gate, axis-only EmptyAxis, before the duplicate check) and kind_ok (phase-2 per-slot kind gate). The error total order now lives in exactly one place. Behaviour-preserving: same error for same input, all 231 tests green (the resolve/bind precedence tests pin it). |
||
|
Brummel
|
a8bd52eaff |
refactor(aura-core): own FieldSpec.name to drop the derive_signature heap leak
FieldSpec.name was &'static str, forcing derive_signature to Box::leak every re-exported composite OutField name. Because compile_with_params reruns derive_signature per sweep point, an N-point sweep over an M-output-field composite leaked N×M boundary names unbounded — a scaling wall the moment the World/sweep layer compiles families per point. Pull FieldSpec.name to String, mirroring the already-owned PortSpec.name and ParamSpec.name: a composite re-exports interior fields under runtime boundary names, so ownership is the correct model, not a workaround. FieldSpec loses Copy; the only fallout is one borrow at the render site. leak_name/Box::leak deleted. Behaviour-preserving: signature contents unchanged, all 231 tests green. |
||
|
Brummel
|
21c1621bd0 |
docs,engine: drop coined "compilat", use FlatGraph / "flat graph"
"Compilat" (German "Kompilat") was a coined noun for the product of the
bootstrap compilation — neither English nor a natural fit. The runtime
artifact already has a code identifier for exactly this thing: the
`FlatGraph` struct (harness.rs). Replace the coinage with that identifier:
- prose mentions -> "flat graph" (mirrors the type, reads plainly)
- definitional anchors -> `FlatGraph` (C11, C23, the running-graph line)
- `render_compilat` -> `render_flat_graph` (historical render symbol;
keeps the `render_blueprint` / `render_flat_graph`
source-vs-product pairing)
- `intra-compilat` -> `intra-graph`
- `from_compilat` (test) -> `from_flat`
"compilation" / "re-compilation" / "bootstrap-as-compilation" (the process,
ordinary English) are deliberately left untouched. Behaviour-preserving:
only comments, design ledger, specs/plans, one test-local variable and its
assert messages change. Full workspace test suite green; clippy clean.
|
||
|
Brummel
|
227d004c9d |
feat(aura-engine): reject unwired or double-wired input ports
A graph that leaves an interior input slot unconnected, or wires one slot from more than one producer, is now a compile-time error instead of a silently-wrong run. Until now an unwired interior slot was accepted and bootstrapped to an empty column (harness.rs:137-148), so a forgotten connection ran a deterministic but wrong backtest; two producers into one slot — ill-formed, since a slot holds one column — was likewise uncompiled-against. A single name-free check, `check_ports_connected`, is added to `validate_wiring` after the existing index/kind checks and before the nested-composite recursion. It counts coverage of each (interior-node, slot) uniformly across interior edges and role targets, and rejects zero coverage (new `CompileError::UnconnectedPort`) or >1 (new `CompileError::DoubleWiredPort`). Because `validate_wiring` is called once from `compile_with_params` and recurses, both the raw `Composite::new` path and the `GraphBuilder::build()` path inherit it at every nesting level with no builder-side code (mirrors `check_param_namespace_injective`, the param-side sibling). A composite's own open input roles (source: None) are coverage providers — the wired-by-enclosing boundary, the root already guarded by UnboundRootRole — not consumers. Index-based and name-free: nothing reaches the compilat, so C23 holds. Supporting fix: the check computes `signature()` on every interior node before the recursion validates that node's interior, so `derive_signature` and `interior_slot_kind` are made bounds-total (a placeholder kind for an out-of-range OutField/target, never a panic; the real fault is still reported by validate_wiring's guarded OutputPortOutOfRange/BadInteriorIndex). This latent panic was exposed by the new check, not introduced by it. Test maintenance (blast radius enumerated empirically, not hand-traced): four existing tests relied incidentally on under-wiring being accepted — three negative tests get a covering root role so their intended deep fault still surfaces via the recursion, and one param-order nesting test is fully wired (param order unchanged). Six new tests: five raw-surface (unwired, double-wire via edge+role and edge+edge, nested, open-role boundary) and one builder-surface forgotten-leg (proving the GraphBuilder inherits the check). Narrowed scope of #65: the original "promote names to load-bearing wiring keys / close the exposure-price swap structurally" goal was dropped — #64 already moved authoring to handles + visible port names, so the residual same-kind swap is a valid-but-wrong name choice no structural check catches, and a structural fix would push domain semantics into the domain-free engine (C10/C7). This ships the name-free half that stands on its own: wiring completeness, which catches forgotten and doubled connections. Verified: cargo test --workspace 231 passed / 0 failed; cargo clippy --workspace --all-targets -- -D warnings clean. closes #65 |
||
|
Brummel
|
a6a314bb98 |
feat(aura-engine): name-based blueprint wiring via GraphBuilder
Add an additive, fluent GraphBuilder authoring surface that wires a blueprint's topology by typed node handles and port/field names instead of raw positional indices. Node references are Copy NodeHandle values (a node reference cannot be mistyped); only port/field names are strings, resolved against each node's cached NodeSchema by exactly-one-match at a single fallible build() -> Result<Composite, BuildError> — the Binder posture, one level up. build() lowers to the unchanged Composite::new, so the compilat stays index-wired (C23): names resolve at the authoring boundary and never reach FlatGraph. A new From<Composite> for BlueprintNode lets add() accept nested composites. Verified: builder-authored sma_cross is structurally equal to the hand-wired fixture, and the full builder-authored harness lowers to a byte-identical FlatGraph (equal edges + sources). The five BuildError variants (Unknown/Ambiguous In/Out, BadHandle) are covered, and the SimBroker exposure/price legs are now addressable by name (the #21 legibility win). The structural close of the exposure/price swap (#21) is deliberately out of scope — this builder makes the swap legible, not impossible — tracked as #65. Two plan-test corrections applied during implementation (both in-scope, no behaviour change): the harness parity test uses compile_with_params with a topology-invariant param vector on both sides (the harness declares three params, so a no-param compile() would trip ParamArity), and the error asserts use .err()/Some(...) rather than unwrap_err() (Composite is not Debug). aura-engine 123 tests green (+9); full workspace green; clippy --all-targets -D warnings clean. Existing index-form Composite::new sites and tests untouched (coexistence). closes #64 |
||
|
Brummel
|
1f01badac7 |
refactor(aura-engine): share the SMA-cross harness test fixtures
Collapse the three verbatim-duplicated #[cfg(test)] fixtures — synthetic_prices, sma_cross, composite_sma_cross_harness — that lived in both blueprint.rs's and sweep.rs's test modules into one shared crate-root `#[cfg(test)] mod test_fixtures` (pub(crate) free fns). The two consumers now import from `crate::test_fixtures`; sweep.rs's test-mod `use` block sheds the imports that were only needed by the moved fns. Why: the topology had to move in lockstep across the two copies but no test pinned them equal, so an edit to one silently diverged the other (#53). One definition removes the hazard structurally — better than a pin-them-equal test, which loses its premise once there is a single definition. Test-only; no production code, public API, or runtime path changes. C16 untouched (a #[cfg(test)] module ships in no artifact). The CLI sample is no longer a third peer (cycle 0036 enriched it into a trend+momentum blend), so this is an engine-internal two-copy dedup, not the three-way the issue originally named. graph_model.rs keeps its own intentionally-minimal golden copy, out of scope. Plan glitch fixed in-flight: the plan's literal import list named sma_cross, but neither consumer calls it directly (only transitively via composite_sma_cross_harness), so importing it tripped the plan's own Step-7 clippy -D warnings gate. Trimmed to {composite_sma_cross_harness, synthetic_prices} — within the task's "drop now-unused imports" scope. Verified: cargo test --workspace green (aura-engine 114 passed, count unchanged — behaviour-preserving); the two load-bearing fixture consumers composite_sma_cross_runs_bit_identical_to_hand_wired (C1) and sweep_equals_n_independent_runs (C11/C12) both green; cargo clippy --workspace --all-targets -- -D warnings clean. closes #53 |
||
|
Brummel
|
7d587d0c4e |
feat(aura-core): PrimitiveBuilder::bind — fix a node param as a structural constant
Add `PrimitiveBuilder::bind(slot, value)`: bind a declared param to a structural constant, removed from param_space entirely rather than pinned in the injected vector. This closes the gap #55 names — param-bearing nodes (Sma/Exposure/LinComb) had no constant path, only the no-sweep-mode SimBroker precedent. The motivating case is "SMA2-entry": the 2 is bound to the two-candle construction, so sweeping it would enumerate deformed strategies as valid family members. Approach (Option B, builder-level — the originally-planned direction): bind shrinks the builder's declared param surface (schema.params) and wraps its build closure to re-splice the captured constant at its original positional slot. The construction layer (collect_params / lower_items / param_space / compile_with_params) is byte-unchanged: both dock sites already key off builder.params(), so the shrink propagates for free. Addressing is by param name (the by-name authoring address space, C23/0032); the compilat stays wired by raw index — names never reach it. bind enforces "exactly one open param matches" itself (collect-all-then-reject, panic on zero / on duplicate), mirroring the resolve binder's posture, because a node's own schema.params is not covered by check_param_namespace_injective. Chained binds reconstruct the correct positional vector because each layer computes its slot index relative to the param list it sees (no global position table). Considered and rejected: a per-node builder_const(N) constructor (explodes combinatorially over which subset is constant, not dynamic); a ParamBinding overlay struct on Composite (would need new bookkeeping the builder-level form avoids). Verified: cargo build/clippy/test --workspace all green; new tests cover slot removal, positional reconstruction (chained + partial, via eval), the three panic paths, and the composite param_space projection; blueprint.rs construction layer confirmed byte-unchanged. Deferred (out of scope, follow-up issue): exporting a named frozen strategy as a blueprint-as-values collection — the issue's third fork, needs its own brainstorm. closes #55 |
||
|
Brummel
|
1b7e4ad169 |
feat(aura-engine): param_space() injectivity as a compile invariant
A non-injective param_space() — two slots under one path-qualified name — is the by-name knob address space (C12/C19) being broken: no binding can select one slot without the other. This cycle makes it a structural compile error. One check, check_param_namespace_injective over the param_space() names, replaces the whole fan-in machinery (signature_of, leaf_has_param, check_fan_in_distinguishability, check_composite_fan_in) and runs from compile_with_params AND from both binders before resolve/resolve_axes — so the canonical by-name author sees the structural cure (CompileError::DuplicateParamPath, carrying the path and pointing at .named(...)) instead of the AmbiguousKnob symptom (#59). With an injective space guaranteed before resolve, no bound name can multi-match, so BindError::AmbiguousKnob is dead and removed (both resolver arms -> unreachable!). param_space() stays infallible; the invariant lives in the compile step (C23). The check is strictly the by-name-addressability property. The old fan-in predicate (equal signature AND >=1 param) also rejected two collisions that are NOT param_space duplicates — an asymmetric param/paramless collision and a role-vs-leg collision — which guarded render identity, dead since the renderer was retired in 0026. Both are intentionally dropped; a future node-/wiring-name check, if wanted, is decoupled from injectivity. The new invariant correctly rejected a pre-existing fixture (multi_output_composite's two unnamed Sma legs); cured in place with the .named(fast/slow) the invariant mandates. A new E2E test drives the cured cross through the by-name binder end-to-end (resolve + bootstrap + run to a populated exposure trace). C9/C12/C19/C23 ledger notes amended. Verified: cargo build/test --workspace green (0 failed), clippy --workspace --all-targets -D warnings clean. closes #59 |
||
|
Brummel
|
ffed8cc612 |
feat(aura-core,aura-engine,aura-cli): node-instance naming retires ParamAlias
Every blueprint node now carries a name. A primitive builder gains an optional
instance name (Sma::builder().named("fast")); when omitted it defaults to the
node's type label, ASCII-lowercased verbatim ("SMA"->"sma", "SimBroker"->
"simbroker"). param_space() is now uniformly <node>.<param> at every level
including the root (sma_cross.fast.length, exposure.scale). Fan-in
distinguishability (C9) and signature_of re-key onto the node name: two same-type
siblings both defaulting to "sma" collide and IndistinguishableFanIn fires, so
one act -- naming the legs "fast"/"slow" -- fixes both the naming collision and
the fan-in check. The index-addressed ParamAlias overlay, Composite.params, the
Composite::new 5th argument, aliases_on, and check_alias_indices are removed
(Role.name and OutField.name untouched). Ledger C9 and C23 amended.
This is why the change is correct, not merely convenient: blueprints are
value-empty (C11), so the thing that would otherwise distinguish two SMAs --
their length -- is not present at construction; identity must come from a name,
not a deferred value. Closes the #56 friction surfaced by the param-space &
sweep milestone fieldtest (a freshly-authored 2-SMA cross was unbindable and
would not bootstrap without two hand-counted ParamAlias overlays).
Two plan defects were corrected during implementation and verified:
- the three new fan-in/path tests authored sma_cross at root level, which would
qualify to "fast.length" (root prefix is empty) and is not the nested case the
fan-in check inspects; nesting sma_cross under a root (a shared
sma_cross_under_root helper) restores the asserted "sma_cross.fast.length" and
IndistinguishableFanIn{node:2};
- three cycle-0030 named-binder tests bound the real harness by the old names
(sma_cross.fast / scale) and were migrated to the new path strings, surfaced by
the workspace test gate.
Verified by the orchestrator: cargo build/test --workspace green (198 tests,
0 red), clippy --all-targets -D warnings clean. model_to_json emits the type
label + factory param names (node-name-independent), so sample-model.json and the
graph_model golden are byte-identical (untouched). NodeSchema gained a Default
derive (the builder default-name test constructs an empty schema). fieldtests/
are frozen non-workspace records, not migrated.
closes #56
|
||
|
Brummel
|
acaa6b0df8 |
audit: cycle 0030 (#35) — drift-clean; named param binding
cycle 0030 tidy (clean). The architect drift review against docs/design/INDEX.md
and CLAUDE.md found the cycle drift-clean: every load-bearing contract the named
binding layer touches holds.
- C1 / C12 / C19 / C23: a pure authoring layer — sweep.rs, the lowering path,
collect_params, and param_space() are unchanged; with/axis/Binder/SweepBinder/
resolve/resolve_axes all terminate into the existing bootstrap_with_params /
GridSpace::new / sweep. No new state, no concurrency, no path into the run loop.
Equivalence pinned by named_binder_runs_bit_identical_to_positional and
named_axes_grid_parity_with_positional. The match key is the exact param_space()
name (p.name == name), read-only.
- C7: pure-equality kind check (v.kind() != p.kind), no coercion.
- C16 / C10: std-only (the
|
||
|
Brummel
|
937257e368 |
feat(aura-engine,aura-cli): named param binding — sweep axes .axis()/.sweep() (0030 iter 2)
Bind a sweep's axes by name instead of by a positional GridSpace:
bp.axis("sma_cross.fast", [2, 3]).axis("sma_cross.slow", [4, 5]).axis("scale", [0.5]).sweep(run)
The sweep half of spec 0030, completing the named-binding feature (#35). A pure
authoring layer over the existing GridSpace::new / sweep primitives; engine core
untouched (C1/C12/C19/C23).
This iteration:
- resolve_axes(): shares resolve's name->slot mapping (iter 1), adds the EmptyAxis
check (the variant defined in iter 1 is now first constructed here) and a
per-element axis kind-check (every element of each axis, first offending element
in axis order). Its validation is a strict superset of GridSpace::new's
(arity / non-empty / per-element kind), so SweepBinder::sweep's downstream
GridSpace::new(...).expect(...) is infallible by construction — it can never
panic on author input.
- Composite::axis -> SweepBinder -> SweepBinder::sweep (Result<SweepFamily, BindError>).
- The CLI sample sweep converted to the named form (behaviour-preserving).
Plan correction (verified by the orchestrator): the iteration plan said only the
GridSpace import was orphaned by the CLI conversion, but replacing the free
sweep(&grid, ..) call with the .sweep(..) method also orphans the free `sweep`
import — both were dropped from aura-cli to satisfy clippy -D warnings. The Scalar
import (from aura_core) stays used and was untouched.
Verified (run by the orchestrator): 6 new aura-engine tests green —
resolve_axes round-trip, EmptyAxis, MissingKnob, the per-element KindMismatch on a
mixed axis (the superset/no-panic guarantee), a builder no-panic wrong-kind test,
and a GridSpace .points() parity test; the two sweep JSON goldens
(sweep_report_renders_four_points_in_odometer_order + the cli_run integration
golden) green unchanged (name/value/order-preserving conversion); full
`cargo test --workspace` green and `cargo clippy --workspace --all-targets
-- -D warnings` clean.
refs #35
|
||
|
Brummel
|
64b19ab3d5 |
feat(aura-engine,aura-cli): named param binding — single-run .with()/.bootstrap() (0030 iter 1)
Bind a blueprint's open knobs by name for a single run instead of by a positional,
Scalar-wrapped Vec in param_space() order:
bp.with("sma_cross.fast", 2).with("sma_cross.slow", 4).with("scale", 0.5).bootstrap()
A pure authoring layer over the existing Composite::param_space / bootstrap_with_params
primitives; the engine run loop and the construction core are untouched (C1/C12/C19/C23
preserved). Raw literals lower via Into<Scalar> (the literal fixes the variant: 2->I64,
0.5->F64, pinned by
|
||
|
Brummel
|
e304dbaae1 |
feat(aura-core): name input ports
PortSpec gains a non-load-bearing `name: String`, so an input port is named just as FieldSpec.name (output) and ParamSpec.name (param) already are — input ports were the lone unnamed member of the node signature. Identity stays positional by slot (C23); the name is render/debug only, never read by bootstrap or the run loop. PortSpec drops Copy (String is not Copy), exactly as ParamSpec already does. - Every aura-std node names its input slots: SMA/EMA "series", Sub/Add "lhs"/"rhs", Exposure "signal", SimBroker "exposure"/"price" (the slots become self-documenting), LinComb "term[i]" and Recorder "col[i]" generated in their build loops (mirroring LinComb's existing weights[i] param loop). - derive_signature carries a composite's Role.name into the derived input port (it was dropped before — the output side already carried FieldSpec.name), so the graph model is homogeneously named at both levels. - model_to_json (port_json + the composite-header inputs in scope_json) emits the name as a third tuple element: ["f64","any","exposure"]. The byte golden was re-captured (machine bytes) and its substring twins updated; the model is now fully named across inputs/outputs/params. - All 16 PortSpec construction sites threaded in one compile-gate change; test fixtures carry fixture names. C8 realization note added to the design ledger. Why name-only, no validation: the name is a pure debug symbol. Wire-by-name was rejected (it would be a C23 contract change). Bootstrap slot-wiring validation (which would close #21's same-kind swap footgun) is deferred to its own cycle — a name alone does not catch the swap; it makes the slots self-documenting and gives a future validation something to check against. Verified: cargo test --workspace 168 green; clippy --all-targets -D warnings clean; cargo build --workspace clean. Read-only render path (C9), no serde (C14), scalar kinds unchanged (C4). closes #50 refs #21 refs #51 |
||
|
Brummel
|
1b3909316e |
feat(aura): node signature lives in the blueprint; collapse Blueprint into Composite
Consolidate the node data structure so every node's signature (NodeSchema:
inputs/output/params) is declared once and exists in the blueprint pre-build,
and dissolve the special "root graph" type. Behaviour-preserving (C1).
Signature vs sizing
- NodeSchema is now the static signature only: InputSpec -> PortSpec{kind,firing},
with lookback removed. The signature is fully static per blueprint (input
kinds/firing, output fields, params); LinComb's variable arity is a builder arg,
not an injected param.
- The one param-dependent quantity, an input's buffer lookback (e.g. Sma's window =
its injected length), moves out of the signature to Node::lookbacks() -> Vec<usize>,
read only by bootstrap for sizing. Node::schema() is removed.
- LeafFactory -> PrimitiveBuilder, which carries the full NodeSchema. The built node
no longer re-declares it: closes the params-declared-twice drift (#36, the 8
per-node factory_params_match_built_node_schema lockstep tests are deleted — their
subject is now structurally impossible) and a value-empty recipe exposes its full
I/O interface pre-build (#43).
Root is just a bound composite
- struct Blueprint is deleted; its compile/bootstrap/param_space methods move onto
Composite. Role gains source: Option<ScalarKind> (None = open interior port,
Some = bound ingestion feed). A composite is runnable iff every root role is bound;
the "main graph" is no longer a category, only the fully-source-bound composite.
New error CompileError::UnboundRootRole for an open root role.
- BlueprintNode::signature() answers uniformly for both arms: Primitive returns the
builder's declared schema, Composite derives it from the interior (role kinds in,
OutField kinds out, aggregated params), pre-build, no build.
compile -> FlatGraph -> bootstrap
- compile validates structure pre-build via signature() (validate_wiring: range +
kind, returning the same variants as before, so an edge kind fault is now caught
before any build closure fires) and emits FlatGraph{nodes,signatures,sources,edges}.
- bootstrap consumes the FlatGraph: kinds/firing/output from the carried signatures,
buffer depth from node.lookbacks(). SourceSpec survives as the flat descriptor.
Renames: BlueprintNode::Leaf -> Primitive, LeafFactory -> PrimitiveBuilder.
Render (aura-cli/src/graph.rs) is migrated compile-only: it takes &Composite, maps
bound roles to the same source-entry shape, so both render goldens reproduce
byte-identical output (no re-capture needed). Render-fidelity tuning is the next cycle.
Verification (orchestrator-run, not agent-reported): cargo build --workspace green;
cargo test --workspace 150 passed / 0 failed; cargo clippy --workspace --all-targets
-D warnings clean. All pinned determinism/run-output tests pass with values unchanged;
no behavioural assertion was altered to go green. 5 new tests assert the signature is
pre-build and uniform, that compile rejects a kind mismatch without building (via a
panicking builder), UnboundRootRole, and lookbacks()/signature arity agreement.
Deferred to cycle-close audit (per plan): docs/design/INDEX.md and some aura-std
module docs still name the old Node::schema()/LeafFactory/BlueprintNode::Leaf/
Blueprint::param_space contracts; prose reconciliation is the architect's at audit.
closes #43 #36
|
||
|
Brummel
|
69d20949c0 |
tidy(aura-engine): single-own alias-validity + shared alias-count anchor
Collapse the two correctness-neutral debt items the 0021 fan-in pre-pass left behind (architect drift review at cycle close). Alias-validity: the `BadInteriorIndex` check was raised in two places — the structural pre-pass `check_composite_fan_in` and `inline_composite`. Since `compile_with_params` runs the pre-pass over every composite before any lowering, `inline_composite`'s copy was dead for that error path. The check now lives in one helper `check_alias_indices`, called solely by the pre-pass; `inline_composite` drops its copy (and the now-unused `param_aliases` binding — the overlay is a pure naming layer, unused in lowering, which is driven by the injected scalar vector). Alias-count: the per-node `a.node == node` predicate was re-derived in three spots (signature base, the unaliased-param test, the CLI render base-length). All three now route through one exported anchor `aliases_on`, so an alias-semantics change touches one site. Behaviour-preserving: the full workspace suite is the guard (15+6 cli, 25 core, 69 engine, 6+1+1 ingest/misc, 29 std — 0 failed), `out_of_range_param_alias_rejected` still green (now raised by the pre-pass), clippy -D warnings clean. No ledger change (the C9 refinement landed in 0021). closes #45 |
||
|
Brummel
|
3f4d756ed2 |
feat(aura): fan-in input distinguishability — construction constraint + source-derived render
A fan-in node (>1 input) whose colliding sources hide an unnamed configuration axis is now illegal at construction, and the definition view renders each fan-in input as a source-derived recursive-signature identifier instead of the positional, meaningless #A/#B. The root defect was sma_cross: two Sma into a Sub with unaliased length params, rendering sma_cross() -> (cross) with two indistinguishable [SMA] and [Sub(#A,#B)] — the author meant fast vs slow but the identity hid it. Now it renders sma_cross(fast:i64, slow:i64) with [SMA(fast)]/[SMA(slow)] and [Sub(#Sf,#Ss)]. Shape (single source of truth for "signature" shared by engine + CLI): - aura-engine signature_of(): a node's recursive authoring identity — type initial + alias initials + each wired input's signature (recursing into interior leaves, stopping at named roles/composites at their initial). - aura-cli leaf_label/fan_in_identifiers: shortest sibling-unique prefix of a source's signature, never below its base (type + alias initials); a role-fed slot renders its name verbatim (#price); equal-signature interchangeable inputs keep the positional letter. - aura-engine IndistinguishableFanIn: a signature collision is a fault only when a colliding source carries an unaliased param (the unnamed axis); param-less interchangeable sources (fan_composite's Pass/Pass) stay legal. Param-aware criterion (refined during design): keeps the blast radius to the param-bearing alias-less fixtures (sma_cross CLI + engine, the nested fast_slow); fan_composite and the hand-wired flat fixtures stay green. Placement decision (deviates from the plan): the constraint runs as a structural pre-pass (check_fan_in_distinguishability) at the head of compile_with_params, BEFORE the param-arity gate — not inside inline_composite as the plan drafted. Reason: the no-param compile() of a param-bearing composite would hit ParamArity first and preempt the fault; the spec mandates a structural check needing no param values, so the pre-pass is the spec-aligned home. Alias-validity ordering (BadInteriorIndex before the fan-in fault) is preserved at the pre-pass head. C23 untouched: the check is construction-phase; the compilat stays name-free (compiled_view_golden byte-stable). Ledger C9 carries the refinement. Known debt (non-gating): the alias-index validity check now exists in both inline_composite and the pre-pass (the pre-pass reaches every composite first, making inline_composite's copy effectively dead). Left for a separate tidy — a 5-line, correctness-neutral removal with its own test surface. closes #44 |
||
|
Brummel
|
41cbb5506f |
feat(aura-engine): name the composite boundary — input roles + param aliases
Brings the other two composite-boundary edge-kinds to the named-projection shape #40 gave outputs. input_roles changes from a bare Vec<Vec<Target>> to Vec<Role { name, targets }> (rendered [in:<name>]); a composite gains params: Vec<ParamAlias { name, node, slot }> that relabels an interior leaf param slot's surface name in param_space() (rendered [param:<name>]). Param aliasing is a PURE NAMING OVERLAY, not curation (the load-bearing decision, per spec 0019): every interior param slot stays in param_space() and sweepable; the alias only relabels in place, never reorders or hides. Proven empirically — the MACD run is byte-identical (total_pips 0.1637945563898923, 3 sign flips), only the param labels improved: param_space() now surfaces [macd.fast, macd.slow, macd.signal] instead of three indistinguishable macd.length, and the manifest reads ema_fast/ ema_slow/ema_signal. C23 honoured: role/param/output names are non-load-bearing debug symbols dropped at lowering; identity is positional (role index, param slot, output field). compiled_view_golden is byte-identical (verified: the golden region is untouched in the diff). An out-of-range alias (missing/ non-leaf node or slot past the leaf's param count) is rejected at compile_with_params as BadInteriorIndex, mirroring the output range-check (no new variant). Orthogonal to #36 — purely additive at the composite level. Aliasing is demonstrated on the CLI MACD site only (the spec's worked example + a new E2E test macd_param_space_surfaces_the_three_named_aliases); sma_cross, the engine test fixtures, and the construction-layer fieldtests get the forced role-name + empty params, so the param_space C23 anchor goldens (param_space_mirrors_compiled_flat_node_param_order + siblings) stay byte-identical. Verification (orchestrator-run, not trusted from the agent report): cargo build/test/clippy --workspace -D warnings all green (engine 66, cli 12); the separate-workspace construction-layer fieldtest crate builds (guards the #42 latent-drift recurrence); compiled_view_golden + MACD determinism unchanged. Two faithful repairs to the plan's literal test/code bodies, no semantic change: the out-of-range test uses .err()/Some(BadInteriorIndex) (the Ok arm Vec<Box<dyn Node>> is not Debug, so .unwrap_err() would not compile), and the inline_composite destructure binds params as `param_aliases` to avoid shadowing the injected `params: &[Scalar]` arg. closes #41 |
||
|
Brummel
|
8bc429f367 |
audit: cycle 0018 (#40) — drift-clean; C9 multi-output realization note
Architect drift review over a4cfe5c..HEAD (covers the previously-unaudited EMA node and MACD PoC plus the #40 composite multi-output cycle): status clean. C8/C7/C4 preserved (one port, one row per eval, K co-fresh base columns — structurally identical to OHLCV); C23 honoured (ItemLowering:: Composite.output is Vec<(usize,usize)>, no name in the compilat; the compiled-view render stayed bit-identical). EMA and MACD PoC carry no ledger drift. Resolution of the three minors: - Ledger lagged the new behaviour: added a "Realization (cycle 0018)" note under C9 recording that a composite's output is now a named K-field record (Vec<OutField>), the same arity C8 already grants a leaf, names dropped at lowering (C23). Amended here (ledger amendment is the audit's job). - blueprint.rs module-doc / Composite struct-doc / new-doc still said "one output port" singular — updated to "output record". - Non-workspace fieldtest From<Sma> debt is pre-existing (cycle-0016) and already tracked as #42; not re-filed. No regression scripts configured (profile regression: []), so the architect review is the sole gate. Carry-on: the next natural iteration is the #41 legibility sibling (input-role + param naming at the composite boundary). |
||
|
Brummel
|
784e6c917a |
feat(aura-engine): composite output is a named multi-field record
Replace a composite's single output port (OutPort { node, field }) with
an ordered, named record (Vec<OutField { node, field, name }>). A
composite can now re-export K interior fields as one output; a consumer
selects one via the existing Edge::from_field — the same field-wise
selector that already works for multi-output leaf producers (OHLCV).
Why: multi-line indicators (MACD, Bollinger, Stochastic, Ichimoku) could
not be authored as a composition and re-exported as a unit — only one
line escaped the boundary. The MACD PoC was forced to expose only the
histogram. This lifts the three `field == 0` / `from_field == 0` caps at
the composite boundary (inline_composite nested arm, rewrite_edge
composite arm), range-checking the index instead.
Boundary completion, not an invariant change:
- C8/C7/C4 untouched — one port, one row per eval, K base columns (a
3-line MACD is identical in kind to OHLCV).
- C23 honoured — re-export names live at the blueprint boundary only and
are dropped in the compilat (raw index wiring). compiled_view_golden
stayed byte-identical (the regression guard): no name leaked into the
flat graph.
The MACD PoC now re-exports macd / signal / histogram as a record; the
strategy still trades the histogram, selected via from_field: 2. The
render shows K [out:<name>] markers per multi-output composite (input
roles stay [in:<index>] — naming them is the dependent #41).
Output naming ships with the capability (OutField is born name-ready) so
#41 — composite param aliasing + input-role naming — does not reopen the
type.
Verification: cargo build/test/clippy --workspace -- -D warnings all
green; aura-engine 62 tests (+3 capability, +1 through-run E2E), aura-cli
unchanged-count green; compiled_view_golden byte-identical.
Known debt (out of scope, pre-existing): the non-workspace construction-
layer fieldtests (fieldtests/milestone-construction-layer/mc_1..mc_4)
carry their OutField sweep but still do not compile standalone — they use
the Sma::new / BlueprintNode::from(Sma) API retired in the cycle-0016
value-empty migration, never propagated to these fixtures. Tracked as a
follow-up.
closes #40
|
||
|
Brummel
|
4b64409036 |
feat(aura-core,aura-std,aura-engine,aura-cli): inject a param-set vector at bootstrap (#31)
Cycle B of milestone "The World — parameter-space & sweep". A blueprint is now value-empty: a leaf holds a param-generic recipe, not a built node, and a positional Scalar vector is bound slot-by-slot at bootstrap — so one blueprint bootstraps into many distinct instances under different vectors, with no cdylib rebuild (C12/C19). This is the binding primitive a sweep (#32) drives. Ratified design (brainstorm): value-empty reconstruct-through-new() over mutate-in-place and over a default-bearing variant. The value lives only in the injected vector (no baked default), keeping the blueprint a pure param-generic recipe (C19); every injected value flows through the node's own constructor (the single sizing/validation gate). - aura-core: LeafFactory { name, params, build } — the recipe (params -> sized node through `new`); Scalar::as_i64/as_f64 value accessors. Node trait unchanged. - aura-std: each of the 7 nodes exposes factory() (SMA length:I64, Exposure scale:F64, LinComb arity x weights[i]:F64; Sub/Add/SimBroker/Recorder paramless, capturing their non-param construction args — pip_size, the Recorder channel). - aura-engine: BlueprintNode::Leaf(LeafFactory), From<LeafFactory>; param_space() reads factory.params() pre-build; compile_with_params/bootstrap_with_params build each leaf from its kind-checked slice while lowering (build-then-wire), arity checked up front via param_space().len(); CompileError::{ParamKindMismatch, ParamArity}. compile/inline/edge-rewrite are structurally unchanged, so the compilat stays bit-identical for a given point (the bit-identity and both param_space mirror tests stay green). The vestigial pre-build Composite::schema / BlueprintNode::schema (no live caller — interface resolution is structural on the built flat nodes) are removed. - aura-cli: the blueprint view labels leaves by bare type via LeafFactory::label() (`[SMA]`) — the ascii-dag renderer cannot render wide cluster-sibling labels (see spec); the compiled view still labels valued (`SMA(2)`). The sample supplies its point as a vector; the mis-wiring swap moved to the compiled view. The #34 dual-traversal drift hazard is subsumed: compile_with_params consumes the vector in the same recipe walk param_space() reports, so the two share one traversal. Verified: cargo build/test --workspace green (127 tests, incl. bit-identity, both mirror tests, and 4 new injection tests — different-vector-different-run, kind mismatch, arity, determinism); clippy --workspace --all-targets -D warnings clean; `aura graph` blueprint view renders cleanly. Scope: one vector -> one instance. Deferred: sweep enumeration (#32), domain validation (#32/C20), single-run authoring convenience (#35). closes #31 |
||
|
Brummel
|
a31d91453c |
test(aura-engine): mirror param_space against compile() under nesting (#34)
The cycle-0015 E2E guard param_space_mirrors_compiled_flat_node_param_order pins the load-bearing C23/#31 invariant — Blueprint::param_space() slot order mirrors compile()'s flat-node param order kind-by-slot — but only on the single-level composite_sma_cross_harness. collect_params (blueprint.rs:217) duplicates lower_items' depth-first traversal rather than sharing it (kept a parallel read-only projection so compile/inline_composite stay untouched and the compilat bit-identical, C9/C23), so the two orders must stay in lockstep forever. No mirror test ever compiled a composite-inside-a-composite: a future inliner reorder that desynced the projections only under nesting could pass the single-level mirror and the isolated nested param_space order test, yet break the slot-by-slot premise #31's binding rests on. Adds param_space_mirrors_compiled_flat_node_param_order_under_nesting: compiles a strategy -> { fast_slow -> [Sma, Sma, Sub], LinComb } nest and asserts param_space() equals the compiled flat-node param order kind-by-slot, pinning the concrete [I64, I64, F64, F64] shape. Green immediately — coverage-hardening, no desync in the current tree. Test-only; collect_params/lower_items/compile untouched. Verified: cargo test -p aura-engine green (55 tests), clippy --all-targets -D warnings clean. closes #34 |
||
|
Brummel
|
931109df58 |
feat(aura-core,aura-std,aura-engine): declare node tunable params (#30)
Cycle A of milestone 'The World — parameter-space & sweep'. A node now declares its tunable parameters in its C8 schema, and a blueprint aggregates them into one flat, inspectable param-space — the root that unlocks the C12 orchestration axes (#31 bind, #32 sweep), filling the gap C8/C23 name 'deliberately not in the schema yet'. - aura-core: ParamSpec { name: String, kind: ScalarKind } as a third schema- declaration type; NodeSchema gains a third field 'params'. name is String (not &'static like FieldSpec) because a vector knob carries a runtime index and aggregation prefixes the composite path. - aura-std: Sma declares [length:I64], Exposure [scale:F64], LinComb expands to N flat [weights[i]:F64] (N = its input arity, topology-fixed per C19); Sub/Add/ SimBroker/Recorder declare none (pip_size is metadata, C10/C15; Recorder is wiring). - aura-engine: Blueprint::param_space() walks the graph-as-data depth-first in lower_items order, path-qualifying via the already-public Composite::name() — a read-only projection (C9). compile/inline_composite/lower_items are untouched, so the compilat stays bit-identical (composite_sma_cross_runs_bit_identical_to_hand_wired and the golden render tests stay green). Design: param identity is positional (slot after the deterministic inline order, C23 'by index not name'); the path-qualified name is a non-load-bearing debug symbol — same-type siblings in one composite share a name, uniqueness is at the slot. Flat over a structured arity-bearing ParamSpec because flattening is unavoidable (a sweep enumerates a flat point-space) and structure-in-the-runtime is the nested-composite reading C23 rejects (see spec 0015 for the full rationale). Scope is declaration + aggregation + inspection only; binding (#31), sweep enumeration (#32), search-range, validity-constraint, and default-range/slider are deferred. An E2E test pins the load-bearing C23/#31 invariant: param_space() slot order mirrors compile()'s flat-node param order, kind-by-slot, on the SMA-cross harness. fieldtests/ (excluded crates) left as frozen snapshots. Verified: cargo build/test --workspace green (64 tests), clippy --all-targets -D warnings clean; compile/inline diff empty. closes #30 |
||
|
Brummel
|
0a855c3943 |
feat(aura-cli): aura graph renders a wired graph as an ASCII DAG (#13)
Make a wired graph introspectable so a mis-wiring becomes visible. Two
selectable views, both authored from one built-in sample blueprint:
aura graph clustered blueprint view — composites drawn as named
ascii-dag cluster boxes (pre-inline, C9)
aura graph --compiled flat compilat view — composite boundaries dissolved
(C23); the graph the run loop actually runs
The payoff is intrinsic, param-carrying node labels: two SMAs read SMA(2) /
SMA(4), so a swapped fast/slow input reads back differently. This is realized
by a `label()` default method on the core Node trait — a non-load-bearing
render symbol the run loop never reads (wiring is by index), the symbol C23
already reserves citing #13. Recorded as a C8 refinement in the ledger.
Rejected the alternatives in brainstorm: a separate Describe trait (forces a
combined trait-object dance for a label the ledger calls non-load-bearing) and
extrinsic parallel labels (decoupled from the node, so an author can label the
slow SMA "fast" and mask the very bug the render exists to surface).
Mechanism:
- aura-core: Node::label() default method (additive, object-safe, single-line).
- aura-std: per-node overrides — SMA(n)/Exposure(s)/SimBroker(p) carry params;
Sub/Add/LinComb/Recorder are bare (their identity is not a mis-wiring axis).
- aura-engine: Composite gains an authored `name` (cluster title; dissolves at
inline) + read-only graph-as-data accessors on Blueprint/Composite. No
external dependency — the engine stays dependency-pure (C16); ascii-dag lives
only in aura-cli. The label-free index-wired compilat (C23) is unchanged.
- aura-cli: ascii-dag v0.9.1 + a graph.rs adapter (Vertical mode; labels
materialized into an owned Vec<String> that outlives the borrow-based Graph).
`aura run` is untouched (the sample duplication is dedup idea #14).
Tests: a concern-defining test (swapped sample renders != correct), structure
pins (cluster present in blueprint view, absent in compiled view), per-node
label disambiguation, and two frozen byte-goldens of the deterministic render.
The cycle-0012 bit-identical and run-sample non-regression tests stay green.
Verified by the orchestrator (not the agent report): cargo build --workspace
clean; cargo test --workspace all green; cargo clippy --workspace --all-targets
-D warnings clean; both views run live and render as expected.
closes #13
|
||
|
Brummel
|
a4fb5d7182 |
feat(aura-engine): blueprint construction layer with composite inlining
Add the construction layer (C9/C19/C23): a named, param-generic graph-as-data (`Blueprint`) that *compiles* to the flat, type-erased instance the run loop already runs. The unit of reuse is the `Composite` — a nestable sub-graph fragment exposing one output port (C8) and named input roles — which `Blueprint::compile()` **inlines** by raw-index lowering into the flat `(nodes, sources, edges)` the unchanged `Harness::bootstrap` consumes. Design (settled in spec 0012, ledger C9/C19/C23): a composite is an authoring-level node, NOT a runtime `Box<dyn Node>` sub-engine. The chosen inlining approach keeps the sacrosanct deterministic run loop untouched and leaves the composite interior fully inspectable for the deferred cross-graph optimiser; the rejected "runtime sub-engine" reading would have put throwaway complexity into the run loop and kept the interior opaque. The compilat is wired by raw index, not by name; names survive only as non-load-bearing debug symbols (as `FieldSpec.name` already is). Lowering is recursive index rewriting: interior items append at an offset, interior edges rewrite by that offset, an edge into a composite fans through its input roles (one blueprint edge -> several flat edges), an edge out resolves to the interior output port, and nesting recurses inside-out. Construction-phase faults are caught as a typed `CompileError` (BadInteriorIndex / RoleKindMismatch / OutputPortOutOfRange); the lowered flat compilat is validated by bootstrap's existing kind- and Kahn-cycle-check (wrapped as `CompileError::Bootstrap`), with no re-implementation. Non-goals (deferred per C23/C16): no optimisation pass (CSE/DCE, sweep-invariant hoisting), no external optimisation crate, no named-handle ergonomic wiring, no `aura graph` render (#13). Verification: 48 engine tests green (8 blueprint: derived-schema, single + nested composite inlining, the three CompileError paths, bootstrap-error wrap, and the headline `composite_sma_cross_runs_bit_identical_to_hand_wired` C1 demonstrator — the SMA-cross composite lowers to a flat graph byte-identical to the hand-wired sample harness, producing bit-for-bit identical equity + exposure traces); `cargo clippy --workspace --all-targets -D warnings` clean. `Node`, `Harness::bootstrap`, the run loop, and `Edge`/`Target`/`SourceSpec` are unchanged. closes #12 |