e01bdc3d5e
Cycle-close tidy for visual-world-cut-2 (decimation #108 + run-context header #102).
Architect drift review (53eeba5..HEAD): the only drift was a STALE LEDGER — the C22
section still listed #102 (run-context header) and #108 (decimation) as deferred/open
after they shipped in 476342d. Resolved in this commit:
- the #101 first-cut note no longer claims "not yet a run-context header";
- a new "served-page hardening — cut 2" amendment records #102 (ChartMeta +
buildHeader, family window = span across members) and #108 (the pure decimate
min-max transform) as landed, both CLI-side (C14);
- the "Open architectural threads" list drops the now-landed run-context header.
Substantive gates (self-verified): C14 honoured — all decimation/header/pixel logic
is in aura-cli (render.rs/main.rs/chart-viewer.js); the only engine touch in range is
the one-line #100 cell.rs doc. C1 preserved — decimate is pure, deterministic
bucketing over the sorted-deduped spine, no-op under budget, monotonic spine (pinned
by a unit test). Regression: cargo test --workspace = 446 passed / 0 failed; cargo
clippy --workspace --all-targets -D warnings clean. No separate baseline script.
Residual debt filed as #110 (idea): the decimation budget is a fixed const (no
--width flag), and the min-then-max sub-pixel ordering is a real ts-displacement in
the non-default continuous x-mode. Both are local aura-cli refinements, neither a bug.
Ephemeral spec/plan removed at cycle close (docs/specs/0062, docs/plans/0062).
Cycle 0062 tidy: drift-clean after this ledger amendment.
1223 lines
86 KiB
Markdown
1223 lines
86 KiB
Markdown
# aura design ledger — INDEX
|
||
|
||
The ledger records the load-bearing design contracts and their rationale. Each
|
||
contract states what it **guarantees**, what it **forbids**, and **why**. A
|
||
change that breaks a contract is a design decision (amend the contract here with
|
||
its new rationale), never a silent refactor.
|
||
|
||
Provenance: contracts C1–C18 were settled in the initial rough-sketch design
|
||
interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the
|
||
C10 refinement to a broker-independent position table came in follow-up turns;
|
||
C10 was later reframed in cycle 0007 — the intent/exposure stream is the primary
|
||
output, the position table a derived layer — see C10). C19–C22 were added as the
|
||
construction / World / playground layer; C23 and the C9/C19 compilation
|
||
refinements were settled 2026-06-05 for the **Construction-layer** milestone (the
|
||
blueprint→flat-graph reading of composites and graph optimisation).
|
||
The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded
|
||
summary of the
|
||
subset that agents must never violate; this file is the fuller form with
|
||
rationale.
|
||
|
||
Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
|
||
*milestone* is a tracker container spanning many cycles (the first milestone is
|
||
the **walking skeleton**: ingest → one signal → exposure/intent → sim-optimal
|
||
broker → synthetic pip-equity signal-quality metric).
|
||
|
||
---
|
||
|
||
## Foundation — what aura is
|
||
|
||
aura is a framework **and** a playground for traders. A human and (primarily)
|
||
LLMs author trading **nodes** directly in Rust; the engine backtests them
|
||
deterministically and massively in parallel, composes them fractally, validates
|
||
them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into
|
||
a standalone bot with a broker connection.
|
||
|
||
The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow,
|
||
too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura
|
||
inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts*
|
||
(synchronous reactive streams, bounded-lookback series, run-counting, SoA).
|
||
RustAst is a conceptual reference, not a dependency. The one reused component is
|
||
`data-server` (the first data source).
|
||
|
||
What sets aura apart is **not** the single backtest — every quant system does
|
||
that — but the **World**: the meta-level where harnesses are dynamically
|
||
constructed and *families* of them orchestrated and explored (walk-forward,
|
||
sweeps, Monte-Carlo, comparison). The deterministic single-harness engine is the
|
||
*substrate*; the World is the *product* (C20–C22).
|
||
|
||
---
|
||
|
||
## External components
|
||
|
||
Two sibling projects live outside this repo and are named throughout the
|
||
contracts. Their concrete location is recorded here so the repo is the single
|
||
source of truth — not session memory.
|
||
|
||
- **`data-server`** — `~/dev/libs/data-server`, Gitea `Brummel/data-server`
|
||
(`http://192.168.178.103:3000/Brummel/data-server.git`). aura's **first data
|
||
source** and the one reused component (Foundation; C3, C11, C12). A standalone
|
||
leaf crate (deps: `chrono`, `regex`, `zip`) that loads Pepperstone **M1/tick**
|
||
binary files and shares them lock-free as `Arc<[T]>` chunks
|
||
(`CHUNK_SIZE = 1024`) via `SymbolChunkIter::next_chunk` (an inherent method
|
||
driven by `while let`, not the `Iterator` trait); `stream_*_windowed(from_ms,
|
||
to_ms)` provides C12's data-window with inclusive Unix-ms bounds. Records are
|
||
**AoS** (`M1Parsed`/`TickParsed`, `time_ms: i64` Unix-ms). The ingestion
|
||
boundary (C3) therefore **transposes** these AoS records into aura's SoA
|
||
columns (C7) and normalizes `time_ms` → canonical epoch-ns at that one
|
||
boundary. Pulled in as a cargo git dependency by the **`aura-ingest`** crate
|
||
(cycle 0011) — the **data-source ingestion edge**, where the `data-server`
|
||
external tree (and its transitive `chrono`/`regex`/`zip`) enters the workspace.
|
||
Under the amended C16 **per-case dependency policy** (cycle 0029), `aura-ingest`
|
||
is no longer a zero-dep *firewall* for the rest: the engine crates link
|
||
standard vetted crates where they earn their place (e.g. `serde`/`serde_json`
|
||
for the run registry, cycle 0029), with particular scrutiny for the frozen
|
||
deploy artifact (C13). Consequence: `cargo build/test --workspace` resolves
|
||
from a populated cargo cache (a Gitea fetch for `data-server` + crates.io for
|
||
the standard crates) rather than fully offline.
|
||
- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst`
|
||
(`http://192.168.178.103:3000/Brummel/RustAst.git`). The predecessor DSL attempt
|
||
(Foundation): **a conceptual reference, never a dependency** — its DSL authoring
|
||
surface and `Value`/HM-inference machinery are exactly what aura rejects (C17).
|
||
But its **`src/ast/rtl/` layer is a working reference implementation of the very
|
||
streaming substrate aura rebuilds**, and is worth reading before authoring
|
||
`aura-core` — these are not just concepts, they exist as code:
|
||
- `rtl/series/data.rs` — `RingBuffer<T>` with financial-style indexing (index 0
|
||
= newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's
|
||
pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no
|
||
String/Record" = C7's closed scalar set); `ScalarSeries<f64|i64|bool>` and the
|
||
**SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns").
|
||
- `rtl/series/mod.rs` — `create_typed_series`, dispatching element type → storage
|
||
backend (the type-specialized **factory** of C19).
|
||
- `rtl/streams/mod.rs` — `Signal { cycle_id, value }` (C4's cycle clock) and the
|
||
`Stream` / `Observer` / `ObservableStream` push traits (the reactive model of
|
||
C4/C5).
|
||
- `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one
|
||
explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator
|
||
(C12's seed-as-input).
|
||
|
||
aura reimplements these natively and **sharpens** them: types are monomorphized
|
||
and edges type-erased to the four scalar kinds with direct dispatch (C7) instead
|
||
of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>`
|
||
(C12) instead of a `VecDeque<Value>`. RustAst shows the *shape*; aura makes it
|
||
fast and deterministic.
|
||
|
||
---
|
||
|
||
## Contracts
|
||
|
||
### C1 — Determinism and disjoint parallelism
|
||
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
|
||
loop that reaches a unique state after each input tick. Same input (incl. seed)
|
||
→ bit-identical run. Two backtests are fully disjoint and run concurrently
|
||
without locking.
|
||
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
|
||
is not captured as an explicit input (see C11, C12).
|
||
**Why.** Real money rides on backtest results; reproducibility and an audit
|
||
trail are non-negotiable. Speed comes from parallelism *across* sims, which
|
||
disjointness makes lock-free.
|
||
|
||
### C2 — Causality / no look-ahead
|
||
**Guarantee.** A node sees only the past. Input history is a read-only window
|
||
that ends at the current cursor; a resampler emits a bar only once it is
|
||
complete.
|
||
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
|
||
/ still-forming bar.
|
||
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
|
||
leaks the future is worse than none. Making the future *physically absent* from
|
||
what a node receives beats merely discouraging it.
|
||
|
||
### C3 — One merge, at ingestion only
|
||
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp
|
||
into one chronological cycle stream at the ingestion boundary; source-native
|
||
time units (e.g. data-server's Unix-`time_ms`) are normalized there to the
|
||
canonical epoch-ns `timestamp` of C7.
|
||
**Forbids.** Any merge / as-of join *inside* the graph.
|
||
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
|
||
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
|
||
future. Keeping the merge at one boundary keeps the graph semantics simple.
|
||
|
||
### C4 — Cycle granularity
|
||
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
|
||
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
|
||
multiple sources) break by source declaration order.
|
||
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
|
||
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
|
||
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
|
||
origin of records, not the cycle semantics. Tie determinism preserves C1.
|
||
|
||
### C5 — Freshness-gated recompute and sample-and-hold
|
||
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
|
||
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
|
||
run-count); otherwise it holds its last output. Stale inputs contribute their
|
||
last (held) value.
|
||
**Forbids.** Recomputing every node every cycle ("push all" is true for the
|
||
*clock*, not for *recompute"); treating a held value as missing.
|
||
**Why.** Total recompute does not scale to many sparse high-frequency sources;
|
||
freshness-gating is the performance discipline that keeps the synchronous model
|
||
fast.
|
||
|
||
### C6 — Firing policy A and B, per input group
|
||
**Guarantee.** A node declares, per input group, one of two firing policies:
|
||
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
|
||
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
|
||
four separate 15m sources: the candle is complete only when all four are fresh).
|
||
A single node may mix an A input and a B group.
|
||
**Forbids.** A single global firing mode; forcing per-node-only granularity.
|
||
**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group
|
||
granularity is required by the OHLC-plus-bias case where one node needs both.
|
||
**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any,
|
||
Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a
|
||
mode-A input is its own trivial group, so "per input group" and the per-input tag
|
||
coincide. The barrier's synchronization token is the cycle **timestamp**, not the
|
||
`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so
|
||
RustAst's `cycle_id`-equality barrier could never fire across them. A group fires
|
||
when every member's last push carries the current cycle's timestamp, guarded by
|
||
"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This
|
||
fires both the multi-source bar and the within-source diamond rejoin (every push
|
||
in a cycle carries that cycle's timestamp).
|
||
|
||
### C7 — Four scalar base types, streamed as SoA
|
||
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
|
||
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
|
||
(OHLCV) are bundles of base columns — this is the **node-output model** too: a
|
||
node emits a record of 1..K base columns (C8), each forwarded field-wise to a
|
||
consumer slot; the bundle is structural, never a fifth scalar type. Edges are
|
||
type-erased to these four kinds;
|
||
the type check is paid once at wiring/sim-start, then the topology is frozen per
|
||
sim → direct dispatch, no per-event allocation.
|
||
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
|
||
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
|
||
allocation; topology mutation mid-sim.
|
||
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
|
||
set and SoA. The open set is composites (schemas of columns), not scalar types.
|
||
Type-erasure at the edge is also forced by the cdylib boundary (C13).
|
||
**Realization (`Cell` carrier split, 2026-06).** The streamed value is now split
|
||
into a tag-free 64-bit word and its kind. `Cell` (`crates/aura-core/src/cell.rs`)
|
||
is a type-erased `u64`: constructed per base type (`from_i64/f64/bool/ts`) and
|
||
read only by naming the type at the call site (`i64()/f64()/bool()/ts()` —
|
||
branch-free bit-casts). The kind is therefore resolved once at the boundary and
|
||
the value itself carries no tag — C7's "the type lives at the column/edge, not in
|
||
the value" made explicit on the single-value carrier (and a single 8-byte word
|
||
vs. the 16-byte tagged enum). `Scalar` becomes `{ kind: ScalarKind, cell: Cell }`
|
||
— the self-describing form for the *dynamic* boundaries (builder binding, serde,
|
||
rendering) — with `debug_assert`-guarded native accessors (caller asserts the
|
||
kind; free in release) and a hand-written **value** `PartialEq` that preserves
|
||
the former enum's IEEE-754 semantics (`NaN != NaN`, `+0.0 == -0.0`), pinned by
|
||
the `scalar_eq_is_value_not_bitwise` fixture. Behaviour-preserving.
|
||
|
||
**Realization (`Cell` becomes the hot-path carrier, 2026-06, #74).** The carrier
|
||
swap deferred above has landed: `Node::eval` now returns `Option<&[Cell]>` and
|
||
every node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free
|
||
8-byte words. The kind lives only at the schema/column: the harness forwards each
|
||
field via the new branch-free `AnyColumn::push_cell` (infallible — the edge
|
||
kind match is verified once at bootstrap, the surviving guard
|
||
`bootstrap_rejects_*_kind_mismatch`). `Scalar` remains on the self-describing
|
||
dynamic boundaries: the param plane (`build`/`bind`/`compile_with_params`/sweep
|
||
points/`RunManifest`), `AnyColumn::get` (the type-erased read for sinks/serde),
|
||
and source ingestion (`Source::next`, the heterogeneous C3 merge). The removed
|
||
per-value runtime kind check on node output is the same authoring-bug class C8
|
||
already leaves to a `debug_assert` (output width); node-output-kind correctness
|
||
is the node's declared `FieldSpec` contract, caught by each node's own test.
|
||
Behaviour-preserving (C1).
|
||
|
||
### C8 — The node contract
|
||
**Guarantee.** A node has a **signature** — its `NodeSchema`: each input's scalar
|
||
type and firing group, its output record, **and the node's own tunable parameters —
|
||
typed, with ranges**, which aggregate into the blueprint's param-space the optimizer
|
||
sweeps (C12/C19/C20). The signature is **declared pre-build on the value-empty
|
||
recipe** (C19), so the whole interface is legible without building (cycle 0024).
|
||
The built node implements `lookbacks()` — the per-input buffer depth, the one
|
||
quantity that may depend on an injected param (e.g. an `Sma`'s window = its
|
||
`length`), read once at bootstrap to size the windows — and `eval(ctx) ->
|
||
Option<&[Cell]>`. The
|
||
engine provides read-only, zero-copy windows into each input's SoA ring buffer
|
||
(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own
|
||
mutable series for derived/intermediate state. `None`/Void return = filter /
|
||
not-yet-warmed-up. A node is a **producer, a consumer, or both**: a
|
||
producer/transformer exposes **one output port**, whose payload is a **record of
|
||
1..K base-scalar columns** (a scalar is the degenerate K=1 record; an `eval`
|
||
returns a borrowed row, one value per column); a **pure consumer (sink)** — chart,
|
||
equity, logger — has **no** output. Sources are pure producers; sinks are pure
|
||
consumers.
|
||
**Forbids.** A node sizing/growing its input lookback at runtime; more than one
|
||
output **port** per node; a fifth scalar type or a heterogeneous output payload
|
||
(a record is a bundle of base columns, C7); copy-on-read of input history.
|
||
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage
|
||
lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized
|
||
buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).
|
||
**Realization (cycle 0005).** `NodeSchema.output` is a `Vec<FieldSpec>` (named base
|
||
columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field`
|
||
selects one producer column per edge; consuming a whole record is N edges (no
|
||
"bind whole record" mechanism). The K fields of one record are **co-fresh by
|
||
construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns
|
||
`Option<&[Cell]>` — a borrowed row into a node-owned buffer — so the forward
|
||
path allocates nothing per cycle (C7) (the carrier is now a tag-free `Cell` — see
|
||
the C7 carrier note).
|
||
**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is
|
||
now realized at the substrate: **recording is a node role, not a type.** A
|
||
recording node reads its typed input windows + `ctx.now()` in `eval` and pushes
|
||
the record to a destination it holds as a field (a channel, a chart handle) — an
|
||
**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a
|
||
node that only records returns `None` (pure consumer), and a node may record
|
||
**and** return an output the engine forwards in the same `eval` (the "both"
|
||
case). **Encoding & return contract.** A pure consumer declares `output: vec![]`
|
||
— the empty record *is* the sink declaration; there is no separate type, trait,
|
||
or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run
|
||
loop debug-asserts the returned row's width equals the declared output width
|
||
(`row.len() == schema.output.len()`). Field-wise wiring resolves
|
||
`Edge::from_field` against the producer's `output` at bootstrap, so no edge can
|
||
bind a field of a zero-output node — it fails with `BadIndex` — making a sink
|
||
structurally unwireable as an in-graph producer; its only output is the
|
||
out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out
|
||
of the graph is the node's own side effect — and that boundary is the
|
||
determinism / graph-as-data boundary (C1/C7).
|
||
|
||
**Refinement (Construction-layer milestone — render labels, 2026-06-05).** A node
|
||
additionally exposes `label() -> String`, a **single-line, non-load-bearing**
|
||
render symbol: a default trait method the run loop never calls (wiring is by
|
||
index, C23). Overrides carry the node's identifying params (`SMA(2)` vs `SMA(4)`)
|
||
so a graph render (C9 graph-as-data, #13) disambiguates identical node types and
|
||
surfaces a mis-wiring. Like `FieldSpec.name`, it is an informative debug symbol,
|
||
not part of the C8 dataflow contract — adding it changes no run behaviour.
|
||
|
||
**Realization (cycle 0015 — param declaration).** The tunable-parameter half of
|
||
this contract is now realized: a node declares its knobs in `schema()` as
|
||
`params: Vec<ParamSpec>` (`ParamSpec { name: String, kind: ScalarKind }`), and
|
||
`Composite::param_space()` (cycle 0024; was `Blueprint::param_space()`) aggregates
|
||
them into one flat, path-qualified list — a
|
||
read-only projection of the graph-as-data (C9), mirroring the inline order
|
||
(C19/C23) so a param's slot matches the later flat-node order. Two refinements to
|
||
the guarantee's "typed, with ranges": (1) the declaration carries `name` + `kind`
|
||
only — the **search-range is the run's, not the node's** (which subset / grid a
|
||
sweep covers is an experiment axis, #32/C20; the node declares the knob's existence
|
||
and type, never its search interval). (2) Identity is **positional** (the slot,
|
||
C23 "by index, not name"); the path-qualified `name` is a non-load-bearing debug
|
||
symbol (like `FieldSpec.name`) — uniqueness is at the slot. **Refinement (cycle
|
||
0032):** identity in the **flat graph** stays positional (C23, unchanged), but the
|
||
`param_space()` **name projection** — the authoring / by-name address space (the
|
||
surface a sweep axis or single-run binding addresses) — must be **injective** for a
|
||
blueprint to compile (a duplicated path is unaddressable; see C9). Two layers:
|
||
positional wiring below (the flat graph), an injective name address space above (the
|
||
authoring boundary). So same-type siblings in one composite no longer silently
|
||
share a name — they must be `.named(...)` apart, which is also what disambiguates a
|
||
same-type fan-in (C9). A vector knob (`LinComb.weights`) expands to `N` flat
|
||
`weights[i]` entries, `N` topology-fixed (C19). Permitted kinds are `i64`/`f64`/
|
||
`bool` (a `timestamp` knob is a structural axis, C20, never a numeric sweep param).
|
||
Binding a value to a slot landed in cycle 0016 (#31, see C19/C23); enumerating a
|
||
sweep (#32) is the deferred next layer. The 0016 binding also moved the param
|
||
declaration's *authoring* home into the value-empty recipe, whose `params()` is read
|
||
pre-build. In 0016–0023 the built node's `schema().params` still reported the same
|
||
slots, kept in lockstep by a per-node test (a duplication filed as debt, #36); **cycle
|
||
0024 dissolved this** — the signature is declared *once* on the recipe and the built
|
||
node no longer carries `schema()` (#36 closed). See the C8 cycle-0024 realization.
|
||
|
||
**Realization (cycle 0024 — the signature lives in the blueprint, #43/#36).** The
|
||
node's whole declared interface — its `NodeSchema` (input scalar types + firing,
|
||
output record, params) — now lives **once**, on the value-empty recipe
|
||
`PrimitiveBuilder` (ex-`LeafFactory`), read pre-build by `param_space()`, the
|
||
compile-time structural validation, and the render. This closes two debts: **#43**
|
||
(a value-empty recipe used to declare *no* input/output interface pre-build — only
|
||
params) and **#36** (params declared twice, recipe vs built node, kept in lockstep
|
||
by a per-node test — those 8 tests are deleted, the duplication structurally gone).
|
||
The split that makes this work: a node's signature is **fully static** per blueprint
|
||
(input kinds/firing, output fields, params — verified across the roster; a variable-
|
||
arity node like `LinComb` takes its arity as a *recipe argument*, not an injected
|
||
param), so it can be declared without building; the **one** param-dependent quantity,
|
||
an input's buffer **lookback** depth, is no longer in the signature but answered by
|
||
`Node::lookbacks() -> Vec<usize>`, read only by bootstrap to size the windows.
|
||
`Node::schema()` is therefore **removed** — the signature is pre-build data, not a
|
||
built-node method. `BlueprintNode::signature()` answers uniformly for both arms (a
|
||
primitive returns its recipe's schema; a composite *derives* it from the interior:
|
||
role kinds in, re-exported field kinds out, aggregated params), so "every node has a
|
||
signature in the blueprint" holds for composites too.
|
||
|
||
**Realization (cycle 0027 — name input ports, refs #21/#51).** `PortSpec` now
|
||
carries a `name: String` (it drops `Copy`, like `ParamSpec`), so an input port is
|
||
named just as `FieldSpec.name` (output) and `ParamSpec.name` (param) already are.
|
||
The name is **non-load-bearing** (C23): wiring stays positional by slot, bootstrap
|
||
and the run loop never read it; it exists for tracing / graph rendering (#13). Leaf
|
||
primitives declare their slot names (`SimBroker`'s `exposure`/`price`, etc.);
|
||
`derive_signature` carries a composite's `Role.name` into the derived input port,
|
||
so the graph model (`model_to_json`) is homogeneously named across inputs, outputs,
|
||
and params and across both graph levels. This does **not** close #21 (a swapped
|
||
same-kind wiring is still only kind-checked) — it makes those slots
|
||
self-documenting; a name-consuming validation is its own future cycle.
|
||
|
||
**Realization (cycle 0040 — wiring totality: every input slot connected exactly
|
||
once, #65).** The node contract's "the engine provides a window into each input"
|
||
presupposes each input is actually fed; until now an unwired interior input slot was
|
||
accepted and bootstrapped to a silent empty column (`harness.rs`), and two producers
|
||
into one slot — ill-formed, since a slot holds one column — was likewise
|
||
uncompiled-against. Cycle 0040 makes a valid graph **total and single-valued** over
|
||
its interior input slots: `check_ports_connected` (in `validate_wiring`, run at every
|
||
nesting level) requires every interior node's every declared input slot to be covered
|
||
by **exactly one** wiring act — one interior `Edge { to, slot }` or one role
|
||
`Target { node, slot }`, edges and role targets counted uniformly. Zero coverage is
|
||
`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. A
|
||
composite's **own** input roles (`source: None`) are coverage *providers* — the
|
||
wired-by-enclosing boundary, the root case already guarded by `UnboundRootRole` —
|
||
never consumers, so only interior input slots are subject to the rule. There is **no
|
||
optional-input concept**: every declared input port is required (no shipped node runs
|
||
meaningfully without one; an unwarmed mode-A input is *wired-but-not-yet-valued*, not
|
||
unwired). The check is **index-based and name-free** — it touches no name machinery
|
||
and emits nothing into the flat graph, so C23 is untouched (it proves the existing
|
||
raw-index wiring is total and single-valued). Inherited identically by the raw
|
||
`Composite::new` path and the `GraphBuilder::build()` path (both compile via
|
||
`compile_with_params`).
|
||
|
||
### C9 — Fractal, acyclic composition
|
||
**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes
|
||
one output; signal, combined signal, and (with execution) strategy are all the
|
||
same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only
|
||
feedback path is an explicit delay/state node (the RTL "register"). Wiring is
|
||
written in Rust (builder API); the built graph is introspectable runtime data.
|
||
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
|
||
"signal-of-signals" as separate mechanics.
|
||
**Why.** Self-application of one contract gives unlimited composition with no
|
||
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined;
|
||
forcing feedback through a visible delay node keeps the per-cycle determinism
|
||
intact and the one legitimate feedback path explicit. Graph-as-data enables
|
||
visualization, freezing, and re-parameterization for sweeps.
|
||
**Refinement (Construction-layer milestone, 2026-06-05).** "A composite is itself
|
||
a `Node`" is an **authoring-level** identity: a composite declares the same
|
||
interface (typed inputs + ≤1 output, C8) and is wireable wherever a node is, but it
|
||
is **not** a runtime object. The bootstrap **compiles it away by inlining** its
|
||
sub-graph into the one flat instance (C19/C23): the composite *boundary* dissolves
|
||
into the raw index wiring the run loop already consumes, and names stay
|
||
**non-load-bearing** (informative debug symbols only, as `FieldSpec.name` already
|
||
is). So "nestable arbitrarily" and "graph-as-data" hold at the **blueprint
|
||
(source)** level; the running graph is the flat, index-wired **`FlatGraph`**. The earlier reading — *a composite survives as a `Box<dyn Node>`
|
||
driving a nested sub-engine* — is **explicitly rejected**: it would keep the
|
||
interior opaque to the cross-graph optimiser (C23) and add a runtime sub-loop the
|
||
flat model does not need. Inlining is what makes the composite boundary free.
|
||
**Realization (cycle 0018 — composite multi-output record, #40).** "Exposes one
|
||
output" is one output **port** carrying a **record of 1..K re-exported fields**, not
|
||
one field: `Composite.output` is a `Vec<OutField { node, field, name }>` (was a
|
||
single `OutPort`). Each entry is a **named projection** of one interior
|
||
`(node, output-field)`; a consumer selects which re-exported field it reads via the
|
||
same `Edge::from_field` that already binds leaf record columns (C8 realization,
|
||
cycle 0005). This is the **same arity** C8 already grants a leaf (OHLCV = one port,
|
||
5 columns): a multi-line indicator (MACD = macd/signal/histogram) is **one** record
|
||
of K fields, **one** port, **one** row per `eval` — C8/C7/C4 untouched, a boundary
|
||
completion, not a contract change. A strategy composite is simply the K=1 case
|
||
(one exposure field, C10). The re-export **names** are **non-load-bearing** (C23):
|
||
they live at the blueprint boundary and in render (cycle 0022/#46 folds each onto
|
||
its producing node as a `name := …` binding; originally `[out:<name>]` markers, #13)
|
||
but are dropped at lowering — `ItemLowering::Composite.output` is `Vec<(usize,
|
||
usize)>`, raw index pairs only, so the flat graph is name-free (verified: the
|
||
compiled-view render stayed bit-identical across this change).
|
||
**Realization (cycle 0019 — name the composite boundary, #41; param-overlay retired,
|
||
cycle 0031).** The named-projection shape covers the surviving boundary edge-kinds:
|
||
`input_roles` is a `Vec<Role { name, targets }>` (was a bare `Vec<Vec<Target>>`),
|
||
alongside `output: Vec<OutField>`. Each is an ordered, positionally-indexed
|
||
**named projection** of interior handles. **Param projection is no longer a
|
||
composite overlay** (the index-addressed `ParamAlias` was retired in cycle 0031):
|
||
a node's surface param name flows from its own **instance name** — every node
|
||
carries a name (default = its lowercased type label, override via `.named()`), and
|
||
`param_space()` is uniformly `<node>.<param>` at every level including the root. A
|
||
same-type fan-in is distinguished by naming the colliding legs, the same single act
|
||
that qualifies their param paths. Like the output and role names, **node names are
|
||
non-load-bearing** (C23): they live at the blueprint boundary and in render but are
|
||
dropped at lowering — the flat graph is wired by raw index. The full composite
|
||
boundary signature (named inputs, multi-outputs) and the per-node param path are
|
||
legible without changing the flat graph.
|
||
**Refinement (param-namespace injectivity, cycle 0032; supersedes the fan-in
|
||
distinguishability check).** A blueprint compiles only if its `param_space()` name
|
||
projection is **injective** — every path-qualified knob name is unique. A duplicated
|
||
path is a knob no binding can select alone (it has no distinct by-name address,
|
||
C12/C19), so it is a `CompileError::DuplicateParamPath` carrying the offending path;
|
||
the cure is to give the colliding same-type sibling nodes distinct names with
|
||
`.named(...)`, the same single act that qualifies their param paths. The
|
||
param-bearing indistinguishable fan-in is *one instance* of a duplicated path (two
|
||
default-named same-type legs share a leaf path). `signature_of`, `leaf_has_param`,
|
||
and the fan-in-specific check (`check_fan_in_distinguishability` /
|
||
`check_composite_fan_in`) are **retired** (cycle 0032), replaced by one structural
|
||
`check_param_namespace_injective` over the `param_space()` names, run before name
|
||
resolution in `compile_with_params` and both binders — so the canonical by-name
|
||
author sees the structural cure rather than a downstream `AmbiguousKnob` symptom
|
||
(that `BindError` arm is retired too: an injective space can never multi-match).
|
||
Paramless interchangeable same-name sources stay legal (no path, no duplicate).
|
||
The old signature-collision predicate had extra breadth — it also rejected an
|
||
**asymmetric param/paramless** collision and a **role-vs-leg** collision, *neither*
|
||
a path duplicate (each colliding configuration keeps a unique param path, or none).
|
||
That breadth guarded **render identity**, dead since the renderer was retired in
|
||
0026; both extra rejections are **intentionally dropped**. A future node-/wiring-name
|
||
distinguishability check, if ever wanted (e.g. for the WASM graph view's #21
|
||
thread), is decoupled from param-space injectivity. Construction-phase only; the
|
||
flat graph stays name-free (C23).
|
||
|
||
### C10 — Strategy output is an intent/exposure stream; position management is a decoupled derived layer; brokers are downstream nodes
|
||
**Guarantee.** A strategy's primary, backtestable output is **not** an equity
|
||
curve, **nor a position-event table**, but an **intent / exposure stream**: the
|
||
DAG expresses exactly one *state* at time t (C8 — a node emits at most one record
|
||
per `eval`), so a strategy emits one **signed, bounded exposure** `f64 ∈
|
||
[-1, +1]` per cycle (per instrument; the portfolio is multi-instrument). Exposure
|
||
is the desired fractional position; **position sizing and risk live in the
|
||
decision/sizing node** that shapes a raw signal score into exposure. The chain is
|
||
`signals (scores) → decision/sizing node → exposure stream`.
|
||
|
||
**Signal quality** is evaluated by the **(a) sim-optimal broker** — a downstream
|
||
consumer node (C8/C9) that consumes the **exposure stream** plus the relevant
|
||
price stream and integrates `exposure(t-1) · (price(t) − price(t-1))` into a
|
||
**synthetic equity stream in pips**: deterministic, frictionless, perfect-fill
|
||
(no currency, no real-broker constraints). The exposure held *into* a cycle
|
||
(decided at t-1) earns that cycle's return, so the integration is causal (C2 — no
|
||
look-ahead). Pip PnL uses per-instrument pip metadata (reference data beside the
|
||
hot path, C7/C15). This pip-equity curve measures the **signal's quality**, not
|
||
an execution-modelled P&L; it is the primary research loop — backtest one signal,
|
||
combine it with another, backtest the combination.
|
||
|
||
**Position management** — turning the exposure stream into a broker-independent,
|
||
time-ordered **table of position events** (pure scalar columns, C7: `event_ts:
|
||
timestamp`, `action: i64` buy / sell / close, `position_id: i64`,
|
||
`instrument_id: i64`, `volume: f64` unsigned — direction is the `action`; a
|
||
position's open time is its opening event's `event_ts`, no separate `open_ts`; a
|
||
`close` references a `position_id` and may be partial via its own `volume`) — is a
|
||
**decoupled, derivable, downstream** layer. The events are the **first difference
|
||
of the exposure state**, materialized as a *computed table* (where multiple
|
||
events may share one `event_ts` — e.g. a reversal's close + open), **never as a
|
||
per-`eval` node output** (which C8 caps at one record per cycle). This layer feeds
|
||
**(b) realistic broker nodes** (Pepperstone, …): downstream consumer nodes that
|
||
consume the position-event table plus prices, apply real spread / commission /
|
||
slippage / lot / margin, may reject or modify positions, and produce a
|
||
**currency** equity stream for viability and deployment. The position-event table
|
||
stays broker-independent — one table feeds many realistic brokers, giving
|
||
directly comparable currency curves. Live: a realistic broker node consumes the
|
||
events in real time and routes orders as a side effect; reconciliation with the
|
||
real account is an external adapter.
|
||
|
||
So **the two broker classes attach at two different points**: the sim-optimal
|
||
broker on the **exposure stream** (signal quality, pips, now), realistic brokers
|
||
on the **derived position-event table** (execution viability, currency, later).
|
||
Both are ordinary downstream nodes (C8/C9); several can attach at once for
|
||
directly comparable curves.
|
||
**Forbids.** Treating an equity curve as the strategy's output; making the
|
||
**position-event table the strategy's direct DAG output** (it is derived, not
|
||
emitted per `eval` — a decision instant may need >1 event, which C8 forbids) or
|
||
the measure of signal quality; baking a broker into the strategy; a special
|
||
external broker subsystem (a broker is an ordinary node); storing `open_ts`
|
||
(derive it from the opening event); a signed-volume direction trick in the event
|
||
table (use `action`); broker-specific assumptions leaking into the strategy logic.
|
||
**Why.** The DAG is a synchronous reactive graph: at time t it holds exactly one
|
||
state, and a node emits at most one record per `eval` (C8). A *sequence of
|
||
position events* — where one decision instant (a stop-and-reverse) needs a close
|
||
**and** an open at the same `event_ts` — cannot be the DAG's per-cycle output
|
||
without violating C8. The state the DAG *can* express faithfully is the **desired
|
||
exposure** (one value per cycle); the position events are its first difference, a
|
||
derived consequence. Decoupling them resolves the C8↔C10 impedance and matches
|
||
how signal research actually proceeds: you first measure a signal's *quality*
|
||
(does this exposure, held over time, make pips?) independently of any execution
|
||
model; only later do you model position management and real-broker frictions for
|
||
deployment. The sim-optimal pip curve is a level, currency-free playing field for
|
||
comparing and combining signals; the derived position-event table + realistic
|
||
broker nodes then test real-world viability. Modelling brokers as nodes (not a
|
||
bespoke subsystem) keeps them within the one Node/graph abstraction (C9). This
|
||
supersedes the earlier "the strategy's output is the position-event table"
|
||
framing (cycle 0007 reframe).
|
||
**Realization (cycle 0007).** The signal-quality half of this contract is now
|
||
realized at the substrate, as two `aura-std` nodes composed on the unchanged
|
||
engine (the engine stays domain-free — it routes only `f64` records, never
|
||
"exposure" or "equity"). The **exposure stream** is realized as `Exposure { scale
|
||
}`: the decision/sizing node, `clamp(signal / scale, -1, +1)`, one `f64` per fired
|
||
cycle (`None` until warmed up). The **sim-optimal broker** is realized as
|
||
`SimBroker { pip_size }`: a two-input node (exposure, price) that accumulates
|
||
`prev_exposure · (price − prev_price) / pip_size` and emits cumulative pip equity
|
||
— the exposure held *into* a cycle (decided at t-1) earns that cycle's return, so
|
||
the integration is causal (C2), and `pip_size` is held reference metadata
|
||
(C7/C15), never streamed. An end-to-end harness (SMA-cross → `Exposure` →
|
||
`SimBroker` → recording sink) produces a recorded pip-equity curve, bit-identical
|
||
across runs (C1). The **position-management half** — deriving the position-event
|
||
table (buy/sell/close, `position_id`, partial closes) from the exposure history,
|
||
and the realistic broker nodes that consume it — is deliberately **not** built
|
||
this cycle; it remains the decoupled, derived, deferred layer described above.
|
||
**Realization (per-instrument pip channel, 2026-06, #22).** `SimBroker`'s
|
||
`pip_size` is now sourced **per instrument**, not from one global literal. A typed
|
||
`InstrumentSpec { pip_size }` + `instrument_spec(symbol) -> Option<InstrumentSpec>`
|
||
lookup — a Rust-authored vetted table in `aura-ingest`, at the ingestion/source
|
||
edge where the symbol still exists (never `Aura.toml`, never `Ctx`) — supplies the
|
||
divisor; the engine stays domain-free (no instrument identity reaches the hot path).
|
||
The honesty rule is **refuse, don't guess**: a real-data run for a symbol with no
|
||
vetted spec is a usage error (`exit 2`), so cross-asset pip equity is comparable by
|
||
construction rather than by researcher discipline. Threaded through the CLI
|
||
`aura run --real` path; the manifest broker label records the looked-up pip. The
|
||
runnable GER40 examples (already at the correct `1.0`) are routed through the same
|
||
lookup separately (#98).
|
||
|
||
### C11 — Generalized sources; record-then-replay determinism boundary
|
||
**Guarantee.** A source is anything that produces timestamped scalar streams —
|
||
market data (`data-server`) and non-financial sources (e.g. a news-agent node
|
||
emitting a bias) are treated identically. Anything nondeterministic, external,
|
||
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
|
||
*before* it enters the engine; backtest replays the recording, live computes
|
||
fresh in real time and records it for future backtests. A bias enters as a value
|
||
held until the next event (firing policy A).
|
||
**Forbids.** Any live external call *inside* a backtest replay.
|
||
**Why.** It is the only model compatible with reproducible backtests — LLM calls
|
||
are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`,
|
||
external LLM (IONOS) calls happen only at the recording/live-source edge, with
|
||
explicit per-session consent, never inside a sim.
|
||
|
||
### C12 — The atomic sim unit and the four orchestration axes
|
||
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
|
||
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
|
||
values injected at graph build (no recompile per param-set; the optimizer sees a
|
||
generic vector of typed ranges). Raw data is shared read-only across sims via
|
||
`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic
|
||
unit: (1) param-sweep (grid/random), (2) optimization (argmax metric),
|
||
(3) walk-forward (rolling in-sample optimize + out-of-sample test),
|
||
(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over
|
||
seeds**; each realization is itself deterministic given its seed.
|
||
**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the
|
||
only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait
|
||
(`peek`/`next`, object-safe) the k-way merge drives — and a streaming
|
||
**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing
|
||
**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and
|
||
decoding each `Scalar` on demand: the **source ring** is resident O(one chunk),
|
||
not O(window length) — the measured `resident_records()` predicate, a *per-source*
|
||
bound, not whole-process RSS. (Data-server's `FileCache` retains each window's
|
||
parsed chunks read-only for the pass — ~56 B/record — so process residency is
|
||
O(records-touched); that is the **replay-many** sharing C12 wants — one window
|
||
parsed once across a sweep family — not a leak. The single-pass cost is tracked as
|
||
#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the
|
||
gap closes by a streaming path *existing*, not by deleting the eager one). **Still open:**
|
||
cross-*sim* `Arc<[T]>` sharing — one window shared zero-copy across many disjoint
|
||
sweep sims — has no consumer until the orchestration families (axes 2–4 above:
|
||
#66/#68/#69) are built, and remains the target for those cycles.
|
||
**Realization (cycles 0028, 0049).** Axis 1 (param-sweep) is built. `GridSpace`
|
||
(0028) enumerates a cartesian lattice over discrete per-slot value-lists;
|
||
`RandomSpace` (0049) draws `N` seeded points over typed continuous `ParamRange`s
|
||
(I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), validated against the
|
||
param-space before any run. Both implement the `Space` trait the disjoint
|
||
`sweep` / `run_indexed` core is generic over, so either enumeration runs through
|
||
one execution path (C1: results in enumeration order, not completion order). The
|
||
seeded sampler reuses the bit-stable `SplitMix64` as a **code-path-disjoint**
|
||
instance from the data-edge seed RNG (the source-seam firewall, #52/#71: they
|
||
share only the `u64` type, never a path).
|
||
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
|
||
primitive — those are pluggable policies atop the atomic unit; recompiling on a
|
||
param change.
|
||
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
|
||
(embarrassingly parallel across the unit) cleanly separated from search policy.
|
||
Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the
|
||
atomic unit is one harness instance, selected by the harness's **structural
|
||
axes** (C20); the structural experiment matrix is the outer orchestration over
|
||
this dimension, the tuning sweep the inner (C19/C20).
|
||
|
||
### C13 — Hot-reload is authoring-only; deploy is frozen
|
||
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
|
||
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
|
||
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
|
||
artifact.
|
||
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
|
||
toolchain plugins.
|
||
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
|
||
and reproducible (audit trail: this bot = this commit). A sweep pays no
|
||
hot-reload tax — params are runtime data (C12), so the cdylib loads once.
|
||
|
||
### C14 — Headless core, two faces
|
||
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
|
||
**programmatic/CLI** face (the primary surface for the LLM and automation —
|
||
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
|
||
for human exploration. Visualization is only a downstream consumer node on the
|
||
streams.
|
||
**Forbids.** Any UI/pixel knowledge inside the engine.
|
||
**Why.** The LLM drives programmatically, the human visually; a headless core
|
||
serves both. The visual face is the **playground** (C22) — a **web frontend
|
||
served from disk-persisted traces** (revised 2026-06, issue #101: the engine
|
||
writes recorded traces to disk, a browser charts them; supersedes the earlier
|
||
egui-native / in-process-zero-copy-from-SoA direction). It is staged after the
|
||
runnable substrate but is core to aura's identity, not optional. The pivot keeps
|
||
this contract's core intact — and arguably tightens it: a browser reading a
|
||
serialized trace file is a stricter "no UI knowledge in the engine" than egui
|
||
reaching zero-copy into the live SoA columns.
|
||
|
||
### C15 — Resampling-as-node; sessions/calendars
|
||
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
|
||
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
|
||
and instrument specs are metadata (non-scalar, beside the hot path); session
|
||
*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open:
|
||
i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after
|
||
session open" is then a plain node checking `bars_since_open == 3`.
|
||
**Forbids.** Streaming the calendar; special-casing session logic outside the
|
||
stream model.
|
||
**Why.** Keeps the line consistent — everything a signal needs arrives as a
|
||
stream; reference data feeds source/session nodes from beside the hot path.
|
||
**Realization (instrument specs, 2026-06, #22).** The "instrument specs are
|
||
metadata" half of this contract is first realized by `aura-ingest`'s
|
||
`InstrumentSpec { pip_size }` + `instrument_spec(symbol)` — non-scalar reference
|
||
data held beside the hot path, keyed by symbol, feeding the sim-optimal broker's
|
||
pip divisor (C10). Minimal today (pip only); extensible to tick size / digits /
|
||
quote currency without a signature break.
|
||
|
||
### C16 — Engine / project separation; three-tier node reuse
|
||
**Guarantee.** aura is the reusable **engine**; each research project is a
|
||
separate external repo that depends on aura via cargo (the game-engine / game
|
||
split). Node reuse is cargo-native, in three tiers: **`aura-std`** (universal
|
||
blocks, ship with the engine) / **shared node crates** (cross-project-reusable,
|
||
their own repos, pulled as cargo git deps) / **project-local `nodes/`**
|
||
(experimental, project-specific). A reusable node is an `rlib` dependency; the
|
||
hot-reload unit stays the project-side `cdylib` that composes it (consistent
|
||
with C13). Concretely a project is a **Rust crate** — a cdylib library of node /
|
||
strategy / experiment blueprints — plus a static `Aura.toml` (project context:
|
||
data paths, instrument/pip metadata, default broker & window, runs dir). During
|
||
research the `aura` host loads and runs it (C13 hot-reload); for deploy the
|
||
chosen strategy + broker freeze into a standalone binary. **A project is
|
||
therefore always a Rust program built on the engine.** Beyond the node-reuse
|
||
tiers, the engine workspace also carries non-node crates — `aura-engine` (the
|
||
run loop), `aura-cli` (the `aura` binary), and **`aura-ingest`** (the
|
||
data-source ingestion edge, cycle 0011, where the `data-server` external tree
|
||
enters). **Dependency policy (amended 2026-06-10, cycle 0029).** Dependencies
|
||
are admitted by deliberate, **per-case review** — what a crate pulls in weighed
|
||
against what it buys — with **particular scrutiny for anything that enters the
|
||
frozen deploy artifact** (C13: this bot = this commit). Well-established
|
||
standard crates (`serde`, `rayon`, …) pass that review and are used wherever
|
||
they do the job, including in the bot. There is **no blanket zero-dependency
|
||
commitment** and **no blanket admission**; hand-rolling what a vetted standard
|
||
crate already does is the anti-pattern, not the dependency. This strikes the
|
||
original *"zero-external-dependency by commitment"* clause and the
|
||
*"`aura-ingest` is the sole external-dependency firewall"* framing; `aura-ingest`
|
||
remains the data-source ingestion edge, no longer a dependency wall.
|
||
**Forbids.** Project-specific signals in the aura repo (it keeps at most
|
||
example/fixture nodes under `examples/` for its own tests); a multi-project
|
||
manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the
|
||
package mechanism). (Dependency admission is governed by the per-case policy
|
||
above, not a blanket ban.)
|
||
**Why.** The engine/game split keeps the engine sharp and reusable while each
|
||
project versions its own research with its own forward-queue. Promotion
|
||
(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism.
|
||
|
||
### C17 — Authoring surface
|
||
**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** —
|
||
is authored in native Rust through **Claude Code + the skills pipeline**: the
|
||
human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI,
|
||
and reports metrics. Declarative config (`Aura.toml`) carries only **static
|
||
project context** (data paths, instrument/pip metadata, defaults, runs dir),
|
||
never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a
|
||
*runtime data source* (news-agent bias, C11), gated by per-session consent,
|
||
never in the code path.
|
||
**Forbids.** An in-app LLM chat that generates node code inside aura; using
|
||
IONOS (weaker models) as the authoring brain.
|
||
**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's
|
||
failure; making weaker models the coding brain reintroduces the very problem.
|
||
Keeps aura's scope an engine + playground, not an LLM-IDE.
|
||
|
||
### C18 — Project management: one repo = one project, plus a run registry
|
||
**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git
|
||
(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as
|
||
the forward-queue, a research thrust = a milestone, the
|
||
`idea → experimental → validated → deployed` label gradient). (2) **Experiments
|
||
& results:** an Aura-native **run registry** — one record per run = a *manifest*
|
||
(node-commit + params + data-window + seed + broker profile) + *metrics*,
|
||
queryable, with *lineage* (composite ← signals; run ← inputs). Determinism
|
||
(C1/C12) makes a run reproducible from its tiny manifest, so the registry stores
|
||
manifests + metrics and re-derives full results on demand. Depth: **structured**
|
||
(promotion/status, lineage, run-diff).
|
||
**Forbids.** Storing results not reproducible from a recorded manifest;
|
||
duplicating git/Gitea inside aura; a multi-project workspace manager.
|
||
**Why.** Comparing experiments over time is the heart of the research loop and
|
||
has no home in git/Gitea; determinism makes a structured registry cheap.
|
||
Sequencing: the walking skeleton emits a manifest + metrics per run from day
|
||
one; the registry/index is a later milestone over manifests that already exist.
|
||
|
||
**Realization (cycle 0029 — the flat run registry).** The experiments-&-results
|
||
plane shipped as `aura-registry`: an append-only JSONL store (`runs/runs.jsonl`),
|
||
one serde_json `RunReport` (`RunManifest{commit, params, window, seed, broker}` +
|
||
`RunMetrics`) per line, with a typed read-path (`load`) and best-first ranking
|
||
(`rank_by`/`optimize`). C9 holds — the registry depends on `aura-engine`, never
|
||
the reverse.
|
||
|
||
**Realization (cycle 0045 — lineage as related records, #70).** The *lineage*
|
||
depth is now realized as a **family store**: a sweep / Monte-Carlo / walk-forward
|
||
run (the C12 axes) is persisted as a *set of related records* — each a
|
||
`FamilyRunRecord` (a `RunReport` stamped with its `family` + `run` + `kind` +
|
||
`ordinal`) — in a sibling JSONL (`families.jsonl`), leaving the flat `runs.jsonl`
|
||
path and its `append`/`load`/`rank_by`/`optimize` API byte-for-byte unchanged.
|
||
the user-facing `family_id = "{family}-{run}"` handle is **derived** from the
|
||
stored `family` name plus a per-name `run` index (assigned as a numeric max+1 —
|
||
not a content hash; re-running the same family mints a fresh id). `group_families` is the round-trip that re-derives a family
|
||
from the stored links (re-listable / rankable as a unit — C21). The manifest *is*
|
||
the re-derivation recipe (#71): no input-stream blob / path / payload enters a
|
||
record, and a member's window is **producer-supplied** via
|
||
`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage),
|
||
never a materialized-`Vec` scan at the call site. CLI surface: `aura mc`,
|
||
`aura runs families`, `aura runs family <id> [rank <metric>]`; `aura sweep` /
|
||
`walkforward` / `mc` persist via `append_family` with an optional `--name`.
|
||
Deferred (Non-goals): content-addressed identity + replay-dedup; the "run-diff"
|
||
depth and ranking families against each other (cross-family, vs. within-family);
|
||
and a live producer for the flat `runs.jsonl` standalone-run path — no CLI command
|
||
writes it (sweep/walkforward persist to the family store; `aura run` does not
|
||
persist). **Resolved (#73, 2026-06): retired** — `aura runs list` / `rank` dropped;
|
||
families (C21) subsume standalone over-time comparison. The `aura-registry` flat
|
||
lib API is retained: `rank_by`/`optimize` keep live consumers (`optimize` backs
|
||
walk-forward's in-sample step, `rank_by` backs `runs family … rank`); `append`/
|
||
`load` (the flat-store half) remain public API with **no in-tree caller** after
|
||
this retire — tested, available to external consumers, a latent dead-code surface
|
||
a later sweep may revisit. **Unknown-id contract (ratified, Runway fieldtest
|
||
2026-06).** `aura runs family <id>` treats an unknown-but-well-formed id as an
|
||
*empty family* (prints nothing, exit 0) — the same treat-as-empty discipline as
|
||
`Registry::load` reading a missing store as `Ok(empty)`, and deliberately
|
||
distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection
|
||
*before* any store access, not a found-nothing lookup. Tightening to a non-zero
|
||
`no such family <id>` exit (typo-safety) is an available future UX choice, not a
|
||
current contract.
|
||
|
||
### C19 — Bootstrap: blueprint → instance (recursive)
|
||
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
|
||
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
|
||
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
|
||
graph-as-data produced by running a Rust builder (C9); it carries *free* numeric
|
||
params (declared ranges) and *free* input roles. The **bootstrap** binds
|
||
`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen
|
||
instance** — buffers sized, topology fixed. This is precisely the "wiring /
|
||
graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12
|
||
("params injected at graph build") already reference. The same machinery applies
|
||
recursively up to the harness (C20). A sweep builds many instances from one
|
||
blueprint; instances are disjoint (C1).
|
||
**Forbids.** Params that change topology (a topology change is a *different*
|
||
blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a
|
||
sim against an un-bootstrapped blueprint.
|
||
**Why.** Separating the param-generic blueprint from the param-bound instance is
|
||
what makes one strategy reusable across a whole sweep and lets the optimizer
|
||
mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead
|
||
of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12
|
||
explicit.
|
||
**Refinement (Construction-layer milestone, 2026-06-05).** This binding is a
|
||
**compilation**: the param-generic, named blueprint (source) is lowered to a flat,
|
||
type-erased **flat graph** (C7) **wired by raw index, not by name**
|
||
(`Edge { from, to, slot, from_field }`): composite boundaries dissolve entirely,
|
||
and field / role names are demoted to **non-load-bearing** debug symbols (as
|
||
`FieldSpec.name` already is). "No recompile" above means no **Rust /
|
||
cdylib** rebuild (C12/C13: the cdylib loads once); re-deriving an instance per
|
||
param-set is a cheap **graph re-compilation**, not a code recompile. Naming the
|
||
build phase a compilation makes its successor explicit: the flat graph is the target
|
||
of behaviour-preserving optimisation (C23).
|
||
**Realization (cycle 0016 — param-set injection).** The bootstrap now binds an
|
||
injected param-set, realizing C12's "params injected at graph build (the optimizer
|
||
sees a generic vector of typed ranges)" and C19's "factory `params → sized node`"
|
||
*literally*: a blueprint leaf is **value-empty** — `BlueprintNode::Leaf` holds a
|
||
`LeafFactory { name, params, build }` recipe, not a built node — and the value lives
|
||
only in the injected vector (no baked default), so the blueprint stays a pure
|
||
param-generic recipe. (Renamed in cycle 0024: `BlueprintNode::Primitive` holds a
|
||
`PrimitiveBuilder { name, schema, build }` — the recipe now carries the full
|
||
signature, see the C8 0024 realization; `bootstrap_with_params`/`compile_with_params`
|
||
moved onto `Composite` when `struct Blueprint` collapsed into it, see the C19 0024
|
||
realization.) `bootstrap_with_params(Vec<Scalar>)` /
|
||
`compile_with_params` **build each leaf through its own constructor** (the single
|
||
sizing/validation gate) from its kind-checked slice **while lowering** (build-then-
|
||
wire), consuming the vector slot-by-slot in the *same* depth-first walk
|
||
`param_space()` projects — so the two share one traversal (subsuming the #34 dual-
|
||
traversal hazard) and the value reaches the node at the slot the sweep enumerates.
|
||
Arity is checked up front (`param_space().len()`); a wrong-kind or wrong-length
|
||
vector is a typed `CompileError::{ParamKindMismatch, ParamArity}` (the typed-value
|
||
check C8 deferred). The lowering/edge/source rewrite is structurally unchanged, so
|
||
the **flat graph stays bit-identical** for a given point (C23, the correctness
|
||
invariant). The value *domain* (e.g. `length ≥ 1`) stays the constructor's own
|
||
`assert`; the search-range is still the run's (#32/C20). One value-empty leaf
|
||
detail for C22: the blueprint view (pre-run, param-generic) labels a leaf by **bare
|
||
type** (`PrimitiveBuilder::label`, was `LeafFactory::label`, → `[SMA]`) — the
|
||
value-bearing `SMA(2)` of C8's render-
|
||
label refinement now appears only in the *compiled* view (built nodes, `Node::label`).
|
||
**Realization (cycle 0017 — blueprint render = main graph + definitions).** The
|
||
`aura graph` blueprint view (C9 graph-as-data, #13) renders the **authored
|
||
structure** as a *program with subroutines*: a flat **main graph** wiring the
|
||
harness with each composite shown as a **single opaque node** `[name]`, plus a
|
||
`where:` section that defines each **distinct** composite type **once** (its
|
||
interior with named input-entry nodes and outputs folded onto their producers as
|
||
`name := …` bindings (render refined through cycles 0019–0022; originally
|
||
`[in:k]`/`[out]` port markers); deduped by `name()`, collected
|
||
recursively so nested composites are opaque nodes with their own definitions). This
|
||
supersedes #13's original cluster-box model and is the durable split this view
|
||
realizes: **blueprint = source** (composites as named subroutines, body once) vs.
|
||
**compiled = inlined machine form** (C23, boundary dissolved). The substantive cause
|
||
for retiring cluster boxes is a real renderer defect — `ascii-dag` 0.9.1's subgraph
|
||
level-centering rounds sibling x-positions with `/2`, overlapping wide sibling
|
||
labels (width/parity-sensitive, no config/padding dodge surviving unequal-width
|
||
siblings); the **flat** layout is collision-free, and both views now build flat
|
||
graphs only. The model also scales (blueprint size tracks top-level wiring, not
|
||
inlined node count) and removes #13's nested-composite `unimplemented!` (the
|
||
definitions pass recurses). The interactive *enter/focus* counterpart (a composite
|
||
collapsed to a navigable node) is the playground's, parked as a separate concern
|
||
(#37); the static CLI keeps the all-at-once definitions form (#38).
|
||
|
||
**Realization (cycle 0024 — the root is the fully-bound composite; the flat graph is
|
||
a named type).** `struct Blueprint` is **deleted**: the root graph IS a `Composite`,
|
||
and `compile_with_params`/`bootstrap_with_params`/`param_space` are its methods.
|
||
What distinguished the root — its bound data sources — is now a property of its input
|
||
roles: `Role` carries `source: Option<ScalarKind>` (`None` = an open interior port,
|
||
wired by the enclosing graph; `Some(kind)` = a bound ingestion feed). A composite is
|
||
**runnable iff every root role is bound** (C3: sources bind at ingestion only); an
|
||
open root role is a compile-time `CompileError::UnboundRootRole`. So the "main graph"
|
||
is no longer a separate kind — only the composite all of whose roles are
|
||
source-bound, governed by the same conditions as any other node. Compilation now
|
||
targets a **named type**: `compile` validates structurally **pre-build** (via
|
||
`signature()`, no node constructed — an ill-typed wiring is caught before any build
|
||
closure fires) and emits `FlatGraph { nodes, signatures, sources, edges }` — the
|
||
C23 flat graph, now first-class — which `Harness::bootstrap` consumes (kinds/firing
|
||
from the carried signatures, buffer depth from `lookbacks()`). The per-flat-node
|
||
signature travels beside the node, so `bootstrap` reads it without a built-node
|
||
`schema()` call (which no longer exists, see the C8 0024 realization).
|
||
|
||
**Realization (cycle 0026 — graph render redesign: model + WASM-Graphviz viewer,
|
||
#51).** `aura graph` no longer renders ASCII. The render path is now two pieces:
|
||
a read-only **model serializer** (`aura_engine::model_to_json`, iteration 1) that
|
||
walks the root composite + every distinct composite type into a deterministic,
|
||
hand-rolled JSON model (C14, golden-tested; the engine's last hand-rolled JSON
|
||
writer after `RunReport::to_json` moved to serde in cycle 0033; the
|
||
swapped-param mis-wire property moved here from the old compiled-view test), and a
|
||
**self-contained HTML viewer** (`aura-cli::render::render_html`, iteration 2) that
|
||
inlines that model, the ported prototype viewer JS, and a vendored Graphviz-WASM
|
||
blob into one page emitted to stdout. Layout/SVG happen in the browser via
|
||
WebAssembly — aura ships no layout engine and stays a serializer (C9: graph-as-data,
|
||
no `eval`/build on the path). The viewer is a render asset (C10 — no node/strategy
|
||
logic, no DSL); it labels every input pin from the model's now-real names (C23
|
||
debug symbols, named in cycle 0027) and colours wires by the four scalar base
|
||
types (C4). This **retires `ascii-dag`** and its adapter (`graph.rs`), the
|
||
`--compiled`/`--macd` flag plumbing, and the invented `#Sf`/`:=`/`histogram →`
|
||
notation — superseding the cycle-0017 flat-ascii model and its renderer-defect
|
||
workaround. The DOT/SVG are Graphviz-version-dependent and not golden-tested; the
|
||
deterministic JSON model is the asserted contract.
|
||
|
||
**Realization (cycle 0034 — structural-constant bind: a knob *removed* from
|
||
param_space, #55).** `PrimitiveBuilder::bind(slot, value)` adds the **third** param
|
||
category beside the topology factory-arg (C7/C19) and the tuning param (the
|
||
cycle-0016 value-pin): a **structural constant**. The cycle-0016 binding *pins* a
|
||
value in the injected vector while the knob **stays** in `param_space` (a tuning
|
||
param the sweep varies); `bind` instead **removes** the slot from `param_space`
|
||
entirely — the knob is gone, not fixed. The discriminator is the #55
|
||
deform-vs-tune test: a value whose variation yields another valid point of the
|
||
*same* strategy is a **tuning param** (stays in `param_space`); a value whose
|
||
variation *deforms* the strategy into a different one (e.g. the `2` of an
|
||
"SMA2-entry" bound to its two-candle construction) is a **structural constant**
|
||
(bound out), so a sweep never enumerates deformed strategies as valid family
|
||
members. Mechanically `bind` shrinks the builder's declared param surface
|
||
(`schema.params`) and wraps its build closure to re-splice the constant at its
|
||
original positional slot; the construction layer
|
||
(`collect_params`/`lower_items`/`param_space`/`compile_with_params`) is
|
||
**byte-unchanged** — both dock sites already key off `builder.params()`, so the
|
||
shrink propagates for free, and chained binds reconstruct the correct positional
|
||
vector because each layer computes its slot index relative to the param list it
|
||
sees. C23 is unaffected: `bind` resolves the param **name** to a position at
|
||
**authoring** time (the by-name authoring address space, the 0032 amendment) and
|
||
the flat graph stays wired by raw index — the name never reaches it. The
|
||
complementary question — exporting a **named frozen** strategy (all/most knobs
|
||
bound) as a reusable blueprint *value* — is deferred (#60); `bind` ships only the
|
||
per-knob overlay, no registry (C9/C10 intact).
|
||
|
||
### C20 — Strategy ↔ harness; the harness is the root sim graph
|
||
**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9):
|
||
broker-, data-, and viz-independent, with inputs declared as named **roles**
|
||
(symbol-agnostic where possible) and the intent/exposure stream (C10) as output.
|
||
A **harness** (the experimental setup) is the **root sim graph** — sources bound
|
||
to the strategy's input roles + the strategy + attached broker node(s) + sinks —
|
||
and is itself produced by the bootstrap (C19). A harness *instance* is C1's
|
||
disjoint unit (RustAst's "root scope"). The harness has **two kinds of
|
||
parameterization**: **structural axes** (which strategy, which instrument(s),
|
||
which broker(s), which window) whose variation selects *different* instances —
|
||
the **experiment matrix**; and **tuning params** (the strategy's numeric params)
|
||
swept *within* a fixed structure (Fork A). The same strategy blueprint is reused
|
||
across backtest, sweep, visual workspaces, and the frozen live bot — each a
|
||
different harness. **Both strategy and harness/experiment are authored in Rust**
|
||
via builder APIs (C17); the experiment matrix is ordinary Rust control flow
|
||
(loops/conditionals), not a config schema. Ontologically, a node (incl. a
|
||
strategy composite) is an **open** fragment — free input roles + ≤1 output
|
||
(C8/C9) — that does not run alone; a **harness is the closed root graph**: a
|
||
strategy with its input roles bound to sources and its output terminated in
|
||
broker/sink nodes, under a clock. A harness is therefore **not a node** (no free
|
||
inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's
|
||
disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21)
|
||
orchestrates them as objects.
|
||
**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a
|
||
declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as
|
||
a node (it is the closed root scope, not an open composable node).
|
||
**Why.** Reusability needs the strategy to be a context-free blueprint that many
|
||
harnesses embed. Modelling the harness as a root graph keeps it within the one
|
||
Node/graph abstraction (C9) and makes "10 strategies in one environment" and
|
||
"one strategy × N instruments" plain nested loops over the structural axes. Rust
|
||
authoring (not config) preserves full programmatic power — conditional/adaptive
|
||
matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap
|
||
C17 rejects.
|
||
**Realization (GER40 session-breakout blueprint milestone, 2026-06-17 — refs
|
||
#94/#96/#97, spec 0051).** Made concrete on the first real-data strategy: a
|
||
**hand-wired `FlatGraph` is not a shippable strategy** — it carries no
|
||
`param_space()`, so the World families (sweep / walk_forward / compare, C21)
|
||
cannot consume it without a hand re-author (the friction the GER40 deep-dive
|
||
fieldtest surfaced). The **canonical shippable form is the `Composite`
|
||
blueprint** (the authoring/source level, C9/C19); the `FlatGraph` is only its
|
||
compiled substrate (C23). The breakout now ships as
|
||
`ger40_breakout_blueprint(bar_period, …)` whose `param_space()` is exactly its
|
||
tuning knobs (`{entry_bar.target, exit_bar.target}`); the **bar period is a
|
||
construction argument** binding Resample + Session together — a structural
|
||
matrix axis (C12: a different period is a *different strategy*, not a sweep
|
||
point), never a `param_space` entry, so a sweep cannot desync the two clocks.
|
||
This is the concrete instance of the structural-axis-vs-tuning-param split
|
||
above (and `delay.lag`, a C8 structural constant, is bound out of the space).
|
||
|
||
### C21 — The World: the meta-level is the product
|
||
**Guarantee.** Above the harness sits the **World** — the project's program /
|
||
"game" (C16: a Rust crate). Within it, **harnesses are dynamically constructible,
|
||
first-class objects**: meta-programs (walk-forward, sweep, optimize,
|
||
Monte-Carlo — C12's axes) construct harness instances at runtime via the
|
||
bootstrap (C19), run them disjointly in parallel (C1), aggregate / compare their
|
||
results, and discard the transient instances. A walk-forward rolls windows →
|
||
bootstraps a harness per window → stitches out-of-sample equity + parameter
|
||
stability into one meta-result. Orchestrating *families* of harnesses is
|
||
**first-class**, not a headless afterthought; the run registry (C18) is the
|
||
World's memory.
|
||
**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep /
|
||
comparison) to second-class headless-only status; treating the single backtest
|
||
as the product.
|
||
**Why.** What happens *within* one harness — backtest a strategy → equity — is
|
||
commodity; every quant system covers it. aura exists for the meta-level: a
|
||
programmable space where harnesses are dynamically built and families of them
|
||
orchestrated and explored. The deterministic single-harness engine (C1–C20) is
|
||
the **substrate**; the World is the **product**.
|
||
|
||
### C22 — The playground is a trace explorer; sinks are the recording mechanism
|
||
**Guarantee.** The World is a *program*: nothing is displayable until it runs,
|
||
and its harnesses are transient machinery (built, run, discarded — C21). The only
|
||
durable, displayable substance is the **recorded trace**, captured by **sinks** —
|
||
pure consumer nodes (C8) that persist a stream (equity, position events, a node's
|
||
output) into the run registry (C18). **Displayable = exactly what a sink
|
||
recorded**; with no sink, only input params + summary metrics remain. The
|
||
**playground** is therefore an **execution viewer / trace explorer**, and it
|
||
plays *any* harness (it is the harness *player*, not a harness, never bound to a
|
||
default one): before a run it shows the program *structure* (graph-as-data, C9) +
|
||
param knobs; during a run, live sink streams; after a run, recorded traces +
|
||
metrics from the registry — including meta-views (stitched walk-forward, sweep
|
||
surfaces, multi-strategy / instrument comparison). Observability is **explicit
|
||
and selective** — you instrument what you want to see; the choice of sinks is
|
||
part of the experiment. The engine ships sample harnesses *with* sinks so a
|
||
newcomer sees a populated trace immediately; a fresh project is empty until
|
||
built, run, and instrumented.
|
||
**Forbids.** A scene-editor model that assumes a persistent populated world;
|
||
constructing or wiring topology in the UI (topology is Rust + hot-reload,
|
||
C9/C17 — the UI reflects the live graph-as-data and tunes runtime params via
|
||
sliders, C12); retaining un-sinked data; binding the playground to a single /
|
||
default harness.
|
||
**Why.** A program has no scene-at-rest to inspect; what persists is what it
|
||
records. Making sinks the one recording-and-observability mechanism keeps "what
|
||
can I see?" answerable by "what did I instrument?", and keeps the engine
|
||
UI-agnostic (C14). Live param tuning (runtime values, no topology change —
|
||
C12 / C19 Fork A) gives the interactive feel without a wiring DSL.
|
||
**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the
|
||
substrate level: a recorded trace is exactly what a recording node pushed out of
|
||
the graph (no engine recording registry; the constructing World holds each
|
||
recording node's destination). The engine's single `observe: usize` affordance is
|
||
removed — `Harness::run` returns `()` and recording is a node-side concern, so one
|
||
run records *many* streams (one per recording node) instead of exactly one row.
|
||
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
|
||
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
|
||
`Harness` API change (observe removed, `run -> ()`) is recorded here.
|
||
**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped
|
||
(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA,
|
||
C7) `ColumnarTrace` to `runs/traces/<name>/` (`aura-registry::TraceStore`, beside the
|
||
run registry's `runs.jsonl`), reachable via `aura run [--macd] [--real <SYM> …]
|
||
--trace <name>`; `aura chart <name> [--panels]` reads them back, aligns all taps on a
|
||
synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph;
|
||
C1 — pure, no live external call), and emits a static self-contained uPlot page
|
||
(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14):
|
||
encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in
|
||
`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`).
|
||
First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served
|
||
page injected the chart *series* but no run-context header (the run manifest is
|
||
persisted on disk in `index.json`, deliberately not surfaced in that first cut — a
|
||
ratified scope call). The header (#102) and serve-time decimation (#108) landed in
|
||
cut 2 (see the served-page-hardening amendment below); the families-comparison view
|
||
landed too (#107). A local server and replay-clock controls remain open (see "Open
|
||
architectural threads").
|
||
**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends
|
||
to family runs: `aura sweep|mc|walkforward --trace <name>` persists *each member* as
|
||
a nested standalone run-dir `runs/traces/<name>/<member_key>/`, reusing
|
||
`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a
|
||
side-effect inside each per-member closure). The `member_key` is content-derived and
|
||
deterministic — sweep: the *varying* axes rendered as a filesystem-portable
|
||
directory component (`<axis-name>-<value>` tokens joined by `_`, charset
|
||
`[A-Za-z0-9._-]`, case-less values, length-capped with an FNV fallback), MC
|
||
`seed{N}`, walk-forward `oos{ns}` — never
|
||
a runtime ordinal, because members run in parallel under the engine's `Fn + Sync`
|
||
HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write`
|
||
targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`,
|
||
stdout/registry are byte-unchanged. Because `TraceStore` resolves `<name>/<member_key>`
|
||
as a subpath, `aura chart <name>/<member_key>` charts any single member with no
|
||
view-side change — the write-side precondition for the family-comparison **view**
|
||
(overlay / small-multiples across members) — now realized (#107; see the
|
||
families-comparison amendment below).
|
||
|
||
**Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an
|
||
**opt-in real-data source axis**: `aura sweep|walkforward --real <SYMBOL> [--from
|
||
<ms>] [--to <ms>]` streams real M1 close bars from the data-server archive (the #71
|
||
`M1FieldSource` seam — C6 record-then-replay: a pre-recorded archive, never a live
|
||
call mid-sim) instead of the built-in synthetic stream, per family member. A
|
||
CLI-side `DataSource` provider (synthetic | real) supplies each member's source,
|
||
pip (`instrument_spec` — C10 refuse-don't-guess; an un-vetted symbol exits 2 before
|
||
any data access), window, and — for walk-forward — its `WindowRoller` sizes
|
||
(synthetic: bar-index 24/12/12; real: fixed calendar-time 90d/30d/30d in ns; CLI
|
||
flags for these are deferred). The engine, ingest, and registry are untouched (C9 —
|
||
the whole change is in `aura-cli`); the synthetic path is byte-unchanged.
|
||
**MC is excluded** (#106 Fork A): its seed varies a *synthetic* price-walk
|
||
realization (C12 axis 4), which is undefined over real data's single realization —
|
||
`aura mc --real` refuses with exit 2; a real MC needs a bootstrap-resampling axis
|
||
(its own cycle). The real walk-forward member key `oos{from.0}` widens from a small
|
||
synthetic bar index to an epoch-ns integer (still portable, collision-free). The
|
||
built-in demo strategy's **length grid is likewise data-kind-dependent**
|
||
(`DataSource::strategy_lengths`): synthetic keeps the short lengths that fit the
|
||
18/60-bar demo streams (trend SMA `{2,3}×{4,5}`, MACD `2/4/3`), real uses realistic
|
||
M1 lengths (trend `{50,100}×{200,400}`, MACD `12/26/9`) so the cross is a genuine
|
||
trend signal over tens of thousands of bars, not noise. This — like the roller
|
||
sizes — is a *demo-strategy calibration* patch; the real answer is project-authored
|
||
strategies (C9: a project crate owns its own grid), deferred to the project-env
|
||
work.
|
||
|
||
**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison
|
||
**view** the #104 amendment left open is now realized. `aura chart <name>`
|
||
classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a
|
||
single run; else member subdirs with `index.json` are a family; else not-found) and,
|
||
for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every
|
||
member** in one self-contained uPlot page: `build_comparison_chart_data` emits one
|
||
labelled series per member, all on a **single shared y-scale** — members measure one
|
||
identical quantity, so a shared scale is what makes them comparable, unlike the
|
||
single-run overlay of *different* taps (per-series scales). The series align on the
|
||
same union-ts spine via `join_on_ts`, so **one mechanism yields three correct
|
||
readings**: sweep/MC members share the data window (a true overlay), walk-forward
|
||
members are disjoint OOS windows (null-complementary → the stitched curve). The view
|
||
consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted
|
||
by key for determinism, C1); a family is its *first producer* — the deliberate first
|
||
step toward the programmable analysis meta-level (C21) **without** rebuilding the
|
||
orchestration axes (that is its own later cycle). Name resolution is made a **total
|
||
function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing
|
||
command (`run`/`run --macd`/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family)
|
||
to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by
|
||
both a run and a family) is unreachable. Every error path exits 2, never panics
|
||
(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is
|
||
`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder +
|
||
`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when
|
||
no `--tap` is given. **Still deferred:** multi-column tap selection
|
||
(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are
|
||
single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as
|
||
composable meta-level tools).
|
||
|
||
**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred
|
||
follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine
|
||
+ registry untouched, C14). **Run-context header (#102):** a `ChartMeta`
|
||
(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params
|
||
for a single run) is built from the `RunManifest` in
|
||
`build_chart_data`/`build_comparison_chart_data`, injected into
|
||
`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js`
|
||
(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across
|
||
members `(min from, max to)` — the only reading that labels a disjoint walk-forward
|
||
family's true OOS coverage (it collapses to the shared window for sweep/MC).
|
||
**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max
|
||
transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots
|
||
(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a
|
||
multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full
|
||
recorded data stays on disk (view-only), and the walk-forward null-fill page collapses
|
||
for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under
|
||
budget. Deferred refinements (#110): a `--width` budget flag (the budget is a fixed
|
||
const) and true intra-bucket min/max ordering for the non-default continuous x-mode.
|
||
|
||
### C23 — Graph compilation and behaviour-preserving optimisation
|
||
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
|
||
named **blueprint** (the authoring source — nodes, composites, strategy, harness;
|
||
C8/C9/C20) into a flat, type-erased **`FlatGraph`** — the frozen runnable
|
||
instance (C7) — **wired by raw index, not by name** (`Edge { from, to, slot,
|
||
from_field }`). Composites are **inlined** at this step (C9): the composite
|
||
*boundary* dissolves entirely (there is no composite in the flat graph). The
|
||
blueprint's field / input-role *names* are **non-load-bearing** — the wiring
|
||
resolves by index, and names survive at most as **informative debug symbols**
|
||
(exactly as `FieldSpec.name` already is, C8), kept for tracing / rendering (C9
|
||
graph-as-data, #13) but carrying no run semantics. The flat graph is then the target of **behaviour-preserving
|
||
optimisation** — any transform that leaves every observable sink trace
|
||
bit-identical (C1 is the correctness invariant) — on two levels:
|
||
- **intra-graph** (within one graph): **common-subexpression elimination** —
|
||
two identical nodes (same type, same bound params, same input source) merge to
|
||
one with output fan-out, so fractal composition (C9) costs no redundant compute —
|
||
and **dead-node elimination** — a node on no path to a sink is dropped.
|
||
Pure-consumer sinks (C8, no output) are never CSE candidates, so their
|
||
out-of-graph side effects are preserved by construction.
|
||
- **across the sweep family** (over the family of flat graphs one blueprint yields
|
||
under a param sweep, C12): the sub-graph whose nodes carry **no swept param** and
|
||
whose inputs are **all sweep-invariant** (transitively from the sources — same
|
||
data window) is **loop-invariant** w.r.t. the sweep and is computed **once**; its
|
||
output is materialised as a recorded stream (C11) shared read-only across the
|
||
sweep instances (`Arc<[T]>`, C12). Only the parameter-dependent suffix re-runs
|
||
per sweep point — loop-invariant code motion over the sweep loop.
|
||
**Forbids.** Any "optimisation" that changes an observable trace (it breaks C1);
|
||
optimising over a representation that is not graph-as-data — an opaque
|
||
trait-object interior (the rejected nested-composite reading of C9) cannot be
|
||
analysed or rewritten across its boundary, so it forecloses both levels.
|
||
**Why.** Determinism (C1) is not merely an audit property; it is the **licence**
|
||
for these rewrites — a behaviour-preserving transform is only meaningful because
|
||
"same input → bit-identical run" makes "same result" decidable, and a
|
||
sweep-invariant sub-graph is reproducible enough to compute once and share. The
|
||
flat graph representation (C19) is the necessary condition: only a flat, inspectable
|
||
graph-as-data — with each node's declared param-ranges (C8) — lets the compiler
|
||
identify identical sub-expressions, dead nodes, and the sweep-invariant frontier.
|
||
This is precisely why a composite is an inlining (C9) and not a runtime
|
||
sub-engine: the flat graph is what makes the whole graph optimisable.
|
||
**Status (Construction-layer milestone).** The flat graph *representation* —
|
||
blueprint → inline / compile → flat instance — is the load-bearing substrate built
|
||
**first**; the optimisation passes (intra-graph CSE/DCE, sweep-invariant
|
||
hoisting) are **deferred**, behaviour-preserving follow-on work, each gated by a
|
||
"flat graph with pass ≡ flat graph without pass, bit-identical run" test (C1). The
|
||
sweep-level pass further presupposes per-node param declarations (C8 — landed in
|
||
cycle 0015 as `name`+`kind`; the swept *range* still pending #32/C20) and the sweep
|
||
orchestration itself (C12/C21).
|
||
**Amendment (cycle 0032 — param-namespace injectivity is part of the compilation).**
|
||
The bootstrap-as-compilation now includes one structural gate:
|
||
`check_param_namespace_injective` over the `param_space()` name projection (see C9 /
|
||
C12-C19), run before name resolution. It reads the **boundary** name projection — the
|
||
authoring address space — and rejects a duplicated path (`DuplicateParamPath`). Node
|
||
names stay **non-load-bearing**: they qualify the param path at construction and are
|
||
dropped at lowering (the flat graph is wired by raw index, unchanged). The check makes
|
||
the by-name *authoring address space* injective; it does **not** make names
|
||
load-bearing in the flat graph.
|
||
|
||
---
|
||
|
||
## Open architectural threads not yet resolved
|
||
|
||
- **Playground & World UI surface** — the playground is core (C22); the surface
|
||
is a **web frontend served from disk-persisted traces** (revised 2026-06, issue
|
||
#101; not egui). Settled for the first cut: raw per-tap trace persistence to
|
||
disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static
|
||
self-contained HTML chart page (uPlot, vendored like the `render_html`
|
||
Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The
|
||
families-comparison **view** — `aura chart <family>` overlaying one tap across a
|
||
family's members on a shared y-scale — shipped (#107, see the C22 amendment).
|
||
Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family /
|
||
multi-strategy comparison, multi-column tap selection #47), a local server, and the
|
||
run/replay clock controls.
|
||
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
|
||
atop the atomic sim unit (C12), not yet designed.
|
||
- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s
|
||
static-context schema** — `aura new` scaffolds a Rust project *crate* (node /
|
||
strategy / experiment blueprints) against the engine (C16/C20); the
|
||
experiment-builder API surface (harness wiring, structural axes, sweep
|
||
combinators) and `Aura.toml`'s schema (data paths, instrument/pip metadata,
|
||
default broker & window, runs dir) are not yet designed.
|
||
- **The analysis meta-level — composable orchestration + project-as-crate authoring
|
||
(tracked: #109).** aura's differentiator (C21) has two unbuilt halves: (1) the
|
||
orchestration axes (sweep / Monte-Carlo / walk-forward) becoming *composable tools*
|
||
a user wires into an analysis workflow, rather than today's hard-wired CLI verbs
|
||
(`sweep_family` / `walkforward_family` in `aura-cli`); and (2) the project-as-crate
|
||
authoring layer above (the `aura new` / experiment-builder API / `Aura.toml` thread
|
||
+ cdylib loading, C16/C17). On the meta-level, authoring a strategy is *wiring
|
||
existing nodes* (C9 composition) + *defining the analysis framework* (C21), not
|
||
writing new nodes; the user wants it driven via the CLI, later the interactive
|
||
server / playground (C14/C22). **Load-bearing open fork — resolve in the brainstorm
|
||
before any cycle:** "driven via the CLI" = (a) *running* a Rust-authored experiment
|
||
via the CLI / playground (compatible with C16/C17/C22) vs (b) *wiring nodes through
|
||
the CLI itself* (a deliberate amendment to C17's no-DSL, C22's no-UI-topology, and
|
||
C9's no-by-name-node-registry). Deferred by the user (2026-06-22); #109 is its
|
||
durable home until promoted to a milestone seeded by that brainstorm.
|
||
- **`aura-std` contents** — the crate exists (doc-only); which universal blocks
|
||
land first follows the walking-skeleton's needs.
|
||
- **`strategies/` split** — a later split, *inside a project*, of top-level
|
||
strategies from reusable building blocks in `nodes/`; not a day-1 cut.
|
||
- **Sequencing** — the runnable single-harness *substrate* comes first (walking
|
||
skeleton: a closed harness that runs deterministically and records via a sink);
|
||
the World/meta layer (C21) and the playground trace-explorer (C22) are the
|
||
differentiating layers that follow — you orchestrate and visualize a thing that
|
||
must first run once.
|