Files
Aura/docs/design/INDEX.md
T
Brummel 7e4b91ab5f feat(0074): source real-path pip from the geometry sidecar; drop the authored floor
The data-server now ships per-symbol geometry sidecars, so aura no longer
hard-codes instrument geometry. A consumption grep showed production reads
exactly one `InstrumentSpec` field — `pip_size` — at the real-data path; the
other eight (incl. `instrument_id`) were constructed but never read.

- aura-cli resolves the real-path pip from the recorded sidecar
  (`instrument_geometry` -> `InstrumentGeometry.pip_size`), refusing (exit 2)
  on absent geometry. The authored table no longer gates: any symbol with a
  sidecar + bars now runs (new test: USDJPY, never vetted, resolves its JPY
  pip 0.01 from the sidecar).
- Removed the redundant authored floor: `InstrumentSpec`, `instrument_spec`,
  `VETTED_SYMBOLS`, and the table-only tests. The `InstrumentGeometry` seam
  (the surviving pip source) and its nameability guard are kept.
- `instrument_id` (the "find out"): no production consumer. The position-event
  table's `instrument_id` is a decoupled caller parameter; the engine never
  imports an instrument spec. Dropped with the struct.
- The example `pip_size_of` is re-sourced to the sidecar; the pure-structural
  blueprint tests use a literal pip to stay archive-independent.

Speculative deploy-grade fields and the #124 override/floor tiers are deferred
to the Stage-2 broker that will consume them, read from the sidecar geometry
then. Ledger (C8/C15/#22/#106) updated; #124 collapses to "recorded sidecar
geometry -> refuse" in the meantime.

Verified: cargo test --workspace (all green), clippy -D warnings clean, cargo
doc clean, with the local archive present so the sidecar paths run.

refs #124
2026-06-25 19:06:12 +02:00

1417 lines
100 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# aura design ledger — INDEX
The ledger records the load-bearing design contracts and their rationale. Each
contract states what it **guarantees**, what it **forbids**, and **why**. A
change that breaks a contract is a design decision (amend the contract here with
its new rationale), never a silent refactor.
Provenance: contracts C1C18 were settled in the initial rough-sketch design
interview (2026-06-03), walking the design tree root-to-leaf (C16C18 and the
C10 refinement to a broker-independent position table came in follow-up turns;
C10 was reframed in cycle 0007 to an exposure stream, then again 2026-06-23
(#117) to a **bias** stream with signal quality in **R**, the position table a
derived layer — see C10). C19C22 were added as the
construction / World / playground layer; C23 and the C9/C19 compilation
refinements were settled 2026-06-05 for the **Construction-layer** milestone (the
blueprint→flat-graph reading of composites and graph optimisation).
The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded
summary of the
subset that agents must never violate; this file is the fuller form with
rationale.
Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
*milestone* is a tracker container spanning many cycles (the first milestone is
the **walking skeleton**: ingest → one signal → exposure → sim-optimal broker →
synthetic pip-equity signal-quality metric — the *pre-reframe* substrate, now
reframed to bias → R-evaluator → R-expectancy, see C10).
---
## Foundation — what aura is
aura is a framework **and** a playground for traders. A human and (primarily)
LLMs author trading **nodes** directly in Rust; the engine backtests them
deterministically and massively in parallel, composes them fractally, validates
them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into
a standalone bot with a broker connection.
The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow,
too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura
inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts*
(synchronous reactive streams, bounded-lookback series, run-counting, SoA).
RustAst is a conceptual reference, not a dependency. The one reused component is
`data-server` (the first data source).
What sets aura apart is **not** the single backtest — every quant system does
that — but the **World**: the meta-level where harnesses are dynamically
constructed and *families* of them orchestrated and explored (walk-forward,
sweeps, Monte-Carlo, comparison). The deterministic single-harness engine is the
*substrate*; the World is the *product* (C20C22).
---
## External components
Two sibling projects live outside this repo and are named throughout the
contracts. Their concrete location is recorded here so the repo is the single
source of truth — not session memory.
- **`data-server`** — `~/dev/libs/data-server`, Gitea `Brummel/data-server`
(`http://192.168.178.103:3000/Brummel/data-server.git`). aura's **first data
source** and the one reused component (Foundation; C3, C11, C12). A standalone
leaf crate (deps: `chrono`, `regex`, `zip`) that loads Pepperstone **M1/tick**
binary files and shares them lock-free as `Arc<[T]>` chunks
(`CHUNK_SIZE = 1024`) via `SymbolChunkIter::next_chunk` (an inherent method
driven by `while let`, not the `Iterator` trait); `stream_*_windowed(from_ms,
to_ms)` provides C12's data-window with inclusive Unix-ms bounds. Records are
**AoS** (`M1Parsed`/`TickParsed`, `time_ms: i64` Unix-ms). The ingestion
boundary (C3) therefore **transposes** these AoS records into aura's SoA
columns (C7) and normalizes `time_ms` → canonical epoch-ns at that one
boundary. Pulled in as a cargo git dependency by the **`aura-ingest`** crate
(cycle 0011) — the **data-source ingestion edge**, where the `data-server`
external tree (and its transitive `chrono`/`regex`/`zip`) enters the workspace.
Under the amended C16 **per-case dependency policy** (cycle 0029), `aura-ingest`
is no longer a zero-dep *firewall* for the rest: the engine crates link
standard vetted crates where they earn their place (e.g. `serde`/`serde_json`
for the run registry, cycle 0029), with particular scrutiny for the frozen
deploy artifact (C13). Consequence: `cargo build/test --workspace` resolves
from a populated cargo cache (a Gitea fetch for `data-server` + crates.io for
the standard crates) rather than fully offline.
- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst`
(`http://192.168.178.103:3000/Brummel/RustAst.git`). The predecessor DSL attempt
(Foundation): **a conceptual reference, never a dependency** — its DSL authoring
surface and `Value`/HM-inference machinery are exactly what aura rejects (C17).
But its **`src/ast/rtl/` layer is a working reference implementation of the very
streaming substrate aura rebuilds**, and is worth reading before authoring
`aura-core` — these are not just concepts, they exist as code:
- `rtl/series/data.rs``RingBuffer<T>` with financial-style indexing (index 0
= newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's
pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no
String/Record" = C7's closed scalar set); `ScalarSeries<f64|i64|bool>` and the
**SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns").
- `rtl/series/mod.rs``create_typed_series`, dispatching element type → storage
backend (the type-specialized **factory** of C19).
- `rtl/streams/mod.rs``Signal { cycle_id, value }` (C4's cycle clock) and the
`Stream` / `Observer` / `ObservableStream` push traits (the reactive model of
C4/C5).
- `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one
explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator
(C12's seed-as-input).
aura reimplements these natively and **sharpens** them: types are monomorphized
and edges type-erased to the four scalar kinds with direct dispatch (C7) instead
of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>`
(C12) instead of a `VecDeque<Value>`. RustAst shows the *shape*; aura makes it
fast and deterministic.
---
## Contracts
### C1 — Determinism and disjoint parallelism
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
loop that reaches a unique state after each input tick. Same input (incl. seed)
→ bit-identical run. Two backtests are fully disjoint and run concurrently
without locking.
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
is not captured as an explicit input (see C11, C12).
**Why.** Real money rides on backtest results; reproducibility and an audit
trail are non-negotiable. Speed comes from parallelism *across* sims, which
disjointness makes lock-free.
### C2 — Causality / no look-ahead
**Guarantee.** A node sees only the past. Input history is a read-only window
that ends at the current cursor; a resampler emits a bar only once it is
complete.
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
/ still-forming bar.
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
leaks the future is worse than none. Making the future *physically absent* from
what a node receives beats merely discouraging it.
### C3 — One merge, at ingestion only
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp
into one chronological cycle stream at the ingestion boundary; source-native
time units (e.g. data-server's Unix-`time_ms`) are normalized there to the
canonical epoch-ns `timestamp` of C7.
**Forbids.** Any merge / as-of join *inside* the graph.
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
future. Keeping the merge at one boundary keeps the graph semantics simple.
### C4 — Cycle granularity
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
multiple sources) break by source declaration order.
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
origin of records, not the cycle semantics. Tie determinism preserves C1.
### C5 — Freshness-gated recompute and sample-and-hold
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
run-count); otherwise it holds its last output. Stale inputs contribute their
last (held) value.
**Forbids.** Recomputing every node every cycle ("push all" is true for the
*clock*, not for *recompute"); treating a held value as missing.
**Why.** Total recompute does not scale to many sparse high-frequency sources;
freshness-gating is the performance discipline that keeps the synchronous model
fast.
### C6 — Firing policy A and B, per input group
**Guarantee.** A node declares, per input group, one of two firing policies:
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
four separate 15m sources: the candle is complete only when all four are fresh).
A single node may mix an A input and a B group.
**Forbids.** A single global firing mode; forcing per-node-only granularity.
**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group
granularity is required by the OHLC-plus-bias case where one node needs both.
**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any,
Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a
mode-A input is its own trivial group, so "per input group" and the per-input tag
coincide. The barrier's synchronization token is the cycle **timestamp**, not the
`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so
RustAst's `cycle_id`-equality barrier could never fire across them. A group fires
when every member's last push carries the current cycle's timestamp, guarded by
"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This
fires both the multi-source bar and the within-source diamond rejoin (every push
in a cycle carries that cycle's timestamp).
### C7 — Four scalar base types, streamed as SoA
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
(OHLCV) are bundles of base columns — this is the **node-output model** too: a
node emits a record of 1..K base columns (C8), each forwarded field-wise to a
consumer slot; the bundle is structural, never a fifth scalar type. Edges are
type-erased to these four kinds;
the type check is paid once at wiring/sim-start, then the topology is frozen per
sim → direct dispatch, no per-event allocation.
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
allocation; topology mutation mid-sim.
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
set and SoA. The open set is composites (schemas of columns), not scalar types.
Type-erasure at the edge is also forced by the cdylib boundary (C13).
**Realization (`Cell` carrier split, 2026-06).** The streamed value is now split
into a tag-free 64-bit word and its kind. `Cell` (`crates/aura-core/src/cell.rs`)
is a type-erased `u64`: constructed per base type (`from_i64/f64/bool/ts`) and
read only by naming the type at the call site (`i64()/f64()/bool()/ts()`
branch-free bit-casts). The kind is therefore resolved once at the boundary and
the value itself carries no tag — C7's "the type lives at the column/edge, not in
the value" made explicit on the single-value carrier (and a single 8-byte word
vs. the 16-byte tagged enum). `Scalar` becomes `{ kind: ScalarKind, cell: Cell }`
— the self-describing form for the *dynamic* boundaries (builder binding, serde,
rendering) — with `debug_assert`-guarded native accessors (caller asserts the
kind; free in release) and a hand-written **value** `PartialEq` that preserves
the former enum's IEEE-754 semantics (`NaN != NaN`, `+0.0 == -0.0`), pinned by
the `scalar_eq_is_value_not_bitwise` fixture. Behaviour-preserving.
**Realization (`Cell` becomes the hot-path carrier, 2026-06, #74).** The carrier
swap deferred above has landed: `Node::eval` now returns `Option<&[Cell]>` and
every node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free
8-byte words. The kind lives only at the schema/column: the harness forwards each
field via the new branch-free `AnyColumn::push_cell` (infallible — the edge
kind match is verified once at bootstrap, the surviving guard
`bootstrap_rejects_*_kind_mismatch`). `Scalar` remains on the self-describing
dynamic boundaries: the param plane (`build`/`bind`/`compile_with_params`/sweep
points/`RunManifest`), `AnyColumn::get` (the type-erased read for sinks/serde),
and source ingestion (`Source::next`, the heterogeneous C3 merge). The removed
per-value runtime kind check on node output is the same authoring-bug class C8
already leaves to a `debug_assert` (output width); node-output-kind correctness
is the node's declared `FieldSpec` contract, caught by each node's own test.
Behaviour-preserving (C1).
### C8 — The node contract
**Guarantee.** A node has a **signature** — its `NodeSchema`: each input's scalar
type and firing group, its output record, **and the node's own tunable parameters —
typed, with ranges**, which aggregate into the blueprint's param-space the optimizer
sweeps (C12/C19/C20). The signature is **declared pre-build on the value-empty
recipe** (C19), so the whole interface is legible without building (cycle 0024).
The built node implements `lookbacks()` — the per-input buffer depth, the one
quantity that may depend on an injected param (e.g. an `Sma`'s window = its
`length`), read once at bootstrap to size the windows — and `eval(ctx) ->
Option<&[Cell]>`. The
engine provides read-only, zero-copy windows into each input's SoA ring buffer
(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own
mutable series for derived/intermediate state. `None`/Void return = filter /
not-yet-warmed-up. A node is a **producer, a consumer, or both**: a
producer/transformer exposes **one output port**, whose payload is a **record of
1..K base-scalar columns** (a scalar is the degenerate K=1 record; an `eval`
returns a borrowed row, one value per column); a **pure consumer (sink)** — chart,
equity, logger — has **no** output. Sources are pure producers; sinks are pure
consumers.
**Forbids.** A node sizing/growing its input lookback at runtime; more than one
output **port** per node; a fifth scalar type or a heterogeneous output payload
(a record is a bundle of base columns, C7); copy-on-read of input history.
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage
lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized
buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).
**Realization (cycle 0005).** `NodeSchema.output` is a `Vec<FieldSpec>` (named base
columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field`
selects one producer column per edge; consuming a whole record is N edges (no
"bind whole record" mechanism). The K fields of one record are **co-fresh by
construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns
`Option<&[Cell]>` — a borrowed row into a node-owned buffer — so the forward
path allocates nothing per cycle (C7) (the carrier is now a tag-free `Cell` — see
the C7 carrier note).
**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is
now realized at the substrate: **recording is a node role, not a type.** A
recording node reads its typed input windows + `ctx.now()` in `eval` and pushes
the record to a destination it holds as a field (a channel, a chart handle) — an
**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a
node that only records returns `None` (pure consumer), and a node may record
**and** return an output the engine forwards in the same `eval` (the "both"
case). **Encoding & return contract.** A pure consumer declares `output: vec![]`
— the empty record *is* the sink declaration; there is no separate type, trait,
or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run
loop debug-asserts the returned row's width equals the declared output width
(`row.len() == schema.output.len()`). Field-wise wiring resolves
`Edge::from_field` against the producer's `output` at bootstrap, so no edge can
bind a field of a zero-output node — it fails with `BadIndex` — making a sink
structurally unwireable as an in-graph producer; its only output is the
out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out
of the graph is the node's own side effect — and that boundary is the
determinism / graph-as-data boundary (C1/C7).
**Refinement (cycle 0070 — end-of-stream `finalize` lifecycle, 2026-06-25).** The
node contract gains a second lifecycle phase beside `eval`: `finalize(&mut self)`,
a **default-no-op** trait method the engine calls **once per node, in topological
order, after the source loop drains** (the `Harness::run` epilogue). It lets a sink
**fold online** — accumulate per `eval` into O(trades)/O(1) owned state and flush
one compact summary at stream end — instead of pushing a record every fired cycle
into an unbounded channel that buffers O(cycles) rows until the run ends (the
retention BLOCKER #138 hit: a full-history Stage-1 sweep held ~2 GiB/member over
~5.5M one-minute bars). C1/C7/C8 are preserved: `finalize` runs once *after* the
deterministic event loop, adding no within-sim concurrency (C1); a folding sink
holds only owned accumulator state, no interior mutability (C7); it still declares
`output: vec![]` and its flush is the same out-of-graph side effect (C8) — one
summary row, not a per-cycle stream. Realized by `GatedRecorder` (emits only gated
rows + the genuine final row) and `SeriesReducer` (folds one column, emits one
summary row), aura-std siblings of the per-cycle `Recorder`, which survives for the
live / `--trace` / test-tap path. The recording sink's *own* per-cycle allocation
(the `Recorder``Probe` rename + its accumulate-vs-stream choice, #77) stays open;
`finalize` is the "non-channel exit from the graph" that issue's
accumulate-then-read option named as missing.
**Refinement (Construction-layer milestone — render labels, 2026-06-05).** A node
additionally exposes `label() -> String`, a **single-line, non-load-bearing**
render symbol: a default trait method the run loop never calls (wiring is by
index, C23). Overrides carry the node's identifying params (`SMA(2)` vs `SMA(4)`)
so a graph render (C9 graph-as-data, #13) disambiguates identical node types and
surfaces a mis-wiring. Like `FieldSpec.name`, it is an informative debug symbol,
not part of the C8 dataflow contract — adding it changes no run behaviour.
**Realization (cycle 0015 — param declaration).** The tunable-parameter half of
this contract is now realized: a node declares its knobs in `schema()` as
`params: Vec<ParamSpec>` (`ParamSpec { name: String, kind: ScalarKind }`), and
`Composite::param_space()` (cycle 0024; was `Blueprint::param_space()`) aggregates
them into one flat, path-qualified list — a
read-only projection of the graph-as-data (C9), mirroring the inline order
(C19/C23) so a param's slot matches the later flat-node order. Two refinements to
the guarantee's "typed, with ranges": (1) the declaration carries `name` + `kind`
only — the **search-range is the run's, not the node's** (which subset / grid a
sweep covers is an experiment axis, #32/C20; the node declares the knob's existence
and type, never its search interval). (2) Identity is **positional** (the slot,
C23 "by index, not name"); the path-qualified `name` is a non-load-bearing debug
symbol (like `FieldSpec.name`) — uniqueness is at the slot. **Refinement (cycle
0032):** identity in the **flat graph** stays positional (C23, unchanged), but the
`param_space()` **name projection** — the authoring / by-name address space (the
surface a sweep axis or single-run binding addresses) — must be **injective** for a
blueprint to compile (a duplicated path is unaddressable; see C9). Two layers:
positional wiring below (the flat graph), an injective name address space above (the
authoring boundary). So same-type siblings in one composite no longer silently
share a name — they must be `.named(...)` apart, which is also what disambiguates a
same-type fan-in (C9). A vector knob (`LinComb.weights`) expands to `N` flat
`weights[i]` entries, `N` topology-fixed (C19). Permitted kinds are `i64`/`f64`/
`bool` (a `timestamp` knob is a structural axis, C20, never a numeric sweep param).
Binding a value to a slot landed in cycle 0016 (#31, see C19/C23); enumerating a
sweep (#32) is the deferred next layer. The 0016 binding also moved the param
declaration's *authoring* home into the value-empty recipe, whose `params()` is read
pre-build. In 00160023 the built node's `schema().params` still reported the same
slots, kept in lockstep by a per-node test (a duplication filed as debt, #36); **cycle
0024 dissolved this** — the signature is declared *once* on the recipe and the built
node no longer carries `schema()` (#36 closed). See the C8 cycle-0024 realization.
**Realization (cycle 0024 — the signature lives in the blueprint, #43/#36).** The
node's whole declared interface — its `NodeSchema` (input scalar types + firing,
output record, params) — now lives **once**, on the value-empty recipe
`PrimitiveBuilder` (ex-`LeafFactory`), read pre-build by `param_space()`, the
compile-time structural validation, and the render. This closes two debts: **#43**
(a value-empty recipe used to declare *no* input/output interface pre-build — only
params) and **#36** (params declared twice, recipe vs built node, kept in lockstep
by a per-node test — those 8 tests are deleted, the duplication structurally gone).
The split that makes this work: a node's signature is **fully static** per blueprint
(input kinds/firing, output fields, params — verified across the roster; a variable-
arity node like `LinComb` takes its arity as a *recipe argument*, not an injected
param), so it can be declared without building; the **one** param-dependent quantity,
an input's buffer **lookback** depth, is no longer in the signature but answered by
`Node::lookbacks() -> Vec<usize>`, read only by bootstrap to size the windows.
`Node::schema()` is therefore **removed** — the signature is pre-build data, not a
built-node method. `BlueprintNode::signature()` answers uniformly for both arms (a
primitive returns its recipe's schema; a composite *derives* it from the interior:
role kinds in, re-exported field kinds out, aggregated params), so "every node has a
signature in the blueprint" holds for composites too.
**Realization (cycle 0027 — name input ports, refs #21/#51).** `PortSpec` now
carries a `name: String` (it drops `Copy`, like `ParamSpec`), so an input port is
named just as `FieldSpec.name` (output) and `ParamSpec.name` (param) already are.
The name is **non-load-bearing** (C23): wiring stays positional by slot, bootstrap
and the run loop never read it; it exists for tracing / graph rendering (#13). Leaf
primitives declare their slot names (`SimBroker`'s `exposure`/`price`, etc.);
`derive_signature` carries a composite's `Role.name` into the derived input port,
so the graph model (`model_to_json`) is homogeneously named across inputs, outputs,
and params and across both graph levels. This does **not** close #21 (a swapped
same-kind wiring is still only kind-checked) — it makes those slots
self-documenting; a name-consuming validation is its own future cycle.
**Realization (cycle 0040 — wiring totality: every input slot connected exactly
once, #65).** The node contract's "the engine provides a window into each input"
presupposes each input is actually fed; until now an unwired interior input slot was
accepted and bootstrapped to a silent empty column (`harness.rs`), and two producers
into one slot — ill-formed, since a slot holds one column — was likewise
uncompiled-against. Cycle 0040 makes a valid graph **total and single-valued** over
its interior input slots: `check_ports_connected` (in `validate_wiring`, run at every
nesting level) requires every interior node's every declared input slot to be covered
by **exactly one** wiring act — one interior `Edge { to, slot }` or one role
`Target { node, slot }`, edges and role targets counted uniformly. Zero coverage is
`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. A
composite's **own** input roles (`source: None`) are coverage *providers* — the
wired-by-enclosing boundary, the root case already guarded by `UnboundRootRole`
never consumers, so only interior input slots are subject to the rule. There is **no
optional-input concept**: every declared input port is required (no shipped node runs
meaningfully without one; an unwarmed mode-A input is *wired-but-not-yet-valued*, not
unwired). The check is **index-based and name-free** — it touches no name machinery
and emits nothing into the flat graph, so C23 is untouched (it proves the existing
raw-index wiring is total and single-valued). Inherited identically by the raw
`Composite::new` path and the `GraphBuilder::build()` path (both compile via
`compile_with_params`).
### C9 — Fractal, acyclic composition
**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes
one output; signal, combined signal, and (with execution) strategy are all the
same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only
feedback path is an explicit delay/state node (the RTL "register"). Wiring is
written in Rust (builder API); the built graph is introspectable runtime data.
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
"signal-of-signals" as separate mechanics.
**Why.** Self-application of one contract gives unlimited composition with no
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined;
forcing feedback through a visible delay node keeps the per-cycle determinism
intact and the one legitimate feedback path explicit. Graph-as-data enables
visualization, freezing, and re-parameterization for sweeps.
**Refinement (Construction-layer milestone, 2026-06-05).** "A composite is itself
a `Node`" is an **authoring-level** identity: a composite declares the same
interface (typed inputs + ≤1 output, C8) and is wireable wherever a node is, but it
is **not** a runtime object. The bootstrap **compiles it away by inlining** its
sub-graph into the one flat instance (C19/C23): the composite *boundary* dissolves
into the raw index wiring the run loop already consumes, and names stay
**non-load-bearing** (informative debug symbols only, as `FieldSpec.name` already
is). So "nestable arbitrarily" and "graph-as-data" hold at the **blueprint
(source)** level; the running graph is the flat, index-wired **`FlatGraph`**. The earlier reading — *a composite survives as a `Box<dyn Node>`
driving a nested sub-engine* — is **explicitly rejected**: it would keep the
interior opaque to the cross-graph optimiser (C23) and add a runtime sub-loop the
flat model does not need. Inlining is what makes the composite boundary free.
**Realization (cycle 0018 — composite multi-output record, #40).** "Exposes one
output" is one output **port** carrying a **record of 1..K re-exported fields**, not
one field: `Composite.output` is a `Vec<OutField { node, field, name }>` (was a
single `OutPort`). Each entry is a **named projection** of one interior
`(node, output-field)`; a consumer selects which re-exported field it reads via the
same `Edge::from_field` that already binds leaf record columns (C8 realization,
cycle 0005). This is the **same arity** C8 already grants a leaf (OHLCV = one port,
5 columns): a multi-line indicator (MACD = macd/signal/histogram) is **one** record
of K fields, **one** port, **one** row per `eval` — C8/C7/C4 untouched, a boundary
completion, not a contract change. A strategy composite is simply the K=1 case
(one bias field, C10). The re-export **names** are **non-load-bearing** (C23):
they live at the blueprint boundary and in render (cycle 0022/#46 folds each onto
its producing node as a `name := …` binding; originally `[out:<name>]` markers, #13)
but are dropped at lowering — `ItemLowering::Composite.output` is `Vec<(usize,
usize)>`, raw index pairs only, so the flat graph is name-free (verified: the
compiled-view render stayed bit-identical across this change).
**Realization (cycle 0019 — name the composite boundary, #41; param-overlay retired,
cycle 0031).** The named-projection shape covers the surviving boundary edge-kinds:
`input_roles` is a `Vec<Role { name, targets }>` (was a bare `Vec<Vec<Target>>`),
alongside `output: Vec<OutField>`. Each is an ordered, positionally-indexed
**named projection** of interior handles. **Param projection is no longer a
composite overlay** (the index-addressed `ParamAlias` was retired in cycle 0031):
a node's surface param name flows from its own **instance name** — every node
carries a name (default = its lowercased type label, override via `.named()`), and
`param_space()` is uniformly `<node>.<param>` at every level including the root. A
same-type fan-in is distinguished by naming the colliding legs, the same single act
that qualifies their param paths. Like the output and role names, **node names are
non-load-bearing** (C23): they live at the blueprint boundary and in render but are
dropped at lowering — the flat graph is wired by raw index. The full composite
boundary signature (named inputs, multi-outputs) and the per-node param path are
legible without changing the flat graph.
**Refinement (param-namespace injectivity, cycle 0032; supersedes the fan-in
distinguishability check).** A blueprint compiles only if its `param_space()` name
projection is **injective** — every path-qualified knob name is unique. A duplicated
path is a knob no binding can select alone (it has no distinct by-name address,
C12/C19), so it is a `CompileError::DuplicateParamPath` carrying the offending path;
the cure is to give the colliding same-type sibling nodes distinct names with
`.named(...)`, the same single act that qualifies their param paths. The
param-bearing indistinguishable fan-in is *one instance* of a duplicated path (two
default-named same-type legs share a leaf path). `signature_of`, `leaf_has_param`,
and the fan-in-specific check (`check_fan_in_distinguishability` /
`check_composite_fan_in`) are **retired** (cycle 0032), replaced by one structural
`check_param_namespace_injective` over the `param_space()` names, run before name
resolution in `compile_with_params` and both binders — so the canonical by-name
author sees the structural cure rather than a downstream `AmbiguousKnob` symptom
(that `BindError` arm is retired too: an injective space can never multi-match).
Paramless interchangeable same-name sources stay legal (no path, no duplicate).
The old signature-collision predicate had extra breadth — it also rejected an
**asymmetric param/paramless** collision and a **role-vs-leg** collision, *neither*
a path duplicate (each colliding configuration keeps a unique param path, or none).
That breadth guarded **render identity**, dead since the renderer was retired in
0026; both extra rejections are **intentionally dropped**. A future node-/wiring-name
distinguishability check, if ever wanted (e.g. for the WASM graph view's #21
thread), is decoupled from param-space injectivity. Construction-phase only; the
flat graph stays name-free (C23).
### C10 — Strategy output is a bias stream; signal quality is measured in R; risk-based execution and realistic brokers are decoupled downstream layers
**Guarantee.** A strategy's primary, backtestable output is **not** an equity
curve, **nor a position-event table**, **nor a position size**, but a **bias
stream**: the DAG expresses exactly one *state* at time t (C8 — a node emits at
most one record per `eval`), so a strategy emits one **signed, bounded bias**
`f64 ∈ [-1, +1]` per cycle (per instrument) — the **sign is direction, the
magnitude is conviction**, and conviction is optional (a bare ±1 / 0 is the modal
case). Bias is **unsized**: position sizing and the protective stop do **not**
live in the strategy. The chain is `signals (scores) → decision node → bias
stream`.
**Risk-based execution is a decoupled downstream layer.** Turning a bias into a
position is the job of a **risk-based execution** chain — `bias → stop-rule →
Sizer → Veto → position-management` — never the strategy's. The **stop-rule** sets
a protective stop, which **defines the risk unit R** (1R = the loss taken if
stopped). The **Sizer** sizes the trade in R; the **Veto** is a distinct
pre-trade gate (position / exposure caps), kept separate from sizing. These wire
into a **RiskExecutor** composite (per symbol); RiskExecutors nest inside a
**Broker / Account** composite that aggregates per-symbol P&L into account equity.
The account **mode is a composition constraint**: a *netting* account allows ≤1
RiskExecutor per (strategy, symbol); a *hedging* account (the European CFD
default) allows many per symbol, even oppositely directed.
**Signal quality is measured in R — Stage 1.** A downstream **R-evaluator**
consumes a RiskExecutor run under **flat-1R** sizing (every trade risks exactly
1R) and integrates the per-trade R-outcomes into an **R-expectancy** / R-curve:
the account- and instrument-agnostic yardstick for *"how much R out per 1R
risked?"*. Flat-1R needs no equity, so this stage is **feed-forward** (no
feedback, no cycle) — the primary research loop, maximally parallel and
deterministic (C1). R, not pips, is the unit: pips are not risk-normalized.
**Currency P&L is Stage 2 — deploy viability.** Entered only once `E[R] > 0` is
established. Here sizing is **fixed-fractional** (size = risk-fraction · equity /
stop-distance), so it **reads account equity** and **compounds**; **realistic
broker** nodes apply real spread / commission / slippage / lot / margin and emit a
**currency** equity stream. This introduces the *only* feedback in the whole
design — equity → Sizer — cut by a **`z⁻¹` register on the fill edge** (the
realized/fill component is delayed one cycle; mark-to-market off the current price
stays a legal same-cycle read, C2), **encapsulated inside the executor
composite**. Whether that register is live is the **flat-1R vs compounding
structural axis** (instantiating a feedback edge is topology, fixed at bootstrap
(C19), hence a structural axis (C20), never a numeric sweep param): flat-1R DCEs
it away (Stage 1), compounding keeps it (Stage 2). The explicit register is
**mandatory** — C9's "only feedback is an explicit delay node" made concrete —
because the bootstrap's optimiser reorders the flat graph (C23 CSE/DCE): the
implicit read-before-fill ordering by which conventional sequential backtesters
cut this loop would not survive that reordering.
**The position-event table** — turning the executed book into a broker-independent,
time-ordered **table of position events** (pure scalar columns, C7: `event_ts:
timestamp`, `action: i64` buy / sell / close, `position_id: i64`,
`instrument_id: i64`, `volume: f64` unsigned — direction is the `action`; a
position's open time is its opening event's `event_ts`, no separate `open_ts`; a
`close` references a `position_id` and may be partial via its own `volume`) — is a
**decoupled, derived, downstream** Stage-2 layer. The events are the **first
difference of the book** under book-tracking (`deal = target book in_flight`;
a close sizes the actual book, never an exposure delta), materialized as a
*computed table* (multiple events may share one `event_ts` — e.g. a reversal's
close + open), **never as a per-`eval` node output** (C8 caps one record per
cycle). This table feeds the realistic broker nodes; one table feeds many, giving
directly comparable currency curves. Live: a realistic broker node consumes the
events in real time and routes orders as a side effect; reconciliation with the
real account is an external adapter.
So the layers attach at distinct points: the **R-evaluator** on a flat-1R
RiskExecutor (signal quality, R, Stage 1); **realistic brokers** on the derived
position-event table (execution viability, currency, Stage 2). All are ordinary
downstream nodes (C8/C9); several can attach at once for directly comparable
curves.
**Forbids.** Putting **sizing or the stop in the strategy** (bias is unsized; the
Sizer and stop-rule own them); treating an equity curve as the strategy's output;
making the **position-event table the strategy's direct DAG output** (it is
derived, not emitted per `eval` — a decision instant may need >1 event, which C8
forbids) or the measure of signal quality; **measuring signal quality in currency
before R is established** (Stage 2 before Stage 1); a **combinational equity → size
cycle** (the fill-edge register is mandatory — an implicit execution ordering does
not survive C23 reordering); **fusing the Sizer and the Veto** into one node (the
size / veto / fill roles stay distinct); baking a broker into the strategy; a
special external broker subsystem (a broker is an ordinary node); storing
`open_ts` (derive it from the opening event); a signed-volume direction trick in
the event table (use `action`); broker-specific assumptions leaking into the
strategy logic.
**Why.** A strategy's edge is a *procedure*, not a currency outcome ("focus on the
procedure, not the money"): the right primary question is *"how much R out per 1R
risked?"*, and R — defined by the stop — is the only account- and
instrument-agnostic, risk-normalized unit (pips are not). Separating **direction
(bias) from sizing (Sizer) from fill (broker)** is the decomposition every mature
system converges on (LEAN's Alpha → Portfolio-Construction → Execution is a near
isomorphism; backtrader, QSTrader, zipline all emit an unsized directional signal
sized downstream). The **Stage-1 / Stage-2 cleave** quarantines the *only*
feedback (equity → sizing) into Stage 2, leaving the signal-quality layer
feed-forward, parallel and deterministic (C1); its closest external precedent is
vectorbt's split of vectorized signal generation from sequential money-management
— it is aura's own synthesis, not an industry standard. The DAG holds exactly one
state at t and a node emits ≤1 record per `eval` (C8), so the faithful per-cycle
output is the **bias** (one value); position *events* are its book's first
difference, a derived consequence — decoupling them resolves the C8↔C10 impedance.
Modelling brokers and executors as nodes (not a bespoke subsystem) keeps them
within the one Node/graph abstraction (C9). This supersedes the **exposure**
framing (the strategy's output was a signed fractional *position*) of the
cycle-0007 reframe and its realization notes below; that in turn superseded the
still-earlier "the strategy's output is the position-event table" framing.
**Reframe (2026-06-23, #117 — exposure → bias, R as the signal-quality unit).**
The contract above is the R-reframe; the realization notes below describe the
**pre-reframe code** (`Exposure`, `SimBroker`, pip-equity) and are retained as
history and as the **Stage-1 ancestor**: `Exposure { scale }` is the ancestor of
the unsized `bias` node, and `SimBroker`'s pip integral is the ancestor of the
R-evaluator (which additionally requires a stop, since R is stop-defined, so it
consumes a flat-1R RiskExecutor rather than raw bias). The `exposure → bias`
rename, the RiskExecutor / Sizer / Veto nodes, and the R-evaluator **landed in
cycle 0065** (Stage-1; see the realization note below); the position-event schema (0063, #114) survives
unchanged as the Stage-2 audit layer, though its derivation is now the **book**
first-difference (`deal = target book in_flight`), not the flawed 0064
exposure-integral derive (abandoned). Industry grounding for this reframe: LEAN /
nautilus_trader / backtrader / QSTrader / vectorbt / zipline (see #117 decision
log).
**Realization (cycle 0007).** The signal-quality half of this contract is now
realized at the substrate, as two `aura-std` nodes composed on the unchanged
engine (the engine stays domain-free — it routes only `f64` records, never
"exposure" or "equity"). The **exposure stream** is realized as `Exposure { scale
}`: the decision/sizing node, `clamp(signal / scale, -1, +1)`, one `f64` per fired
cycle (`None` until warmed up). The **sim-optimal broker** is realized as
`SimBroker { pip_size }`: a two-input node (exposure, price) that accumulates
`prev_exposure · (price prev_price) / pip_size` and emits cumulative pip equity
— the exposure held *into* a cycle (decided at t-1) earns that cycle's return, so
the integration is causal (C2), and `pip_size` is held reference metadata
(C7/C15), never streamed. An end-to-end harness (SMA-cross → `Exposure`
`SimBroker` → recording sink) produces a recorded pip-equity curve, bit-identical
across runs (C1). The **position-management half** — deriving the position-event
table (buy/sell/close, `position_id`, partial closes) from the exposure history,
and the realistic broker nodes that consume it — is deliberately **not** built
this cycle; it remains the decoupled, derived, deferred layer described above.
**Realization (per-instrument pip channel, 2026-06, #22).** `SimBroker`'s
`pip_size` is now sourced **per instrument**, not from one global literal. The
divisor is resolved from the recorded geometry sidecar (`instrument_geometry`, over
data-server's `symbol_meta`), at the ingestion/source edge where the symbol still
exists (never `Aura.toml`, never `Ctx`); the engine stays domain-free (no instrument
identity reaches the hot path). (The original cycle-0022 form was a Rust-authored
vetted floor `InstrumentSpec { pip_size }` + `instrument_spec(symbol)`; cycle 0074
removed that floor — see the C15 note below — once the sidecar geometry made it
redundant for the real path. Refuse-don't-guess on absent geometry.)
**Realization (position-event schema, cycle 0063, #114).** The position-management
half's **schema** now landed (its derivation and the brokers remain deferred): a
closed `PositionAction { Buy, Sell, Close }` enum + the `PositionEvent` row
(`event_ts`, `action`, `position_id`, `instrument_id`, unsigned `volume`; no
`open_ts`; direction *is* the action) live in `aura-engine` beside `RunMetrics` as a
post-run value type (not a per-`eval` node — C8). `action` serde-encodes as a bare
`i64` (Buy=0, Sell=1, Close=2), the C7 scalar column form the ledger's table spec
requires, with an out-of-range code rejected on read. Still deferred:
`derive_position_events` (the first-difference reduction over the exposure history,
#115) and the realistic broker nodes that consume the table (#116). The table stays
broker-independent.
The honesty rule is **refuse, don't guess**: a real-data run for a symbol with no
vetted spec is a usage error (`exit 2`), so cross-asset pip equity is comparable by
construction rather than by researcher discipline. Threaded through the CLI
`aura run --real` path; the manifest broker label records the looked-up pip. The
runnable GER40 examples (already at the correct `1.0`) are routed through the same
lookup separately (#98).
**Realization (cycle 0065 — Stage-1 R signal quality, #119/#126/#127/#128/#129).**
The Stage-1 chain above is now built. `Exposure → Bias` renames the unsized strategy
output (the node + its output field); the persisted `exposure_sign_flips` metric key
(kept as a serde alias) and the `SimBroker` `exposure` input slot + the on-disk
`exposure` **tap** label (a deliberate permanent keep, #117) retain the old name. The
strategy-output **param namespace** — the `Bias` instance, its `bias.scale` knob, and
the `bias_scale` manifest param — was the deferred rename tail, **completed in #134**
(the knob path is a runtime address, so no on-disk back-compat break). A **stop-rule**
defines 1R: `FixedStop` (a triggered-constant primitive) and a `vol_stop(length, k)`
**composition** `k·√EMA(Δ²)` (the originally-fused node was corrected to a composition
of new `Mul`/`Sqrt` primitives — a node is a primitive only if not DAG-expressible
from others). **`PositionManagement`** (`aura-std`) is the stateful heart: it latches
the entry-cycle stop distance as the immutable R-denominator, marks against the
one-cycle-lagged fill (no look-ahead, C2), and emits a **dense 14-column per-cycle
R-record** (one row per eval, C8; the trade ledger is the `closed_this_cycle` subset,
the R-equity is `cum_realized_r + unrealized_r`, the window-end open trade is the last
`open=true` row). The **`Sizer`** (`size = risk_budget / stop_distance`, flat-1R) is
the feed-forward sizing seam, and **R is size-invariant** — scaling `risk_budget`
leaves every `realized_r` unchanged (pinned by a RED test). **`summarize_r`** is a
post-run fold (sibling of `summarize`, **not** an in-graph node) → `RMetrics` (E[R],
SQN, win-rate, profit-factor, max-R-drawdown, conviction terciles, net-of-cost). The
**RiskExecutor** ships as a public `aura-engine` composite-builder
(`risk_executor(StopRule, risk_budget)` — bias+price roles embedding
`stop-rule → Sizer → PositionManagement`, beside `vol_stop`) with a `StopRule{Fixed,Vol}`
**structural axis** (C11); the **Veto** stays a documented seam, not a runtime node (a
pass-through identity is exactly what C19/C23 DCE deletes). The layer is **operable
from the CLI**: `aura run --harness <sma|macd|stage1-r>` — a **compile-time** selector
over Rust-authored harnesses (C9/C17: the CLI *runs*, it does not wire) — folds
`summarize_r` into `RunMetrics.r` (additive; `skip_serializing_if` keeps pip-only and
legacy `runs.jsonl` JSON byte-unchanged), the `stage1-r` harness fanning one bias into
both `SimBroker` (pip) and the RiskExecutor (R) for an honest dual yardstick; an
`r_equity` tap charts the by-trade R-equity through the existing `aura chart --tap`.
The R-record redundancy `debug_assert` uses a **scale-robust relative tolerance** (an
absolute `1e-9` panicked on tiny-pip FX where the `entry stop` denominator
reconstruction loses precision; the stored `realized_r` is exact). Composites live in
`aura-engine` (the engine's convenience layer over the standard nodes, sibling of
`summarize_r`), making `aura-engine → aura-std` a normal dependency — the graph stays
acyclic. **Stage 2** (currency P&L, fixed-fractional compounding through the z⁻¹
fill-edge register, realistic brokers consuming the position-event table) remains the
deferred downstream layer, entered only after `E[R] > 0`.
**Realization (cycle 0066).** SQN is now the operational single-number objective for
ranking a Stage-1 sweep family by signal quality — C12 **axis-2 (argmax-metric)** over
the C18 family store. `metric_cmp` (`aura-registry`) learns the higher-is-better R
metrics `sqn`, `expectancy_r`, `net_expectancy_r`; a member with no `r` block sorts
last (`NEG_INFINITY`), so a pip-only family ranked by an R metric degrades to ordinal
order rather than erroring. `aura sweep --strategy stage1-r` produces the rankable R
family, each member folding `summarize_r` into `RunMetrics.r`. The default grid varies
**only the signal** (`fast`/`slow` SMA lengths), holding the stop and sizing fixed:
`risk_budget` is R-invariant and `bias.scale` is sign-only under flat-1R (both
degenerate axes for the ranked metric), and — load-bearing — the **stop defines 1R**,
so varying it would change what R *means* per member and break cross-member SQN
comparability (the motivation for the deferred n-normalized SQN100, #130). Each swept
member's manifest records the fixed R-defining params (stop, scale) beside the floated
knobs, so a member is reproducible from its own manifest (C18) and the constant-stop
basis of comparability is auditable from the family record. Per-member `--trace` for
the stage1-r sweep is refused explicitly (not a silent no-op) pending #135.
**Realization (cycle 0067, #130 + #135).** The two follow-ups the 0066 note flagged
are now shipped. **#130 (SQN100):** `RMetrics` gains `sqn_normalized =
(mean_R/stdev_R)·√(min(n, 100))` — the n-normalized "SQN score" (van Tharp's cap,
`SQN_CAP = 100`), turnover-robust where the raw `sqn` rewards sheer trade count. It is
an **opt-in** rank key (`Metric::SqnNormalized`); the raw `sqn` and the default ranker
stay byte-unchanged, and the field carries `#[serde(default)]` (C18 — a pre-0067 `r:`
block reads back with `sqn_normalized = 0`). Below the cap (`n ≤ 100`) it equals the raw
`sqn` exactly. **#135 (stage1-r `--trace`):** `stage1_r_sweep_family` now persists each
member's equity/exposure/r_equity under `runs/traces/<n>/<member_key>/` via the same
`persist_traces_r` the single run uses (mirroring `momentum_sweep_family`); the 0066
refusal guard is gone, so per-member `--trace` is symmetric across all three sweep
strategies and a swept member charts (`chart <n>/<member> --tap r_equity`).
**Realization (cycle 0068, #115 — position-event derive).** The Stage-2 audit layer's
*derivation* now landed (the schema was 0063, #114; the realistic brokers consuming it
remain #116). `derive_position_events(record, instrument_id) -> Vec<PositionEvent>`
(`aura-engine`, beside `summarize_r`) is the **first difference of the executed book**:
a pure post-run reduction over the `PositionManagement` dense record (read positionally
as type-erased `Scalar`s, C7 SoA — no in-graph node, so the hot path stays domain-free,
C14), emitting a `Buy`/`Sell` at each open and a `Close` at each exit, a reversal (or a
stop-then-same-cycle reopen) emitting **Close then the opposite open at one `event_ts`**
(close first — the C8 ">1 event per instant" case that forces this to be a *derived*
table, not a per-`eval` output). The close sizes the **actual book** (the closed
position's stored volume), never an exposure delta — the post-reframe replacement for the
rolled-back 0064 exposure-integral derive (#117). `instrument_id` is a caller-supplied
scalar (`aura-engine` depends only on `aura-core`, so it never imports an instrument
spec from `aura-ingest`).
A position open at window end emits its open with **no synthetic `Close`** (the table
records actual executed events; `summarize_r`'s force-close is for the R metric only).
The `r_col` ⟷ PM-record lockstep is now guard-pinned for `direction` too. **Still #116:**
fixed-fractional / currency / equity-feedback sizing (the equity→Sizer z⁻¹ fill-edge
register) and the realistic brokers that consume the table.
### C11 — Generalized sources; record-then-replay determinism boundary
**Guarantee.** A source is anything that produces timestamped scalar streams —
market data (`data-server`) and non-financial sources (e.g. a news-agent node
emitting a bias) are treated identically. Anything nondeterministic, external,
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
*before* it enters the engine; backtest replays the recording, live computes
fresh in real time and records it for future backtests. A bias enters as a value
held until the next event (firing policy A).
**Forbids.** Any live external call *inside* a backtest replay.
**Why.** It is the only model compatible with reproducible backtests — LLM calls
are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`,
external LLM (IONOS) calls happen only at the recording/live-source edge, with
explicit per-session consent, never inside a sim.
### C12 — The atomic sim unit and the four orchestration axes
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
values injected at graph build (no recompile per param-set; the optimizer sees a
generic vector of typed ranges). Raw data is shared read-only across sims via
`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic
unit: (1) param-sweep (grid/random), (2) optimization (argmax metric),
(3) walk-forward (rolling in-sample optimize + out-of-sample test),
(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over
seeds**; each realization is itself deterministic given its seed.
**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the
only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait
(`peek`/`next`, object-safe) the k-way merge drives — and a streaming
**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing
**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and
decoding each `Scalar` on demand: the **source ring** is resident O(one chunk),
not O(window length) — the measured `resident_records()` predicate, a *per-source*
bound, not whole-process RSS. (Data-server's `FileCache` retains each window's
parsed chunks read-only for the pass — ~56 B/record — so process residency is
O(records-touched); that is the **replay-many** sharing C12 wants — one window
parsed once across a sweep family — not a leak. The single-pass cost is tracked as
#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the
gap closes by a streaming path *existing*, not by deleting the eager one). **Still open:**
cross-*sim* `Arc<[T]>` sharing — one window shared zero-copy across many disjoint
sweep sims — has no consumer until the orchestration families (axes 24 above:
#66/#68/#69) are built, and remains the target for those cycles.
**Realization (cycles 0028, 0049).** Axis 1 (param-sweep) is built. `GridSpace`
(0028) enumerates a cartesian lattice over discrete per-slot value-lists;
`RandomSpace` (0049) draws `N` seeded points over typed continuous `ParamRange`s
(I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), validated against the
param-space before any run. Both implement the `Space` trait the disjoint
`sweep` / `run_indexed` core is generic over, so either enumeration runs through
one execution path (C1: results in enumeration order, not completion order). The
seeded sampler reuses the bit-stable `SplitMix64` as a **code-path-disjoint**
instance from the data-edge seed RNG (the source-seam firewall, #52/#71: they
share only the `u64` type, never a path).
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
primitive — those are pluggable policies atop the atomic unit; recompiling on a
param change.
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
(embarrassingly parallel across the unit) cleanly separated from search policy.
Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the
atomic unit is one harness instance, selected by the harness's **structural
axes** (C20); the structural experiment matrix is the outer orchestration over
this dimension, the tuning sweep the inner (C19/C20).
### C13 — Hot-reload is authoring-only; deploy is frozen
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
artifact.
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
toolchain plugins.
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
and reproducible (audit trail: this bot = this commit). A sweep pays no
hot-reload tax — params are runtime data (C12), so the cdylib loads once.
### C14 — Headless core, two faces
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
**programmatic/CLI** face (the primary surface for the LLM and automation —
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
for human exploration. Visualization is only a downstream consumer node on the
streams.
**Forbids.** Any UI/pixel knowledge inside the engine.
**Why.** The LLM drives programmatically, the human visually; a headless core
serves both. The visual face is the **playground** (C22) — a **web frontend
served from disk-persisted traces** (revised 2026-06, issue #101: the engine
writes recorded traces to disk, a browser charts them; supersedes the earlier
egui-native / in-process-zero-copy-from-SoA direction). It is staged after the
runnable substrate but is core to aura's identity, not optional. The pivot keeps
this contract's core intact — and arguably tightens it: a browser reading a
serialized trace file is a stricter "no UI knowledge in the engine" than egui
reaching zero-copy into the live SoA columns.
### C15 — Resampling-as-node; sessions/calendars
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
and instrument specs are metadata (non-scalar, beside the hot path); session
*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open:
i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after
session open" is then a plain node checking `bars_since_open == 3`.
**Forbids.** Streaming the calendar; special-casing session logic outside the
stream model.
**Why.** Keeps the line consistent — everything a signal needs arrives as a
stream; reference data feeds source/session nodes from beside the hot path.
**Realization (instrument specs, 2026-06, #22 → #124).** The "instrument specs are
metadata" half of this contract is realized by the **recorded geometry sidecar**:
`instrument_geometry(server, symbol)`, over data-server's `symbol_meta`, returns
neutral broker-agnostic `InstrumentGeometry` (the raw provider JSON never enters this
repo) — non-scalar reference data held beside the hot path, keyed by symbol, feeding
the sim-optimal broker's pip divisor (C10). The real-path pip is sourced from
`InstrumentGeometry.pip_size`; refuse-don't-guess on absent geometry. **History:**
cycles 0022/0063 first carried a Rust-authored vetted floor (`InstrumentSpec` — a
single `pip_size`, later a six-field deploy-grade row `instrument_id`,
`contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`, `quote_currency`, plus
`tick_size`/`digits`), cross-checked against the sidecar geometry. **Cycle 0074
removed that floor**: with the sidecar geometry supplying the real-path pip, the
authored table was redundant, so `InstrumentSpec` / `instrument_spec` / the vetted
list were deleted. The speculative deploy-grade fields and the override/floor tiers
of the #124 hierarchy (tier 1 authored override, tier 3 authored floor) are
**deferred** until a consumer — the Stage-2 realistic broker — needs them, read from
the sidecar geometry then. The `quote_currency` runtime-type transition is likewise
deferred to that consumer; #124 collapses to **recorded sidecar geometry → refuse**
in the meantime.
### C16 — Engine / project separation; three-tier node reuse
**Guarantee.** aura is the reusable **engine**; each research project is a
separate external repo that depends on aura via cargo (the game-engine / game
split). Node reuse is cargo-native, in three tiers: **`aura-std`** (universal
blocks, ship with the engine) / **shared node crates** (cross-project-reusable,
their own repos, pulled as cargo git deps) / **project-local `nodes/`**
(experimental, project-specific). A reusable node is an `rlib` dependency; the
hot-reload unit stays the project-side `cdylib` that composes it (consistent
with C13). Concretely a project is a **Rust crate** — a cdylib library of node /
strategy / experiment blueprints — plus a static `Aura.toml` (project context:
data paths, instrument/pip metadata, default broker & window, runs dir). During
research the `aura` host loads and runs it (C13 hot-reload); for deploy the
chosen strategy + broker freeze into a standalone binary. **A project is
therefore always a Rust program built on the engine.** Beyond the node-reuse
tiers, the engine workspace also carries non-node crates — `aura-engine` (the
run loop), `aura-cli` (the `aura` binary), and **`aura-ingest`** (the
data-source ingestion edge, cycle 0011, where the `data-server` external tree
enters). **Dependency policy (amended 2026-06-10, cycle 0029).** Dependencies
are admitted by deliberate, **per-case review** — what a crate pulls in weighed
against what it buys — with **particular scrutiny for anything that enters the
frozen deploy artifact** (C13: this bot = this commit). Well-established
standard crates (`serde`, `rayon`, …) pass that review and are used wherever
they do the job, including in the bot. There is **no blanket zero-dependency
commitment** and **no blanket admission**; hand-rolling what a vetted standard
crate already does is the anti-pattern, not the dependency. This strikes the
original *"zero-external-dependency by commitment"* clause and the
*"`aura-ingest` is the sole external-dependency firewall"* framing; `aura-ingest`
remains the data-source ingestion edge, no longer a dependency wall.
**Forbids.** Project-specific signals in the aura repo (it keeps at most
example/fixture nodes under `examples/` for its own tests); a multi-project
manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the
package mechanism). (Dependency admission is governed by the per-case policy
above, not a blanket ban.)
**Why.** The engine/game split keeps the engine sharp and reusable while each
project versions its own research with its own forward-queue. Promotion
(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism.
### C17 — Authoring surface
**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses**
is authored in native Rust through **Claude Code + the skills pipeline**: the
human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI,
and reports metrics. Declarative config (`Aura.toml`) carries only **static
project context** (data paths, instrument/pip metadata, defaults, runs dir),
never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a
*runtime data source* (news-agent bias, C11), gated by per-session consent,
never in the code path.
**Forbids.** An in-app LLM chat that generates node code inside aura; using
IONOS (weaker models) as the authoring brain.
**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's
failure; making weaker models the coding brain reintroduces the very problem.
Keeps aura's scope an engine + playground, not an LLM-IDE.
### C18 — Project management: one repo = one project, plus a run registry
**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git
(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as
the forward-queue, a research thrust = a milestone, the
`idea → experimental → validated → deployed` label gradient). (2) **Experiments
& results:** an Aura-native **run registry** — one record per run = a *manifest*
(node-commit + params + data-window + seed + broker profile) + *metrics*,
queryable, with *lineage* (composite ← signals; run ← inputs). Determinism
(C1/C12) makes a run reproducible from its tiny manifest, so the registry stores
manifests + metrics and re-derives full results on demand. Depth: **structured**
(promotion/status, lineage, run-diff).
**Forbids.** Storing results not reproducible from a recorded manifest;
duplicating git/Gitea inside aura; a multi-project workspace manager.
**Why.** Comparing experiments over time is the heart of the research loop and
has no home in git/Gitea; determinism makes a structured registry cheap.
Sequencing: the walking skeleton emits a manifest + metrics per run from day
one; the registry/index is a later milestone over manifests that already exist.
**Realization (cycle 0029 — the flat run registry).** The experiments-&-results
plane shipped as `aura-registry`: an append-only JSONL store (`runs/runs.jsonl`),
one serde_json `RunReport` (`RunManifest{commit, params, window, seed, broker}` +
`RunMetrics`) per line, with a typed read-path (`load`) and best-first ranking
(`rank_by`/`optimize`). C9 holds — the registry depends on `aura-engine`, never
the reverse.
**Realization (cycle 0045 — lineage as related records, #70).** The *lineage*
depth is now realized as a **family store**: a sweep / Monte-Carlo / walk-forward
run (the C12 axes) is persisted as a *set of related records* — each a
`FamilyRunRecord` (a `RunReport` stamped with its `family` + `run` + `kind` +
`ordinal`) — in a sibling JSONL (`families.jsonl`), leaving the flat `runs.jsonl`
path and its `append`/`load`/`rank_by`/`optimize` API byte-for-byte unchanged.
the user-facing `family_id = "{family}-{run}"` handle is **derived** from the
stored `family` name plus a per-name `run` index (assigned as a numeric max+1 —
not a content hash; re-running the same family mints a fresh id). `group_families` is the round-trip that re-derives a family
from the stored links (re-listable / rankable as a unit — C21). The manifest *is*
the re-derivation recipe (#71): no input-stream blob / path / payload enters a
record, and a member's window is **producer-supplied** via
`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage),
never a materialized-`Vec` scan at the call site. CLI surface: `aura mc`,
`aura runs families`, `aura runs family <id> [rank <metric>]`; `aura sweep` /
`walkforward` / `mc` persist via `append_family` with an optional `--name`.
Deferred (Non-goals): content-addressed identity + replay-dedup; the "run-diff"
depth and ranking families against each other (cross-family, vs. within-family);
and a live producer for the flat `runs.jsonl` standalone-run path — no CLI command
writes it (sweep/walkforward persist to the family store; `aura run` does not
persist). **Resolved (#73, 2026-06): retired**`aura runs list` / `rank` dropped;
families (C21) subsume standalone over-time comparison. The `aura-registry` flat
lib API is retained: `rank_by`/`optimize` keep live consumers (`optimize` backs
walk-forward's in-sample step, `rank_by` backs `runs family … rank`); `append`/
`load` (the flat-store half) remain public API with **no in-tree caller** after
this retire — tested, available to external consumers, a latent dead-code surface
a later sweep may revisit. **Unknown-id contract (ratified, Runway fieldtest
2026-06).** `aura runs family <id>` treats an unknown-but-well-formed id as an
*empty family* (prints nothing, exit 0) — the same treat-as-empty discipline as
`Registry::load` reading a missing store as `Ok(empty)`, and deliberately
distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection
*before* any store access, not a found-nothing lookup. Tightening to a non-zero
`no such family <id>` exit (typo-safety) is an available future UX choice, not a
current contract.
### C19 — Bootstrap: blueprint → instance (recursive)
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
graph-as-data produced by running a Rust builder (C9); it carries *free* numeric
params (declared ranges) and *free* input roles. The **bootstrap** binds
`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen
instance** — buffers sized, topology fixed. This is precisely the "wiring /
graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12
("params injected at graph build") already reference. The same machinery applies
recursively up to the harness (C20). A sweep builds many instances from one
blueprint; instances are disjoint (C1).
**Forbids.** Params that change topology (a topology change is a *different*
blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a
sim against an un-bootstrapped blueprint.
**Why.** Separating the param-generic blueprint from the param-bound instance is
what makes one strategy reusable across a whole sweep and lets the optimizer
mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead
of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12
explicit.
**Refinement (Construction-layer milestone, 2026-06-05).** This binding is a
**compilation**: the param-generic, named blueprint (source) is lowered to a flat,
type-erased **flat graph** (C7) **wired by raw index, not by name**
(`Edge { from, to, slot, from_field }`): composite boundaries dissolve entirely,
and field / role names are demoted to **non-load-bearing** debug symbols (as
`FieldSpec.name` already is). "No recompile" above means no **Rust /
cdylib** rebuild (C12/C13: the cdylib loads once); re-deriving an instance per
param-set is a cheap **graph re-compilation**, not a code recompile. Naming the
build phase a compilation makes its successor explicit: the flat graph is the target
of behaviour-preserving optimisation (C23).
**Realization (cycle 0016 — param-set injection).** The bootstrap now binds an
injected param-set, realizing C12's "params injected at graph build (the optimizer
sees a generic vector of typed ranges)" and C19's "factory `params → sized node`"
*literally*: a blueprint leaf is **value-empty**`BlueprintNode::Leaf` holds a
`LeafFactory { name, params, build }` recipe, not a built node — and the value lives
only in the injected vector (no baked default), so the blueprint stays a pure
param-generic recipe. (Renamed in cycle 0024: `BlueprintNode::Primitive` holds a
`PrimitiveBuilder { name, schema, build }` — the recipe now carries the full
signature, see the C8 0024 realization; `bootstrap_with_params`/`compile_with_params`
moved onto `Composite` when `struct Blueprint` collapsed into it, see the C19 0024
realization.) `bootstrap_with_params(Vec<Scalar>)` /
`compile_with_params` **build each leaf through its own constructor** (the single
sizing/validation gate) from its kind-checked slice **while lowering** (build-then-
wire), consuming the vector slot-by-slot in the *same* depth-first walk
`param_space()` projects — so the two share one traversal (subsuming the #34 dual-
traversal hazard) and the value reaches the node at the slot the sweep enumerates.
Arity is checked up front (`param_space().len()`); a wrong-kind or wrong-length
vector is a typed `CompileError::{ParamKindMismatch, ParamArity}` (the typed-value
check C8 deferred). The lowering/edge/source rewrite is structurally unchanged, so
the **flat graph stays bit-identical** for a given point (C23, the correctness
invariant). The value *domain* (e.g. `length ≥ 1`) stays the constructor's own
`assert`; the search-range is still the run's (#32/C20). One value-empty leaf
detail for C22: the blueprint view (pre-run, param-generic) labels a leaf by **bare
type** (`PrimitiveBuilder::label`, was `LeafFactory::label`, → `[SMA]`) — the
value-bearing `SMA(2)` of C8's render-
label refinement now appears only in the *compiled* view (built nodes, `Node::label`).
**Realization (cycle 0017 — blueprint render = main graph + definitions).** The
`aura graph` blueprint view (C9 graph-as-data, #13) renders the **authored
structure** as a *program with subroutines*: a flat **main graph** wiring the
harness with each composite shown as a **single opaque node** `[name]`, plus a
`where:` section that defines each **distinct** composite type **once** (its
interior with named input-entry nodes and outputs folded onto their producers as
`name := …` bindings (render refined through cycles 00190022; originally
`[in:k]`/`[out]` port markers); deduped by `name()`, collected
recursively so nested composites are opaque nodes with their own definitions). This
supersedes #13's original cluster-box model and is the durable split this view
realizes: **blueprint = source** (composites as named subroutines, body once) vs.
**compiled = inlined machine form** (C23, boundary dissolved). The substantive cause
for retiring cluster boxes is a real renderer defect — `ascii-dag` 0.9.1's subgraph
level-centering rounds sibling x-positions with `/2`, overlapping wide sibling
labels (width/parity-sensitive, no config/padding dodge surviving unequal-width
siblings); the **flat** layout is collision-free, and both views now build flat
graphs only. The model also scales (blueprint size tracks top-level wiring, not
inlined node count) and removes #13's nested-composite `unimplemented!` (the
definitions pass recurses). The interactive *enter/focus* counterpart (a composite
collapsed to a navigable node) is the playground's, parked as a separate concern
(#37); the static CLI keeps the all-at-once definitions form (#38).
**Realization (cycle 0024 — the root is the fully-bound composite; the flat graph is
a named type).** `struct Blueprint` is **deleted**: the root graph IS a `Composite`,
and `compile_with_params`/`bootstrap_with_params`/`param_space` are its methods.
What distinguished the root — its bound data sources — is now a property of its input
roles: `Role` carries `source: Option<ScalarKind>` (`None` = an open interior port,
wired by the enclosing graph; `Some(kind)` = a bound ingestion feed). A composite is
**runnable iff every root role is bound** (C3: sources bind at ingestion only); an
open root role is a compile-time `CompileError::UnboundRootRole`. So the "main graph"
is no longer a separate kind — only the composite all of whose roles are
source-bound, governed by the same conditions as any other node. Compilation now
targets a **named type**: `compile` validates structurally **pre-build** (via
`signature()`, no node constructed — an ill-typed wiring is caught before any build
closure fires) and emits `FlatGraph { nodes, signatures, sources, edges }` — the
C23 flat graph, now first-class — which `Harness::bootstrap` consumes (kinds/firing
from the carried signatures, buffer depth from `lookbacks()`). The per-flat-node
signature travels beside the node, so `bootstrap` reads it without a built-node
`schema()` call (which no longer exists, see the C8 0024 realization).
**Realization (cycle 0026 — graph render redesign: model + WASM-Graphviz viewer,
#51).** `aura graph` no longer renders ASCII. The render path is now two pieces:
a read-only **model serializer** (`aura_engine::model_to_json`, iteration 1) that
walks the root composite + every distinct composite type into a deterministic,
hand-rolled JSON model (C14, golden-tested; the engine's last hand-rolled JSON
writer after `RunReport::to_json` moved to serde in cycle 0033; the
swapped-param mis-wire property moved here from the old compiled-view test), and a
**self-contained HTML viewer** (`aura-cli::render::render_html`, iteration 2) that
inlines that model, the ported prototype viewer JS, and a vendored Graphviz-WASM
blob into one page emitted to stdout. Layout/SVG happen in the browser via
WebAssembly — aura ships no layout engine and stays a serializer (C9: graph-as-data,
no `eval`/build on the path). The viewer is a render asset (C10 — no node/strategy
logic, no DSL); it labels every input pin from the model's now-real names (C23
debug symbols, named in cycle 0027) and colours wires by the four scalar base
types (C4). This **retires `ascii-dag`** and its adapter (`graph.rs`), the
`--compiled`/`--macd` flag plumbing, and the invented `#Sf`/`:=`/`histogram →`
notation — superseding the cycle-0017 flat-ascii model and its renderer-defect
workaround. The DOT/SVG are Graphviz-version-dependent and not golden-tested; the
deterministic JSON model is the asserted contract.
**Realization (cycle 0034 — structural-constant bind: a knob *removed* from
param_space, #55).** `PrimitiveBuilder::bind(slot, value)` adds the **third** param
category beside the topology factory-arg (C7/C19) and the tuning param (the
cycle-0016 value-pin): a **structural constant**. The cycle-0016 binding *pins* a
value in the injected vector while the knob **stays** in `param_space` (a tuning
param the sweep varies); `bind` instead **removes** the slot from `param_space`
entirely — the knob is gone, not fixed. The discriminator is the #55
deform-vs-tune test: a value whose variation yields another valid point of the
*same* strategy is a **tuning param** (stays in `param_space`); a value whose
variation *deforms* the strategy into a different one (e.g. the `2` of an
"SMA2-entry" bound to its two-candle construction) is a **structural constant**
(bound out), so a sweep never enumerates deformed strategies as valid family
members. Mechanically `bind` shrinks the builder's declared param surface
(`schema.params`) and wraps its build closure to re-splice the constant at its
original positional slot; the construction layer
(`collect_params`/`lower_items`/`param_space`/`compile_with_params`) is
**byte-unchanged** — both dock sites already key off `builder.params()`, so the
shrink propagates for free, and chained binds reconstruct the correct positional
vector because each layer computes its slot index relative to the param list it
sees. C23 is unaffected: `bind` resolves the param **name** to a position at
**authoring** time (the by-name authoring address space, the 0032 amendment) and
the flat graph stays wired by raw index — the name never reaches it. The
complementary question — exporting a **named frozen** strategy (all/most knobs
bound) as a reusable blueprint *value* — is deferred (#60); `bind` ships only the
per-knob overlay, no registry (C9/C10 intact).
### C20 — Strategy ↔ harness; the harness is the root sim graph
**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9):
broker-, data-, and viz-independent, with inputs declared as named **roles**
(symbol-agnostic where possible) and the **bias** stream (C10) as output.
A **harness** (the experimental setup) is the **root sim graph** — sources bound
to the strategy's input roles + the strategy + attached broker node(s) + sinks —
and is itself produced by the bootstrap (C19). A harness *instance* is C1's
disjoint unit (RustAst's "root scope"). The harness has **two kinds of
parameterization**: **structural axes** (which strategy, which instrument(s),
which broker(s), which window) whose variation selects *different* instances —
the **experiment matrix**; and **tuning params** (the strategy's numeric params)
swept *within* a fixed structure (Fork A). The same strategy blueprint is reused
across backtest, sweep, visual workspaces, and the frozen live bot — each a
different harness. **Both strategy and harness/experiment are authored in Rust**
via builder APIs (C17); the experiment matrix is ordinary Rust control flow
(loops/conditionals), not a config schema. Ontologically, a node (incl. a
strategy composite) is an **open** fragment — free input roles + ≤1 output
(C8/C9) — that does not run alone; a **harness is the closed root graph**: a
strategy with its input roles bound to sources and its output terminated in
broker/sink nodes, under a clock. A harness is therefore **not a node** (no free
inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's
disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21)
orchestrates them as objects.
**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a
declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as
a node (it is the closed root scope, not an open composable node).
**Why.** Reusability needs the strategy to be a context-free blueprint that many
harnesses embed. Modelling the harness as a root graph keeps it within the one
Node/graph abstraction (C9) and makes "10 strategies in one environment" and
"one strategy × N instruments" plain nested loops over the structural axes. Rust
authoring (not config) preserves full programmatic power — conditional/adaptive
matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap
C17 rejects.
**Realization (GER40 session-breakout blueprint milestone, 2026-06-17 — refs
#94/#96/#97, spec 0051).** Made concrete on the first real-data strategy: a
**hand-wired `FlatGraph` is not a shippable strategy** — it carries no
`param_space()`, so the World families (sweep / walk_forward / compare, C21)
cannot consume it without a hand re-author (the friction the GER40 deep-dive
fieldtest surfaced). The **canonical shippable form is the `Composite`
blueprint** (the authoring/source level, C9/C19); the `FlatGraph` is only its
compiled substrate (C23). The breakout now ships as
`ger40_breakout_blueprint(bar_period, …)` whose `param_space()` is exactly its
tuning knobs (`{entry_bar.target, exit_bar.target}`); the **bar period is a
construction argument** binding Resample + Session together — a structural
matrix axis (C12: a different period is a *different strategy*, not a sweep
point), never a `param_space` entry, so a sweep cannot desync the two clocks.
This is the concrete instance of the structural-axis-vs-tuning-param split
above (and `delay.lag`, a C8 structural constant, is bound out of the space).
### C21 — The World: the meta-level is the product
**Guarantee.** Above the harness sits the **World** — the project's program /
"game" (C16: a Rust crate). Within it, **harnesses are dynamically constructible,
first-class objects**: meta-programs (walk-forward, sweep, optimize,
Monte-Carlo — C12's axes) construct harness instances at runtime via the
bootstrap (C19), run them disjointly in parallel (C1), aggregate / compare their
results, and discard the transient instances. A walk-forward rolls windows →
bootstraps a harness per window → stitches out-of-sample equity + parameter
stability into one meta-result. Orchestrating *families* of harnesses is
**first-class**, not a headless afterthought; the run registry (C18) is the
World's memory.
**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep /
comparison) to second-class headless-only status; treating the single backtest
as the product.
**Why.** What happens *within* one harness — backtest a strategy → equity — is
commodity; every quant system covers it. aura exists for the meta-level: a
programmable space where harnesses are dynamically built and families of them
orchestrated and explored. The deterministic single-harness engine (C1C20) is
the **substrate**; the World is the **product**.
### C22 — The playground is a trace explorer; sinks are the recording mechanism
**Guarantee.** The World is a *program*: nothing is displayable until it runs,
and its harnesses are transient machinery (built, run, discarded — C21). The only
durable, displayable substance is the **recorded trace**, captured by **sinks**
pure consumer nodes (C8) that persist a stream (equity, position events, a node's
output) into the run registry (C18). **Displayable = exactly what a sink
recorded**; with no sink, only input params + summary metrics remain. The
**playground** is therefore an **execution viewer / trace explorer**, and it
plays *any* harness (it is the harness *player*, not a harness, never bound to a
default one): before a run it shows the program *structure* (graph-as-data, C9) +
param knobs; during a run, live sink streams; after a run, recorded traces +
metrics from the registry — including meta-views (stitched walk-forward, sweep
surfaces, multi-strategy / instrument comparison). Observability is **explicit
and selective** — you instrument what you want to see; the choice of sinks is
part of the experiment. The engine ships sample harnesses *with* sinks so a
newcomer sees a populated trace immediately; a fresh project is empty until
built, run, and instrumented.
**Forbids.** A scene-editor model that assumes a persistent populated world;
constructing or wiring topology in the UI (topology is Rust + hot-reload,
C9/C17 — the UI reflects the live graph-as-data and tunes runtime params via
sliders, C12); retaining un-sinked data; binding the playground to a single /
default harness.
**Why.** A program has no scene-at-rest to inspect; what persists is what it
records. Making sinks the one recording-and-observability mechanism keeps "what
can I see?" answerable by "what did I instrument?", and keeps the engine
UI-agnostic (C14). Live param tuning (runtime values, no topology change —
C12 / C19 Fork A) gives the interactive feel without a wiring DSL.
**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the
substrate level: a recorded trace is exactly what a recording node pushed out of
the graph (no engine recording registry; the constructing World holds each
recording node's destination). The engine's single `observe: usize` affordance is
removed — `Harness::run` returns `()` and recording is a node-side concern, so one
run records *many* streams (one per recording node) instead of exactly one row.
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
`Harness` API change (observe removed, `run -> ()`) is recorded here.
**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped
(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA,
C7) `ColumnarTrace` to `runs/traces/<name>/` (`aura-registry::TraceStore`, beside the
run registry's `runs.jsonl`), reachable via `aura run [--real <SYM> …]
--trace <name>`; `aura chart <name> [--panels]` reads them back, aligns all taps on a
synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph;
C1 — pure, no live external call), and emits a static self-contained uPlot page
(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14):
encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in
`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`).
First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served
page injected the chart *series* but no run-context header (the run manifest is
persisted on disk in `index.json`, deliberately not surfaced in that first cut — a
ratified scope call). The header (#102) and serve-time decimation (#108) landed in
cut 2 (see the served-page-hardening amendment below); the families-comparison view
landed too (#107). A local server and replay-clock controls remain open (see "Open
architectural threads").
**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends
to family runs: `aura sweep|mc|walkforward --trace <name>` persists *each member* as
a nested standalone run-dir `runs/traces/<name>/<member_key>/`, reusing
`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a
side-effect inside each per-member closure). The `member_key` is content-derived and
deterministic — sweep: the *varying* axes rendered as a filesystem-portable
directory component (`<axis-name>-<value>` tokens joined by `_`, charset
`[A-Za-z0-9._-]`, case-less values, length-capped with an FNV fallback), MC
`seed{N}`, walk-forward `oos{ns}` — never
a runtime ordinal, because members run in parallel under the engine's `Fn + Sync`
HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write`
targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`,
stdout/registry are byte-unchanged. Because `TraceStore` resolves `<name>/<member_key>`
as a subpath, `aura chart <name>/<member_key>` charts any single member with no
view-side change — the write-side precondition for the family-comparison **view**
(overlay / small-multiples across members) — now realized (#107; see the
families-comparison amendment below).
**Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an
**opt-in real-data source axis**: `aura sweep|walkforward --real <SYMBOL> [--from
<ms>] [--to <ms>]` streams real M1 close bars from the data-server archive (the #71
`M1FieldSource` seam — C6 record-then-replay: a pre-recorded archive, never a live
call mid-sim) instead of the built-in synthetic stream, per family member. A
CLI-side `DataSource` provider (synthetic | real) supplies each member's source,
pip (resolved from the recorded geometry sidecar (`instrument_geometry`) — C10
refuse-don't-guess; a symbol with no recorded geometry exits 2 before any data
access), window, and — for walk-forward — its `WindowRoller` sizes
(synthetic: bar-index 24/12/12; real: fixed calendar-time 90d/30d/30d in ns; CLI
flags for these are deferred). The engine, ingest, and registry are untouched (C9 —
the whole change is in `aura-cli`); the synthetic path is byte-unchanged.
**MC is excluded** (#106 Fork A): its seed varies a *synthetic* price-walk
realization (C12 axis 4), which is undefined over real data's single realization —
`aura mc --real` refuses with exit 2; a real MC needs a bootstrap-resampling axis
(its own cycle). The real walk-forward member key `oos{from.0}` widens from a small
synthetic bar index to an epoch-ns integer (still portable, collision-free). The
built-in demo strategy's **length grid is likewise data-kind-dependent**
(`DataSource::strategy_lengths`): synthetic keeps the short lengths that fit the
18/60-bar demo streams (trend SMA `{2,3}×{4,5}`, MACD `2/4/3`), real uses realistic
M1 lengths (trend `{50,100}×{200,400}`, MACD `12/26/9`) so the cross is a genuine
trend signal over tens of thousands of bars, not noise. This — like the roller
sizes — is a *demo-strategy calibration* patch; the real answer is project-authored
strategies (C9: a project crate owns its own grid), deferred to the project-env
work.
**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison
**view** the #104 amendment left open is now realized. `aura chart <name>`
classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a
single run; else member subdirs with `index.json` are a family; else not-found) and,
for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every
member** in one self-contained uPlot page: `build_comparison_chart_data` emits one
labelled series per member, all on a **single shared y-scale** — members measure one
identical quantity, so a shared scale is what makes them comparable, unlike the
single-run overlay of *different* taps (per-series scales). The series align on the
same union-ts spine via `join_on_ts`, so **one mechanism yields three correct
readings**: sweep/MC members share the data window (a true overlay), walk-forward
members are disjoint OOS windows (null-complementary → the stitched curve). The view
consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted
by key for determinism, C1); a family is its *first producer* — the deliberate first
step toward the programmable analysis meta-level (C21) **without** rebuilding the
orchestration axes (that is its own later cycle). Name resolution is made a **total
function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing
command (`run`/`run --harness macd`/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family)
to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by
both a run and a family) is unreachable. Every error path exits 2, never panics
(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is
`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder +
`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when
no `--tap` is given. **Still deferred:** multi-column tap selection
(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are
single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as
composable meta-level tools).
**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred
follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine
+ registry untouched, C14). **Run-context header (#102):** a `ChartMeta`
(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params
for a single run) is built from the `RunManifest` in
`build_chart_data`/`build_comparison_chart_data`, injected into
`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js`
(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across
members `(min from, max to)` — the only reading that labels a disjoint walk-forward
family's true OOS coverage (it collapses to the shared window for sweep/MC).
**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max
transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots
(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a
multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full
recorded data stays on disk (view-only), and the walk-forward null-fill page collapses
for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under
budget. **Refinement (#111, 2957561):** the per-bucket reduction is **tap-aware** — an
*envelope* series (equity) keeps min/max, but a *bounded level* series (the C10
bias level — pre-reframe: exposure — ∈[-1,+1]) reduces by per-bucket **mean** (a `ReduceKind` on each `Series`,
set by `reduce_for_tap`). Without it min/max made every multi-year exposure a solid
-1..+1 band (each bucket straddles many sign flips), reading as per-point oscillation;
the mean shows the net/duty-cycle level. Deferred refinements: rendering the min/max
envelope honestly as range bars / OHLC rather than a polyline (#112); a `--width`
budget flag and true intra-bucket min/max ordering for the non-default continuous
x-mode (#110).
### C23 — Graph compilation and behaviour-preserving optimisation
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
named **blueprint** (the authoring source — nodes, composites, strategy, harness;
C8/C9/C20) into a flat, type-erased **`FlatGraph`** — the frozen runnable
instance (C7) — **wired by raw index, not by name** (`Edge { from, to, slot,
from_field }`). Composites are **inlined** at this step (C9): the composite
*boundary* dissolves entirely (there is no composite in the flat graph). The
blueprint's field / input-role *names* are **non-load-bearing** — the wiring
resolves by index, and names survive at most as **informative debug symbols**
(exactly as `FieldSpec.name` already is, C8), kept for tracing / rendering (C9
graph-as-data, #13) but carrying no run semantics. The flat graph is then the target of **behaviour-preserving
optimisation** — any transform that leaves every observable sink trace
bit-identical (C1 is the correctness invariant) — on two levels:
- **intra-graph** (within one graph): **common-subexpression elimination**
two identical nodes (same type, same bound params, same input source) merge to
one with output fan-out, so fractal composition (C9) costs no redundant compute —
and **dead-node elimination** — a node on no path to a sink is dropped.
Pure-consumer sinks (C8, no output) are never CSE candidates, so their
out-of-graph side effects are preserved by construction.
- **across the sweep family** (over the family of flat graphs one blueprint yields
under a param sweep, C12): the sub-graph whose nodes carry **no swept param** and
whose inputs are **all sweep-invariant** (transitively from the sources — same
data window) is **loop-invariant** w.r.t. the sweep and is computed **once**; its
output is materialised as a recorded stream (C11) shared read-only across the
sweep instances (`Arc<[T]>`, C12). Only the parameter-dependent suffix re-runs
per sweep point — loop-invariant code motion over the sweep loop.
**Forbids.** Any "optimisation" that changes an observable trace (it breaks C1);
optimising over a representation that is not graph-as-data — an opaque
trait-object interior (the rejected nested-composite reading of C9) cannot be
analysed or rewritten across its boundary, so it forecloses both levels.
**Why.** Determinism (C1) is not merely an audit property; it is the **licence**
for these rewrites — a behaviour-preserving transform is only meaningful because
"same input → bit-identical run" makes "same result" decidable, and a
sweep-invariant sub-graph is reproducible enough to compute once and share. The
flat graph representation (C19) is the necessary condition: only a flat, inspectable
graph-as-data — with each node's declared param-ranges (C8) — lets the compiler
identify identical sub-expressions, dead nodes, and the sweep-invariant frontier.
This is precisely why a composite is an inlining (C9) and not a runtime
sub-engine: the flat graph is what makes the whole graph optimisable.
**Status (Construction-layer milestone).** The flat graph *representation*
blueprint → inline / compile → flat instance — is the load-bearing substrate built
**first**; the optimisation passes (intra-graph CSE/DCE, sweep-invariant
hoisting) are **deferred**, behaviour-preserving follow-on work, each gated by a
"flat graph with pass ≡ flat graph without pass, bit-identical run" test (C1). The
sweep-level pass further presupposes per-node param declarations (C8 — landed in
cycle 0015 as `name`+`kind`; the swept *range* still pending #32/C20) and the sweep
orchestration itself (C12/C21).
**Amendment (cycle 0032 — param-namespace injectivity is part of the compilation).**
The bootstrap-as-compilation now includes one structural gate:
`check_param_namespace_injective` over the `param_space()` name projection (see C9 /
C12-C19), run before name resolution. It reads the **boundary** name projection — the
authoring address space — and rejects a duplicated path (`DuplicateParamPath`). Node
names stay **non-load-bearing**: they qualify the param path at construction and are
dropped at lowering (the flat graph is wired by raw index, unchanged). The check makes
the by-name *authoring address space* injective; it does **not** make names
load-bearing in the flat graph.
---
## Open architectural threads not yet resolved
- **Playground & World UI surface** — the playground is core (C22); the surface
is a **web frontend served from disk-persisted traces** (revised 2026-06, issue
#101; not egui). Settled for the first cut: raw per-tap trace persistence to
disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static
self-contained HTML chart page (uPlot, vendored like the `render_html`
Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The
families-comparison **view**`aura chart <family>` overlaying one tap across a
family's members on a shared y-scale — shipped (#107, see the C22 amendment).
Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family /
multi-strategy comparison, multi-column tap selection #47), a local server, and the
run/replay clock controls.
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
atop the atomic sim unit (C12), not yet designed.
- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s
static-context schema** — `aura new` scaffolds a Rust project *crate* (node /
strategy / experiment blueprints) against the engine (C16/C20); the
experiment-builder API surface (harness wiring, structural axes, sweep
combinators) and `Aura.toml`'s schema (data paths, instrument/pip metadata,
default broker & window, runs dir) are not yet designed.
- **The analysis meta-level — composable orchestration + project-as-crate authoring
(tracked: #109).** aura's differentiator (C21) has two unbuilt halves: (1) the
orchestration axes (sweep / Monte-Carlo / walk-forward) becoming *composable tools*
a user wires into an analysis workflow, rather than today's hard-wired CLI verbs
(`sweep_family` / `walkforward_family` in `aura-cli`); and (2) the project-as-crate
authoring layer above (the `aura new` / experiment-builder API / `Aura.toml` thread
+ cdylib loading, C16/C17). On the meta-level, authoring a strategy is *wiring
existing nodes* (C9 composition) + *defining the analysis framework* (C21), not
writing new nodes; the user wants it driven via the CLI, later the interactive
server / playground (C14/C22). **Load-bearing open fork — resolve in the brainstorm
before any cycle:** "driven via the CLI" = (a) *running* a Rust-authored experiment
via the CLI / playground (compatible with C16/C17/C22) vs (b) *wiring nodes through
the CLI itself* (a deliberate amendment to C17's no-DSL, C22's no-UI-topology, and
C9's no-by-name-node-registry). Deferred by the user (2026-06-22); #109 is its
durable home until promoted to a milestone seeded by that brainstorm.
- **`aura-std` contents** — the crate exists (doc-only); which universal blocks
land first follows the walking-skeleton's needs.
- **`strategies/` split** — a later split, *inside a project*, of top-level
strategies from reusable building blocks in `nodes/`; not a day-1 cut.
- **Sequencing** — the runnable single-harness *substrate* comes first (walking
skeleton: a closed harness that runs deterministically and records via a sink);
the World/meta layer (C21) and the playground trace-explorer (C22) are the
differentiating layers that follow — you orchestrate and visualize a thing that
must first run once.