Files
Aura/docs/design/INDEX.md
T
Brummel 63ea7eb3b1 audit: cycle 0011 tidy — record the external-dependency firewall invariant
Architect drift review (f68258b..HEAD) found no code drift — the firewall is
real at the dependency-graph level (aura-core/std/engine/cli carry no external
dep; the chrono/regex/zip tree is confined to aura-ingest), and C3/C7/C1 are
faithfully realized. Four ledger-level items, all resolved here:

- [ledger-drift] The zero-external-dependency commitment — the load-bearing
  rationale for making aura-ingest a crate rather than a feature-gate — lived
  only in spec 0011. Recorded in the ledger: C16 now states the engine workspace
  is zero-external-dependency by commitment, names aura-ingest as the ingestion
  edge / external-dependency firewall, and forbids an external dep in any engine
  crate other than aura-ingest.
- [ledger-drift] C16's reuse taxonomy did not place aura-ingest. C16 now lists
  the non-node engine crates (aura-engine, aura-cli, aura-ingest).
- [ledger-debt] C12's eager-materialization-now / Arc<[T]>-sharing-later choice
  was only in the spec. C12 now carries a "Status (cycle 0011)" note recording
  the deliberate gap.
- [debt] aura-engine/Cargo.toml carried a stale comment ("data-server enters
  when the ingestion task starts") that contradicted the firewall it should
  protect. Replaced with the dependency-pure / firewall-in-aura-ingest note.

The External components data-server bullet is updated to reality (pulled in by
aura-ingest; workspace now resolves from a populated cargo cache, not fully
offline).

Regression gate: no-op (commands.regression empty); architect is the sole gate.
Cycle 0011 is drift-clean. (Drift-clean, not a milestone close — the
Walking-skeleton milestone close additionally needs its end-to-end milestone
fieldtest, a separate deliberate act.)
2026-06-04 21:56:29 +02:00

612 lines
40 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# aura design ledger — INDEX
The ledger records the load-bearing design contracts and their rationale. Each
contract states what it **guarantees**, what it **forbids**, and **why**. A
change that breaks a contract is a design decision (amend the contract here with
its new rationale), never a silent refactor.
Provenance: contracts C1C18 were settled in the initial rough-sketch design
interview (2026-06-03), walking the design tree root-to-leaf (C16C18 and the
C10 refinement to a broker-independent position table came in follow-up turns;
C10 was later reframed in cycle 0007 — the intent/exposure stream is the primary
output, the position table a derived layer — see C10).
The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded
summary of the
subset that agents must never violate; this file is the fuller form with
rationale.
Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
*milestone* is a tracker container spanning many cycles (the first milestone is
the **walking skeleton**: ingest → one signal → exposure/intent → sim-optimal
broker → synthetic pip-equity signal-quality metric).
---
## Foundation — what aura is
aura is a framework **and** a playground for traders. A human and (primarily)
LLMs author trading **nodes** directly in Rust; the engine backtests them
deterministically and massively in parallel, composes them fractally, validates
them (sweep / Monte-Carlo / walk-forward), and freezes a validated strategy into
a standalone bot with a broker connection.
The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow,
too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura
inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts*
(synchronous reactive streams, bounded-lookback series, run-counting, SoA).
RustAst is a conceptual reference, not a dependency. The one reused component is
`data-server` (the first data source).
What sets aura apart is **not** the single backtest — every quant system does
that — but the **World**: the meta-level where harnesses are dynamically
constructed and *families* of them orchestrated and explored (walk-forward,
sweeps, Monte-Carlo, comparison). The deterministic single-harness engine is the
*substrate*; the World is the *product* (C20C22).
---
## External components
Two sibling projects live outside this repo and are named throughout the
contracts. Their concrete location is recorded here so the repo is the single
source of truth — not session memory.
- **`data-server`** — `~/dev/libs/data-server`, Gitea `Brummel/data-server`
(`http://192.168.178.103:3000/Brummel/data-server.git`). aura's **first data
source** and the one reused component (Foundation; C3, C11, C12). A standalone
leaf crate (deps: `chrono`, `regex`, `zip`) that loads Pepperstone **M1/tick**
binary files and shares them lock-free as `Arc<[T]>` chunks
(`CHUNK_SIZE = 1024`) via `SymbolChunkIter::next_chunk` (an inherent method
driven by `while let`, not the `Iterator` trait); `stream_*_windowed(from_ms,
to_ms)` provides C12's data-window with inclusive Unix-ms bounds. Records are
**AoS** (`M1Parsed`/`TickParsed`, `time_ms: i64` Unix-ms). The ingestion
boundary (C3) therefore **transposes** these AoS records into aura's SoA
columns (C7) and normalizes `time_ms` → canonical epoch-ns at that one
boundary. Pulled in as a cargo git dependency by the **`aura-ingest`** crate
(cycle 0011) — the ingestion edge and the workspace's **external-dependency
firewall**: `aura-ingest` is the *only* crate that links `data-server` (hence
its transitive `chrono`/`regex`/`zip`), so the engine crates (`aura-core` /
`aura-std` / `aura-engine` / `aura-cli`) and the frozen deploy artifact (C13)
stay external-dependency-free. Consequence: `cargo build/test --workspace` now
resolves from a populated cargo cache (one Gitea fetch) rather than fully
offline.
- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst`
(`http://192.168.178.103:3000/Brummel/RustAst.git`). The predecessor DSL attempt
(Foundation): **a conceptual reference, never a dependency** — its DSL authoring
surface and `Value`/HM-inference machinery are exactly what aura rejects (C17).
But its **`src/ast/rtl/` layer is a working reference implementation of the very
streaming substrate aura rebuilds**, and is worth reading before authoring
`aura-core` — these are not just concepts, they exist as code:
- `rtl/series/data.rs``RingBuffer<T>` with financial-style indexing (index 0
= newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's
pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no
String/Record" = C7's closed scalar set); `ScalarSeries<f64|i64|bool>` and the
**SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns").
- `rtl/series/mod.rs``create_typed_series`, dispatching element type → storage
backend (the type-specialized **factory** of C19).
- `rtl/streams/mod.rs``Signal { cycle_id, value }` (C4's cycle clock) and the
`Stream` / `Observer` / `ObservableStream` push traits (the reactive model of
C4/C5).
- `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one
explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator
(C12's seed-as-input).
aura reimplements these natively and **sharpens** them: types are monomorphized
and edges type-erased to the four scalar kinds with direct dispatch (C7) instead
of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>`
(C12) instead of a `VecDeque<Value>`. RustAst shows the *shape*; aura makes it
fast and deterministic.
---
## Contracts
### C1 — Determinism and disjoint parallelism
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
loop that reaches a unique state after each input tick. Same input (incl. seed)
→ bit-identical run. Two backtests are fully disjoint and run concurrently
without locking.
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
is not captured as an explicit input (see C11, C12).
**Why.** Real money rides on backtest results; reproducibility and an audit
trail are non-negotiable. Speed comes from parallelism *across* sims, which
disjointness makes lock-free.
### C2 — Causality / no look-ahead
**Guarantee.** A node sees only the past. Input history is a read-only window
that ends at the current cursor; a resampler emits a bar only once it is
complete.
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
/ still-forming bar.
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
leaks the future is worse than none. Making the future *physically absent* from
what a node receives beats merely discouraging it.
### C3 — One merge, at ingestion only
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp
into one chronological cycle stream at the ingestion boundary; source-native
time units (e.g. data-server's Unix-`time_ms`) are normalized there to the
canonical epoch-ns `timestamp` of C7.
**Forbids.** Any merge / as-of join *inside* the graph.
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
future. Keeping the merge at one boundary keeps the graph semantics simple.
### C4 — Cycle granularity
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
multiple sources) break by source declaration order.
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
origin of records, not the cycle semantics. Tie determinism preserves C1.
### C5 — Freshness-gated recompute and sample-and-hold
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
run-count); otherwise it holds its last output. Stale inputs contribute their
last (held) value.
**Forbids.** Recomputing every node every cycle ("push all" is true for the
*clock*, not for *recompute"); treating a held value as missing.
**Why.** Total recompute does not scale to many sparse high-frequency sources;
freshness-gating is the performance discipline that keeps the synchronous model
fast.
### C6 — Firing policy A and B, per input group
**Guarantee.** A node declares, per input group, one of two firing policies:
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
four separate 15m sources: the candle is complete only when all four are fresh).
A single node may mix an A input and a B group.
**Forbids.** A single global firing mode; forcing per-node-only granularity.
**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group
granularity is required by the OHLC-plus-bias case where one node needs both.
**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any,
Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a
mode-A input is its own trivial group, so "per input group" and the per-input tag
coincide. The barrier's synchronization token is the cycle **timestamp**, not the
`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so
RustAst's `cycle_id`-equality barrier could never fire across them. A group fires
when every member's last push carries the current cycle's timestamp, guarded by
"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This
fires both the multi-source bar and the within-source diamond rejoin (every push
in a cycle carries that cycle's timestamp).
### C7 — Four scalar base types, streamed as SoA
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
(OHLCV) are bundles of base columns — this is the **node-output model** too: a
node emits a record of 1..K base columns (C8), each forwarded field-wise to a
consumer slot; the bundle is structural, never a fifth scalar type. Edges are
type-erased to these four kinds;
the type check is paid once at wiring/sim-start, then the topology is frozen per
sim → direct dispatch, no per-event allocation.
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
allocation; topology mutation mid-sim.
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
set and SoA. The open set is composites (schemas of columns), not scalar types.
Type-erasure at the edge is also forced by the cdylib boundary (C13).
### C8 — The node contract
**Guarantee.** A node implements `schema()` (declares each input's scalar type,
required lookback depth, and firing group, **and the node's own tunable
parameters — typed, with ranges**, which aggregate into the blueprint's
param-space the optimizer sweeps, C12/C19/C20) + `eval(ctx) -> Option<&[Scalar]>`. The
engine provides read-only, zero-copy windows into each input's SoA ring buffer
(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own
mutable series for derived/intermediate state. `None`/Void return = filter /
not-yet-warmed-up. A node is a **producer, a consumer, or both**: a
producer/transformer exposes **one output port**, whose payload is a **record of
1..K base-scalar columns** (a scalar is the degenerate K=1 record; an `eval`
returns a borrowed row, one value per column); a **pure consumer (sink)** — chart,
equity, logger — has **no** output. Sources are pure producers; sinks are pure
consumers.
**Forbids.** A node sizing/growing its input lookback at runtime; more than one
output **port** per node; a fifth scalar type or a heterogeneous output payload
(a record is a bundle of base columns, C7); copy-on-read of input history.
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage
lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized
buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop).
**Realization (cycle 0005).** `NodeSchema.output` is a `Vec<FieldSpec>` (named base
columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field`
selects one producer column per edge; consuming a whole record is N edges (no
"bind whole record" mechanism). The K fields of one record are **co-fresh by
construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns
`Option<&[Scalar]>` — a borrowed row into a node-owned buffer — so the forward
path allocates nothing per cycle (C7).
**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is
now realized at the substrate: **recording is a node role, not a type.** A
recording node reads its typed input windows + `ctx.now()` in `eval` and pushes
the record to a destination it holds as a field (a channel, a chart handle) — an
**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a
node that only records returns `None` (pure consumer), and a node may record
**and** return an output the engine forwards in the same `eval` (the "both"
case). **Encoding & return contract.** A pure consumer declares `output: vec![]`
— the empty record *is* the sink declaration; there is no separate type, trait,
or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run
loop debug-asserts the returned row's width equals the declared output width
(`row.len() == schema.output.len()`). Field-wise wiring resolves
`Edge::from_field` against the producer's `output` at bootstrap, so no edge can
bind a field of a zero-output node — it fails with `BadIndex` — making a sink
structurally unwireable as an in-graph producer; its only output is the
out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out
of the graph is the node's own side effect — and that boundary is the
determinism / graph-as-data boundary (C1/C7).
### C9 — Fractal, acyclic composition
**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes
one output; signal, combined signal, and (with execution) strategy are all the
same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only
feedback path is an explicit delay/state node (the RTL "register"). Wiring is
written in Rust (builder API); the built graph is introspectable runtime data.
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
"signal-of-signals" as separate mechanics.
**Why.** Self-application of one contract gives unlimited composition with no
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined;
forcing feedback through a visible delay node keeps the per-cycle determinism
intact and the one legitimate feedback path explicit. Graph-as-data enables
visualization, freezing, and re-parameterization for sweeps.
### C10 — Strategy output is an intent/exposure stream; position management is a decoupled derived layer; brokers are downstream nodes
**Guarantee.** A strategy's primary, backtestable output is **not** an equity
curve, **nor a position-event table**, but an **intent / exposure stream**: the
DAG expresses exactly one *state* at time t (C8 — a node emits at most one record
per `eval`), so a strategy emits one **signed, bounded exposure** `f64 ∈
[-1, +1]` per cycle (per instrument; the portfolio is multi-instrument). Exposure
is the desired fractional position; **position sizing and risk live in the
decision/sizing node** that shapes a raw signal score into exposure. The chain is
`signals (scores) → decision/sizing node → exposure stream`.
**Signal quality** is evaluated by the **(a) sim-optimal broker** — a downstream
consumer node (C8/C9) that consumes the **exposure stream** plus the relevant
price stream and integrates `exposure(t-1) · (price(t) price(t-1))` into a
**synthetic equity stream in pips**: deterministic, frictionless, perfect-fill
(no currency, no real-broker constraints). The exposure held *into* a cycle
(decided at t-1) earns that cycle's return, so the integration is causal (C2 — no
look-ahead). Pip PnL uses per-instrument pip metadata (reference data beside the
hot path, C7/C15). This pip-equity curve measures the **signal's quality**, not
an execution-modelled P&L; it is the primary research loop — backtest one signal,
combine it with another, backtest the combination.
**Position management** — turning the exposure stream into a broker-independent,
time-ordered **table of position events** (pure scalar columns, C7: `event_ts:
timestamp`, `action: i64` buy / sell / close, `position_id: i64`,
`instrument_id: i64`, `volume: f64` unsigned — direction is the `action`; a
position's open time is its opening event's `event_ts`, no separate `open_ts`; a
`close` references a `position_id` and may be partial via its own `volume`) — is a
**decoupled, derivable, downstream** layer. The events are the **first difference
of the exposure state**, materialized as a *computed table* (where multiple
events may share one `event_ts` — e.g. a reversal's close + open), **never as a
per-`eval` node output** (which C8 caps at one record per cycle). This layer feeds
**(b) realistic broker nodes** (Pepperstone, …): downstream consumer nodes that
consume the position-event table plus prices, apply real spread / commission /
slippage / lot / margin, may reject or modify positions, and produce a
**currency** equity stream for viability and deployment. The position-event table
stays broker-independent — one table feeds many realistic brokers, giving
directly comparable currency curves. Live: a realistic broker node consumes the
events in real time and routes orders as a side effect; reconciliation with the
real account is an external adapter.
So **the two broker classes attach at two different points**: the sim-optimal
broker on the **exposure stream** (signal quality, pips, now), realistic brokers
on the **derived position-event table** (execution viability, currency, later).
Both are ordinary downstream nodes (C8/C9); several can attach at once for
directly comparable curves.
**Forbids.** Treating an equity curve as the strategy's output; making the
**position-event table the strategy's direct DAG output** (it is derived, not
emitted per `eval` — a decision instant may need >1 event, which C8 forbids) or
the measure of signal quality; baking a broker into the strategy; a special
external broker subsystem (a broker is an ordinary node); storing `open_ts`
(derive it from the opening event); a signed-volume direction trick in the event
table (use `action`); broker-specific assumptions leaking into the strategy logic.
**Why.** The DAG is a synchronous reactive graph: at time t it holds exactly one
state, and a node emits at most one record per `eval` (C8). A *sequence of
position events* — where one decision instant (a stop-and-reverse) needs a close
**and** an open at the same `event_ts` — cannot be the DAG's per-cycle output
without violating C8. The state the DAG *can* express faithfully is the **desired
exposure** (one value per cycle); the position events are its first difference, a
derived consequence. Decoupling them resolves the C8↔C10 impedance and matches
how signal research actually proceeds: you first measure a signal's *quality*
(does this exposure, held over time, make pips?) independently of any execution
model; only later do you model position management and real-broker frictions for
deployment. The sim-optimal pip curve is a level, currency-free playing field for
comparing and combining signals; the derived position-event table + realistic
broker nodes then test real-world viability. Modelling brokers as nodes (not a
bespoke subsystem) keeps them within the one Node/graph abstraction (C9). This
supersedes the earlier "the strategy's output is the position-event table"
framing (cycle 0007 reframe).
**Realization (cycle 0007).** The signal-quality half of this contract is now
realized at the substrate, as two `aura-std` nodes composed on the unchanged
engine (the engine stays domain-free — it routes only `f64` records, never
"exposure" or "equity"). The **exposure stream** is realized as `Exposure { scale
}`: the decision/sizing node, `clamp(signal / scale, -1, +1)`, one `f64` per fired
cycle (`None` until warmed up). The **sim-optimal broker** is realized as
`SimBroker { pip_size }`: a two-input node (exposure, price) that accumulates
`prev_exposure · (price prev_price) / pip_size` and emits cumulative pip equity
— the exposure held *into* a cycle (decided at t-1) earns that cycle's return, so
the integration is causal (C2), and `pip_size` is held reference metadata
(C7/C15), never streamed. An end-to-end harness (SMA-cross → `Exposure`
`SimBroker` → recording sink) produces a recorded pip-equity curve, bit-identical
across runs (C1). The **position-management half** — deriving the position-event
table (buy/sell/close, `position_id`, partial closes) from the exposure history,
and the realistic broker nodes that consume it — is deliberately **not** built
this cycle; it remains the decoupled, derived, deferred layer described above.
### C11 — Generalized sources; record-then-replay determinism boundary
**Guarantee.** A source is anything that produces timestamped scalar streams —
market data (`data-server`) and non-financial sources (e.g. a news-agent node
emitting a bias) are treated identically. Anything nondeterministic, external,
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
*before* it enters the engine; backtest replays the recording, live computes
fresh in real time and records it for future backtests. A bias enters as a value
held until the next event (firing policy A).
**Forbids.** Any live external call *inside* a backtest replay.
**Why.** It is the only model compatible with reproducible backtests — LLM calls
are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`,
external LLM (IONOS) calls happen only at the recording/live-source edge, with
explicit per-session consent, never inside a sim.
### C12 — The atomic sim unit and the four orchestration axes
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
values injected at graph build (no recompile per param-set; the optimizer sees a
generic vector of typed ranges). Raw data is shared read-only across sims via
`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic
unit: (1) param-sweep (grid/random), (2) optimization (argmax metric),
(3) walk-forward (rolling in-sample optimize + out-of-sample test),
(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over
seeds**; each realization is itself deterministic given its seed.
*Status (cycle 0011).* The first ingestion (`aura-ingest`) **eagerly
materializes** a data-server window into owned source streams rather than
sharing `data-server`'s `Arc<[T]>` chunks zero-copy across sims — the cross-sim
`Arc<[T]>` sharing has no consumer until the orchestration axes above are built.
That sharing remains the target for the orchestration cycle that introduces
them; the transpose logic is unchanged by it. A known, deliberate gap, not drift.
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
primitive — those are pluggable policies atop the atomic unit; recompiling on a
param change.
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
(embarrassingly parallel across the unit) cleanly separated from search policy.
Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the
atomic unit is one harness instance, selected by the harness's **structural
axes** (C20); the structural experiment matrix is the outer orchestration over
this dimension, the tuning sweep the inner (C19/C20).
### C13 — Hot-reload is authoring-only; deploy is frozen
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
artifact.
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
toolchain plugins.
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
and reproducible (audit trail: this bot = this commit). A sweep pays no
hot-reload tax — params are runtime data (C12), so the cdylib loads once.
### C14 — Headless core, two faces
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
**programmatic/CLI** face (the primary surface for the LLM and automation —
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
for human exploration. Visualization is only a downstream consumer node on the
streams.
**Forbids.** Any UI/pixel knowledge inside the engine.
**Why.** The LLM drives programmatically, the human visually; a headless core
serves both. The visual face is the **playground** (C22) — egui-native,
in-process zero-copy from the SoA columns; it is staged after the runnable
substrate but is core to aura's identity, not optional.
### C15 — Resampling-as-node; sessions/calendars
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
and instrument specs are metadata (non-scalar, beside the hot path); session
*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open:
i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after
session open" is then a plain node checking `bars_since_open == 3`.
**Forbids.** Streaming the calendar; special-casing session logic outside the
stream model.
**Why.** Keeps the line consistent — everything a signal needs arrives as a
stream; reference data feeds source/session nodes from beside the hot path.
### C16 — Engine / project separation; three-tier node reuse
**Guarantee.** aura is the reusable **engine**; each research project is a
separate external repo that depends on aura via cargo (the game-engine / game
split). Node reuse is cargo-native, in three tiers: **`aura-std`** (universal
blocks, ship with the engine) / **shared node crates** (cross-project-reusable,
their own repos, pulled as cargo git deps) / **project-local `nodes/`**
(experimental, project-specific). A reusable node is an `rlib` dependency; the
hot-reload unit stays the project-side `cdylib` that composes it (consistent
with C13). Concretely a project is a **Rust crate** — a cdylib library of node /
strategy / experiment blueprints — plus a static `Aura.toml` (project context:
data paths, instrument/pip metadata, default broker & window, runs dir). During
research the `aura` host loads and runs it (C13 hot-reload); for deploy the
chosen strategy + broker freeze into a standalone binary. **A project is
therefore always a Rust program built on the engine.** Beyond the node-reuse
tiers, the engine workspace also carries non-node crates — `aura-engine` (the
run loop), `aura-cli` (the `aura` binary), and **`aura-ingest`** (the
data-source ingestion edge, cycle 0011). `aura-ingest` is the
**external-dependency firewall**: it alone links external crates (via
`data-server`), keeping the other engine crates and the frozen deploy artifact
(C13) dependency-pure (see External components). The engine workspace is
**zero-external-dependency** by commitment — the hand-rolled JSON of C14/C18 and
the from-scratch `aura-core` substrate exist precisely to honour it; the one
sanctioned external tree enters at the ingestion edge and is firewalled there.
**Forbids.** Project-specific signals in the aura repo (it keeps at most
example/fixture nodes under `examples/` for its own tests); a multi-project
manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the
package mechanism); an external (crates.io / git) dependency in any engine crate
other than the ingestion-edge crate `aura-ingest`.
**Why.** The engine/game split keeps the engine sharp and reusable while each
project versions its own research with its own forward-queue. Promotion
(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism.
### C17 — Authoring surface
**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses**
is authored in native Rust through **Claude Code + the skills pipeline**: the
human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI,
and reports metrics. Declarative config (`Aura.toml`) carries only **static
project context** (data paths, instrument/pip metadata, defaults, runs dir),
never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a
*runtime data source* (news-agent bias, C11), gated by per-session consent,
never in the code path.
**Forbids.** An in-app LLM chat that generates node code inside aura; using
IONOS (weaker models) as the authoring brain.
**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's
failure; making weaker models the coding brain reintroduces the very problem.
Keeps aura's scope an engine + playground, not an LLM-IDE.
### C18 — Project management: one repo = one project, plus a run registry
**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git
(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as
the forward-queue, a research thrust = a milestone, the
`idea → experimental → validated → deployed` label gradient). (2) **Experiments
& results:** an Aura-native **run registry** — one record per run = a *manifest*
(node-commit + params + data-window + seed + broker profile) + *metrics*,
queryable, with *lineage* (composite ← signals; run ← inputs). Determinism
(C1/C12) makes a run reproducible from its tiny manifest, so the registry stores
manifests + metrics and re-derives full results on demand. Depth: **structured**
(promotion/status, lineage, run-diff).
**Forbids.** Storing results not reproducible from a recorded manifest;
duplicating git/Gitea inside aura; a multi-project workspace manager.
**Why.** Comparing experiments over time is the heart of the research loop and
has no home in git/Gitea; determinism makes a structured registry cheap.
Sequencing: the walking skeleton emits a manifest + metrics per run from day
one; the registry/index is a later milestone over manifests that already exist.
### C19 — Bootstrap: blueprint → instance (recursive)
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
graph-as-data produced by running a Rust builder (C9); it carries *free* numeric
params (declared ranges) and *free* input roles. The **bootstrap** binds
`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen
instance** — buffers sized, topology fixed. This is precisely the "wiring /
graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12
("params injected at graph build") already reference. The same machinery applies
recursively up to the harness (C20). A sweep builds many instances from one
blueprint; instances are disjoint (C1).
**Forbids.** Params that change topology (a topology change is a *different*
blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a
sim against an un-bootstrapped blueprint.
**Why.** Separating the param-generic blueprint from the param-bound instance is
what makes one strategy reusable across a whole sweep and lets the optimizer
mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead
of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12
explicit.
### C20 — Strategy ↔ harness; the harness is the root sim graph
**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9):
broker-, data-, and viz-independent, with inputs declared as named **roles**
(symbol-agnostic where possible) and the intent/exposure stream (C10) as output.
A **harness** (the experimental setup) is the **root sim graph** — sources bound
to the strategy's input roles + the strategy + attached broker node(s) + sinks —
and is itself produced by the bootstrap (C19). A harness *instance* is C1's
disjoint unit (RustAst's "root scope"). The harness has **two kinds of
parameterization**: **structural axes** (which strategy, which instrument(s),
which broker(s), which window) whose variation selects *different* instances —
the **experiment matrix**; and **tuning params** (the strategy's numeric params)
swept *within* a fixed structure (Fork A). The same strategy blueprint is reused
across backtest, sweep, visual workspaces, and the frozen live bot — each a
different harness. **Both strategy and harness/experiment are authored in Rust**
via builder APIs (C17); the experiment matrix is ordinary Rust control flow
(loops/conditionals), not a config schema. Ontologically, a node (incl. a
strategy composite) is an **open** fragment — free input roles + ≤1 output
(C8/C9) — that does not run alone; a **harness is the closed root graph**: a
strategy with its input roles bound to sources and its output terminated in
broker/sink nodes, under a clock. A harness is therefore **not a node** (no free
inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's
disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21)
orchestrates them as objects.
**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a
declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as
a node (it is the closed root scope, not an open composable node).
**Why.** Reusability needs the strategy to be a context-free blueprint that many
harnesses embed. Modelling the harness as a root graph keeps it within the one
Node/graph abstraction (C9) and makes "10 strategies in one environment" and
"one strategy × N instruments" plain nested loops over the structural axes. Rust
authoring (not config) preserves full programmatic power — conditional/adaptive
matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap
C17 rejects.
### C21 — The World: the meta-level is the product
**Guarantee.** Above the harness sits the **World** — the project's program /
"game" (C16: a Rust crate). Within it, **harnesses are dynamically constructible,
first-class objects**: meta-programs (walk-forward, sweep, optimize,
Monte-Carlo — C12's axes) construct harness instances at runtime via the
bootstrap (C19), run them disjointly in parallel (C1), aggregate / compare their
results, and discard the transient instances. A walk-forward rolls windows →
bootstraps a harness per window → stitches out-of-sample equity + parameter
stability into one meta-result. Orchestrating *families* of harnesses is
**first-class**, not a headless afterthought; the run registry (C18) is the
World's memory.
**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep /
comparison) to second-class headless-only status; treating the single backtest
as the product.
**Why.** What happens *within* one harness — backtest a strategy → equity — is
commodity; every quant system covers it. aura exists for the meta-level: a
programmable space where harnesses are dynamically built and families of them
orchestrated and explored. The deterministic single-harness engine (C1C20) is
the **substrate**; the World is the **product**.
### C22 — The playground is a trace explorer; sinks are the recording mechanism
**Guarantee.** The World is a *program*: nothing is displayable until it runs,
and its harnesses are transient machinery (built, run, discarded — C21). The only
durable, displayable substance is the **recorded trace**, captured by **sinks**
pure consumer nodes (C8) that persist a stream (equity, position events, a node's
output) into the run registry (C18). **Displayable = exactly what a sink
recorded**; with no sink, only input params + summary metrics remain. The
**playground** is therefore an **execution viewer / trace explorer**, and it
plays *any* harness (it is the harness *player*, not a harness, never bound to a
default one): before a run it shows the program *structure* (graph-as-data, C9) +
param knobs; during a run, live sink streams; after a run, recorded traces +
metrics from the registry — including meta-views (stitched walk-forward, sweep
surfaces, multi-strategy / instrument comparison). Observability is **explicit
and selective** — you instrument what you want to see; the choice of sinks is
part of the experiment. The engine ships sample harnesses *with* sinks so a
newcomer sees a populated trace immediately; a fresh project is empty until
built, run, and instrumented.
**Forbids.** A scene-editor model that assumes a persistent populated world;
constructing or wiring topology in the UI (topology is Rust + hot-reload,
C9/C17 — the UI reflects the live graph-as-data and tunes runtime params via
sliders, C12); retaining un-sinked data; binding the playground to a single /
default harness.
**Why.** A program has no scene-at-rest to inspect; what persists is what it
records. Making sinks the one recording-and-observability mechanism keeps "what
can I see?" answerable by "what did I instrument?", and keeps the engine
UI-agnostic (C14). Live param tuning (runtime values, no topology change —
C12 / C19 Fork A) gives the interactive feel without a wiring DSL.
**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the
substrate level: a recorded trace is exactly what a recording node pushed out of
the graph (no engine recording registry; the constructing World holds each
recording node's destination). The engine's single `observe: usize` affordance is
removed — `Harness::run` returns `()` and recording is a node-side concern, so one
run records *many* streams (one per recording node) instead of exactly one row.
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
`Harness` API change (observe removed, `run -> ()`) is recorded here.
---
## Open architectural threads not yet resolved
- **Playground & World UI surface** — the playground is core (C22), egui-native;
open is the concrete UI of the meta-views (walk-forward, sweep surfaces,
comparison) and the run/replay clock controls.
- **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies
atop the atomic sim unit (C12), not yet designed.
- **`aura new` scaffolder, the experiment-builder API, and `Aura.toml`'s
static-context schema** — `aura new` scaffolds a Rust project *crate* (node /
strategy / experiment blueprints) against the engine (C16/C20); the
experiment-builder API surface (harness wiring, structural axes, sweep
combinators) and `Aura.toml`'s schema (data paths, instrument/pip metadata,
default broker & window, runs dir) are not yet designed.
- **`aura-std` contents** — the crate exists (doc-only); which universal blocks
land first follows the walking-skeleton's needs.
- **`strategies/` split** — a later split, *inside a project*, of top-level
strategies from reusable building blocks in `nodes/`; not a day-1 cut.
- **Sequencing** — the runnable single-harness *substrate* comes first (walking
skeleton: a closed harness that runs deterministically and records via a sink);
the World/meta layer (C21) and the playground trace-explorer (C22) are the
differentiating layers that follow — you orchestrate and visualize a thing that
must first run once.