From 8688a60deda149ab252bb47329acabe9962e4a60 Mon Sep 17 00:00:00 2001 From: claude Date: Tue, 21 Jul 2026 16:40:36 +0200 Subject: [PATCH] docs(ledger): split the design ledger into an INDEX map, per-contract live files, and history sidecars The single-file ledger had grown to 2968 lines / ~42k tokens, mixing current design law with accreted history: 59 cycle-stamped realization blocks, 18 [HISTORY] passages, 22 supersession markers, and the C10 / C22 / C24 reframe sagas layered several supersessions deep. A code-grounding audit (31 agents, adversarially verified) confirmed 11 defects stated as current truth: stale crate homes from the C28 #288 roster split (cost nodes, PositionManagement, PositionEvent, Session), the renamed InputSpec->PortSpec, the pre-#241 project model in C16 and the open-threads section, a stale HarnessKind retirement deferral in C24, and three C28-internal inconsistencies. New shape, per the ailang precedent: - INDEX.md stays the sole addressable entry point: foundation, external components, a C-id-keyed contract map (one line per contract), and only the genuinely open architectural threads. - contracts/cNN-.md carries each contract's current truth only: Guarantee / Forbids / Why with ratified refinements integrated, plus a code-anchored Current state. All confirmed defects are fixed here; crate anchors were re-verified against the tree. - contracts/cNN-.history.md (18 sidecars) and INDEX.history.md preserve every superseded block verbatim, stamps and issue refs intact, under a frozen-record banner. Nothing was deleted: superseded design intent remains an addressable working-tree artifact, off the per-cycle audit walk. - Ledger discipline is now stated in INDEX.md: live files are edited in place at cycle close, superseded text moves verbatim to the sidecar, and a supersession marker in a live file is itself an audit finding. Every contract file was verified against its old text by an independent zero-loss pass (statement-by-statement) plus a code-accuracy spot check; C-ids and contract titles are unchanged, so existing C-id citations in code, tests, and issues resolve as before. --- docs/design/INDEX.history.md | 233 ++ docs/design/INDEX.md | 3100 +---------------- .../contracts/c01-determinism.history.md | 25 + docs/design/contracts/c01-determinism.md | 69 + docs/design/contracts/c02-causality.md | 36 + docs/design/contracts/c03-single-merge.md | 36 + .../c04-cycle-granularity.history.md | 15 + .../design/contracts/c04-cycle-granularity.md | 43 + docs/design/contracts/c05-freshness-gating.md | 35 + .../contracts/c06-firing-policy.history.md | 14 + docs/design/contracts/c06-firing-policy.md | 42 + .../contracts/c07-scalar-soa.history.md | 32 + docs/design/contracts/c07-scalar-soa.md | 64 + .../contracts/c08-node-contract.history.md | 159 + docs/design/contracts/c08-node-contract.md | 146 + .../c09-fractal-composition.history.md | 85 + .../contracts/c09-fractal-composition.md | 88 + .../contracts/c10-bias-r-cost.history.md | 386 ++ docs/design/contracts/c10-bias-r-cost.md | 361 ++ .../contracts/c11-sources-record-replay.md | 39 + .../contracts/c12-atomic-sim-unit.history.md | 36 + docs/design/contracts/c12-atomic-sim-unit.md | 82 + .../c13-hot-reload-frozen-deploy.history.md | 25 + .../contracts/c13-hot-reload-frozen-deploy.md | 63 + .../c14-headless-two-faces.history.md | 29 + .../contracts/c14-headless-two-faces.md | 89 + .../c15-resampling-sessions.history.md | 38 + .../contracts/c15-resampling-sessions.md | 55 + .../c16-engine-project-split.history.md | 60 + .../contracts/c16-engine-project-split.md | 100 + .../design/contracts/c17-authoring-surface.md | 52 + docs/design/contracts/c18-registry.history.md | 273 ++ docs/design/contracts/c18-registry.md | 287 ++ .../design/contracts/c19-bootstrap.history.md | 118 + docs/design/contracts/c19-bootstrap.md | 118 + .../contracts/c20-strategy-harness.history.md | 36 + docs/design/contracts/c20-strategy-harness.md | 95 + docs/design/contracts/c21-world.md | 59 + .../c22-playground-traces.history.md | 137 + .../design/contracts/c22-playground-traces.md | 144 + .../design/contracts/c23-graph-compilation.md | 77 + .../contracts/c24-blueprint-data.history.md | 160 + docs/design/contracts/c24-blueprint-data.md | 300 ++ docs/design/contracts/c25-role-model.md | 90 + docs/design/contracts/c26-input-binding.md | 73 + docs/design/contracts/c27-declared-taps.md | 69 + .../contracts/c28-stratification.history.md | 85 + docs/design/contracts/c28-stratification.md | 179 + 48 files changed, 5013 insertions(+), 2924 deletions(-) create mode 100644 docs/design/INDEX.history.md create mode 100644 docs/design/contracts/c01-determinism.history.md create mode 100644 docs/design/contracts/c01-determinism.md create mode 100644 docs/design/contracts/c02-causality.md create mode 100644 docs/design/contracts/c03-single-merge.md create mode 100644 docs/design/contracts/c04-cycle-granularity.history.md create mode 100644 docs/design/contracts/c04-cycle-granularity.md create mode 100644 docs/design/contracts/c05-freshness-gating.md create mode 100644 docs/design/contracts/c06-firing-policy.history.md create mode 100644 docs/design/contracts/c06-firing-policy.md create mode 100644 docs/design/contracts/c07-scalar-soa.history.md create mode 100644 docs/design/contracts/c07-scalar-soa.md create mode 100644 docs/design/contracts/c08-node-contract.history.md create mode 100644 docs/design/contracts/c08-node-contract.md create mode 100644 docs/design/contracts/c09-fractal-composition.history.md create mode 100644 docs/design/contracts/c09-fractal-composition.md create mode 100644 docs/design/contracts/c10-bias-r-cost.history.md create mode 100644 docs/design/contracts/c10-bias-r-cost.md create mode 100644 docs/design/contracts/c11-sources-record-replay.md create mode 100644 docs/design/contracts/c12-atomic-sim-unit.history.md create mode 100644 docs/design/contracts/c12-atomic-sim-unit.md create mode 100644 docs/design/contracts/c13-hot-reload-frozen-deploy.history.md create mode 100644 docs/design/contracts/c13-hot-reload-frozen-deploy.md create mode 100644 docs/design/contracts/c14-headless-two-faces.history.md create mode 100644 docs/design/contracts/c14-headless-two-faces.md create mode 100644 docs/design/contracts/c15-resampling-sessions.history.md create mode 100644 docs/design/contracts/c15-resampling-sessions.md create mode 100644 docs/design/contracts/c16-engine-project-split.history.md create mode 100644 docs/design/contracts/c16-engine-project-split.md create mode 100644 docs/design/contracts/c17-authoring-surface.md create mode 100644 docs/design/contracts/c18-registry.history.md create mode 100644 docs/design/contracts/c18-registry.md create mode 100644 docs/design/contracts/c19-bootstrap.history.md create mode 100644 docs/design/contracts/c19-bootstrap.md create mode 100644 docs/design/contracts/c20-strategy-harness.history.md create mode 100644 docs/design/contracts/c20-strategy-harness.md create mode 100644 docs/design/contracts/c21-world.md create mode 100644 docs/design/contracts/c22-playground-traces.history.md create mode 100644 docs/design/contracts/c22-playground-traces.md create mode 100644 docs/design/contracts/c23-graph-compilation.md create mode 100644 docs/design/contracts/c24-blueprint-data.history.md create mode 100644 docs/design/contracts/c24-blueprint-data.md create mode 100644 docs/design/contracts/c25-role-model.md create mode 100644 docs/design/contracts/c26-input-binding.md create mode 100644 docs/design/contracts/c27-declared-taps.md create mode 100644 docs/design/contracts/c28-stratification.history.md create mode 100644 docs/design/contracts/c28-stratification.md diff --git a/docs/design/INDEX.history.md b/docs/design/INDEX.history.md new file mode 100644 index 0000000..dbd30d9 --- /dev/null +++ b/docs/design/INDEX.history.md @@ -0,0 +1,233 @@ +# aura design ledger — INDEX history (frozen record) + +> FROZEN HISTORICAL RECORD. Each block below was true as of its stamp and may +> be superseded; this file is NOT current truth and NOT a grounding surface. +> Current ledger: [INDEX.md](INDEX.md). Per-contract history lives in +> `contracts/*.history.md`. + +## Header provenance and vocabulary gloss (pre-refactor wording) + +Moved out of the INDEX header in the 2026-07-21 ledger refactor; the +walking-skeleton milestone and the C1–C18 interview are completed history. + +Provenance: contracts C1–C18 were settled in the initial rough-sketch design +interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the +C10 refinement to a broker-independent position table came in follow-up turns; +C10 was reframed in cycle 0007 to an exposure stream, then again 2026-06-23 +(#117) to a **bias** stream with signal quality in **R**, the position table a +derived layer — see C10). C19–C22 were added as the +construction / World / playground layer; C23 and the C9/C19 compilation +refinements were settled 2026-06-05 for the **Construction-layer** milestone (the +blueprint→flat-graph reading of composites and graph optimisation). C24 (the +blueprint as a serializable, World-owned data value) was settled 2026-06-29, +resolving the long-deferred #109 fork in favour of **topology-as-data** (the +game-engine principle, C16): the engine owns topology as content it serializes / +loads / generates, not as Rust source baked into the binary. + +Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a +*milestone* is a tracker container spanning many cycles (the first milestone is +the **walking skeleton**: ingest → one signal → exposure → sim-optimal broker → +synthetic pip-equity signal-quality metric — the *pre-reframe* substrate, now +reframed to bias → R-evaluator → R-expectancy, see C10). + +## External components — the RustAst pre-authoring reading list + +Written before `aura-core` existed as a guide for authoring it; `aura-core` +has long been built, so the file-by-file reading list is a historical aid. +RustAst's living role (conceptual reference, never a dependency) stays in +[INDEX.md](INDEX.md). + + But its **`src/ast/rtl/` layer is a working reference implementation of the very + streaming substrate aura rebuilds**, and is worth reading before authoring + `aura-core` — these are not just concepts, they exist as code: + - `rtl/series/data.rs` — `RingBuffer` with financial-style indexing (index 0 + = newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's + pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no + String/Record" = C7's closed scalar set); `ScalarSeries` and the + **SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns"). + - `rtl/series/mod.rs` — `create_typed_series`, dispatching element type → storage + backend (the type-specialized **factory** of C19). + - `rtl/streams/mod.rs` — `Signal { cycle_id, value }` (C4's cycle clock) and the + `Stream` / `Observer` / `ObservableStream` push traits (the reactive model of + C4/C5). + - `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one + explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator + (C12's seed-as-input). + + aura reimplements these natively and **sharpens** them: types are monomorphized + and edges type-erased to the four scalar kinds with direct dispatch (C7) instead + of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>` + (C12) instead of a `VecDeque`. RustAst shows the *shape*; aura makes it + fast and deterministic. + +## Open threads — resolved or superseded entries (pre-refactor wording) + +Each entry below is the pre-refactor text. Where a thread stays genuinely +open, INDEX.md carries a terse live successor; resolved threads live only +here (their outcomes are recorded in the named contracts). + +- **Playground & World UI surface** — the playground is core (C22); the surface + is a **web frontend served from disk-persisted traces** (revised 2026-06, issue + #101; not egui). Settled for the first cut: raw per-tap trace persistence to + disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static + self-contained HTML chart page (uPlot, vendored like the `render_html` + Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The + families-comparison **view** — `aura chart ` overlaying one tap across a + family's members on a shared y-scale — shipped (#107, see the C22 amendment). + Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family / + multi-strategy comparison, multi-column tap selection #47), a local server, and the + run/replay clock controls. + +- **`aura new` scaffolder and the experiment-builder API** — `aura new` + scaffolds a Rust project *crate* (node / strategy blueprints) against the + engine (C16/C20); the experiment-builder API surface (harness wiring, + structural axes, sweep combinators) is not yet designed. `Aura.toml`'s + schema was settled paths-only in cycle 0102 (data archive root, runs dir — + see the C13 realization note); the load boundary itself shipped there. + +- **The analysis meta-level — composable orchestration + project-as-crate authoring + (tracked: #109).** aura's differentiator (C21) had two unbuilt halves: (1) the + orchestration axes (sweep / Monte-Carlo / walk-forward) becoming *composable tools* + a user wires into an analysis workflow, rather than today's hard-wired CLI verbs + (`sweep_family` / `walkforward_family` in `aura-cli`) — still open; and (2) the + project-as-crate authoring layer above — its **load boundary landed in cycle 0102** + (`Aura.toml` discovery + cdylib loading + merged vocabulary, C13 realization note) + and the **`aura new` scaffolder in cycle 0103**; of that layer only the + experiment-builder API remains open (C16/C17). + On the meta-level, authoring a strategy is *wiring + existing nodes* (C9 composition) + *defining the analysis framework* (C21), not + writing new nodes; the user wants it driven via the CLI, later the interactive + server / playground (C14/C22). **Resolved (2026-06-29) → C24.** The fork was + mis-posed as "(a) run-a-Rust-experiment-via-CLI vs (b) wire-nodes-via-CLI". The real + axis is **topology = compile-time Rust source vs runtime, serializable, World-owned + data value**, settled as the **value** (C24, the game-engine principle): node + *logic* stays Rust (C17), *topology* becomes data the World generates / serializes / + reproduces. "Wire-nodes-via-CLI" is dropped — interactive wiring through the CLI is a + non-goal; it collapses to a file + parser, which *is* C24's load path. What + **remains** under #109: the **blueprint data format** itself (C24 Status — its own + brainstorm / milestone), and the **composable-orchestration** half — the axes + (sweep / MC / walk-forward) becoming tools applied to a World-constructed harness + instead of the hard-wired `aura-cli` verbs (`sweep_family` / `walkforward_family`). + **Status (cycle 0106, #189 — the artifact half shipped v1).** The #188 role-model pass + re-cut the "experiment-builder API" reading of this thread (role-6b work mis-typed in + role-2 technique): experiment intent is now **data**, not a Rust API. Cycle 0106 shipped + both document vocabularies (process document, campaign document — `aura-research`), + two-tier validation, the op-script-precedent introspection contract + (`aura process|campaign introspect --vocabulary/--block/--unwired/--content-id`), and + content-addressed stores (C18 realization note) — authorable and checkable headless, no + compile, no run. **Status update (cycles 0107–0110).** The executor question resolved and + shipped (cycles 0107/0108, #198/#200: `aura-campaign` executes the v2 pipeline shape over + the `MemberRunner` seam); the #188 amendment package landed 2026-07-03 (C25 role-model + entry, C20/C22 refinements, invariant-10 clarification). The "once it carries" dissolution + of the hard-wired verbs (user decision 2026-07-03 on #188) is **running as the milestone + "Verb dissolution" (#210)**: the fork triage of 2026-07-04 decided its design (dissolve + the real-data blueprint branches only, built-in/synthetic branches stay verb-wired until + #159; generated documents auto-registered; full behaviour parity modulo the recorded + additive instrument stamp), and the verb set dissolved in the #210-decision-7 order: cycle 0110 took `aura sweep + --real …` (enabled by the `std::sweep` selection group becoming optional — + all-or-nothing, selection-free = terminal-stage-only; wire form and every stored content + id unchanged), then generalize, walkforward, and mc's R-bootstrap path, each now sugar over + a generated, content-addressed campaign document through the one executor, plus a structural + risk-regime axis (`CampaignDoc.risk: [RiskRegime]`, sole `vol{length,k}` variant — the stop + compared as a matrix dimension, stamped into every member manifest, never argmax-swept). + Ad-hoc verb intent no longer evaporates into shell history — the #188 diagnosis's cure + applied to the verbs themselves. **The dissolved form is per-verb, by intended scope** + (ratified at milestone close, fieldtest 2026-07-07): for sweep it is the blueprint file + (` --real`), while `aura sweep --strategy r-sma --real` stays the inline built-in + path (its member lines carry no instrument/topology_hash/selection stamp and register no + campaign document) — the built-in `--strategy` demo surface is #159's hard-wired-harness + retirement target, not the dissolution's; for generalize/walkforward/mc the dissolved + form is now the same generic grammar as sweep — an arbitrary blueprint positional plus + `--real` and repeatable `--axis =` (#220). [HISTORY — superseded in + two steps. #159 (cuts 1b-4, post-2026-07-07) retired the built-in `--strategy` + sweep/demo surface: the inline `aura sweep --strategy r-sma --real` path no longer + exists. #220 then removed the verbs' weld — at milestone close the dissolved form of + generalize/walkforward/mc was still the welded `--strategy r-sma --real`; #220 made + the three verbs blueprint-generic (arbitrary blueprint + arbitrary `--axis`; mc keeps + the synthetic `--seeds` family unchanged beside the new `--real --axis` campaign mode; + generalize takes `--real `, >=2, with one value per axis), deleted the + welded `--strategy` grammar (clap rejects it, exit 2), dissolved `RGrid`, and unified + the verbs on the #214 invocation struct. The surviving form everywhere is `aura + --real … --axis …` over examples/r_*.json.] Milestone closed 2026-07-07 on a green end-to-end fieldtest + (0 bugs; behaviour preservation, campaign-substrate reach-through, and the risk-regime axis + all confirmed); residual findings are discoverability/ergonomics follow-ups (#216 risk-axis + discoverability, #217 verb knob asymmetry, #218 no-project store litter). + **Amendment (2026-07-14, #256 fork B):** the dissolved walkforward/mc + translations' leading sweep became the enumerate-only **`std::grid`** stage — + a fieldless vocabulary block, legal only as the first stage and immediately + before `std::walk_forward`. Only the grid's parameter points ever crossed the + stage seam (the wf stage re-sweeps them per IS window itself), so the leading + stage no longer executes members or persists a `Sweep` registry family: + persisted dissolved-walkforward/mc campaign documents lose that family, while + the visible grades are byte-identical (the exact-grade E2E pins are + unchanged). The executor's inter-stage seam is now a typed two-armed flow + (points-only vs executed members); report-consuming stages are fenced by + preflight. `translate_generalize` keeps its executed `std::sweep(argmax)` + leading stage — its generalize stage consumes the argmax winner report as the + cell nominee. + +- **Inferential honesty of the World — family-selection without false-discovery + control (tracked: milestone "Inferential validation (defend against false + discovery at sweep scale)", #144 / #145 / #146; adjacent #139).** The World's + massively-parallel sweep (C12 axes 1–2, C21) is aura's differentiator *and* its + most direct route to false discovery: `optimize` / `rank_by` select the single + best family member by a bare argmax, with no penalty for the number of + configurations tried, no preference for a robust parameter neighbourhood over a + sharp in-sample peak, and no cross-instrument generalization read; `mc` is a + descriptive seed-sweep, not an inferential significance test. The hygiene + invariants keep one backtest honest (C1 determinism, C2 no look-ahead); they do + **not** make a *selection across a family* honest. Recognized direction: the + selection/aggregation layer must deflate a winner for the family size (#144), + prefer a plateau over a peak (#145), and score cross-instrument generalization + (#146), with a per-candidate out-of-sample bootstrap as the adjacent + significance read (#139, landed cycle 0075). Distinct from the two threads above — + neither orchestration *composability* (#109) nor a search *policy* (Bayesian/ + genetic), but the statistical *validity* of the selection itself. Not a + C-invariant until built; recorded as the World's third half — **now structurally + complete** (all three pieces built, cycle 0078). + **Status (cycle 0076):** the trials-deflation piece (#144) landed — + `optimize_deflated` (aura-registry) wraps `optimize` and records, additively on + the winning member's manifest (C18, the new `RunManifest.selection`), a deflated + score + an overfit probability for the family size, **without changing which + member wins** (C23; `optimize` / `rank_by` stay a bare argmax — the deflation is + recorded provenance, not a re-ranking). The R arm is a centred moving-block + reality-check (reusing the `r_bootstrap` kernel); the `total_pips` arm a + closed-form expected-max-of-K dispersion floor. **Status (cycle 0077):** the + plateau-over-peak piece (#145) landed — `optimize_plateau` (aura-registry) + argmaxes the **neighbourhood-smoothed** metric surface (mean or worst-case over + each member's closed mixed-radix grid neighbourhood) instead of the bare peak, + recorded on the same `RunManifest.selection` carrier, now an orthogonal *rule × + annotation*: `SelectionMode::{Argmax, PlateauMean, PlateauWorst}` with either the + deflation annotation (#144) or the plateau annotation (`neighbourhood_score` / + `n_neighbours`). It is opt-in via a `--select` flag — default argmax stays + byte-identical (C23). The grid lattice surfaces from the engine + (`SweepBinder::sweep_with_lattice`); the policy stays in aura-registry with + `walk_forward` selection-agnostic (C9). Selection is now a *pluggable objective* + (bare argmax → trials-deflated → plateau), not a single hard-wired argmax. + **Status (cycle 0078):** the last piece — cross-instrument generalization (#146) — + landed, completing the inferential half. Unlike #144/#145 (which *select + annotate* + a within-family winner on `RunManifest.selection`), #146 is an **aggregator/ + validator**, not a selector: the `aura generalize` subcommand runs a *brought* + candidate across an instrument list and `generalization` (aura-registry) reduces the + per-instrument R-metrics to a **worst-case floor** (min over instruments — the + cross-instrument sibling of `PlateauMode::Worst`, R-only per C10) + a sign-agreement + count + the per-instrument breakdown. It is a *recomputable aggregate* over a new + `FamilyKind::CrossInstrument` family (C12's comparison axis, realized), each member + self-identifying via the new `RunManifest.instrument` lineage field (C18) — *not* + stamped on `FamilySelection` (there is no within-family winner to annotate). The + inferential half is now structurally built; the per-candidate OOS bootstrap (#139, + cycle 0075) is its adjacent significance read. + +- **`aura-std` contents** — substantively populated (~30 node modules): SMA/EMA, + arithmetic (`Add`/`Sub`/`Mul`/`Sqrt`/`LinComb`), logic (`And`/`Gt`/`Latch`/`EqConst`), + the `Delay` z⁻¹ register, `Resample`, `RollingMin`/`RollingMax`, `Session`, the R chain + (`Bias`/`FixedStop`/`Sizer`/`PositionManagement`), the legacy `SimBroker` pip yardstick, + the cost-model graph (`CostNode`/`CostRunner`/`CostSum` + `ConstantCost`/`VolSlippageCost`/ + `CarryCost`), and the `Recorder`/`GatedRecorder`/`SeriesReducer` sinks. Further blocks + land demand-driven as the walking-skeleton needs them. + +- **Sequencing** — the runnable single-harness *substrate* comes first (walking + skeleton: a closed harness that runs deterministically and records via a sink); + the World/meta layer (C21) and the playground trace-explorer (C22) are the + differentiating layers that follow — you orchestrate and visualize a thing that + must first run once. diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index 7414138..1790c51 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -2,32 +2,32 @@ The ledger records the load-bearing design contracts and their rationale. Each contract states what it **guarantees**, what it **forbids**, and **why**. A -change that breaks a contract is a design decision (amend the contract here with -its new rationale), never a silent refactor. +change that breaks a contract is a design decision (amend the contract with its +new rationale), never a silent refactor. -Provenance: contracts C1–C18 were settled in the initial rough-sketch design -interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the -C10 refinement to a broker-independent position table came in follow-up turns; -C10 was reframed in cycle 0007 to an exposure stream, then again 2026-06-23 -(#117) to a **bias** stream with signal quality in **R**, the position table a -derived layer — see C10). C19–C22 were added as the -construction / World / playground layer; C23 and the C9/C19 compilation -refinements were settled 2026-06-05 for the **Construction-layer** milestone (the -blueprint→flat-graph reading of composites and graph optimisation). C24 (the -blueprint as a serializable, World-owned data value) was settled 2026-06-29, -resolving the long-deferred #109 fork in favour of **topology-as-data** (the -game-engine principle, C16): the engine owns topology as content it serializes / -loads / generates, not as Rust source baked into the binary. -The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded -summary of the -subset that agents must never violate; this file is the fuller form with -rationale. +This file is the sole addressable entry point. Each contract lives in its own +file under [`contracts/`](contracts/) as `cNN-.md`; durable artifacts +cite contracts by C-id (`C10`), which resolves here. Contracts C1–C18 were +settled in the initial design interview (2026-06-03); later contracts carry +their ratification refs inline. The `CLAUDE.md` **Domain invariants** section +is the compressed, always-loaded summary of the subset agents must never +violate; the contract files are the fuller form with rationale. -Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a -*milestone* is a tracker container spanning many cycles (the first milestone is -the **walking skeleton**: ingest → one signal → exposure → sim-optimal broker → -synthetic pip-equity signal-quality metric — the *pre-reframe* substrate, now -reframed to bias → R-evaluator → R-expectancy, see C10). +**Ledger discipline** (adopted with the 2026-07-21 refactor): + +- A live contract file carries **current truth only** — present-tense design + law plus a code-anchored *Current state*. A supersession marker + (`[HISTORY]`, "superseded", "pre-reframe") appearing in a live file is + itself an audit finding: it signals a move to the history sidecar was + skipped. +- **At cycle close**, realization notes are edited **in place** in the + contract's *Current state*; text they supersede moves **verbatim** (stamps + and issue refs intact) to the contract's `cNN-.history.md` sidecar, + appended newest-last. The sidecars — not specs, plans, or git archaeology — + are the durable record of superseded design intent. +- **The audit walk** is this file plus every `contracts/cNN-*.md`. History + sidecars are read on demand (why-questions, retired-design lookups), never + as part of the per-cycle drift walk, and are never a grounding surface. --- @@ -43,2926 +43,178 @@ The predecessor RustAst (`myc`) tried this as a custom DSL and failed: too slow, too buggy, and LLMs author far better in Rust than in an unfamiliar DSL. aura inverts it — engine in Rust, strategies in Rust — but keeps RustAst's *concepts* (synchronous reactive streams, bounded-lookback series, run-counting, SoA). -RustAst is a conceptual reference, not a dependency. The one reused component is -`data-server` (the first data source). What sets aura apart is **not** the single backtest — every quant system does that — but the **World**: the meta-level where harnesses are dynamically constructed and *families* of them orchestrated and explored (walk-forward, sweeps, Monte-Carlo, comparison). The deterministic single-harness engine is the -*substrate*; the World is the *product* (C20–C22). - ---- +*substrate*; the World is the *product* (C20–C22, C24). ## External components -Two sibling projects live outside this repo and are named throughout the -contracts. Their concrete location is recorded here so the repo is the single -source of truth — not session memory. +Two sibling projects live outside this repo. Their location is recorded here so +the repo, not session memory, is the source of truth. - **`data-server`** — `~/dev/libs/data-server`, Gitea `Brummel/data-server` (`http://192.168.178.103:3000/Brummel/data-server.git`). aura's **first data - source** and the one reused component (Foundation; C3, C11, C12). A standalone - leaf crate (deps: `chrono`, `regex`, `zip`) that loads Pepperstone **M1/tick** - binary files and shares them lock-free as `Arc<[T]>` chunks - (`CHUNK_SIZE = 1024`) via `SymbolChunkIter::next_chunk` (an inherent method - driven by `while let`, not the `Iterator` trait); `stream_*_windowed(from_ms, - to_ms)` provides C12's data-window with inclusive Unix-ms bounds. Records are - **AoS** (`M1Parsed`/`TickParsed`, `time_ms: i64` Unix-ms). The ingestion - boundary (C3) therefore **transposes** these AoS records into aura's SoA - columns (C7) and normalizes `time_ms` → canonical epoch-ns at that one - boundary. Pulled in as a cargo git dependency by the **`aura-ingest`** crate - (cycle 0011) — the **data-source ingestion edge**, where the `data-server` - external tree (and its transitive `chrono`/`regex`/`zip`) enters the workspace. - Under the amended C16 **per-case dependency policy** (cycle 0029), `aura-ingest` - is no longer a zero-dep *firewall* for the rest: the engine crates link - standard vetted crates where they earn their place (e.g. `serde`/`serde_json` - for the run registry, cycle 0029), with particular scrutiny for the frozen - deploy artifact (C13). Consequence: `cargo build/test --workspace` resolves - from a populated cargo cache (a Gitea fetch for `data-server` + crates.io for - the standard crates) rather than fully offline. -- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst` - (`http://192.168.178.103:3000/Brummel/RustAst.git`). The predecessor DSL attempt - (Foundation): **a conceptual reference, never a dependency** — its DSL authoring - surface and `Value`/HM-inference machinery are exactly what aura rejects (C17). - But its **`src/ast/rtl/` layer is a working reference implementation of the very - streaming substrate aura rebuilds**, and is worth reading before authoring - `aura-core` — these are not just concepts, they exist as code: - - `rtl/series/data.rs` — `RingBuffer` with financial-style indexing (index 0 - = newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's - pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no - String/Record" = C7's closed scalar set); `ScalarSeries` and the - **SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns"). - - `rtl/series/mod.rs` — `create_typed_series`, dispatching element type → storage - backend (the type-specialized **factory** of C19). - - `rtl/streams/mod.rs` — `Signal { cycle_id, value }` (C4's cycle clock) and the - `Stream` / `Observer` / `ObservableStream` push traits (the reactive model of - C4/C5). - - `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one - explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator - (C12's seed-as-input). - - aura reimplements these natively and **sharpens** them: types are monomorphized - and edges type-erased to the four scalar kinds with direct dispatch (C7) instead - of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>` - (C12) instead of a `VecDeque`. RustAst shows the *shape*; aura makes it - fast and deterministic. + source**: a standalone leaf crate that loads Pepperstone **M1/tick** binary + archives and shares them lock-free as `Arc<[T]>` chunks; records are **AoS** + with Unix-ms timestamps, transposed to aura's SoA columns and normalized to + canonical epoch-ns at the ingestion boundary (C3/C7). It enters the workspace + as a cargo git dependency at exactly the seams C28 pins: the ingestion edge + (`aura-ingest`), the assembly position (`aura-runner`), and the shell. +- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst`. The + predecessor DSL attempt: **a conceptual reference, never a dependency** — its + DSL authoring surface and `Value`/HM-inference machinery are exactly what aura + rejects (C17). Its `src/ast/rtl/` layer served as the reference + implementation for `aura-core`'s streaming substrate; the file-by-file + reading list from the pre-authoring era is preserved in + [INDEX.history.md](INDEX.history.md). --- ## Contracts -### C1 — Determinism and disjoint parallelism -**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event -loop that reaches a unique state after each input tick. Same input (incl. seed) -→ bit-identical run. Two backtests are fully disjoint and run concurrently -without locking. -**Scope (ratified, fieldtest 0078).** The bit-identity is *per run* — one backtest -of given (inputs, seed) reproduces byte-for-byte. A *derived metric* recomputed for -the same params by two *different command paths* (e.g. a swept member's `sqn` vs the -same cell re-run under `aura generalize`) may differ by floating-point reassociation -(≤1 ULP), since the two paths accumulate the same logical reduction in a different -operation order (IEEE-754 non-associativity). This is not a C1 violation: C1 governs -the determinism of a single run, not the cross-command bit-identity of a re-derived -statistic. -**Realization note (2026-07-16, #277).** Cross-sim parallelism now also spans -campaign cells: the executor flattens the cell matrix, groups it by instrument -ordinal, and walks sequential chunks of K instrument groups -(`--parallel-instruments`, default 4 — a structural bound on distinct resident -instruments, the RAM lever for the external data-server's per-reference file -retention); within a chunk, cells run concurrently on the process-global rayon -pool shared with the member/window fan-out. Results are collected into -document-order slots, so outputs stay byte-identical across worker counts and -bounds. Two deliberate scheduling-dependent carve-outs, both outside this -contract's per-run bit-identity (which governs successful runs): on the -run-fatal path (non-containable faults, e.g. a dead registry store) the -propagated fault is the lowest document-order fault among the cells that -completed before the abort flag latched, and the set of per-cell family lines -already written by then is scheduling-dependent — inert, because no -campaign-run record is written on that path and store reads are name-keyed, -never line-ordered. Duplicate campaign instruments are refused at both the -validate tier and the executor's preflight: the per-cell family name embeds -the raw instrument string, so uniqueness is what keeps concurrent appends from -racing one name's run-index assignment. -**Forbids.** Concurrency *within* a single sim; any nondeterministic input that -is not captured as an explicit input (see C11, C12). -**Why.** Real money rides on backtest results; reproducibility and an audit -trail are non-negotiable. Speed comes from parallelism *across* sims, which -disjointness makes lock-free. - -### C2 — Causality / no look-ahead -**Guarantee.** A node sees only the past. Input history is a read-only window -that ends at the current cursor; a resampler emits a bar only once it is -complete. -**Forbids.** Any node access to data with `timestamp > now`; emitting a partial -/ still-forming bar. -**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that -leaks the future is worse than none. Making the future *physically absent* from -what a node receives beats merely discouraging it. - -### C3 — One merge, at ingestion only -**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp -into one chronological cycle stream at the ingestion boundary; source-native -time units (e.g. data-server's Unix-`time_ms`) are normalized there to the -canonical epoch-ns `timestamp` of C7. -**Forbids.** Any merge / as-of join *inside* the graph. -**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate -sources (news daily-bias + M5 + ticks) causally combinable without leaking the -future. Keeping the merge at one boundary keeps the graph semantics simple. - -### C4 — Cycle granularity -**Guarantee.** The clock is data-driven: one input record = one cycle, advanced -in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp, -multiple sources) break by source declaration order. -**Forbids.** A fixed time-grid clock; nondeterministic tie ordering. -**Why.** The market *is* an irregular event sequence; a grid is arbitrary and -either wastes empty cycles or clumps ticks. Backtest and live differ only in the -origin of records, not the cycle semantics. Tie determinism preserves C1. - -**Realization (#275, 2026-07-15).** Ingestion sources are supplied to `run` **by -role name**, not by list position. `SourceSpec` carries a `role: Option` -(the bound `Role`'s name, load-bearing for source binding), and `Harness::run_bound` -resolves a keyed supply against those roles, emitting sources in `SourceSpec` -declaration order. The C4 tie-break stays "source declaration order" — now -independent of how the caller orders the supply — and a mis-bound feed is a named -`SourceBindError` at start time rather than a silently wrong run. The raw-index -`run(Vec)` primitive (positional, C23) is unchanged; every hand-built graph keeps -`role: None`. - -### C5 — Freshness-gated recompute and sample-and-hold -**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node -re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by -run-count); otherwise it holds its last output. Stale inputs contribute their -last (held) value. -**Forbids.** Recomputing every node every cycle ("push all" is true for the -*clock*, not for *recompute"); treating a held value as missing. -**Why.** Total recompute does not scale to many sparse high-frequency sources; -freshness-gating is the performance discipline that keeps the synchronous model -fast. - -### C6 — Firing policy A and B, per input group -**Guarantee.** A node declares, per input group, one of two firing policies: -**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held -daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from -four separate 15m sources: the candle is complete only when all four are fresh). -A single node may mix an A input and a B group. -**Forbids.** A single global firing mode; forcing per-node-only granularity. -**Why.** Both are genuinely needed; RustAst implemented only B. Per-input-group -granularity is required by the OHLC-plus-bias case where one node needs both. -**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any, -Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a -mode-A input is its own trivial group, so "per input group" and the per-input tag -coincide. The barrier's synchronization token is the cycle **timestamp**, not the -`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so -RustAst's `cycle_id`-equality barrier could never fire across them. A group fires -when every member's last push carries the current cycle's timestamp, guarded by -"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This -fires both the multi-source bar and the within-source diamond rejoin (every push -in a cycle carries that cycle's timestamp). - -### C7 — Four scalar base types, streamed as SoA -**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64, -epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams -(OHLCV) are bundles of base columns — this is the **node-output model** too: a -node emits a record of 1..K base columns (C8), each forwarded field-wise to a -consumer slot; the bundle is structural, never a fifth scalar type. Edges are -type-erased to these four kinds; -the type check is paid once at wiring/sim-start, then the topology is frozen per -sim → direct dispatch, no per-event allocation. -**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those -live as metadata beside the hot path; `dyn Any` payloads; per-event heap -allocation; topology mutation mid-sim. -**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar -set and SoA. The open set is composites (schemas of columns), not scalar types. -Type-erasure at the edge is also forced by the cdylib boundary (C13). -**Realization (`Cell` carrier split, 2026-06).** The streamed value is now split -into a tag-free 64-bit word and its kind. `Cell` (`crates/aura-core/src/cell.rs`) -is a type-erased `u64`: constructed per base type (`from_i64/f64/bool/ts`) and -read only by naming the type at the call site (`i64()/f64()/bool()/ts()` — -branch-free bit-casts). The kind is therefore resolved once at the boundary and -the value itself carries no tag — C7's "the type lives at the column/edge, not in -the value" made explicit on the single-value carrier (and a single 8-byte word -vs. the 16-byte tagged enum). `Scalar` becomes `{ kind: ScalarKind, cell: Cell }` -— the self-describing form for the *dynamic* boundaries (builder binding, serde, -rendering) — with `debug_assert`-guarded native accessors (caller asserts the -kind; free in release) and a hand-written **value** `PartialEq` that preserves -the former enum's IEEE-754 semantics (`NaN != NaN`, `+0.0 == -0.0`), pinned by -the `scalar_eq_is_value_not_bitwise` fixture. Behaviour-preserving. - -**Realization (`Cell` becomes the hot-path carrier, 2026-06, #74).** The carrier -swap deferred above has landed: `Node::eval` now returns `Option<&[Cell]>` and -every node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free -8-byte words. The kind lives only at the schema/column: the harness forwards each -field via the new branch-free `AnyColumn::push_cell` (infallible — the edge -kind match is verified once at bootstrap, the surviving guard -`bootstrap_rejects_*_kind_mismatch`). `Scalar` remains on the self-describing -dynamic boundaries: the param plane (`build`/`bind`/`compile_with_params`/sweep -points/`RunManifest`), `AnyColumn::get` (the type-erased read for sinks/serde), -and source ingestion (`Source::next`, the heterogeneous C3 merge). The removed -per-value runtime kind check on node output is the same authoring-bug class C8 -already leaves to a `debug_assert` (output width); node-output-kind correctness -is the node's declared `FieldSpec` contract, caught by each node's own test. -Behaviour-preserving (C1). - -### C8 — The node contract -**Guarantee.** A node has a **signature** — its `NodeSchema`: each input's scalar -type and firing group, its output record, **and the node's own tunable parameters — -typed, with ranges**, which aggregate into the blueprint's param-space the optimizer -sweeps (C12/C19/C20). The signature is **declared pre-build on the value-empty -recipe** (C19), so the whole interface is legible without building (cycle 0024). -The built node implements `lookbacks()` — the per-input buffer depth, the one -quantity that may depend on an injected param (e.g. an `Sma`'s window = its -`length`), read once at bootstrap to size the windows — and `eval(ctx) -> -Option<&[Cell]>`. The -engine provides read-only, zero-copy windows into each input's SoA ring buffer -(`ctx.f64_in(x)[k]`, sized at wiring); a node may *additionally* keep its own -mutable series for derived/intermediate state. `None`/Void return = filter / -not-yet-warmed-up. A node is a **producer, a consumer, or both**: a -producer/transformer exposes **one output port**, whose payload is a **record of -1..K base-scalar columns** (a scalar is the degenerate K=1 record; an `eval` -returns a borrowed row, one value per column); a **pure consumer (sink)** — chart, -equity, logger — has **no** output. Sources are pure producers; sinks are pure -consumers. -**Forbids.** A node sizing/growing its input lookback at runtime; more than one -output **port** per node; a fifth scalar type or a heterogeneous output payload -(a record is a bundle of base columns, C7); copy-on-read of input history. -**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage -lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized -buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop). -**Realization (cycle 0005).** `NodeSchema.output` is a `Vec` (named base -columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field` -selects one producer column per edge; consuming a whole record is N edges (no -"bind whole record" mechanism). The K fields of one record are **co-fresh by -construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns -`Option<&[Cell]>` — a borrowed row into a node-owned buffer — so the forward -path allocates nothing per cycle (C7) (the carrier is now a tag-free `Cell` — see -the C7 carrier note). -**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is -now realized at the substrate: **recording is a node role, not a type.** A -recording node reads its typed input windows + `ctx.now()` in `eval` and pushes -the record to a destination it holds as a field (a channel, a chart handle) — an -**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a -node that only records returns `None` (pure consumer), and a node may record -**and** return an output the engine forwards in the same `eval` (the "both" -case). **Encoding & return contract.** A pure consumer declares `output: vec![]` -— the empty record *is* the sink declaration; there is no separate type, trait, -or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run -loop debug-asserts the returned row's width equals the declared output width -(`row.len() == schema.output.len()`). Field-wise wiring resolves -`Edge::from_field` against the producer's `output` at bootstrap, so no edge can -bind a field of a zero-output node — it fails with `BadIndex` — making a sink -structurally unwireable as an in-graph producer; its only output is the -out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out -of the graph is the node's own side effect — and that boundary is the -determinism / graph-as-data boundary (C1/C7). - -**Refinement (cycle 0070 — end-of-stream `finalize` lifecycle, 2026-06-25).** The -node contract gains a second lifecycle phase beside `eval`: `finalize(&mut self)`, -a **default-no-op** trait method the engine calls **once per node, in topological -order, after the source loop drains** (the `Harness::run` epilogue). It lets a sink -**fold online** — accumulate per `eval` into O(trades)/O(1) owned state and flush -one compact summary at stream end — instead of pushing a record every fired cycle -into an unbounded channel that buffers O(cycles) rows until the run ends (the -retention BLOCKER #138 hit: a full-history Stage-1 sweep held ~2 GiB/member over -~5.5M one-minute bars). C1/C7/C8 are preserved: `finalize` runs once *after* the -deterministic event loop, adding no within-sim concurrency (C1); a folding sink -holds only owned accumulator state, no interior mutability (C7); it still declares -`output: vec![]` and its flush is the same out-of-graph side effect (C8) — one -summary row, not a per-cycle stream. Realized by `GatedRecorder` (emits only gated -rows + the genuine final row) and `SeriesReducer` (folds one column, emits one -summary row), aura-std siblings of the per-cycle `Recorder`, which survives for the -live / `--trace` / test-tap path. The recording sink's *own* per-cycle allocation -(the `Recorder`→`Probe` rename + its accumulate-vs-stream choice, #77) stays open; -`finalize` is the "non-channel exit from the graph" that issue's -accumulate-then-read option named as missing. - -**Refinement (Construction-layer milestone — render labels, 2026-06-05).** A node -additionally exposes `label() -> String`, a **single-line, non-load-bearing** -render symbol: a default trait method the run loop never calls (wiring is by -index, C23). Overrides carry the node's identifying params (`SMA(2)` vs `SMA(4)`) -so a graph render (C9 graph-as-data, #13) disambiguates identical node types and -surfaces a mis-wiring. Like `FieldSpec.name`, it is an informative debug symbol, -not part of the C8 dataflow contract — adding it changes no run behaviour. - -**Realization (cycle 0015 — param declaration).** The tunable-parameter half of -this contract is now realized: a node declares its knobs in `schema()` as -`params: Vec` (`ParamSpec { name: String, kind: ScalarKind }`), and -`Composite::param_space()` (cycle 0024; was `Blueprint::param_space()`) aggregates -them into one flat, path-qualified list — a -read-only projection of the graph-as-data (C9), mirroring the inline order -(C19/C23) so a param's slot matches the later flat-node order. Two refinements to -the guarantee's "typed, with ranges": (1) the declaration carries `name` + `kind` -only — the **search-range is the run's, not the node's** (which subset / grid a -sweep covers is an experiment axis, #32/C20; the node declares the knob's existence -and type, never its search interval). (2) Identity is **positional** (the slot, -C23 "by index, not name"); the path-qualified `name` is a non-load-bearing debug -symbol (like `FieldSpec.name`) — uniqueness is at the slot. **Refinement (cycle -0032):** identity in the **flat graph** stays positional (C23, unchanged), but the -`param_space()` **name projection** — the authoring / by-name address space (the -surface a sweep axis or single-run binding addresses) — must be **injective** for a -blueprint to compile (a duplicated path is unaddressable; see C9). Two layers: -positional wiring below (the flat graph), an injective name address space above (the -authoring boundary). So same-type siblings in one composite no longer silently -share a name — they must be `.named(...)` apart, which is also what disambiguates a -same-type fan-in (C9). A vector knob (`LinComb.weights`) expands to `N` flat -`weights[i]` entries, `N` topology-fixed (C19). Permitted kinds are `i64`/`f64`/ -`bool` (a `timestamp` knob is a structural axis, C20, never a numeric sweep param). -Binding a value to a slot landed in cycle 0016 (#31, see C19/C23); enumerating a -sweep (#32) is the deferred next layer. The 0016 binding also moved the param -declaration's *authoring* home into the value-empty recipe, whose `params()` is read -pre-build. In 0016–0023 the built node's `schema().params` still reported the same -slots, kept in lockstep by a per-node test (a duplication filed as debt, #36); **cycle -0024 dissolved this** — the signature is declared *once* on the recipe and the built -node no longer carries `schema()` (#36 closed). See the C8 cycle-0024 realization. - -**Realization (cycle 0024 — the signature lives in the blueprint, #43/#36).** The -node's whole declared interface — its `NodeSchema` (input scalar types + firing, -output record, params) — now lives **once**, on the value-empty recipe -`PrimitiveBuilder` (ex-`LeafFactory`), read pre-build by `param_space()`, the -compile-time structural validation, and the render. This closes two debts: **#43** -(a value-empty recipe used to declare *no* input/output interface pre-build — only -params) and **#36** (params declared twice, recipe vs built node, kept in lockstep -by a per-node test — those 8 tests are deleted, the duplication structurally gone). -The split that makes this work: a node's signature is **fully static** per blueprint -(input kinds/firing, output fields, params — verified across the roster; a variable- -arity node like `LinComb` takes its arity as a *recipe argument*, not an injected -param), so it can be declared without building; the **one** param-dependent quantity, -an input's buffer **lookback** depth, is no longer in the signature but answered by -`Node::lookbacks() -> Vec`, read only by bootstrap to size the windows. -`Node::schema()` is therefore **removed** — the signature is pre-build data, not a -built-node method. `BlueprintNode::signature()` answers uniformly for both arms (a -primitive returns its recipe's schema; a composite *derives* it from the interior: -role kinds in, re-exported field kinds out, aggregated params), so "every node has a -signature in the blueprint" holds for composites too. - -**Realization (cycle 0027 — name input ports, refs #21/#51).** `PortSpec` now -carries a `name: String` (it drops `Copy`, like `ParamSpec`), so an input port is -named just as `FieldSpec.name` (output) and `ParamSpec.name` (param) already are. -The name is **non-load-bearing** (C23): wiring stays positional by slot, bootstrap -and the run loop never read it; it exists for tracing / graph rendering (#13). Leaf -primitives declare their slot names (`SimBroker`'s `exposure`/`price`, etc.); -`derive_signature` carries a composite's `Role.name` into the derived input port, -so the graph model (`model_to_json`) is homogeneously named across inputs, outputs, -and params and across both graph levels. This does **not** close #21 (a swapped -same-kind wiring is still only kind-checked) — it makes those slots -self-documenting; a name-consuming validation is its own future cycle. - -**Realization (2026-07-11 — composite `doc`, #125).** `Composite` carries an -optional authored rationale `doc: Option` — the prose twin of its `name` -and the same C23 category: a non-load-bearing debug symbol. Authored via -`Composite::with_doc` / `GraphBuilder::doc`; persisted as a Tier-1 -additive-optional `CompositeData` field (no format-version bump; absent-field -documents keep their exact bytes, so existing content ids are untouched); -**blanked in the identity projection** (`strip_debug_symbols`) while staying -canonical-byte-bearing; dissolved at inline like the name (never reaches -`FlatGraph`). Surfaced read-only (C22): the graph model emits an optional -trailing `"doc"` fragment per scope (`json_str` hardened for multi-line free -text — `\n`/`\t`/`\r` named, other control chars as `\u00XX`), the viewer shows -it in both composite view states (collapsed tooltip, expanded cluster frame) and -the root's as a muted header line. The construction op-script vocabulary -deliberately has no doc-carrying surface yet (scope cut recorded on #125). - -**Realization (cycle 0040 — wiring totality: every input slot connected exactly -once, #65).** The node contract's "the engine provides a window into each input" -presupposes each input is actually fed; until now an unwired interior input slot was -accepted and bootstrapped to a silent empty column (`harness.rs`), and two producers -into one slot — ill-formed, since a slot holds one column — was likewise -uncompiled-against. Cycle 0040 makes a valid graph **total and single-valued** over -its interior input slots: `check_ports_connected` (in `validate_wiring`, run at every -nesting level) requires every interior node's every declared input slot to be covered -by **exactly one** wiring act — one interior `Edge { to, slot }` or one role -`Target { node, slot }`, edges and role targets counted uniformly. Zero coverage is -`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. A -composite's **own** input roles (`source: None`) are coverage *providers* — the -wired-by-enclosing boundary, the root case already guarded by `UnboundRootRole` — -never consumers, so only interior input slots are subject to the rule. There is **no -optional-input concept**: every declared input port is required (no shipped node runs -meaningfully without one; an unwarmed mode-A input is *wired-but-not-yet-valued*, not -unwired). The check is **index-based and name-free** — it touches no name machinery -and emits nothing into the flat graph, so C23 is untouched (it proves the existing -raw-index wiring is total and single-valued). Inherited identically by the raw -`Composite::new` path and the `GraphBuilder::build()` path (both compile via -`compile_with_params`). - -### C9 — Fractal, acyclic composition -**Guarantee.** A composite is itself a `Node` that wires a sub-graph and exposes -one output; signal, combined signal, and (with execution) strategy are all the -same abstraction, nestable arbitrarily. The dataflow graph is a DAG; the only -feedback path is an explicit delay/state node (the RTL "register"). Wiring is -written in Rust (builder API); the built graph is introspectable runtime data. -**Forbids.** Implicit dataflow cycles (combinational loops); special-casing -"signal-of-signals" as separate mechanics. -**Why.** Self-application of one contract gives unlimited composition with no -adapter zoo. Acyclicity keeps the synchronous reactive model well-defined; -forcing feedback through a visible delay node keeps the per-cycle determinism -intact and the one legitimate feedback path explicit. Graph-as-data enables -visualization, freezing, and re-parameterization for sweeps. -**Refinement (Construction-layer milestone, 2026-06-05).** "A composite is itself -a `Node`" is an **authoring-level** identity: a composite declares the same -interface (typed inputs + ≤1 output, C8) and is wireable wherever a node is, but it -is **not** a runtime object. The bootstrap **compiles it away by inlining** its -sub-graph into the one flat instance (C19/C23): the composite *boundary* dissolves -into the raw index wiring the run loop already consumes, and names stay -**non-load-bearing** (informative debug symbols only, as `FieldSpec.name` already -is). So "nestable arbitrarily" and "graph-as-data" hold at the **blueprint -(source)** level; the running graph is the flat, index-wired **`FlatGraph`**. The earlier reading — *a composite survives as a `Box` -driving a nested sub-engine* — is **explicitly rejected**: it would keep the -interior opaque to the cross-graph optimiser (C23) and add a runtime sub-loop the -flat model does not need. Inlining is what makes the composite boundary free. -**Realization (cycle 0018 — composite multi-output record, #40).** "Exposes one -output" is one output **port** carrying a **record of 1..K re-exported fields**, not -one field: `Composite.output` is a `Vec` (was a -single `OutPort`). Each entry is a **named projection** of one interior -`(node, output-field)`; a consumer selects which re-exported field it reads via the -same `Edge::from_field` that already binds leaf record columns (C8 realization, -cycle 0005). This is the **same arity** C8 already grants a leaf (OHLCV = one port, -5 columns): a multi-line indicator (MACD = macd/signal/histogram) is **one** record -of K fields, **one** port, **one** row per `eval` — C8/C7/C4 untouched, a boundary -completion, not a contract change. A strategy composite is simply the K=1 case -(one bias field, C10). The re-export **names** are **non-load-bearing** (C23): -they live at the blueprint boundary and in render (cycle 0022/#46 folds each onto -its producing node as a `name := …` binding; originally `[out:]` markers, #13) -but are dropped at lowering — `ItemLowering::Composite.output` is `Vec<(usize, -usize)>`, raw index pairs only, so the flat graph is name-free (verified: the -compiled-view render stayed bit-identical across this change). -**Realization (cycle 0019 — name the composite boundary, #41; param-overlay retired, -cycle 0031).** The named-projection shape covers the surviving boundary edge-kinds: -`input_roles` is a `Vec` (was a bare `Vec>`), -alongside `output: Vec`. Each is an ordered, positionally-indexed -**named projection** of interior handles. **Param projection is no longer a -composite overlay** (the index-addressed `ParamAlias` was retired in cycle 0031): -a node's surface param name flows from its own **instance name** — every node -carries a name (default = its lowercased type label, override via `.named()`), and -`param_space()` is uniformly `.` at every level including the root. A -same-type fan-in is distinguished by naming the colliding legs, the same single act -that qualifies their param paths. Like the output and role names, **node names are -non-load-bearing** (C23): they live at the blueprint boundary and in render but are -dropped at lowering — the flat graph is wired by raw index. The full composite -boundary signature (named inputs, multi-outputs) and the per-node param path are -legible without changing the flat graph. -**Refinement (param-namespace injectivity, cycle 0032; supersedes the fan-in -distinguishability check).** A blueprint compiles only if its `param_space()` name -projection is **injective** — every path-qualified knob name is unique. A duplicated -path is a knob no binding can select alone (it has no distinct by-name address, -C12/C19), so it is a `CompileError::DuplicateParamPath` carrying the offending path; -the cure is to give the colliding same-type sibling nodes distinct names with -`.named(...)`, the same single act that qualifies their param paths. The -param-bearing indistinguishable fan-in is *one instance* of a duplicated path (two -default-named same-type legs share a leaf path). `signature_of`, `leaf_has_param`, -and the fan-in-specific check (`check_fan_in_distinguishability` / -`check_composite_fan_in`) are **retired** (cycle 0032), replaced by one structural -`check_param_namespace_injective` over the `param_space()` names, run before name -resolution in `compile_with_params` and both binders — so the canonical by-name -author sees the structural cure rather than a downstream `AmbiguousKnob` symptom -(that `BindError` arm is retired too: an injective space can never multi-match). -Paramless interchangeable same-name sources stay legal (no path, no duplicate). -The old signature-collision predicate had extra breadth — it also rejected an -**asymmetric param/paramless** collision and a **role-vs-leg** collision, *neither* -a path duplicate (each colliding configuration keeps a unique param path, or none). -That breadth guarded **render identity**, dead since the renderer was retired in -0026; both extra rejections are **intentionally dropped**. A future node-/wiring-name -distinguishability check, if ever wanted (e.g. for the WASM graph view's #21 -thread), is decoupled from param-space injectivity. Construction-phase only; the -flat graph stays name-free (C23). -**Refinement (2026-06-29 — graph-as-data round-trips both ways, C24).** "The built -graph is introspectable runtime data" is now contract-level **bidirectional**: a -blueprint is not only *emitted* as data (`model_to_json`, the render half) but is -itself a **serializable data value with a load path** (data → blueprint → -`FlatGraph`), so topology is a value the World generates, stores, and reproduces — -see C24. The Rust builder API stays the primary *human / LLM* authoring surface -(C17); the data form is the *machine* surface the World owns. (Load path **realised** -cycle 0087 / #155, `d5602ec`: `aura-engine::blueprint_from_json`.) - -### C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge -**Guarantee.** A strategy's primary, backtestable output is **not** an equity -curve, **nor a position-event table**, **nor a position size**, but a **bias -stream**: the DAG expresses exactly one *state* at time t (C8 — a node emits at -most one record per `eval`), so a strategy emits one **signed, bounded bias** -`f64 ∈ [-1, +1]` per cycle (per instrument) — the **sign is direction, the -magnitude is conviction**, and conviction is optional (a bare ±1 / 0 is the modal -case). Bias is **unsized**: position sizing and the protective stop do **not** -live in the strategy. The chain is `signals (scores) → decision node → bias -stream`. - -**Risk-based execution is a decoupled downstream layer; in research it is Stop + -position-management in R, no Sizer.** Turning a bias into a tracked trade is the -job of a downstream **execution** chain, never the strategy's. The **stop-rule** -sets a protective stop, which **defines the risk unit R** (1R = the loss taken if -stopped). In the research loop the executor is **stop-rule → position-management**, -operating **directly in R**, with the **Veto** an optional documented pre-trade-gate -seam (a pass-through identity DCE'd away under C19/C23 when absent): there is **no -Sizer**. Sizing in *currency* (`size` / `volume`) is a **deploy** concept. C8's -wiring totality (cycle-0040 `check_ports_connected`: no optional-input concept — -every declared port is covered by exactly one wiring act) forbids a *dangling* -`size` port, so the resolution is concrete: research `PositionManagement` either -**drops its `size` input port** from its node signature, or has that port **driven -by a constant unit node** (the flat-1R degenerate) — never an unwired "vestigial" -port; the record's `size` field is held at unit and carries no research -information, and the position-event table's `volume` column likewise. Consequently -the **position-event table is demoted to a deploy / reconciliation artifact** (real -volume lives there), no longer a research artifact and no longer "fed to a broker". -The three-way decoupling of **direction (bias) / sizing / fill** stands as a -*structure*; sizing and fill are simply pushed entirely to the deploy edge, out of -the research loop. - -**Signal quality is measured in R — gross R and net R.** A downstream -**R-evaluator** consumes the executor run and integrates the per-trade R-outcomes -into an **R-expectancy** / R-curve: the account- and instrument-agnostic yardstick -for *"how much R out per 1R risked?"*. R, not pips, is the unit (pips are not -risk-normalized). Two readings of the same unit: **gross R** (signal only) vs **net -R** (after the cost model), with `net R = gross R − cost-in-R`. The headline -artifact is the **net-R equity curve** — the cost-drag drawn onto the R curve, -recorded through a named **`net_r_equity`** tap/sink (sibling of the existing -`r_equity` tap; a sink is the only thing the registry can display, C8/C18/C22). No -new unit is invented: it is R, gross and net, **continuous with the existing -`net_expectancy_r`**. A bare **gross-R run with no cost model attached is valid**: -the cost layer is optional and additively composed-on (the zero-cost baseline is -the "default simple" floor). - -**Cost is a COST MODEL — a composable C9 graph of cost nodes, in R, that -approximates (never claims) realism.** The realistic broker is *retired* (see -Forbids / the 2026-06-28 reframe): real friction — slippage (live -liquidity / order-size / volatility at fill), swaps (broker-set, time-varying), -even recorded feed spread (often a fake constant) — is **not historically -knowable**, so an authored-friction historical broker is "horseshoe-throwing". Its -replacement is a **cost model**: an ordinary downstream **C9 graph of cost nodes** -that *approximates* the cost side a broker would produce, explicitly as an -approximation. The cost nodes live in `aura-std` (the cost-graph composite-builder -in `aura-composites`), never in the domain-free `aura-engine` (C14/C16). They are -**not** "additive on the R stream" in isolation: a cost node **reads the state it -depends on** — the price stream, a realized-volatility tap, a C11-recorded -interest-rate source, and the executor's per-cycle R-record / trade events — and -emits a **cost-in-R** stream that is subtracted from gross R to yield net R. Cost -attaches at the structurally correct grain: **per-trade** factors (commission, a -flat cost-per-trade) deduct from `realized_r` at close; **per-cycle-held** factors -(carry / funding / swap) accrue over the holding duration. The model **generalizes** -the existing scalar `round_trip_cost` / `net_expectancy_r` into a possibly -**state-dependent graph**: the scalar `round_trip_cost` is the degenerate -constant-per-trade special case, **subsumed** by the cost graph — the post-run -`summarize_r` fold no longer recomputes cost independently but folds the -cost-model's net-R stream into `net_expectancy_r` (one home for cost, no -double-count). Discipline: every cost factor is **either** a clearly-labelled -**stress-parameter** (e.g. a flat cost-per-trade is a breakeven-threshold probe) -**or** **data-grounded / falsifiable** (e.g. realized-volatility → slippage; -recorded interest-rate data via C11 → funding / swap). **Default simple; -complexity is earned per grounded factor.** Stacking unfalsifiable guesses -(over-modelling) is the anti-pattern. - -**The research loop is pure feed-forward — compounding is removed.** Flat-1R-style -R accounting needs no equity, and with the Sizer gone there is **no equity → size -edge at all** in research: the loop is **feed-forward, maximally parallel and -deterministic (C1)**, the cost model a feed-forward subtraction on the R stream. -**Compounding is removed from research**: it is a **post-strategy money-management -transform** — a pure function of the per-trade **net-R sequence** and a -bet-fraction *f*, multiplicative and path / order-dependent, the **sole** source of -feedback — so compounding, Kelly-*f*, and drawdown-under-compounding are derived -**post-hoc, analytically, at the deploy / account layer** from the net-R -distribution. Consequently the `z⁻¹` fill-edge register **and** the -flat-1R-vs-compounding structural axis are **gone from the research loop** (with no -in-loop feedback there is nothing for a register to cut, so the C23-reordering -hazard the old text invoked to make the register mandatory no longer exists — this -is strictly C9 / C23-cleaner). - -**Conviction-based risk allocation survives — as an R-aggregation axis, not a -Sizer.** Scaling risk by bias strength is **signal-side and R-denominated**. -Because per-trade R is **size-invariant**, conviction cannot be expressed by -scaling position size (that is invisible to R); it is expressed by **weighting the -per-trade R-contribution** in the R-equity: **flat** (sum of `realized_r`, sign -only) vs **conviction-weighted** (sum of `|bias| · realized_r`, sign + magnitude). -This is a **feed-forward, additive, order-independent research axis** (flat vs -conviction-weighted R-aggregation), distinct from the removed money-Sizer; it is -tested via the existing `conviction_at_entry` field and `conviction_terciles_r` -metric, and may sit in-graph or as a post-hoc fold. - -**Money / real broker / cTrader Open API = a separate, later live / deploy-edge -concern — the only `belastbare` (reliable) ground truth, measured never modelled.** -Reliable friction statistics require **forward-trading against a real broker** -(e.g. cTrader Open API), and are non-stationary even then. This fits C11 -(record-then-replay: real fills are recorded live, then replayable) and the -frozen-deploy invariant (C13: deploy = frozen bot + broker connection); -reconciliation with the real account is an **external I/O adapter** at the -recording / deploy edge, not an in-graph node. **This is the only place account -money appears.** (Currency-denominated *reference geometry* — pip value, -stop-distance-in-currency from the C15 `instrument_geometry` sidecar — is still -read at the **ingestion** edge to normalize a currency / pip cost factor into R; -notional size cancels in `cost_in_R = cost_in_currency / (size · stop_dist)`, so -the cost model is R-pure without ever holding *equity*. It is equity / account -money, not reference geometry, that lives only at the deploy edge.) The -broker-independent **position-event table** (`event_ts, action[buy/sell/close], -position_id, instrument_id, volume`) is the **deploy / reconciliation** record at -this edge (real volume), not a research artifact. - -**Honesty principle.** The net-R curve under the cost model is a **research / -ranking tool and a hypothesis**; the **forward / live run against a real broker is -the ground truth**. The cost model *approximates*; it **never claims realism**. -`SimBroker` (the pip-equity, unsized-exposure node) is the **pre-reframe pip -ancestor** — with the net-R cost model it is **redundant as a quality measure** (R -supersedes pips). It is retained as a **legacy / simple optional pip yardstick** -(still wired in the `r-sma` harness for an honest dual readout), **not part of -the new model and not to be expanded**. - -**Forbids.** Putting **sizing or the stop in the strategy** (bias is unsized; the -stop-rule owns the stop, R is the unit); treating an **equity curve** (R or -currency) as the strategy's direct output; **putting a Sizer / currency size / -`volume` into the research loop** (size is a deploy concept; research is in R); -leaving a **dangling `size` input port** on the research executor (C8 wiring -totality forbids it — drop the port or drive it with a constant unit); **any -equity → size / equity → anything feedback in research** (the research loop is pure -feed-forward; compounding is a post-hoc money-management transform, not an in-loop -edge); modelling an **authored-friction "realistic broker" over historical data** -(real friction is not historically knowable — use the approximating cost model, and -treat the real broker as the live-edge ground truth only); **claiming the cost -model is realism** (it is an explicit approximation); **stacking unfalsifiable cost -guesses** (each cost factor is a labelled stress-parameter *or* data-grounded — -over-modelling is the anti-pattern); **computing cost in two homes** (the cost -graph owns cost; the post-run fold subsumes the old scalar `round_trip_cost`, never -double-counts it); expressing **conviction by scaling position size** -(size-invisible to R; conviction is an R-aggregation weight); making the -**position-event table the strategy's direct DAG output** (it is derived, not -emitted per `eval` — a decision instant may need >1 event, which C8 forbids) or a -**research** measure of signal quality (it is a deploy / reconciliation artifact); -**measuring signal quality in currency / account money** in the research loop at all -(account money lives only at the live deploy edge; reference geometry at ingestion -is not account money); baking a broker into the strategy or an **in-graph broker -subsystem** (the in-graph realistic broker is retired; the only broker is the -live-edge I/O adapter, C11 / C13); a signed-volume direction trick in the event -table (use `action`); storing `open_ts` (derive it from the opening event). - -**Why.** A strategy's edge is a *procedure*, not a currency outcome ("focus on the -procedure, not the money"): the right primary question is *"how much R out per 1R -risked?"*, and R — defined by the stop — is the only account- and -instrument-agnostic, risk-normalized unit (pips are not). Separating **direction -(bias) from sizing from fill** is the decomposition every mature system converges -on (LEAN's Alpha → Portfolio-Construction → Execution is a near isomorphism; -backtrader, QSTrader, zipline all emit an unsized directional signal sized -downstream) — and aura pushes *sizing* and *fill* off the research loop for two -**distinct** reasons: **sizing** is off because per-trade R is **size-invariant** -(size carries no information in R — flat-1R is perfectly knowable, it just does not -matter), and **fill / friction** is off because real friction is **not -historically knowable** and therefore not honest over history. Keeping research -**pure feed-forward** leaves the signal-quality layer parallel and deterministic -(C1); the **only** real feedback (equity → bet-fraction) is **compounding**, a -closed-form, path-dependent transform of the net-R sequence, and therefore belongs -**after** the strategy, at the deploy / account layer, not as an in-loop register. -The **cost model as a C9 graph** keeps cost within the one Node / graph abstraction -(C9) and generalizes the scalar `net_expectancy_r` continuously, while the -**gross-R / net-R** split states the cost-drag honestly without inventing a unit. -Refusing the historical realistic broker is an **honesty** stance: the only -`belastbare` friction is **measured forward** against a real broker (cTrader Open -API) — the live deploy edge, the sole place account money and ground truth appear. -The DAG holds exactly one state at t and a node emits ≤1 record per `eval` (C8), so -the faithful per-cycle output is the **bias** (one value); position *events* are a -derived, deploy-side consequence. This supersedes the **realistic-broker / -currency / compounding / Stage-1-vs-Stage-2** framing of the #117 reframe (see the -2026-06-28 reframe note), which itself superseded the **exposure** framing of -cycle-0007 and the still-earlier "strategy's output is the position-event table" -framing. - -**Reframe (2026-06-28, #116 — realistic broker retired; cost-model graph in R; money to the live edge).** -This is the live contract above. Ratified in an in-context design discussion -(reference issue #116). It **preserves** the durable spine — unsized bias stream -(sign = direction, magnitude = conviction), signal quality in R, the stop defining -R, and the decoupling of direction from sizing from fill — and the shipped Stage-1 -realizations (the `Bias` node, `FixedStop` / `vol_stop`, `PositionManagement`, -`summarize_r` / `RMetrics` / `sqn_normalized`, SQN as ranking objective). The -**RiskExecutor composite survives in shape** (bias + price → stop-rule → -position-management) but with its **Sizer interior removed** and its `risk_budget` -argument dropped / vestigial (it sized the now-removed Sizer); the Veto remains an -optional documented seam. The contract **supersedes** the following, which were -**design intent, largely unbuilt** (#116 — the realistic broker and the whole -Stage-2 currency layer were never implemented; the Stage-1 R chain was) and are -retired: -- the **"realistic broker"** concept — an authored-friction historical broker — - rejected as "horseshoe-throwing" (real friction is not historically knowable), - replaced by the **cost model**: a composable C9 graph of cost nodes (in - `aura-std` / `aura-composites`), approximating not claiming realism, - generalizing / subsuming `round_trip_cost` into `net_expectancy_r`; -- the **"Currency P&L is Stage 2"** paragraph in full, the **Stage-1-vs-Stage-2 - hard sequencing gate**, and **currency / fixed-fractional / compounding** in - research — compounding is now a post-hoc money-management transform of the - net-R sequence at the deploy / account layer; -- the **`z⁻¹` register on the fill edge** and the **flat-1R-vs-compounding - structural axis** as research mechanism — there is no equity → size edge in - research, so no register and no such axis in the loop; -- the **Sizer in research** and currency **size / `volume`** — size is a deploy - concept; the research executor is stop + position-management in R; - `PositionManagement`'s `size` port is dropped or constant-driven (no dangling - port, C8), its `size` field and the event table's `volume` are vestigial in - research; -- the **"a broker is an ordinary in-graph node / no special external broker - subsystem"** forbid, **for the live edge only**: in-graph brokers are retired - outright, and the live broker is now an explicit **I/O adapter** at the C11 - recording / C13 deploy edge — not an in-graph node and not part of the research - graph (the no-in-graph-broker-subsystem prohibition still holds inside the - graph); -- the **position-event table as the realistic-broker input** and its - first-difference-of-the-book (`deal = target − book − in_flight`) execution - framing — the table (schema 0063 #114, derive 0068 #115) **survives** as the - **deploy / reconciliation** artifact (real volume), not a research artifact and - not "fed to a broker" in research. -Money, a real broker, and cTrader Open API are a **separate, later live / -deploy-edge** concern (C11 record-then-replay; C13 frozen-deploy invariant) — the -only `belastbare` ground truth, measured, never modelled. The honesty principle is -explicit: the net-R curve is a research / ranking hypothesis; the forward / live -run is ground truth. `SimBroker` is downgraded to a legacy / optional pip -yardstick, not to be expanded. (Terminology note: with Stage 2 gone, the -**"Stage-1"** label below is no longer one half of a two-stage gate — it survives -only as a historical cycle name, like the `exposure` → `bias` on-disk alias, -denoting the shipped feed-forward R chain; the identifier family that carried it -was renamed to the r-family — `r-sma` / `r-breakout` / `r-meanrev` — in cycle -0100, #174.) - -**Realization (cycle 0081 — cost-model graph, cycle 1, #148).** The first concrete -cost node + the net-R seam shipped. `ConstantCost` (`aura-std`) is an ordinary -downstream node (C9) emitting a 3-field cost-in-R record `{cost_in_r, cum_cost_in_r, -open_cost_in_r}` isomorphic to `PositionManagement`'s R-triple; R-pure -(`cost_per_trade / |entry − stop|`, notional cancels, no equity held). The scalar -`round_trip_cost` argument of `summarize_r` is **retired**: `summarize_r` folds a -co-temporal cost stream (positional 1:1 join — `cost[i]` is `record[i]`'s cycle) into -`net_expectancy_r`, one home for cost / no double-count, byte-identical to the old -cost = 0 baseline on an empty stream. The headline sink is **`net_r_equity`** = -`LinComb(4)[cum_realized_r, unrealized_r, −cum_cost_in_r, −open_cost_in_r]` → Recorder -(C8/C18), a sibling of `r_equity`, emitted only when a cost is authored. Wired on the -**run path** via `--cost-per-trade` (`r_sma_graph`); sweep / walk-forward / mc pass -`None` (cost on the reduce-mode sweep path and cost in OOS pooling deferred — the -positional join holds only while one cost node fires in lockstep with PM). -[Superseded 2026-07-11 (#234): the `--cost-*` run-path flags are gone (#221 removed -that surface); cost is authored as the campaign document's `cost` block and reaches -every family/campaign member — see the cycle-net-r realization below.] Deferred to -later milestone cycles: the general `CostNode` trait + multi-node cost-graph -composite-builder, data-grounded factors (realized-vol → slippage, recorded-rate → -swap), per-cycle-held accrual (carry / funding), and the conviction-weighting -R-aggregation axis. Decision log: #148. - -**Realization (cycle 0082 — cost-graph composition, cycle 2, #148).** The cost graph -**composes**: a second, *state-dependent* cost node plus an aggregator prove that two -cost nodes sum into one net-R curve with `summarize_r` and the `net_r_equity` tap -structurally unchanged. `VolSlippageCost` (`aura-std`) charges `slip_vol_mult · vol / -|entry − stop|` in R, reading an **independent short-horizon realized-range** vol -(`RollingMax − RollingMin`, window distinct from the stop's own vol — scaling slippage -by the stop's vol would collapse cost-in-R to a constant, indistinguishable from -`ConstantCost`). `CostSum` (`aura-std`) is the cost-graph **output node**: it sums `N` -cost nodes' 3-field cost-in-R records **per-field** into one aggregate (`n = 1` is the -identity), so the seam consumes a single cost stream regardless of node count — one -home for cost, the positional join unchanged. **Co-temporality contract (load-bearing, -generalizes to all future factors):** since `summarize_r` positionally joins `cost[i] -↔ record[i]`, a cost node is gated **only by the PM trade-geometry**; any not-yet-warm -state input (the vol proxy warms later than PM) contributes **0 cost** that cycle -rather than withholding — the node still emits its row, so the cost stream stays -co-temporal 1:1 with the PM record. This makes co-temporality structural and -warm-up-independent, preserves the C18 golden, and is honest (no slippage estimate yet -→ no charge). `ConstantCost` satisfies it trivially; only state-dependent nodes need -the missing-factor → 0 rule. Wired on the **run path** via `--slip-vol-mult`, -composable with `--cost-per-trade` (their costs sum); sweep / walk-forward / mc pass -`None`. The 3-field cost triple is now restated by-convention across the two producers -+ `CostSum` (a compiler-unlinked lockstep) — to be unified by the still-deferred -general `CostNode` trait (now justified by two concrete nodes). Decision log: #148. -[Superseded 2026-07-11 (#234): the `--slip-vol-mult`/`--cost-per-trade` flags are gone; -authoring moved to the campaign `cost` block — see the cycle-net-r realization below.] - -**Realization (cycle 0083 — CostNode trait + shared cost-record contract, cycle 3, -#148).** The deferred unifier ships. A new `aura-std/src/cost.rs` owns the cost-model -node abstraction: the 3-field cost triple is now **one source of truth** -(`COST_FIELD_NAMES` / `COST_WIDTH`, mirroring `position_management::{FIELD_NAMES, -WIDTH}`), read by both producers + `CostSum` + the CLI wiring — the cycle-0082 -by-convention lockstep is **structurally gone** (four restatements collapsed to one). -The `CostNode` **factor trait** carries a cost node's only per-node difference — the -price-unit **cost numerator** (`cost_numerator(&mut self, &Ctx) -> f64`), plus -`extra_inputs` (default none) and `label`; everything else is the generic -`CostRunner` **adapter**, which holds the shared `cum`/`out` state and -implements `Node`, writing the co-temporality skeleton (geometry-only gating, -`numerator / latched` R-normalization, the closed/open charge, the running `cum`, the -3-field emit) **once**. `ConstantCost` and `VolSlippageCost` are now thin factors whose -`new()` returns `CostRunner`. Honours C9 (a `CostRunner` is a plain downstream -`Node`, no runtime sub-object) and C23 (`name()` dropped as dead surface — `label()` -remains the non-load-bearing symbol). **Behaviour-preserving**: the builders emit -unchanged schemas, so the wiring / `net_r_equity` seam / `summarize_r` are untouched and -the `numerator / latched` token form is byte-identical; the existing suite passes -verbatim and two new CLI characterization goldens pin the exact flat/composed -`net_expectancy_r` (the prior tests only asserted `net < gross`). Decision log: #148. - -**Realization (cycle 0084 — cost-graph composite-builder, cycle 4, #148).** Decision E -ships. A new `cost_graph(Vec) -> Composite` in `aura-composites` (the -C16 layer that couples the engine builder + `aura-std` nodes) is the cost-model graph's -**authoring primitive**: it fans the 4 PM-geometry inputs to `N` cost nodes, surfaces -each node's extra inputs (discovered via `schema().inputs[GEOMETRY_WIDTH..]`, -`GEOMETRY_WIDTH` now re-exported from `aura-std`) as `cost[k].` composite roles, -sums them through `CostSum`, and exposes the 3-field aggregate. The CLI's manual -slot-indexed cost-wiring + the hardcoded `MAX_RUN_COST_NODES = 2` cap are deleted — the -composite handles arbitrary arity. **Behaviour-preserving** (C11): the composite inlines -at bootstrap to the same flat fan-in, so the cycle-0083 `net_expectancy_r` goldens are -byte-identical (four `aura-composites` unit tests pin the exposed role-set + output -triple, incl. arbitrary-arity per-node namespacing). Honours C9 (ordinary downstream -nodes), C16 (wiring stays out of `aura-engine`), C23 (role/port names are -non-load-bearing). **Carried debt (#152, for the deferred sweep-cost cycle):** the -`cost[k].` index-namespacing is restated across `CostSum` / `cost_graph` / the CLI -(a build-validated lockstep, not the silent positional kind 0083 collapsed), and -`cost_graph` `.leak()`s runtime port names per build — fine for one-shot run-path -construction, but to be interned (the `COL_PORTS` production pattern, not the test-only -`.leak()`) before cost reaches the per-member sweep path. Decision log: #148. -[Discharged 2026-07-11 (#152/#234): `cost_port`/`intern_port` in `aura-std/src/cost.rs` -are the process-global interned single source; both `cost_graph` `.leak()`s are gone.] - -**Realization (cycle 0085 — per-cycle-held accrual, cycle 5, #148).** C10's -"per-trade factors deduct at close; **per-cycle-held factors accrue over the hold**" -clause is first realized. A cost factor now declares *when* it charges via -`ChargeMode { AtClose, PerHeldCycle }` (a defaulted `CostNode::charge_mode()`, default -`AtClose`), read by the **one shared `CostRunner`** — not a second runner type (a -commission is intrinsically at-close, a carry intrinsically per-held-cycle; the timing -belongs to the factor, preserving the cycle-0083 "skeleton written once" win). The -`PerHeldCycle` arm accrues `per` into a per-position `acc` every held cycle, dumps the -accrued total into `cum` at close (resetting `acc`), and marks the open position via a -**growing `open_cost_in_r`**. The first accrual node, **`CarryCost`** (`aura-std`, -a `ConstantCost` twin differing only in `charge_mode()`), is a labelled stress -parameter — the flat base of the accrual family; a run-path `--carry-per-cycle` flag -pushes it into the existing `cost_graph`/`CostSum` aggregation (no new wiring; a -per-trade and a per-held-cycle node compose, the costs summing per-field). **Approach B -(honest bleed), achieved without a `summarize_r` fold change:** the headline -`net_r_equity` curve bleeds continuously over the hold because the bleed lives in -`open_cost_in_r`, which the `net_r_equity` tap already subtracts — so `summarize_r` and -the CLI `net_eq` wiring are **untouched**, and the cycle-0083/0084 `net_expectancy_r` -goldens stay byte-identical (the `AtClose` arm is the pre-cycle-5 eval tokens verbatim). -The 3-field cost record's semantics generalize cleanly: `cost_in_r` = cost realized this -cycle (into `cum`); `open_cost_in_r` = the open position's cost marked-to-market -as-of-now (would-be-close for `AtClose`, accrued-so-far for `PerHeldCycle`) — singly -counted, no `cum`/`open` double-count (the two are mutually exclusive per cycle). Honours -C9 (a `CostRunner` is an ordinary downstream `Node`), C11 (byte-identity), C16 -(factor in `aura-std`, `aura-engine` domain-free), C23 (`label()` carries the rate). The -B-vs-A discriminator (a growing intra-hold `open_cost_in_r`, invisible to the scalar -`net_expectancy_r`) is pinned by unit B-proofs + a `net_r_equity`-bleeds-over-the-hold -integration test. **Deferred (each its own #148 cycle):** a notional-based carry -(`price × rate`, reads the price tap), the calendar-aware **overnight swap** proper -(rollover-boundary timing, 3× Wednesday, long/short asymmetry — deploy-edge realism), and -sweep-path cost. Decision log: #148. - -**Realization (cycle net-r — cost reaches the family/campaign path, #234/#152, -2026-07-11).** The deferred sweep-path cost ships, delivered where C24 put experiment -intent: the campaign document gains an additive `cost: Vec` block — a closed, -externally tagged vocabulary over the three shipped nodes (`constant` / `vol_slippage` -/ `carry`, field names = the builders' own `ParamSpec` names), mirroring the `risk` -block (serde `default` + skip-if-empty: cost-less docs hash byte-identically, C18). -Net is the **default**: an absent block is the explicit zero-cost model and -`summarize_r`'s net family equals gross under it — every result is net, no second -"gross-labelled" result kind. `cost_nodes_for` (beside `stop_rule_for_regime`) is the -one doc→builder binding, every component fully bound (the wrapped param space stays -cost-invariant); `wrap_r` carries an optional cost leg (the #221-deleted wiring -rebuilt: cost_graph off the executor's four geometry outputs, the vol proxy back in -production, a gated cost recorder in reduce mode as the `summarize_r` join input, the -`LinComb(4)` `net_r_equity` curve in trace mode). Member manifests stamp the -components; both re-run sides re-derive them (reproduce via `cost_specs_from_params` — -the #233 stop pattern — and the persist re-run binds the same model, so the C1 drift -alarm compares like with like; costed families reproduce bit-identically, pinned incl. -`Carry`). `TapChannel::Net` routes `net_r_equity` to persisted curves; a cost-less doc -requesting it keeps a remedy-naming skip notice. The `--cost-*` run-path flags do not -return; the verb sugar passes no cost (docs carry it) — the same lean-flag call as the -C26 bindings block. #152's interning ships with it (`cost_port`/`intern_port`, both -`cost_graph` `.leak()`s gone). Decision log: #234. - -**Realization (cost-flag harness scoping, #153).** Cost flags are defined only -against an R-evaluator harness (the gross-R → net-R chain). On a non-R harness -(`--harness sma`/`macd`, which produce no R) a cost flag is a **usage error -(exit 2)**, not a silent no-op — refuse-don't-guess. Negative cost rates are -likewise rejected (exit 2) with a named diagnostic that identifies the offending -flag. CLI ergonomics only; the cost-model graph and the R math are untouched. -[HISTORY — the built-in `--harness` selector (its `sma`/`macd` non-R examples) -was retired with the demos → blueprint-data (#159, cuts 1b-4); the cost-flag -scoping rule survives on the `aura ` r-sma run path over -examples/r_*.json.] - -**Reframe (2026-06-23, #117 — exposure → bias, R as the signal-quality unit).** -[HISTORY — its R spine survives into the 2026-06-28 contract; its Stage-2 currency / -realistic-broker / register / flat-1R-vs-compounding portions are SUPERSEDED by that -reframe.] The contract was reframed from **exposure** (a signed fractional -position) to an **unsized bias** plus **R** as the signal-quality unit. The -realization notes below describe the **pre-reframe code** (`Exposure`, `SimBroker`, -pip-equity), retained as history and as the **ancestor** of the current chain: -`Exposure { scale }` is the ancestor of the unsized `bias` node, and `SimBroker`'s -pip integral is the ancestor of the R-evaluator (which additionally requires a -stop, since R is stop-defined). The `exposure → bias` rename, the RiskExecutor / -Sizer / Veto nodes, and the R-evaluator **landed in cycle 0065**; the -position-event schema (0063, #114) survives as the audit layer. Industry grounding -for this reframe: LEAN / nautilus_trader / backtrader / QSTrader / vectorbt / -zipline (see #117 decision log). - -**Realization (cycle 0007).** [HISTORY — pre-reframe; `SimBroker` is now legacy per -the 2026-06-28 reframe.] The signal-quality half was realized at the substrate as -two `aura-std` nodes on the unchanged engine (the engine stays domain-free — it -routes only `f64` records). The **exposure stream** was realized as `Exposure { -scale }`: `clamp(signal / scale, -1, +1)`, one `f64` per fired cycle. The -**sim-optimal broker** was realized as `SimBroker { pip_size }`: a two-input node -(exposure, price) accumulating `prev_exposure · (price − prev_price) / pip_size` -and emitting cumulative pip equity — the exposure held *into* a cycle (decided at -t-1) earns that cycle's return (causal, C2); `pip_size` is held reference metadata -(C7/C15). An end-to-end harness (SMA-cross → `Exposure` → `SimBroker` → recording -sink) produced a recorded pip-equity curve, bit-identical across runs (C1). - -**Realization (per-instrument pip channel, 2026-06, #22).** [HISTORY — pertains to -the legacy `SimBroker` pip channel.] `SimBroker`'s `pip_size` is sourced **per -instrument** from the recorded geometry sidecar (`instrument_geometry`, over -data-server's `symbol_meta`), at the ingestion / source edge (never `Aura.toml`, -never `Ctx`); the engine stays domain-free. (The original cycle-0022 form was a -Rust-authored vetted floor `InstrumentSpec { pip_size }` + `instrument_spec(symbol)`; -cycle 0074 removed that floor — see the C15 note — once the sidecar geometry made it -redundant for the real path. Refuse-don't-guess on absent geometry.) The honesty -rule is **refuse, don't guess**: a real-data run for a symbol with no vetted spec -is a usage error (`exit 2`). Threaded through the CLI `aura run --real` path; the -manifest broker label records the looked-up pip. - -**Realization (position-event schema, cycle 0063, #114).** [Survives as the -**deploy / reconciliation** schema per the 2026-06-28 reframe — no longer a -broker-input research artifact.] A closed `PositionAction { Buy, Sell, Close }` -enum + the `PositionEvent` row (`event_ts`, `action`, `position_id`, -`instrument_id`, unsigned `volume`; no `open_ts`; direction *is* the action) live -beside `RunMetrics` as a post-run value type (not a per-`eval` node — C8) — both in -the `aura-analysis` crate since cycle 0079 (#136); originally `aura-engine`, see the -C16 cycle-0079 note. `action` serde-encodes as a bare `i64` (Buy=0, Sell=1, -Close=2), the C7 scalar column form, with an out-of-range code rejected on read. -The table stays broker-independent. - -**Realization (cycle 0065 — Stage-1 R signal quality, #119/#126/#127/#128/#129).** -[The R spine here is the live model; the *Stage-2 deferral* clause at its end is -SUPERSEDED by the 2026-06-28 reframe — there is no Stage-2 currency / compounding -layer; cost is now the cost-model graph and money lives only at the live deploy -edge. "Stage-1" reads as a historical identifier, not a gate half.] `Exposure → -Bias` renamed the unsized strategy output (node + output field); the persisted -`exposure_sign_flips` metric key (serde alias), the `SimBroker` `exposure` input -slot, and the on-disk `exposure` **tap** label retain the old name. The -strategy-output **param namespace** (the `Bias` instance, its `bias.scale` knob, -the `bias_scale` manifest param) was completed in #134. A **stop-rule** defines 1R: -`FixedStop` (a triggered-constant primitive) and a `vol_stop(length, k)` -**composition** `k·√EMA(Δ²)` (a composition of `Mul` / `Sqrt` primitives). -**`PositionManagement`** (`aura-std`) is the stateful heart: it latches the -entry-cycle stop distance as the immutable R-denominator, marks against the -one-cycle-lagged fill (no look-ahead, C2), and emits a **dense 14-column per-cycle -R-record** (one row per eval, C8; the trade ledger is the `closed_this_cycle` -subset, the R-equity is `cum_realized_r + unrealized_r`). The **`Sizer`** (`size = -risk_budget / stop_distance`, flat-1R) was the feed-forward sizing seam, and **R is -size-invariant** — scaling `risk_budget` leaves every `realized_r` unchanged -(pinned by a RED test); per the 2026-06-28 reframe the Sizer and `size` / `volume` -are removed from research (size is a deploy concept). **`summarize_r`** is a -post-run fold (sibling of `summarize`, **not** an in-graph node) → `RMetrics` (E[R], -SQN, win-rate, profit-factor, max-R-drawdown, conviction terciles, net-of-cost); -per the 2026-06-28 reframe it folds the cost-model's net-R rather than recomputing -a scalar cost. The **RiskExecutor** ships as a public `aura-composites` -composite-builder (`risk_executor(StopRule, risk_budget)`) with a -`StopRule{Fixed,Vol}` **structural axis** (C20); per the 2026-06-28 reframe its -Sizer interior and `risk_budget` arg are dropped / vestigial, the **Veto** stays a -documented seam, not a runtime node. The layer is operable from the CLI: `aura run ---harness ` — a compile-time selector over Rust-authored -harnesses (C9/C17) — folds `summarize_r` into `RunMetrics.r`, the `r-sma` -harness fanning one bias into both `SimBroker` (legacy pip) and the RiskExecutor -(R); an `r_equity` tap charts the by-trade R-equity. Composites live in the -dedicated `aura-composites` crate, so `aura-engine`'s runtime dependency stays -`aura-core`-only and `aura-std` is an `aura-engine` `[dev-dependencies]` (the graph -stays acyclic). -[HISTORY — the built-in `--harness` selector was retired with the demos → -blueprint-data (#159, cuts 1b-4); `run` is now blueprint-driven — `aura run -` over examples/r_*.json.] - -**Realization (2026-07-06 — the risk regime as a structural campaign axis, #210).** -The `StopRule{Fixed,Vol}` structural axis is realized at the campaign-document -level as `CampaignDoc.risk: [RiskRegime]` (a serializable, content-addressable -mirror of the runtime enum — `aura-research`, variants `Vol{length,k}` and -`VolTf{period_minutes,length,k}` (#262), the fixed-stop rule additive when -needed). It is a **kept-separate** matrix axis, a -peer of instruments and windows: the executor keys the nominee map by -`(strategy, window, regime)`, so generalize aggregates across instruments -*within* a regime, never across regimes. Regimes are therefore **compared** at -presentation, never argmax-**selected** across — a cross-regime E[R] argmax would -compare R-multiples in different R units (the stop defines 1R), so the legacy -verbs' `--stop-length`/`--stop-k` joint stop grid is retired as a campaign -methodology (finding the best regime = the same walk-forward / worst-case-R -validation, run once per regime, and picking the most robust — a comparison, not -a search). Each member manifest stamps its resolved stop (default included), -closing the C18 gap; absent/empty `risk` = one implicit default regime, -absent-serializing for content-id parity. Two default *representations* -coexist by design (#217): a dissolved sweep binds no regime at all -(`risk: []`), late-resolved per member by `stop_rule_for_regime` at run -time, while walkforward/mc/generalize — whose `--stop-length`/`--stop-k` -became optional — bind the default regime *eagerly* into the campaign -document (`risk: [Vol{length:3,k:2.0}]`). Same R behaviour either way, but -deliberately different document content ids: a stop-less verb invocation's -document equals its explicit `--stop-length 3 --stop-k 2.0` spelling, not -a stop-less sweep's document. Deferred (documented): regime-aware -**trace** persistence — the trace re-run and cell-key dir naming still assume the -default stop, since `CellRealization` carries no regime (`#212`); the core -run/stamp/generalize path is unaffected. - -**Realization (cycle 0066).** SQN is the operational single-number objective for -ranking an r-sma sweep family by signal quality — C12 **axis-2 (argmax-metric)** -over the C18 family store. `metric_cmp` (`aura-registry`) learns the -higher-is-better R metrics `sqn`, `expectancy_r`, `net_expectancy_r`; a member -with no `r` block sorts last (`NEG_INFINITY`). `aura sweep --strategy r-sma` -produces the rankable R family, each member folding `summarize_r` into -`RunMetrics.r`. The default grid varies **only the signal** (`fast` / `slow` SMA -lengths), holding the stop and sizing fixed: `risk_budget` is R-invariant and -`bias.scale` is sign-only under flat-1R, and — load-bearing — the **stop defines -1R**, so varying it would change what R *means* per member and break cross-member -SQN comparability. Each swept member's manifest records the fixed R-defining params -beside the floated knobs (reproducible from its own manifest, C18). -[HISTORY — the built-in `--strategy` sweep surface was retired with the demos → -blueprint-data (#159, cuts 1b-4); the rankable R sweep now runs as `aura sweep - --axis …` over r_sma_open.json (an example then; relocated to -`crates/aura-cli/tests/fixtures/` by #248).] - -**Realization (cycle 0067, #130 + #135).** **#130 (SQN100):** `RMetrics` gains -`sqn_normalized = (mean_R/stdev_R)·√(min(n, 100))` — the n-normalized "SQN score" -(`SQN_CAP = 100`), turnover-robust where raw `sqn` rewards trade count. It is an -**opt-in** rank key (`Metric::SqnNormalized`); raw `sqn` and the default ranker -stay byte-unchanged, and the field carries `#[serde(default)]` (C18). Below the cap -(`n ≤ 100`) it equals raw `sqn` exactly. **#135 (r-sma `--trace`):** -`r_sma_sweep_family` persists each member's equity / exposure / r_equity under -`runs/traces///` via the same `persist_traces_r` the single run uses; -per-member `--trace` is symmetric across all three sweep strategies. -[HISTORY — the built-in `--strategy` sweep triple (sma / momentum / r-sma) was -retired with the demos → blueprint-data (#159, cuts 1b-4). The per-member -`--trace` symmetry was NOT carried to the `aura sweep ` path — it -was silently dropped at #159/#220 (`run_blueprint_sweep` never wired `persist`; -`let _ = persist`), and #168 makes the surface honest: `sweep`/`walkforward` (like -the pre-existing `run`/`mc`) now refuse `--trace` outright. See the CLI-`--trace` -retirement amendment below.] - -**Realization (cycle 0068, #115 — position-event derive).** [The derivation -survives as the **deploy / reconciliation** layer per the 2026-06-28 reframe — its -"realistic brokers consuming it" goal is retired; money is the live-edge concern.] -`derive_position_events(record, instrument_id) -> Vec` (in -`aura-analysis` since cycle 0079, #136; originally `aura-engine`, beside -`summarize_r`) is the **first difference of the executed book**: a pure post-run -reduction over the `PositionManagement` dense record (read positionally as -type-erased `Scalar`s, C7 SoA — no in-graph node, so the hot path stays -domain-free, C14), emitting a `Buy` / `Sell` at each open and a `Close` at each -exit, a reversal (or stop-then-same-cycle reopen) emitting **Close then the -opposite open at one `event_ts`** (close first — the C8 ">1 event per instant" case -that forces a *derived* table, not a per-`eval` output). The close sizes the -**actual book** (the closed position's stored volume), never an exposure delta. -`instrument_id` is a caller-supplied scalar (`aura-analysis` depends only on -`aura-core`). A position open at window end emits its open with **no synthetic -`Close`** (the table records actual executed events; `summarize_r`'s force-close is -for the R metric only). The `r_col` ⟷ PM-record lockstep is guard-pinned for -`direction` too. - -### C11 — Generalized sources; record-then-replay determinism boundary -**Guarantee.** A source is anything that produces timestamped scalar streams — -market data (`data-server`) and non-financial sources (e.g. a news-agent node -emitting a bias) are treated identically. Anything nondeterministic, external, -or slow (LLM/news/web) is materialized into a recorded, timestamped stream -*before* it enters the engine; backtest replays the recording, live computes -fresh in real time and records it for future backtests. A bias enters as a value -held until the next event (firing policy A). -**Forbids.** Any live external call *inside* a backtest replay. -**Why.** It is the only model compatible with reproducible backtests — LLM calls -are nondeterministic and far too slow per-cycle. Per `~/.claude/CLAUDE.md`, -external LLM (IONOS) calls happen only at the recording/live-source edge, with -explicit per-session consent, never inside a sim. - -### C12 — The atomic sim unit and the four orchestration axes -**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window + -RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime -values injected at graph build (no recompile per param-set; the optimizer sees a -generic vector of typed ranges). Raw data is shared read-only across sims via -`Arc<[T]>` (data-server is built for this). Four axes orchestrate the atomic -unit: (1) param-sweep (grid/random), (2) optimization (argmax metric), -(3) walk-forward (rolling in-sample optimize + out-of-sample test), -(4) Monte-Carlo (N seeded realizations perturbing input). **MC = sweep over -seeds**; each realization is itself deterministic given its seed. -**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the -only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait -(`peek`/`next`, object-safe) the k-way merge drives — and a streaming -**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing -**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and -decoding each `Scalar` on demand: the **source ring** is resident O(one chunk), -not O(window length) — the measured `resident_records()` predicate, a *per-source* -bound, not whole-process RSS. (Data-server's `FileCache` retains each window's -parsed chunks read-only for the pass — ~56 B/record — so process residency is -O(records-touched); that is the **replay-many** sharing C12 wants — one window -parsed once across a sweep family — not a leak. The single-pass cost is tracked as -#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the -gap closes by a streaming path *existing*, not by deleting the eager one). **Realized -(2026-06-29, family cycles shipped).** cross-*sim* `Arc<[T]>` sharing — one window shared -zero-copy across many disjoint sweep sims — is now in force: the sweep / Monte-Carlo / -walk-forward families (axes 2–4 above) build their members over **one** shared -`Arc` (one `FileCache`), so a window is parsed once and every member's -`M1FieldSource` borrows the same cached `Arc<[M1Parsed]>` chunks -(`crates/aura-ingest/src/lib.rs:316`). The single-pass *parse* cost stays tracked as #95. -**Realization (cycles 0028, 0049).** Axis 1 (param-sweep) is built. `GridSpace` -(0028) enumerates a cartesian lattice over discrete per-slot value-lists; -`RandomSpace` (0049) draws `N` seeded points over typed continuous `ParamRange`s -(I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), validated against the -param-space before any run. Both implement the `Space` trait the disjoint -`sweep` / `run_indexed` core is generic over, so either enumeration runs through -one execution path (C1: results in enumeration order, not completion order). The -seeded sampler reuses the bit-stable `SplitMix64` as a **code-path-disjoint** -instance from the data-edge seed RNG (the source-seam firewall, #52/#71: they -share only the `u64` type, never a path). -**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the -primitive — those are pluggable policies atop the atomic unit; recompiling on a -param change. -**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell" -(embarrassingly parallel across the unit) cleanly separated from search policy. -Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the -atomic unit is one harness instance, selected by the harness's **structural -axes** (C20); the structural experiment matrix is the outer orchestration over -this dimension, the tuning sweep the inner (C19/C20). -**Amendment (2026-07-12, #246).** A bound blueprint param is the param's -**default**: axis 1 (param-sweep) may name a bound param — the family boundary -re-opens it on the probe and on every member reload, and the axis binds it per -cell. "Open" means *must be bound by an axis*; "bound" means *default, -overridable by an axis*. `run`/`mc` still require every param resolved (a truly -open param still refuses). Identity is untouched: `content_id_of` and -`topology_hash` read the authored document, never a re-opened probe, and each -member manifest records its per-cell bindings as before. The restriction this -amends — axes bind only open knobs — was an implementation consequence of -`bind()` shrinking the param surface, not a recorded decision. - -### C13 — Hot-reload is authoring-only; deploy is frozen -**Guarantee.** A node/strategy is authored as a native Rust `cdylib`, -hot-reloaded during the authoring loop (Rust-ABI; host and node built with the -same toolchain). The live/deploy bot is a statically-linked, versioned, frozen -artifact. -**Forbids.** Hot-swapping a running live bot; loading third-party / foreign- -toolchain plugins. -**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen -and reproducible (audit trail: this bot = this commit). A sweep pays no -hot-reload tax — params are runtime data (C12), so the cdylib loads once. -**Realization (cycle 0102 — the load boundary; per-invocation reload).** The -authoring-loop half is realized: a project is an external cdylib crate loaded -per invocation through a two-tier `#[repr(C)]` descriptor (`AURA_PROJECT`, -`aura-core::project`) — a C-ABI stamp prefix (rustc version + aura-core -version, baked per consuming build) validated **before** any Rust-ABI field is -touched, then the vocabulary resolver + enumerable type-id list behind the -stamp gate. "Hot-reload" reads, in v1, as **per-invocation load of the -freshest build**: the author (Claude) runs `cargo build`, the next `aura` -invocation locates the artifact via `cargo metadata` (debug default, -`--release` opt-in) and loads it **load-and-hold** (leaked, never unloaded). -Scope boundary: load-and-hold is trivially sound only because the CLI is a -one-shot process — a future long-running host (the open local-server thread, -C22) must re-solve reload (host restart or subprocess isolation), never -in-process unload. Mismatch of either stamp refuses (exit 1) naming both -sides; the project vocabulary is charter-checked at load (`::`-namespaced ids, -no duplicates against std, list↔resolver cross-check) — the invariant-9 -data-plane discipline of the C24 enforcement-shift note, now enforced at the -one seam. - -### C14 — Headless core, two faces -**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a -**programmatic/CLI** face (the primary surface for the LLM and automation — -author a node, run a sim/sweep, emit structured metrics) and a **visual** face -for human exploration. Visualization is only a downstream consumer node on the -streams. -**Forbids.** Any UI/pixel knowledge inside the engine. -**Why.** The LLM drives programmatically, the human visually; a headless core -serves both. The visual face is the **playground** (C22) — a **web frontend -served from disk-persisted traces** (revised 2026-06, issue #101: the engine -writes recorded traces to disk, a browser charts them; supersedes the earlier -egui-native / in-process-zero-copy-from-SoA direction). It is staged after the -runnable substrate but is core to aura's identity, not optional. The pivot keeps -this contract's core intact — and arguably tightens it: a browser reading a -serialized trace file is a stricter "no UI knowledge in the engine" than egui -reaching zero-copy into the live SoA columns. -**Realization (cycles 0098/0099, #175 — the CLI meets GNU/clig.dev conventions -via clap).** The programmatic/CLI face's argument surface moved from a hand-rolled -argv parser to a `clap` derive parser (admitted under the C16 per-case dependency -policy — research-side, a dev-loop/compile tax, never a frozen-artifact tax; -invariant 8 untouched, the change is confined to `aura-cli`, a leaf binary the -frozen deploy artifact cannot pick up). One declarative source now yields scoped -`aura --help`, `--version`/`-V`, a per-flag Options section, and GNU -`--flag=value` / `--` / long-option abbreviation. The **exit-code partition** is a -durable part of the automation contract, so a caller can branch on the failure -class without parsing stderr: **exit 0 = success**; **exit 2 = usage error** — a -command-line fault (clap parse errors + aura's post-parse argument-structure -validations, *including* the content of an argv-named blueprint file, which is -itself an argument); **exit 1 = runtime failure** — a well-formed command whose -needed environment / recorded state is missing, or bad piped stdin data. The four -dual-grammar subcommands (run/sweep/walkforward/mc) keep both grammars under one -token via an optional `[blueprint]` positional + a post-parse `is_file()` -dispatch; the execution layer is unchanged (arg-plumbing via thin `*_from` -adapters). Error-message casing is normalized to the clap house style — -every hand-rolled usage line reads `Usage: aura …` (#179, cycle 0101); -refusal diagnostics stay unprefixed (diagnostics are not usage lines). The -machine-first help surface (JSON/manifest help, stdin op-scripts) stays on the -#157/C21 track, not this cycle (settled as human/GNU convention compliance). -**Amendment (2026-07-13, #249 — two artifact classes, two redundancy budgets).** -The data layer the programmatic face emits is a public interface read raw (by -humans and LLMs), and its records split into two classes with opposite -redundancy budgets: **generated outcome records** (manifests, metrics, family -reports) have a single writer at run time — redundancy there cannot drift and -is deliberately spent on direct readability; **authored intent artifacts** -(blueprints, op-scripts, campaign documents) are author-maintained — every -redundancy is a drift site and stays out. Consequence shipped with #249: a run -manifest stamps the untouched bound defaults (`defaults`, one-directionally -widened beside `params`, which keeps its "what varied" reproduce semantics) so -a raw reader of a fully-bound run no longer sees a misleading `"params": []`; -the blueprint itself carries no such duplication. -**Amendment (2026-07-20, #295 — the programmatic face is an executor).** The -member-run recipe — the definition of what a standard aura backtest is — is -library content (`aura-runner`/`aura-backtest`/`aura-measurement`, C28 -assembly position), not CLI content: the binary is argv → document → -executor → presentation, and a downstream World program reaches the same -recipe with no binary involved. See C25's control-surface amendment for the -projection rule. - -### C15 — Resampling-as-node; sessions/calendars -**Guarantee.** A resampler is a node (finer stream → coarser bar stream), -clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars -and instrument specs are metadata (non-scalar, beside the hot path); session -*context* is exposed as scalar streams via a `SessionNode` (`bars_since_open: -i64`, `in_session: bool`, `session_open_ts: timestamp`). "3rd 15m candle after -session open" is then a plain node checking `bars_since_open == 3`. -**Forbids.** Streaming the calendar; special-casing session logic outside the -stream model. -**Why.** Keeps the line consistent — everything a signal needs arrives as a -stream; reference data feeds source/session nodes from beside the hot path. -**Realization (instrument specs, 2026-06, #22 → #124).** The "instrument specs are -metadata" half of this contract is realized by the **recorded geometry sidecar**: -`instrument_geometry(server, symbol)`, over data-server's `symbol_meta`, returns -neutral broker-agnostic `InstrumentGeometry` (the raw provider JSON never enters this -repo) — non-scalar reference data held beside the hot path, keyed by symbol, feeding -the sim-optimal broker's pip divisor (C10). The real-path pip is sourced from -`InstrumentGeometry.pip_size`; refuse-don't-guess on absent geometry. **History:** -cycles 0022/0063 first carried a Rust-authored vetted floor (`InstrumentSpec` — a -single `pip_size`, later a six-field deploy-grade row `instrument_id`, -`contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`, `quote_currency`, plus -`tick_size`/`digits`), cross-checked against the sidecar geometry. **Cycle 0074 -removed that floor**: with the sidecar geometry supplying the real-path pip, the -authored table was redundant, so `InstrumentSpec` / `instrument_spec` / the vetted -list were deleted. The speculative deploy-grade fields and the override/floor tiers -of the #124 hierarchy (tier 1 authored override, tier 3 authored floor) are -**deferred** until a consumer — the cost model's currency→R normalization (C10), or -the live deploy edge (real broker) — needs them, read from the sidecar geometry then. The `quote_currency` runtime-type transition is likewise -deferred to that consumer; #124 collapses to **recorded sidecar geometry → refuse** -in the meantime. - -**Realization (SessionNode — `bars_since_open` only, by design; #154).** The -session-context half of this contract ships **one** scalar stream, not the three the -Guarantee lists: `Session` (`crates/aura-std/src/session.rs`) emits only -`bars_since_open: i64` (tz-aware, DST-correct, baked Frankfurt open). This is a deliberate -narrowing pinned in the node's own contract — *"there is no separate in-session bool gate, -`bars_since_open` alone is the contract"*: a downstream `EqConst(== N)` gate subsumes the -`in_session: bool` stream (pre-open instants give `<= 0`, which never match), and -`session_open_ts: timestamp` has no consumer. The two streams the Guarantee names remain -the original design intent, **deferred until a consumer needs them** (default-simple; -forward-queued as #154). The stream model is untouched — session context is still exposed -as scalar streams fed from beside the hot path. - -### C16 — Engine / project separation; three-tier node reuse -**Guarantee.** aura is the reusable **engine**; each research project is a -separate external repo that depends on aura via cargo (the game-engine / game -split). Node reuse is cargo-native, in three tiers: **`aura-std`** (universal -blocks, ship with the engine) / **shared node crates** (cross-project-reusable, -their own repos, pulled as cargo git deps) / **project-local `nodes/`** -(experimental, project-specific). A reusable node is an `rlib` dependency; the -hot-reload unit stays the project-side `cdylib` that composes it (consistent -with C13). Concretely a project is a **Rust crate** — a cdylib library of node / -strategy / experiment blueprints — plus a static `Aura.toml` (project context, -paths only: data archive root, runs dir — cycle 0102). During -research the `aura` host loads and runs it (C13 hot-reload); for deploy the -chosen strategy + broker freeze into a standalone binary. **A project is -therefore always a Rust program built on the engine.** Beyond the node-reuse -tiers, the engine workspace also carries non-node crates — `aura-engine` (the -run loop), `aura-cli` (the `aura` binary), and **`aura-ingest`** (the -data-source ingestion edge, cycle 0011, where the `data-server` external tree -enters). **Dependency policy (amended 2026-06-10, cycle 0029).** Dependencies -are admitted by deliberate, **per-case review** — what a crate pulls in weighed -against what it buys — with **particular scrutiny for anything that enters the -frozen deploy artifact** (C13: this bot = this commit). Well-established -standard crates (`serde`, `rayon`, …) pass that review and are used wherever -they do the job, including in the bot. There is **no blanket zero-dependency -commitment** and **no blanket admission**; hand-rolling what a vetted standard -crate already does is the anti-pattern, not the dependency. This strikes the -original *"zero-external-dependency by commitment"* clause and the -*"`aura-ingest` is the sole external-dependency firewall"* framing; `aura-ingest` -remains the data-source ingestion edge, no longer a dependency wall. -**Forbids.** Project-specific signals in the aura repo (it keeps at most -example/fixture nodes under `examples/` for its own tests); a multi-project -manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the -package mechanism). (Dependency admission is governed by the per-case policy -above, not a blanket ban.) -**Why.** The engine/game split keeps the engine sharp and reusable while each -project versions its own research with its own forward-queue. Promotion -(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism. -**Realization (cycle 0079, #136 — the analysis leaf becomes its own crate).** The -trading-domain **analysis** layer — the post-run reductions that are pure functions of a -run's recorded data (`RunMetrics`, `RMetrics`, the R reduction `summarize_r` / -`r_metrics_from_rs`, the position-event table `PositionEvent` / `PositionAction` / -`derive_position_events`, and the multiple-comparison hurdle math `inv_norm_cdf` / -`expected_max_of_normals`) — is extracted out of `aura-engine`'s `report` module into a -dedicated **`aura-analysis`** crate (deps: `aura-core` + `serde` only; `serde_json` is a -dev-dependency, test-only). This sharpens the engine↔domain seam: the run loop's crate no -longer *defines* the trading metrics — it `pub use`-re-exports them for source -compatibility this cycle, and still hosts the trace-coupled `summarize`, `RunReport`, -`RunManifest`, and the columnar trace utils. The non-node engine-workspace crates are now -`aura-engine` (run loop), `aura-cli` (`aura` binary), `aura-ingest` (ingestion edge), -`aura-composites` (composite-builder convenience layer), and `aura-analysis` (post-run -domain reductions + selection provenance). Behaviour-preserving (C1): every serde shape -is byte-identical (C18 goldens green). -**Realization (cycle 0080, #136 — engine-side extraction complete).** The last -trading-domain types still *defined* in `aura-engine`'s `report` module — the -selection-provenance types `FamilySelection` / `SelectionMode` — also move to -`aura-analysis` (the only non-cyclic home: `RunManifest.selection` needs the type and the -engine already depends on `aura-analysis`; the registry/CLI reach them unchanged via the -re-export). Behaviour-preserving (C1; suite 665/0). **Settled (2026-06-27, user-ratified -in the #136 thread):** the `aura-registry` `Metric` / `metric_cmp` / deflation vocabulary -**stays in the registry by design** — the registry is the trading *selection* layer, not -the domain-free engine, and its deflation statistics branch irreducibly on metric -identity (R-bootstrap vs. `total_pips` dispersion floor); forcing genericity would be a -one-implementor abstraction. *(Superseded 2026-07-20: measurement's IC became the second -implementor — #290 — and the vocabulary moved behind the `MetricVocabulary` trait -supplied by the outer rungs; see the C28 #147 disposition.)* **Deferred to the World / -C21 layer (explicitly *not* #136):** -making `RunReport` generic over a metric type and turning the orchestration/registry layer -into a reusable domain-agnostic substrate — until then `RunReport` / `RunManifest` / -`summarize` stay trace-coupled in the engine. -**Realization (cycle 0107, #198 — the campaign-execution leaf).** The non-node engine -workspace gains **`aura-campaign`**: campaign-execution *semantics* (cell enumeration, -preflight, gate evaluation, winner selection, walk-forward rolling, realization assembly — -see C18) as a leaf library crate whose deps deliberately exclude `aura-ingest` / -`aura-std` / `aura-composites`; harness construction and data binding enter only through -the one-method `MemberRunner` seam, so consumers (the `aura` CLI today; the playground and -tests tomorrow) bind their own runners while the semantics live here once. It is -explicitly NOT C21's World (a project-side program): it realizes one campaign document -and owns no topology, no data sources, no UI. -**Realization (2026-07-12 — wiring-only tier, #241).** The smallest project -is now data-only: `Aura.toml` + `blueprints/` + `runs/`, no crate. The load -boundary tier-selects (a `[nodes]` pointer list in `Aura.toml` → load that -crate; a root `Cargo.toml` → the pre-#241 native project, unchanged; neither -→ std-vocabulary-only). `aura new` scaffolds the data-only tier; -`aura nodes new` scaffolds a node crate **beside** the project and attaches -it — the visible role-2 switch (C25 role model). Provenance widened -additively: a data-only run stamps commit-only. Invariant 9's "a project is -always a Rust crate" was deliberately amended (user decision, 2026-07-12). - -### C17 — Authoring surface -**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** — -is authored in native Rust through **Claude Code + the skills pipeline**: the -human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI, -and reports metrics. Declarative config (`Aura.toml`) carries only **static -project context** (paths only: data archive root, runs dir — cycle 0102), -never logic. aura ships **no embedded coding-LLM**. IONOS LLMs are used only as a -*runtime data source* (news-agent bias, C11), gated by per-session consent, -never in the code path. -**Forbids.** An in-app LLM chat that generates node code inside aura; using -IONOS (weaker models) as the authoring brain. -**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's -failure; making weaker models the coding brain reintroduces the very problem. -Keeps aura's scope an engine + playground, not an LLM-IDE. -**Refinement (2026-06-29, #109 resolved — node *logic* is Rust; *topology* is data, -C24).** "All logic … including experiments/harnesses … is Rust" governs -**computation** — a node's math, a meta-program's control flow — and that stays -native Rust (the RustAst lesson, sharpened: a small LLM cannot reliably *apply* a -computational DSL, and a strong one does not need one). It does **not** bind -**topology** — which node feeds which, the structural axes, the param-space — to -Rust *source*. Topology is a **serializable data value** the World owns (C24): a -non-Turing static DAG over the closed, compiled-in node vocabulary (C8/C16), -carrying no computation. This is *not* the RustAst trap re-opened: that trap is a -DSL the author must *apply*; a topology-data blueprint is machine-generated / owned -and applied by no one. The no-DSL guard therefore stands unbroken — it forbids a -*computational* experiment / strategy language, not a static topology-data format. -The experiment-matrix *generator* may stay Rust (C20); its *output* is -topology-data. - -### C18 — Project management: one repo = one project, plus a run registry -**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git -(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as -the forward-queue, a research thrust = a milestone, the -`idea → experimental → validated → deployed` label gradient). (2) **Experiments -& results:** an Aura-native **run registry** — one record per run = a *manifest* -(node-commit + params + data-window + seed + broker profile) + *metrics*, -queryable, with *lineage* (composite ← signals; run ← inputs). Determinism -(C1/C12) makes a run reproducible from its tiny manifest, so the registry stores -manifests + metrics and re-derives full results on demand. Depth: **structured** -(promotion/status, lineage, run-diff). -**Forbids.** Storing results not reproducible from a recorded manifest; -duplicating git/Gitea inside aura; a multi-project workspace manager. -**Why.** Comparing experiments over time is the heart of the research loop and -has no home in git/Gitea; determinism makes a structured registry cheap. -Sequencing: the walking skeleton emits a manifest + metrics per run from day -one; the registry/index is a later milestone over manifests that already exist. - -**Realization (cycle 0029 — the flat run registry).** The experiments-&-results -plane shipped as `aura-registry`: an append-only JSONL store (`runs/runs.jsonl`), -one serde_json `RunReport` (`RunManifest{commit, params, window, seed, broker}` + -`RunMetrics`) per line, with a typed read-path (`load`) and best-first ranking -(`rank_by`/`optimize`). C9 holds — the registry depends on `aura-engine`, never -the reverse. - -**Realization (cycle 0045 — lineage as related records, #70).** The *lineage* -depth is now realized as a **family store**: a sweep / Monte-Carlo / walk-forward -run (the C12 axes) is persisted as a *set of related records* — each a -`FamilyRunRecord` (a `RunReport` stamped with its `family` + `run` + `kind` + -`ordinal`) — in a sibling JSONL (`families.jsonl`), leaving the flat `runs.jsonl` -path and its `append`/`load`/`rank_by`/`optimize` API byte-for-byte unchanged. -the user-facing `family_id = "{family}-{run}"` handle is **derived** from the -stored `family` name plus a per-name `run` index (assigned as a numeric max+1 — -not a content hash; re-running the same family mints a fresh id). `group_families` is the round-trip that re-derives a family -from the stored links (re-listable / rankable as a unit — C21). The manifest *is* -the re-derivation recipe (#71): no input-stream blob / path / payload enters a -record, and a member's window is **producer-supplied** via -`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage), -never a materialized-`Vec` scan at the call site. CLI surface: `aura mc`, -`aura runs families`, `aura runs family [rank ]`; `aura sweep` / -`walkforward` / `mc` persist via `append_family` with an optional `--name`. -**Realization (cycle 0078 — cross-instrument family + instrument lineage, #146).** -The comparison axis (C12) is realized as a `FamilyKind::CrossInstrument` family: -`aura generalize` runs one candidate across an instrument list and persists the M -per-instrument runs via `append_family`, each member self-identifying through a new -first-class `RunManifest.instrument` lineage field (serde-widened with -`skip_serializing_if`, so legacy lines and every non-cross-instrument path stay -byte-identical — C14/C23). The cross-instrument *generalization score* (worst-case R -floor + sign-agreement + per-instrument breakdown) is a **recomputable aggregate** -over those members, not a persisted family-level record — distinct from #144/#145's -per-winner selection annotation on `RunManifest.selection`. -Deferred (Non-goals): replay-dedup (content-addressed *identity* shipped — #158, -cycle 0094, Realization below); the "run-diff" -depth and ranking families against each other (cross-family, vs. within-family); -and a live producer for the flat `runs.jsonl` standalone-run path — no CLI command -writes it (sweep/walkforward persist to the family store; `aura run` does not -persist). **Resolved (#73, 2026-06): retired** — `aura runs list` / `rank` dropped; -families (C21) subsume standalone over-time comparison. The `aura-registry` flat -lib API is retained: `rank_by`/`optimize` keep live consumers (`optimize` backs -walk-forward's in-sample step, `rank_by` backs `runs family … rank`); `append`/ -`load` (the flat-store half) remain public API with **no in-tree caller** after -this retire — tested, available to external consumers, a latent dead-code surface -a later sweep may revisit. **Unknown-id contract (ratified, Runway fieldtest -2026-06).** `aura runs family ` treats an unknown-but-well-formed id as an -*empty family* (prints nothing, exit 0) — the same treat-as-empty discipline as -`Registry::load` reading a missing store as `Ok(empty)`, and deliberately -distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection -*before* any store access, not a found-nothing lookup. Tightening to a non-zero -`no such family ` exit (typo-safety) is an available future UX choice, not a -current contract. - -**Refinement (2026-06-29 — a generated run's topology lives in / is addressed by -its manifest, C24).** The manifest identifies a run's topology today only via -`commit` (this engine + a hand-coded harness) — sufficient while harnesses are a -finite hand-coded menu. Once the World **generates or structurally searches** -topologies (C21/C24), `commit` no longer identifies the graph, so the manifest must -**carry or content-address the topology-data** to stay the re-derivation recipe -(C18's "reproducible from a recorded manifest"). This pulls the previously-deferred -**content-addressed identity** non-goal forward as the natural home for a generated -topology's identity. The format and the carrier are C24's design; recorded here as -the reproduction requirement it must meet. - -**Realization (2026-07-01, cycle 0094 — content-addressed reproduction of a generated -run, #158).** A blueprint sweep's topology is now **content-addressed**: the canonical -`blueprint_to_json` bytes are stored once, keyed by the `topology_hash` the manifest -already carries, in a **dumb bytes-by-key store** beside the run registry -(`runs/blueprints/.json` — `Registry::put_blueprint`/`get_blueprint`, aura-registry; -no `sha2`, no parse — the caller owns the hash, and reproduction's bit-identical compare is -the integrity check). `aura reproduce ` re-derives every persisted member: load -the member's blueprint by its `topology_hash`, reconstruct the sweep point from the -recorded params, re-run through the **same** `run_blueprint_member` the live sweep uses -(so bit-identity is by construction, C1), and compare metrics — refuse-don't-guess on an -unknown id / missing stored blueprint (exit 1 — recorded state is missing, C14's -runtime-failure class; this paragraph over-claimed exit 2 until #298 recorded the code's -actual, C14-consistent behaviour), DIVERGED → exit 1. The id resolves first as the -derived `{family}-{run}` handle; a bare enumeration name naming exactly one stored run -resolves as fallback, an ambiguous name refuses listing the candidate handles (#298 — -the list-then-reproduce seam). The manifest + the -content-addressed store + the commit are the complete re-derivation recipe (C18). One -blueprint is stored per family (all members share the signal `topology_hash` — C11/C12 -dedup). `aura graph introspect --content-id` exposes the same id for an op-script, via the -one `content_id` primitive `topology_hash` also uses (acc 1); a Tier-1 optional the -blueprint does not use leaves the id byte-stable (acc 3, composing #156/#164). -`serde_json/float_roundtrip` is enabled so stored f64 metrics round-trip exactly through -`families.jsonl` — the precondition for a bit-identical compare (C1). **Signal-only this -cycle**: the id covers the signal blueprint; the fixed r-sma scaffolding stays -commit-identified (it is not yet blueprint-data, C24). Whole-harness / structural-axis -content-addressing remains deferred. The debug-name-in-id question was settled additively -(cycle 0104, #171): the **identity id** — the canonical form with every non-load-bearing -debug symbol blanked (invariant 11), hashed through the same `content_id` primitive and -surfaced as `aura graph introspect --identity-id` beside `--content-id` (combinable) — -makes same-topology blueprints comparable across authoring paths, while the byte-exact -`topology_hash` keeps every role untouched (introspection-only: no manifest field, no -store key until a dedup consumer exists). Reproduction is proven on -synthetic (deterministic) data; recorded-dataset reproduction rides the DataServer seam -(#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` — -branching on the family kind, it reconstructs each member's seed-driven synthetic walk from -the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives -bit-identically through the same `run_blueprint_member` path. **Cycle 0097 (#173):** reproduce -spans the third variant, `FamilyKind::WalkForward` — the same branch rebuilds each OOS member's -windowed slice from `manifest.window` (winner params via the shared `manifest→cells` recovery), -so an `aura walkforward` family re-derives bit-identically too. All three family kinds now -persist *and* reproduce through the one shared `topology_hash`+`put_blueprint` hook. -**Realization (cycle 0106, #189 — research-artifact document stores).** The registry's -content-addressed store family grew two siblings beside `blueprints/`: `processes/` and -`campaigns/` hold the two research-artifact document types shipped by the #188 role-model -pass — the **process document** (role 5: a named validation/eval methodology, a closed std -stage vocabulary wrapping shipped primitives) and the **campaign document** (role 6b: -persisted experiment intent — instruments × windows × strategy refs by content/identity id × -param axes × process ref (content-id-only) × data-level presentation). Documents are -canonical JSON (`format_version` envelope, omit-defaults, no trailing newline) keyed by the -**shared content-id primitive, now library-hosted** (`aura_research::content_id_of`; -`aura-cli`'s `content_id`/`topology_hash` delegate byte-identically — the id goldens pin the -move). Unlike `put_blueprint` (caller owns the hash), the document puts self-key from -canonical bytes; gets are `Ok(None)` treat-as-empty. The **referential tier** -(`validate_campaign_refs`) resolves process/strategy refs against the stores (identity refs -by store scan in this cycle; index-first since #191, below) and checks each campaign axis — name AND declared `ScalarKind`, the axis -carries its kind once — against the referenced blueprint's `param_space`. Campaign P1 -control constructs (axes/gates/ladders per #188) carried intent only in this cycle: no -executor existed yet — executor need was re-tested against the intent-persistence -diagnosis (#189 triage, decided item 6; the cycle-0106 fieldtest F7 verdict was that -evidence), and the v1 executor shipped the next cycle (below). -**Realization (cycle 0107, #198/#196 — the campaign executor and its realization store).** -`aura campaign run ` executes a campaign (a file is register-then-run -sugar; the content id is canonical): zero-fault referential gate, then the process -pipeline — v1 executable shape `std::sweep [std::gate]* [std::walk_forward]?` -(`std::monte_carlo`/`std::generalize` refused loudly at preflight in this cycle; they -execute since cycle 0108, below) — runs once per (strategy, instrument, window) cell in -doc order. Execution -*semantics* live in the **`aura-campaign` library crate** (#198 home decision: reachable -beyond the CLI; NOT C21's project-side World): grid odometer over the campaign axes, -members through the engine `sweep` over a **`ListSpace`** (explicit point set beside -`GridSpace`/`RandomSpace` — a gate's survivor subset has no cartesian structure), -per-member gates via the 14-name `member_metric` roster (an R-predicate over a missing R -block fails conservatively), walk-forward re-rolled in the doc's epoch-ms unit -(`WindowRoller`; IS windows search ONLY the survivor points; OOS winner reports carry -`manifest.selection`), deflation nulls seeded from the doc's `seed` (C1: realization is a -pure function of doc + stores + data). Harness/data binding stays consumer-side behind the -one-method **`MemberRunner`** seam — the CLI binds the shipped loaded-blueprint reduce -convention with unique suffix-join of raw axis names onto the wrapped `param_space`. The -registry grew the **`campaign_runs.jsonl`** JSONL sibling (beside `runs.jsonl` / -`families.jsonl`): one thin `CampaignRunRecord` per run — campaign/process ids, seed, and -per-cell realized stage prefixes linking family ids, gate survivor ordinals, and sweep -selections — over untouched family records, run-counted per campaign id. Zero survivors -truncate a cell's realized prefix and exit 0 (a null result is a valid research result); -`emit` is honored (`family_table`/`selection_report` lines); `persist_taps` was deferred -LOUDLY on stderr in this cycle (the wiring shipped in cycle 0109, below). The **blueprint on-ramp** (#196) closes -the F5 authoring gap: `aura graph register` (store put keyed by content id == topology -hash), `aura graph introspect --params` (the RAW `param_space` namespace campaign axes -are validated against), and a blueprint-file mode on `--content-id`. The -`std::walk_forward` vocabulary was corrected to machinery-true fields -(`in_sample_ms`/`out_of_sample_ms`/`step_ms`/`mode` — `WindowRoller`'s three lengths and -both `RollMode`s; the shipped `folds` slot mapped to nothing the machinery accepts), with -a new `ZeroWalkForwardLength` intrinsic fault; wf-bearing process docs get new content ids -by design (the 0106 fieldtest corpus stays as the historical record). Known debt: -metric-roster triplication (still hand-maintained, but drift from the shipped -`aura-analysis` types is now test-caught by a cross-crate guard, #190; -single-source removal waits on #147), deflation-constant duplication (#199). -#300 (2026-07-21) closed the store's read loop: `aura process|campaign show -` prints a registered document's canonical bytes — the -generate → retrieve → hand-extend → re-register cycle needs no direct store -filesystem access. -**Realization (cycle 0108, #200 — the annotator stages execute).** The v2 executable -shape is `std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? -[std::generalize]?` — an ordered optional annotator suffix, each at most once, -`std::generalize` strictly last; the executor preflight refuses what the intrinsic tier -deliberately admits (`[sweep, mc, walk_forward]` is intrinsically valid — the tier -boundary is test-pinned on both sides), plus three new static guards -(single-instrument generalize, non-R generalize metric via the registry's -`check_r_metric`, zero mc `resamples`/`block_len`). **Both annotators are terminal** -(unanimous #200 triage): nothing flows out of them; filtering stays the gate's monopoly. -`std::monte_carlo` bootstraps the stage's *incoming R-evidence* with one semantics, -input-shaped by position — after a walk_forward, ONE `r_bootstrap` over the wf family's -pooled per-window OOS `net_trade_rs` in roll order (`PooledOos`; #259 materialized this -conduit as the cost-netted per-trade series `r − cost_in_r`, equal to the gross series -bit-for-bit when no cost model is bound); after sweep/gates, one `r_bootstrap` per -surviving member's fresh in-memory series (`PerSurvivor`, ordinals into the population -family; a zero-trade member records the engine's defined all-zero degenerate) — seeded -from the campaign doc's `seed` (C1; `net_trade_rs` is `#[serde(skip)]`, so annotators run -in-executor or not at all). `std::generalize` executes at **campaign -scope**: after all cells, per (strategy, window) the per-cell *nominees* (last wf -window's OOS report, else the sweep winner; none on gate truncation) across instruments -feed the shipped `generalization()` when ≥ 2 exist — divergent per-instrument winners -are exposed via their params, never averaged away; a shortfall is recorded, not computed -around. Realization: `StageRealization.bootstrap` (`StageBootstrap::PerSurvivor | -PooledOos`) and `CampaignRunRecord.generalizations` (`CampaignGeneralization` keyed -strategy × window with `winners`/`missing`), both serde-default sparse — pre-0108 -`campaign_runs.jsonl` lines parse unchanged (C14/C23). No document content id moved -(introspection doc-strings dropped "in v1" only). Noted debt: the mc arm detects the wf -family by the stringly `block == "std::walk_forward"` literal (a pre-existing lockstep -pattern with the realization's block strings). -**Realization (cycle 0109, #201 — persist_taps wired).** Campaign presentation persists -traces: the tap namespace is a **closed vocabulary** of the wrap convention's four sink -names (`equity`/`exposure`/`r_equity`/`net_r_equity`; `aura_research::tap_vocabulary`, -intrinsic `DocFault::UnknownTap` — the escalation for a new observable is a new -vocabulary entry or an authored blueprint sink, never an open node-path namespace). -Scope is the per-cell **nominee only**: after the pipeline settles, the CLI re-runs each -nominee once in non-reduce mode (all four channels drained; windowed to the nominee -manifest's own ns bounds) and **asserts metrics equality** against the recorded nominee — -the C1 drift alarm, hard refusal on divergence (the reproduce precedent, enforced). -Traces land in the existing TraceStore as -`traces/{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json` -(`ensure_name_free(Family)` once; window ordinal doc-derived), chartable by the unchanged -viewer. The record carries ONE sparse pointer, `CampaignRunRecord.trace_name` -(`Some("{campaign8}-{run}")` iff the doc requests taps — the claim-sentinel contract: -`execute` claims, `append_campaign_run` composes the name via the single-sourced -`derive_trace_name`, `execute` mirrors it onto the returned copy). Loud lines replace -the retired deferral: per-cell no-nominee skip, per-run unproducible-tap skip -(`net_r_equity` needs a cost leg; the campaign runner wires none), one summary. -`aura-campaign` stays trace-agnostic (the MemberRunner seam is unchanged; the stamp is a -pure name derivation). Noted debt: `aura chart` over the campaign family ROOT (cells -spanning instruments) is untested/semantically undefined — only per-cell read-back is -pinned. -**Realization (#272, 2026-07-14 — per-cell fault isolation).** A member -fault (no-data, bind, run, or a caught panic) is a recorded per-cell outcome, -never a global abort: `run_cell` returns a fault-annotated `CellRealization` -(`fault: Option`, closed `CellFaultKind`) instead of `Err`, so -`execute`'s existing accumulate-then-append-once tail persists every healthy -cell and the one run record. Containment granularity is the cell for a sweep -stage (a grid hole compromises selection) and the fold for walk_forward -(surviving folds pool; failed folds recorded as `StageRealization. -window_faults`, the summary naming the ratio). `ExecFault::Registry` and -doc-shape preflight faults stay global. The CLI declares holes (per-cell -notes + a completion summary) and a run with ≥1 failed cell exits **3** -("completed with failed cells" — distinct from 0/1/2). A partially-covered -window carries a `CellCoverage` annotation (effective bounds + interior gap -months, from the #264 archive primitives). Generalize already treats a -no-nominee cell as `missing`, so a failed cell surfaces there unchanged. -Member panics are caught with `catch_unwind`(`AssertUnwindSafe`) at the three -member-run sites and recorded as `MemberFault::Panic`; a ref-counted -`SilencedPanic` guard (a process-global panic-hook save/no-op/restore behind a -`static Mutex`, held only around each `catch_unwind`) suppresses the default -crash backtrace so "recorded, campaign continues" is observably true on stderr, -not merely true in the record. The guard's mutex serialises only the -ref-count/hook-swap (O(1)), never the member computation, so C1 disjoint-parallel -execution and determinism are preserved; ref-counting (save on 0→1, restore on -1→0) keeps concurrent sweep/walk-forward threads and any caller-installed hook -correct. -**Realization (#191, 2026-07-17 — identity-ref resolution is index-first).** -`find_blueprint_by_identity` no longer re-loads the whole blueprint store per -reference: a fourth persistent JSONL sidecar, `blueprint_identity_index.jsonl` -(identity id → content id; fixed-name sibling of the runs store, appends under -the #276 lock), is consulted first, and every hit is **verified** by loading -that one blueprint under the current resolver and recomputing its identity id — -the index is a cache, never an oracle, so resolution results stay -scan-identical under roster drift, store surgery, or index corruption (the one -unspecified corner is unchanged in kind: which same-identity twin answers was -`read_dir`-order-dependent before and is index-history-dependent now). Any miss -or failed verification runs the old scan as a **full-store repair pass**, -collect-then-diff-append: the walk's last-wins mapping is diffed against the -pre-walk snapshot after the walk, so a converged index — twin stores included — -appends nothing (the twin-convergence pin). Index reads never fail a lookup -(missing/unreadable file → empty, unparseable lines skipped), repair appends -are best-effort, and a pre-index store backfills itself on its first miss, no -migration; a read-only store keeps scanning as before. Write paths, the engine, -and both callers are untouched; maintenance is lazy-only — put-time indexing -was rejected because it would need a roster-free doc-level identity function -whose equivalence to the loaded-composite path no green test ratifies (decision -log: #191 comments). - -### C19 — Bootstrap: blueprint → instance (recursive) -**Guarantee.** Construction is a distinct phase, recursive at every level. Each -node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)` -sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic -graph-as-data produced by running a Rust builder (C9); it carries *free* numeric -params (declared ranges) and *free* input roles. The **bootstrap** binds -`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen -instance** — buffers sized, topology fixed. This is precisely the "wiring / -graph build" that C7 ("sized at wiring", "topology frozen per sim") and C12 -("params injected at graph build") already reference. The same machinery applies -recursively up to the harness (C20). A sweep builds many instances from one -blueprint; instances are disjoint (C1). -**Forbids.** Params that change topology (a topology change is a *different* -blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a -sim against an un-bootstrapped blueprint. -**Why.** Separating the param-generic blueprint from the param-bound instance is -what makes one strategy reusable across a whole sweep and lets the optimizer -mutate "the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead -of rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12 -explicit. -**Refinement (Construction-layer milestone, 2026-06-05).** This binding is a -**compilation**: the param-generic, named blueprint (source) is lowered to a flat, -type-erased **flat graph** (C7) **wired by raw index, not by name** -(`Edge { from, to, slot, from_field }`): composite boundaries dissolve entirely, -and field / role names are demoted to **non-load-bearing** debug symbols (as -`FieldSpec.name` already is). "No recompile" above means no **Rust / -cdylib** rebuild (C12/C13: the cdylib loads once); re-deriving an instance per -param-set is a cheap **graph re-compilation**, not a code recompile. Naming the -build phase a compilation makes its successor explicit: the flat graph is the target -of behaviour-preserving optimisation (C23). -**Refinement (#275, 2026-07-15).** One narrow exception to "role names demoted to -non-load-bearing": a `SourceSpec.role` (the lowered bound-`Role` name) is -**load-bearing for source binding** — the key `Harness::run_bound` / `bind_sources` -resolve a keyed source supply against. Every other flat-graph name (edges, ports, -composite boundaries) stays a non-load-bearing debug symbol; the raw-index -positional `run` path carries no role. -**Realization (cycle 0016 — param-set injection).** The bootstrap now binds an -injected param-set, realizing C12's "params injected at graph build (the optimizer -sees a generic vector of typed ranges)" and C19's "factory `params → sized node`" -*literally*: a blueprint leaf is **value-empty** — `BlueprintNode::Leaf` holds a -`LeafFactory { name, params, build }` recipe, not a built node — and the value lives -only in the injected vector (no baked default), so the blueprint stays a pure -param-generic recipe. (Renamed in cycle 0024: `BlueprintNode::Primitive` holds a -`PrimitiveBuilder { name, schema, build }` — the recipe now carries the full -signature, see the C8 0024 realization; `bootstrap_with_params`/`compile_with_params` -moved onto `Composite` when `struct Blueprint` collapsed into it, see the C19 0024 -realization.) `bootstrap_with_params(Vec)` / -`compile_with_params` **build each leaf through its own constructor** (the single -sizing/validation gate) from its kind-checked slice **while lowering** (build-then- -wire), consuming the vector slot-by-slot in the *same* depth-first walk -`param_space()` projects — so the two share one traversal (subsuming the #34 dual- -traversal hazard) and the value reaches the node at the slot the sweep enumerates. -Arity is checked up front (`param_space().len()`); a wrong-kind or wrong-length -vector is a typed `CompileError::{ParamKindMismatch, ParamArity}` (the typed-value -check C8 deferred). The lowering/edge/source rewrite is structurally unchanged, so -the **flat graph stays bit-identical** for a given point (C23, the correctness -invariant). The value *domain* (e.g. `length ≥ 1`) stays the constructor's own -`assert`; the search-range is still the run's (#32/C20). One value-empty leaf -detail for C22: the blueprint view (pre-run, param-generic) labels a leaf by **bare -type** (`PrimitiveBuilder::label`, was `LeafFactory::label`, → `[SMA]`) — the -value-bearing `SMA(2)` of C8's render- -label refinement now appears only in the *compiled* view (built nodes, `Node::label`). -**Realization (cycle 0017 — blueprint render = main graph + definitions).** The -`aura graph` blueprint view (C9 graph-as-data, #13) renders the **authored -structure** as a *program with subroutines*: a flat **main graph** wiring the -harness with each composite shown as a **single opaque node** `[name]`, plus a -`where:` section that defines each **distinct** composite type **once** (its -interior with named input-entry nodes and outputs folded onto their producers as -`name := …` bindings (render refined through cycles 0019–0022; originally -`[in:k]`/`[out]` port markers); deduped by `name()`, collected -recursively so nested composites are opaque nodes with their own definitions). This -supersedes #13's original cluster-box model and is the durable split this view -realizes: **blueprint = source** (composites as named subroutines, body once) vs. -**compiled = inlined machine form** (C23, boundary dissolved). The substantive cause -for retiring cluster boxes is a real renderer defect — `ascii-dag` 0.9.1's subgraph -level-centering rounds sibling x-positions with `/2`, overlapping wide sibling -labels (width/parity-sensitive, no config/padding dodge surviving unequal-width -siblings); the **flat** layout is collision-free, and both views now build flat -graphs only. The model also scales (blueprint size tracks top-level wiring, not -inlined node count) and removes #13's nested-composite `unimplemented!` (the -definitions pass recurses). The interactive *enter/focus* counterpart (a composite -collapsed to a navigable node) is the playground's, parked as a separate concern -(#37); the static CLI keeps the all-at-once definitions form (#38). - -**Realization (cycle 0024 — the root is the fully-bound composite; the flat graph is -a named type).** `struct Blueprint` is **deleted**: the root graph IS a `Composite`, -and `compile_with_params`/`bootstrap_with_params`/`param_space` are its methods. -What distinguished the root — its bound data sources — is now a property of its input -roles: `Role` carries `source: Option` (`None` = an open interior port, -wired by the enclosing graph; `Some(kind)` = a bound ingestion feed). A composite is -**runnable iff every root role is bound** (C3: sources bind at ingestion only); an -open root role is a compile-time `CompileError::UnboundRootRole`. So the "main graph" -is no longer a separate kind — only the composite all of whose roles are -source-bound, governed by the same conditions as any other node. Compilation now -targets a **named type**: `compile` validates structurally **pre-build** (via -`signature()`, no node constructed — an ill-typed wiring is caught before any build -closure fires) and emits `FlatGraph { nodes, signatures, sources, edges }` — the -C23 flat graph, now first-class — which `Harness::bootstrap` consumes (kinds/firing -from the carried signatures, buffer depth from `lookbacks()`). The per-flat-node -signature travels beside the node, so `bootstrap` reads it without a built-node -`schema()` call (which no longer exists, see the C8 0024 realization). - -**Realization (cycle 0026 — graph render redesign: model + WASM-Graphviz viewer, -#51).** `aura graph` no longer renders ASCII. The render path is now two pieces: -a read-only **model serializer** (`aura_engine::model_to_json`, iteration 1) that -walks the root composite + every distinct composite type into a deterministic, -hand-rolled JSON model (C14, golden-tested; the engine's last hand-rolled JSON -writer after `RunReport::to_json` moved to serde in cycle 0033; the -swapped-param mis-wire property moved here from the old compiled-view test), and a -**self-contained HTML viewer** (`aura-cli::render::render_html`, iteration 2) that -inlines that model, the ported prototype viewer JS, and a vendored Graphviz-WASM -blob into one page emitted to stdout. Layout/SVG happen in the browser via -WebAssembly — aura ships no layout engine and stays a serializer (C9: graph-as-data, -no `eval`/build on the path). The viewer is a render asset (C10 — no node/strategy -logic, no DSL); it labels every input pin from the model's now-real names (C23 -debug symbols, named in cycle 0027) and colours wires by the four scalar base -types (C4). This **retires `ascii-dag`** and its adapter (`graph.rs`), the -`--compiled`/`--macd` flag plumbing, and the invented `#Sf`/`:=`/`histogram →` -notation — superseding the cycle-0017 flat-ascii model and its renderer-defect -workaround. The DOT/SVG are Graphviz-version-dependent and not golden-tested; the -deterministic JSON model is the asserted contract. - -**Realization (cycle 0034 — structural-constant bind: a knob *removed* from -param_space, #55).** `PrimitiveBuilder::bind(slot, value)` adds the **third** param -category beside the topology factory-arg (C7/C19) and the tuning param (the -cycle-0016 value-pin): a **structural constant**. The cycle-0016 binding *pins* a -value in the injected vector while the knob **stays** in `param_space` (a tuning -param the sweep varies); `bind` instead **removes** the slot from `param_space` -entirely — the knob is gone, not fixed. The discriminator is the #55 -deform-vs-tune test: a value whose variation yields another valid point of the -*same* strategy is a **tuning param** (stays in `param_space`); a value whose -variation *deforms* the strategy into a different one (e.g. the `2` of an -"SMA2-entry" bound to its two-candle construction) is a **structural constant** -(bound out), so a sweep never enumerates deformed strategies as valid family -members. Mechanically `bind` shrinks the builder's declared param surface -(`schema.params`) and wraps its build closure to re-splice the constant at its -original positional slot; the construction layer -(`collect_params`/`lower_items`/`param_space`/`compile_with_params`) is -**byte-unchanged** — both dock sites already key off `builder.params()`, so the -shrink propagates for free, and chained binds reconstruct the correct positional -vector because each layer computes its slot index relative to the param list it -sees. C23 is unaffected: `bind` resolves the param **name** to a position at -**authoring** time (the by-name authoring address space, the 0032 amendment) and -the flat graph stays wired by raw index — the name never reaches it. The -complementary question — exporting a **named frozen** strategy (all/most knobs -bound) as a reusable blueprint *value* — is deferred (#60); `bind` ships only the -per-knob overlay, no registry (C9/C10 intact). - -### C20 — Strategy ↔ harness; the harness is the root sim graph -**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9): -broker-, data-, and viz-independent, with inputs declared as named **roles** -(symbol-agnostic where possible) and the **bias** stream (C10) as output. -A **harness** (the experimental setup) is the **root sim graph** — sources bound -to the strategy's input roles + the strategy + attached broker node(s) + sinks — -and is itself produced by the bootstrap (C19). A harness *instance* is C1's -disjoint unit (RustAst's "root scope"). The harness has **two kinds of -parameterization**: **structural axes** (which strategy, which instrument(s), -which broker(s), which window, which **risk regime**) whose variation selects -*different* instances — the **experiment matrix** (the risk regime is the fourth -axis, realized 2026-07-06 / #210 as `CampaignDoc.risk`; see the C10 realization -for its kept-separate, compared-not-selected semantics); and **tuning params** -(the strategy's numeric params) -swept *within* a fixed structure (Fork A). The same strategy blueprint is reused -across backtest, sweep, visual workspaces, and the frozen live bot — each a -different harness. **Both strategy and harness/experiment are authored in Rust** -via builder APIs (C17); the experiment matrix is ordinary Rust control flow -(loops/conditionals), not a config schema. Ontologically, a node (incl. a -strategy composite) is an **open** fragment — free input roles + ≤1 output -(C8/C9) — that does not run alone; a **harness is the closed root graph**: a -strategy with its input roles bound to sources and its output terminated in -broker/sink nodes, under a clock. A harness is therefore **not a node** (no free -inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's -disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21) -orchestrates them as objects. -**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a -declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as -a node (it is the closed root scope, not an open composable node). -**Why.** Reusability needs the strategy to be a context-free blueprint that many -harnesses embed. Modelling the harness as a root graph keeps it within the one -Node/graph abstraction (C9) and makes "10 strategies in one environment" and -"one strategy × N instruments" plain nested loops over the structural axes. Rust -authoring (not config) preserves full programmatic power — conditional/adaptive -matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap -C17 rejects. -**Realization (GER40 session-breakout blueprint milestone, 2026-06-17 — refs -#94/#96/#97, spec 0051).** Made concrete on the first real-data strategy: a -**hand-wired `FlatGraph` is not a shippable strategy** — it carries no -`param_space()`, so the World families (sweep / walk_forward / compare, C21) -cannot consume it without a hand re-author (the friction the GER40 deep-dive -fieldtest surfaced). The **canonical shippable form is the `Composite` -blueprint** (the authoring/source level, C9/C19); the `FlatGraph` is only its -compiled substrate (C23). The breakout now ships as -`ger40_breakout_blueprint(bar_period, …)` whose `param_space()` is exactly its -tuning knobs (`{entry_bar.target, exit_bar.target}`); the **bar period is a -construction argument** binding Resample + Session together — a structural -matrix axis (C12: a different period is a *different strategy*, not a sweep -point), never a `param_space` entry, so a sweep cannot desync the two clocks. -This is the concrete instance of the structural-axis-vs-tuning-param split -above (and `delay.lag`, a C8 structural constant, is bound out of the space). -**Refinement (2026-06-29 — experiment-matrix *members* are topology-data, C24).** -"The experiment matrix is ordinary Rust control flow, not a config schema" holds for -the **generator** — the loop / conditional that *enumerates* structural variants may -stay Rust (C17). But each **member** it yields is a **topology-data value** (C24), -not a hand-coded blueprint nor engine-baked source: a structural axis selects among -*data* blueprints the World constructs, mutates, and serializes. This is what makes -structural variation **first-class and searchable** rather than a hand-enumerated -menu — `HarnessKind` and the per-strategy `*_sweep_family` functions in `aura-cli` -are the pre-C24 scaffolding (topology-as-engine-source, a C16 tension), retired as -C24 + the project-as-crate layer land. [C26 realization, 2026-07-10 (#231): the -scaffolding's last data weld — `wrap_r`'s hard-wired `price`←close role and the -`M1Field::Close`-only open sites — is retired; a strategy's input roles now bind -archive columns by name (C26). `wrap_r`'s remaining R-scaffolding retirement -stays #159.] -**Refinement (2026-07-03, #188/#189 — experiment INTENT is campaign-document data).** -The "ordinary Rust control flow, not a config schema" clause and the forbids-clause -"declarative experiment mini-DSL" are scoped by the #188 role-model pass: they forbid an -open, logic-bearing experiment language (the RustAst trap), **not** the closed-vocabulary -**campaign document** (C25/C18) that now carries persisted experiment intent — data -windows × strategy ids × param axes × process ref under the total, declarative P1 -construct tier (bounded axes, gates, ladders; no variables, no general recursion, no -unbounded iteration — invariant 5's no-free-feedback move applied one level up). A -*generator* that enumerates structural variants may still be plain Rust (role 2/6a -technique); what it yields, and what a campaign declares, is data. Genuinely new logic -(metrics, analysis blocks) escalates to a new Rust block (role 2), never to a freetext -hole in the artifact. - -### C21 — The World: the meta-level is the product -**Guarantee.** Above the harness sits the **World** — the project's program / -"game" (C16: a Rust crate). Within it, **harnesses are dynamically constructible, -first-class objects**: meta-programs (walk-forward, sweep, optimize, -Monte-Carlo — C12's axes) construct harness instances at runtime via the -bootstrap (C19), run them disjointly in parallel (C1), aggregate / compare their -results, and discard the transient instances. A walk-forward rolls windows → -bootstraps a harness per window → stitches out-of-sample equity + parameter -stability into one meta-result. Orchestrating *families* of harnesses is -**first-class**, not a headless afterthought; the run registry (C18) is the -World's memory. -**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep / -comparison) to second-class headless-only status; treating the single backtest -as the product. -**Why.** What happens *within* one harness — backtest a strategy → equity — is -commodity; every quant system covers it. aura exists for the meta-level: a -programmable space where harnesses are dynamically built and families of them -orchestrated and explored. The deterministic single-harness engine (C1–C20) is -the **substrate**; the World is the **product**. -**Refinement (2026-06-29, #109 resolved — the World owns topology as data, C24).** -"Harnesses are dynamically constructible, first-class objects" is sharpened: their -**topology is a serializable data value the World owns** (C24), not Rust source. -This is the precondition for the World's full ambition — it can **generate**, -**mutate**, **structurally search** (genetic / NEAT-style graph operators, the open -search-policy thread), **compare**, and **serialize-for-reproduction** (C18) -*families that vary structure*, not only numeric params. A param sweep over a fixed -skeleton is the degenerate case; varying the skeleton itself needs -topology-as-value. The game-engine principle (C16) made literal: the engine owns the -scene / blueprint graph as content, instantiates and runs it, the way aura owns -harness topology as data. - -### C22 — The playground is a trace explorer; sinks are the recording mechanism -**Guarantee.** The World is a *program*: nothing is displayable until it runs, -and its harnesses are transient machinery (built, run, discarded — C21). The only -durable, displayable substance is the **recorded trace**, captured by **sinks** — -pure consumer nodes (C8) that persist a stream (equity, position events, a node's -output) into the run registry (C18). **Displayable = exactly what a sink -recorded**; with no sink, only input params + summary metrics remain. The -**playground** is therefore an **execution viewer / trace explorer**, and it -plays *any* harness (it is the harness *player*, not a harness, never bound to a -default one): before a run it shows the program *structure* (graph-as-data, C9) + -param knobs; during a run, live sink streams; after a run, recorded traces + -metrics from the registry — including meta-views (stitched walk-forward, sweep -surfaces, multi-strategy / instrument comparison). Observability is **explicit -and selective** — you instrument what you want to see; the choice of sinks is -part of the experiment. The engine ships sample harnesses *with* sinks so a -newcomer sees a populated trace immediately; a fresh project is empty until -built, run, and instrumented. -**Forbids.** A scene-editor model that assumes a persistent populated world; -constructing or wiring topology in the UI (topology is Rust + hot-reload, -C9/C17 — the UI reflects the live graph-as-data and tunes runtime params via -sliders, C12); retaining un-sinked data; binding the playground to a single / -default harness. -**Why.** A program has no scene-at-rest to inspect; what persists is what it -records. Making sinks the one recording-and-observability mechanism keeps "what -can I see?" answerable by "what did I instrument?", and keeps the engine -UI-agnostic (C14). Live param tuning (runtime values, no topology change — -C12 / C19 Fork A) gives the interactive feel without a wiring DSL. -**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the -substrate level: a recorded trace is exactly what a recording node pushed out of -the graph (no engine recording registry; the constructing World holds each -recording node's destination). The engine's single `observe: usize` affordance is -removed — `Harness::run` returns `()` and recording is a node-side concern, so one -run records *many* streams (one per recording node) instead of exactly one row. -Recorded streams are sparse and timestamped (a record per fired cycle, tagged -`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the -`Harness` API change (observe removed, `run -> ()`) is recorded here. -**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped -(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA, -C7) `ColumnarTrace` to `runs/traces//` (`aura-registry::TraceStore`, beside the -run registry's `runs.jsonl`), reachable via `aura run [--real …] ---trace `; `aura chart [--panels]` reads them back, aligns all taps on a -synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph; -C1 — pure, no live external call), and emits a static self-contained uPlot page -(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14): -encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in -`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`). -First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served -page injected the chart *series* but no run-context header (the run manifest is -persisted on disk in `index.json`, deliberately not surfaced in that first cut — a -ratified scope call). The header (#102) and serve-time decimation (#108) landed in -cut 2 (see the served-page-hardening amendment below); the families-comparison view -landed too (#107). A local server and replay-clock controls remain open (see "Open -architectural threads"). -**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends -to family runs: `aura sweep|mc|walkforward --trace ` persists *each member* as -a nested standalone run-dir `runs/traces///`, reusing -`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a -side-effect inside each per-member closure). The `member_key` is content-derived and -deterministic — sweep: the *varying* axes rendered as a filesystem-portable -directory component (`-` tokens joined by `_`, charset -`[A-Za-z0-9._-]`, case-less values, length-capped with an FNV fallback), MC -`seed{N}`, walk-forward `oos{ns}` — never -a runtime ordinal, because members run in parallel under the engine's `Fn + Sync` -HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write` -targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`, -stdout/registry are byte-unchanged. Because `TraceStore` resolves `/` -as a subpath, `aura chart /` charts any single member with no -view-side change — the write-side precondition for the family-comparison **view** -(overlay / small-multiples across members) — now realized (#107; see the -families-comparison amendment below). - -**Amendment (CLI `--trace` retired, #168 / #224).** The CLI-verb `--trace` story above (single-run `run --trace`, amendment 3b56efb; per-member `sweep|mc|walkforward --trace`, #104) describes capabilities that no longer reach the code. `run` and `mc` refuse `--trace` (structurally parseable, refused at dispatch); the per-member family `--trace` was never wired to the blueprint sweep (`run_blueprint_sweep`'s `let _ = persist`) and was silently dropped when the welded `--strategy` triple retired (#159/#220). #168 makes the surface honest — `sweep`/`walkforward --trace` now refuse up front (exit 2, forward-pointer to #224). The single live `TraceStore::write` site is the campaign `presentation.persist_taps` → `runs/traces//` (`persist_campaign_traces`); `aura chart ` reads that back. Restoring CLI-side per-member trace-writing is deferred to #224. **[Delivered 2026-07-11 (#224 + fieldtest B1 fix): `sweep`/`walkforward --trace` now WRITE per-member traces on the real-data campaign path (the depth-2 fan-out `runs/traces////`), refusing only on the synthetic path; `aura chart ` resolves both the depth-1 campaign layout and the depth-2 fan-out (members keyed `/`, C1-sorted). The refusal story in this amendment is historical.]** - -**Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an -**opt-in real-data source axis**: `aura sweep|walkforward --real [--from -] [--to ]` streams real M1 close bars from the data-server archive (the #71 -`M1FieldSource` seam — C6 record-then-replay: a pre-recorded archive, never a live -call mid-sim) instead of the built-in synthetic stream, per family member. A -CLI-side `DataSource` provider (synthetic | real) supplies each member's source, -pip (resolved from the recorded geometry sidecar (`instrument_geometry`) — C10 -refuse-don't-guess; a symbol with no recorded geometry exits 2 before any data -access), window, and — for walk-forward — its `WindowRoller` sizes -(synthetic: bar-index 24/12/12; real: fixed calendar-time 90d/30d/30d in ns; CLI -flags for these are deferred). **[Amended 2026-07-11 (#239): on the dissolved -`walkforward --real`/`mc --real` sugar path the fixed 90/30/30 sizes are a -*ceiling*, not a constant — `fit_wf_ms_sizes` (aura-cli) passes them through -byte-identically whenever they fit the resolved campaign window, and scales them -down preserving the 3:1 IS:OOS ratio (step = OOS, one roll at the minimum) when -the window is shorter; authored process documents still validate their declared -sizes as-is.]** The engine, ingest, and registry are untouched (C9 — -the whole change is in `aura-cli`); the synthetic path is byte-unchanged. -**MC is excluded** (#106 Fork A): its seed varies a *synthetic* price-walk -realization (C12 axis 4), which is undefined over real data's single realization — -`aura mc --real` refuses with exit 2; a real MC needs a bootstrap-resampling axis -(its own cycle). The real walk-forward member key `oos{from.0}` widens from a small -synthetic bar index to an epoch-ns integer (still portable, collision-free). The -built-in demo strategy's **length grid is likewise data-kind-dependent** -(`DataSource::strategy_lengths`): synthetic keeps the short lengths that fit the -18/60-bar demo streams (trend SMA `{2,3}×{4,5}`, MACD `2/4/3`), real uses realistic -M1 lengths (trend `{50,100}×{200,400}`, MACD `12/26/9`) so the cross is a genuine -trend signal over tens of thousands of bars, not noise. This — like the roller -sizes — is a *demo-strategy calibration* patch; the real answer is project-authored -strategies (C9: a project crate owns its own grid), deferred to the project-env -work. - -**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison -**view** the #104 amendment left open is now realized. `aura chart ` -classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a -single run; else member subdirs with `index.json` are a family; else not-found) and, -for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every -member** in one self-contained uPlot page: `build_comparison_chart_data` emits one -labelled series per member, all on a **single shared y-scale** — members measure one -identical quantity, so a shared scale is what makes them comparable, unlike the -single-run overlay of *different* taps (per-series scales). The series align on the -same union-ts spine via `join_on_ts`, so **one mechanism yields three correct -readings**: sweep/MC members share the data window (a true overlay), walk-forward -members are disjoint OOS windows (null-complementary → the stitched curve). The view -consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted -by key for determinism, C1); a family is its *first producer* — the deliberate first -step toward the programmable analysis meta-level (C21) **without** rebuilding the -orchestration axes (that is its own later cycle). Name resolution is made a **total -function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing -command (`run`/`run `/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family) -to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by -both a run and a family) is unreachable. Every error path exits 2, never panics -(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is -`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder + -`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when -no `--tap` is given. **Still deferred:** multi-column tap selection -(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are -single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as -composable meta-level tools). - -**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred -follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine -+ registry untouched, C14). **Run-context header (#102):** a `ChartMeta` -(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params -for a single run) is built from the `RunManifest` in -`build_chart_data`/`build_comparison_chart_data`, injected into -`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js` -(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across -members `(min from, max to)` — the only reading that labels a disjoint walk-forward -family's true OOS coverage (it collapses to the shared window for sweep/MC). -**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max -transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots -(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a -multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full -recorded data stays on disk (view-only), and the walk-forward null-fill page collapses -for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under -budget. **Refinement (#111, 2957561):** the per-bucket reduction is **tap-aware** — an -*envelope* series (equity) keeps min/max, but a *bounded level* series (the C10 -bias level — pre-reframe: exposure — ∈[-1,+1]) reduces by per-bucket **mean** (a `ReduceKind` on each `Series`, -set by `reduce_for_tap`). Without it min/max made every multi-year exposure a solid --1..+1 band (each bucket straddles many sign flips), reading as per-point oscillation; -the mean shows the net/duty-cycle level. Deferred refinements: rendering the min/max -envelope honestly as range bars / OHLC rather than a polyline (#112); a `--width` -budget flag and true intra-bucket min/max ordering for the non-default continuous -x-mode (#110). -**Refinement (2026-06-29 — the visual face is orthogonal and optional; it does not -motivate C24).** The blueprint data format (C24) is required by C21 (generation / -structural search) and C18 (reproduction of a generated run) **at the CLI level, -autonomously** — independent of any visual surface. The playground's form (web or -egui, **editor or viewer, or not built at all**) is therefore **downstream and -optional**, and topology authoring does **not** live in it. This severs an earlier -conflation that treated the playground as where topology-editing would live. C22's -"never a scene editor" is refined to its true content — **no persistent -scene-at-rest** (a World is a *program*, C21) — which does **not** forbid a future -blueprint *editor* over the World-owned topology-data (C24); should the playground -ever become one, the data format is already the substrate it round-trips, not a new -mechanism. -**Refinement (2026-07-03, #188 — visual surfaces are stateless projections; C25).** -The role-model pass generalizes the point beyond the playground: for **every** -artifact class (blueprints, op-scripts, process/campaign documents), the canonical, -complete form is **text**, every operation is executable headless, and any visual -surface — viewer or editor — is a *stateless projection* that reads/writes that same -canonical form and adds no semantics. Read-only projections (viewers, this -contract's trace explorer) rank far above write editors in value; whether an editor -is ever built is secondary, because the Blockly litmus test (C25) already -disciplines each vocabulary to be palette-generatable without one. The shipped face -is the **web-from-disk** front (revised 2026-06, #101 — not egui): disk-persisted -traces rendered as self-contained HTML chart pages. - -### C23 — Graph compilation and behaviour-preserving optimisation -**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic, -named **blueprint** (the authoring source — nodes, composites, strategy, harness; -C8/C9/C20) into a flat, type-erased **`FlatGraph`** — the frozen runnable -instance (C7) — **wired by raw index, not by name** (`Edge { from, to, slot, -from_field }`). Composites are **inlined** at this step (C9): the composite -*boundary* dissolves entirely (there is no composite in the flat graph). The -blueprint's field / input-role *names* are **non-load-bearing** — the wiring -resolves by index, and names survive at most as **informative debug symbols** -(exactly as `FieldSpec.name` already is, C8), kept for tracing / rendering (C9 -graph-as-data, #13) but carrying no run semantics. The flat graph is then the target of **behaviour-preserving -optimisation** — any transform that leaves every observable sink trace -bit-identical (C1 is the correctness invariant) — on two levels: -- **intra-graph** (within one graph): **common-subexpression elimination** — - two identical nodes (same type, same bound params, same input source) merge to - one with output fan-out, so fractal composition (C9) costs no redundant compute — - and **dead-node elimination** — a node on no path to a sink is dropped. - Pure-consumer sinks (C8, no output) are never CSE candidates, so their - out-of-graph side effects are preserved by construction. -- **across the sweep family** (over the family of flat graphs one blueprint yields - under a param sweep, C12): the sub-graph whose nodes carry **no swept param** and - whose inputs are **all sweep-invariant** (transitively from the sources — same - data window) is **loop-invariant** w.r.t. the sweep and is computed **once**; its - output is materialised as a recorded stream (C11) shared read-only across the - sweep instances (`Arc<[T]>`, C12). Only the parameter-dependent suffix re-runs - per sweep point — loop-invariant code motion over the sweep loop. -**Forbids.** Any "optimisation" that changes an observable trace (it breaks C1); -optimising over a representation that is not graph-as-data — an opaque -trait-object interior (the rejected nested-composite reading of C9) cannot be -analysed or rewritten across its boundary, so it forecloses both levels. -**Why.** Determinism (C1) is not merely an audit property; it is the **licence** -for these rewrites — a behaviour-preserving transform is only meaningful because -"same input → bit-identical run" makes "same result" decidable, and a -sweep-invariant sub-graph is reproducible enough to compute once and share. The -flat graph representation (C19) is the necessary condition: only a flat, inspectable -graph-as-data — with each node's declared param-ranges (C8) — lets the compiler -identify identical sub-expressions, dead nodes, and the sweep-invariant frontier. -This is precisely why a composite is an inlining (C9) and not a runtime -sub-engine: the flat graph is what makes the whole graph optimisable. -**Status (Construction-layer milestone).** The flat graph *representation* — -blueprint → inline / compile → flat instance — is the load-bearing substrate built -**first**; the optimisation passes (intra-graph CSE/DCE, sweep-invariant -hoisting) are **deferred**, behaviour-preserving follow-on work, each gated by a -"flat graph with pass ≡ flat graph without pass, bit-identical run" test (C1). The -sweep-level pass further presupposes per-node param declarations (C8 — landed in -cycle 0015 as `name`+`kind`; the swept *range* still pending #32/C20) and the sweep -orchestration itself (C12/C21). -**Amendment (cycle 0032 — param-namespace injectivity is part of the compilation).** -The bootstrap-as-compilation now includes one structural gate: -`check_param_namespace_injective` over the `param_space()` name projection (see C9 / -C12-C19), run before name resolution. It reads the **boundary** name projection — the -authoring address space — and rejects a duplicated path (`DuplicateParamPath`). Node -names stay **non-load-bearing**: they qualify the param path at construction and are -dropped at lowering (the flat graph is wired by raw index, unchanged). The check makes -the by-name *authoring address space* injective; it does **not** make names -load-bearing in the flat graph. - -### C24 — The blueprint is a serializable, World-owned data value (the topology data format) -**Guarantee.** A **blueprint** — the param-generic, named graph-as-data a Rust -builder produces (C9/C19) — is a **first-class serializable data value** with a -stable format, and the engine has a **load path** (data → blueprint → `FlatGraph`, -C23), not only the Rust-builder construction path. Topology is therefore a **value -the World owns** (C21): constructed, **generated**, mutated, **structurally -searched**, serialized for **reproduction** (C18), and (optionally) rendered or -edited by a visual face (C22). `model_to_json` (C9) is the **existing reverse half** -(blueprint → data, for render); this contract closes the loop. The format carries -**topology + structural axes + the param-space**, referencing nodes by their -**compiled-in type identity** — the closed, typed node vocabulary of `aura-std` + the -project `cdylib` (C8/C16) — and carries **no node logic** (computation is compiled -Rust, C17). It is **non-Turing-complete by construction** (a static DAG, no control -flow), so it structurally cannot become a second RustAst. -**Forbids.** Putting **node logic / computation** in the format (the RustAst trap — -logic is Rust, C17); a **Turing-complete or control-flow-bearing** blueprint format -(it is static data; the *generator* that emits it may be Rust, C20, but the artifact -is not a language); a **by-name node marketplace / registry** (the vocabulary is the -compiled-in closed set referenced by type identity — the format is not a distribution -mechanism, C9/C16); treating the **visual face (C22) as the motivation or the -authoring brain** (the format is required by C21/C18 at the CLI level, independent of -any UI); a **generated / searched run whose topology is not recoverable from its -manifest** (C18 reproduction); baking topology as **engine source** compiled into the -binary (the hard-wired `aura-cli` harnesses are pre-C24 scaffolding, a C16 tension, -retired as this lands). [C26, 2026-07-10 (#231): the single-price data weld inside -the surviving `wrap_r` scaffolding is retired — input roles bind archive columns -by name; the wrapper's remaining R-scaffolding retirement stays #159.] -**Why.** The World (C21) is the product, and it cannot orchestrate, **structurally -search**, or **reproduce** *families that vary topology* while topology stays opaque -Rust source baked into the engine. The **game-engine principle** (C16's engine / game -split) makes it concrete: the engine is compiled native systems; the "game" is -**content** — and topology *is* content, a data value the engine owns, serializes, -instantiates, and mutates, exactly as a game engine owns its scene / prefab / -blueprint graphs. The #109 fork was never "*who types the wiring*" — a small LLM -cannot reliably *apply* a DSL and a strong one does not need one, so a -manifest-as-authoring-surface dies on both horns (C17). It is "**is topology a -compile-time *source* artifact or a runtime, serializable, World-owned *value*?**", -and determinism (C1) + reproduction (C18) + structural search (the open search-policy -thread) force the **value**. The engine already treats topology as a runtime value -(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the -flat graph as the optimisation target); C24 only adds that the value **serializes out -of Rust and loads back in**. -**Status (2026-06-29; first cut shipped — cycle 0087 / #155, `d5602ec`; -construction service shipped — cycle 0088 / #157).** The -**principle** is settled (this contract, ratified in an in-context design discussion -— the #109 resolution). The **first cut now ships**: a `Composite` blueprint -serializes to a **canonical, versioned** data value (`format_version` envelope, -omit-defaults JSON) and **loads back** (data → blueprint → `FlatGraph`) via -`blueprint_to_json` / `blueprint_from_json` (`aura-engine::blueprint_serde`), -referencing nodes by **compiled-in type identity** through an **injected resolver** -whose concrete closed `match` over the `aura-std` vocabulary lives outside the engine -(`aura-std::std_vocabulary`) — the engine stays domain-free, no node registry -(invariant 9). Acceptance met: a serialized blueprint runs **bit-identical** (C1) to -its Rust-built twin. `model_to_json` (C9) remains the render half; this closes the -loop for the round-trippable vocabulary. **Cycle 0088 (#157) adds the -introspectable construction service**: a declarative, replayable by-identifier -op-script (`aura graph build` / `introspect` over a JSON op-list, the engine -`GraphSession` / `replay`) builds a runnable blueprint through validated ops — the -engine's construction gates split into *eager* per-op checks (name resolution, edge -kind-match, the double-wire arm, param bind, and acyclicity — a `connect` that would close a -cycle is rejected eagerly, #161) and *holistic* finalize checks (wiring -totality, param-namespace injectivity, root-role boundness), **both cadences calling -the same extracted predicates** (`edge_kind_check`, the shared resolution helpers, -`check_root_roles_bound` — no second validator) — plus build-free introspection over -the closed vocabulary (types, a node's ports/kinds, a partial document's unwired -slots). It **emits** the #155 blueprint; acceptance met: a graph built purely through -the ops compiles identical (C1) to its Rust-built twin, an invalid op is rejected at -the op naming the cause, introspection answers without a build. The engine `Op` stays -serde-free (the wire DTO is CLI-side); the vocabulary's enumerable companion -(`std_vocabulary_types`) lives in `aura-std` (no registry, invariant 9). **Cycle -0089 follow-up (#161 / #162, after the cycle-0088 fieldtest).** The eager acyclicity -gate above (`GraphSession::closes_cycle`, a reachability check at the closing -`connect`) closed a fieldtest-found hole where `graph build` accepted a non-DAG -op-list (invariant 5 / C9). **Lockstep:** it is a *second* home of invariant-5 -alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two -must co-evolve when the explicit delay/register node — invariant-5's sole legal -feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be -rejected at construction. The holistic *finalize* faults now also read -**by-identifier** (#162): `finish()` translates the index-carrying `CompileError`s -(`UnconnectedPort` / `RoleKindMismatch` / `UnboundRootRole`) into by-identifier -`OpError` variants — still *calling* the unchanged holistic gates (the -no-second-validator lockstep preserved), only translating their result. -**Cycle 0090 (#156) codifies the forward-compat two-tier discipline.** Tier-1 -(additive-optional) is serde-default silent-ignore with no `format_version` bump -(a new optional field defaults to prior behaviour, C1) — now proven by -`unknown_optional_field_is_tolerated_byte_identically` (an unknown optional key -loads byte-identically and runs bit-identically). Tier-2 (must-understand: a new -node type, edge semantics, or structural-axis kind) bumps `format_version` so an -old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`, -already green). The per-section required-flag scheme is deferred (no current -Tier-2 section to validate it; recorded on #156). -Pre-ship dormancy (#61, 2026-07-10): until the first external ship there are -no out-of-repo readers — reader and writer change atomically in one commit — -so the Tier-2 bump discipline is dormant and structurally-semantic additions -(the `gangs` section) land as additive-optional fields of v1; the first ship -consciously freezes v1, gangs included, and activates the bump discipline. -**Milestone delivered — 2026-06-30, cycle 0090 — the serializable format + loader + construction service.** A green end-to-end milestone fieldtest (`fieldtests/milestone-topology-as-data/`) proves the full author → serialize → load → construct → introspect → reproduce story from the public surface alone, 0 behavioural bugs; the round-trippable format (#155), the construction service (#157), and the forward-compat two-tier discipline (#156) all shipped. Polish filed forward: op-script grammar docs + a stale example (#163), the canonical trailing-newline / `Composite` value ergonomics (#164), CLI discoverability of `build`/`introspect` (#159). -**Realization (2026-06-30, cycle 0092 — runs are built FROM blueprint-data, #165).** -`aura run ` now loads a serialized **signal** blueprint -(`blueprint_from_json` → the closed `std_vocabulary`; an unknown type fails clean as -`UnknownNodeType`, the data-plane face of invariant 9), wraps it in the r-sma run -scaffolding (sinks / broker / data supplied **at run**, not serialized — C24's deferred -set), runs it, and emits a `RunReport` **bit-identical** (C1) to its Rust-built twin — -proven by `loaded_signal_runs_bit_identical_to_rust_built`. The `RunManifest` now carries -a **`topology_hash`**: SHA256 of the canonical (#164) `blueprint_to_json`, the #158 -reproducibility anchor, a Tier-1 optional field (#156). The hash + helper live -research-side (`aura-cli` + `sha2`), off the frozen engine (invariant 8). This is the -**keystone of the World/C21 milestone**: topology-as-data is now *runnable*, not only -serializable. The harness wrap was made shareable — `r_sma_graph()` = -`wrap_r(sma_signal(...))`, so the Rust path and the data path are the same seam -over the same signal (a behaviour-preserving C19/C23 restructure). Scope note: the hash -covers the **signal** only (the fixed scaffolding is identified by `commit`); a content-id -distinguishing harness-structural variants is a #158/#166 concern once scaffolding varies. - -**Realization (2026-07-01, cycle 0093 — the World orchestrates FAMILIES from blueprint-data, -#166).** `aura sweep --axis =` loads an **open** signal blueprint, -grids the by-name axes against its `param_space()`, builds each member through cycle-1's -`wrap_r` seam, and aggregates to a `FamilyKind::Sweep` family — byte-identical in shape -to the hard-wired sweep (`append_family` / `sweep_member_reports` reused verbatim). This is the -C21 step beyond a single run: the World now constructs and orchestrates **families** of -harnesses from topology-data, not just one. Each member manifest carries the **shared** -`topology_hash` (one signal topology, only params vary; `member_key` distinguishes members) — -reproducible per C18. Two design facts: a sweep needs an **open** blueprint (a fully-bound one -has an empty `param_space`, distinct from cycle-1's bound run fixture), and the signal is -**re-loaded per member** from its serialized doc (the `Composite` is `!Clone` — its -`PrimitiveBuilder`s hold `Box` build closures, #164 — and `bootstrap_with_cells` -consumes the graph). **Monte-Carlo over a loaded blueprint shipped (cycle 0095, #170).** `aura mc - --seeds N` runs a **closed** blueprint across N seeds — each seed a distinct -synthetic price walk (`synthetic_walk_sources`, the `mc_family` `SyntheticSpec` pattern), the -draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and aggregates to a -`FamilyKind::MonteCarlo` family with one stored blueprint per family (the C18 hook), so it -`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint -(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered -exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the -pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward - --axis = [--select argmax|plateau:mean|plateau:worst]` re-optimizes -the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each -24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric + -`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a -`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical -(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One -substitution deep from the hard-wired r-sma WF arm (the loaded blueprint via -`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure -exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form -honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the -meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP -is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the -`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint -warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle -0096, #169):** `aura sweep --list-axes` lists a loaded blueprint's open -sweepable knobs (one `:` per line, `param_space()` order), then exits — the names it -prints are exactly what `--axis` binds. Every listed name is **mandatory** on a `sweep` / -`walkforward`: the blueprint must be fully bound before it runs, so a subset grid is refused with -the missing knob named (`BindError::MissingKnob`) and there is no default — pin a knob you do not -want to vary with a single-value axis (`--axis =`). A single `blueprint_axis_probe` helper now single-sources -the wrapped probe (`wrap_r(loaded_signal).param_space()`) for the sweep terminal, the MC -closed-check, AND the listing (three former inline copies → one), so **listed == swept by -construction** (and stays so across #159's harness retirement — the listing tracks whatever the -sweep actually resolves, never a second source of truth). The names are prefixed by the current -r-sma wrapping (`sma_signal.fast.length`, the nested-composite prefix), not the raw -`param_space` — which is why the discovery lives on the sweep verb (it owns the wrapping), not -`graph introspect`. CLI `--trace` is refused on all verbs (#168 for sweep/walkforward; run/mc already refused); the live trace-writer is the campaign `presentation.persist_taps` (`persist_campaign_traces`), and restoring per-member CLI traces is tracked by #224. **[Delivered 2026-07-11 (#224): `sweep`/`walkforward --trace` write per-member traces on the real-data campaign path (depth-2 fan-out, chartable by the printed family handle); only the synthetic path still refuses.]** - -**Content-addressed reproduction shipped (cycle 0094, #158, C18)**: `topology_hash` -landed cycle 0092; re-deriving a member's FlatGraph from a stored manifest + the -content-addressed blueprint store (`aura reproduce`) shipped cycle 0094 (see C18 -Realization). What **remains** is a content-id that covers **structural-axis / whole-harness -variants** (the scaffolding is not yet blueprint-data); the debug-name-in-id question is -settled by cycle 0104's **identity id** (#171: `blueprint_identity_json` + `graph -introspect --identity-id`, an additive sibling — the byte-exact content id keeps the -store/reproduce roles; introspection-only until a dedup consumer exists). Still deferred: retiring the pre-C24 -hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `*_sweep_family`) once -the project-as-crate layer lands (#159, paired with #157's data-authoring surface). **Out of the first cut's round-trippable set** -(deliberate; fails clean as `UnknownNodeType`, never a silent wrong graph): recording -sinks (capture an `mpsc::Sender` — runtime identity, not param-generic data, C19) and -construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / `Session` — -structural-axis args, a C20 concern), additively addable later (#156). The -**project-as-crate load boundary landed in cycle 0102** (`Aura.toml` discovery + -`cdylib` loading + merged project ∪ std vocabulary — see the C13 realization -note; the `aura new` scaffolder followed in cycle 0103 — one command emits a -buildable project whose blueprint runs through the merged vocabulary); of the -two layers this paragraph used to name as sequencing-coupled, what remains -open is the **composable-orchestration** thread (#109): topology-as-data is -the substrate it stands on. **Canonical project shape (#181, resolved -2026-07-02):** the `aura new` templates (`scaffold.rs`) are the canonical -authoring shape and evolve with the engine; the cycle-0102 `demo-project` -fixture is an intentionally frozen known-good twin for the load-boundary -tests. The two are deliberately **not** lockstep-guarded: no consumer requires -them to match, each is e2e-guarded on its fitness for purpose (build → -descriptor load → charter check → deterministic run — blueprint *wiring* -content is pinned in neither, stated honestly), and an equality guard would -convert every deliberate template improvement into forced churn of the frozen -fixture — the same cross-purpose coupling that rules out regenerating the -fixture from the scaffolder. - -**Op-script grammar (`aura graph build`, the #157 wire surface).** The construction -op-script is a JSON **array of ops**, each object internally tagged by `"op"`, -replayed in order; nodes are referenced **by identifier**, ports as dotted -`.`. The eight verbs: -- `source` — `{"op":"source","role":,"kind":}` — declare a root - source role producing a base column of `kind`. -- `input` — `{"op":"input","role":}` — declare a root input role (kind inferred - from the slots it feeds). -- `add` — `{"op":"add","type":,"name":?,"bind":{:}?}` — - add a node of compiled-in type identity `type`; **`name` is its identifier** - (mirrors the builder's `.named(...)`; defaults to the lowercased type label, so - two unnamed nodes of one type collide); `bind` sets params. -- `feed` — `{"op":"feed","role":,"into":[,…]}` — fan a root role into - interior input slots. -- `connect` — `{"op":"connect","from":,"to":}` — wire an interior output - field to an input slot; a `connect` closing a cycle is rejected eagerly - (invariant 5). -- `expose` — `{"op":"expose","from":,"as":}` — promote an interior output - field to a boundary output, aliased by `as` (a real alias, not a naming; contrast - `add`'s `name`) — one of the two verbs that keep `as` (with `tap`). -- `tap` — `{"op":"tap","from":,"as":}` — declare a measurement **tap** on - an interior output field under `as` (#284) — the output-side twin of `expose`: a - recorded observation point (a `Composite.taps` entry, C27), not a boundary output. - Name-addressed like every other op (no raw index); tap names are their own - namespace (a duplicate refuses). The `finish` gate threads op-declared taps into - the built `Composite` (`.with_taps`); a single `aura run` records each, a sweep - leaves them inert. -- `gang` — `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}` - — fuse two or more sibling params into ONE public knob: the member addresses - leave the sweepable param space and `as` replaces them; the bound or swept - value fans out to every member at bootstrap. Members must share one scalar - kind and stay open (un-bound). - -Value forms are the #155 representations: a `Scalar` bind value is the typed tag form -`{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` / `{"Timestamp":}`, and `kind` is -the capitalized `ScalarKind` (`"F64"`). Worked example: -`fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json` -(an SMA-crossover bias). The surface is introspectable build-free: -`aura graph introspect --vocabulary | --node | --unwired`. (Surfacing this same -grammar in `aura graph build --help` rides with the CLI-discoverability work, #159.) - -**Canonical form (#164).** The canonical blueprint artifact is exactly the bytes -`blueprint_to_json` returns — a JSON value with **no trailing newline**. -`aura graph build` emits those bytes verbatim (it does not frame them with a display -newline via `println!`), so the CLI and library emit paths are **byte-identical** — a -prerequisite for content-addressed topology identity (#158, a hash over the canonical -form). The **canonical JSON is also the blueprint's equality / identity surface**: -`Composite` / `PrimitiveBuilder` deliberately carry **no in-memory `PartialEq` / -`Debug`** (the recipe holds a `build: Box` closure — non-comparable, -non-printable; equality could only be defined over the serialized *data*, of which -`blueprint_to_json` is already the single source — a second in-memory notion would be -a drift hazard against the very form #158 content-addresses). The loader stays lenient -(a trailing newline on input still parses, Tier-1 robustness). - -**Enforcement shift — invariant 9 on the data plane (2026-06-30, cycle-0091 analysis, -adversarially verified).** Lifting topology onto the data plane relocates *where* the -engine/project boundary (invariant 9, no bespoke node registry/marketplace) is enforced, -without changing what it forbids. **Pre-C24** a blueprint referencing another project's -node was a **compile error** — topology was in-process Rust, so cross-boundary node -resolution was *structurally impossible*, compiler-guaranteed for free. **Post-C24** that -same reference is a **type-id string in a serialized blueprint**, refused only by the -injected resolver's closed set (`UnknownNodeType`). The anti-NIH rationale survives intact -(node *logic* stays cargo+Gitea-only; the format distributes **topology-as-content**, never -logic — C17), and `std_vocabulary` stays a compiled-in dispatch table, not a registry — but -its enforcement **degrades from compiler-guaranteed to resolver-seam discipline**. Two -consequences: (1) **#160** (the guard that the closed vocabulary stays honest) is no longer -droppable hygiene — it guards the data-plane face of the boundary (its own drift direction -still fails *safe*, `UnknownNodeType`). **Delivered cycle 0105:** the -`std_vocabulary_roster!` macro expands the resolver match and the enumerable -`std_vocabulary_types()` list from one roster, closing the resolver-vs-list drift mode by -construction; the residual (a new zero-arg node never rostered at all) stays fail-safe, -guarded by the one-line maintainer surface plus the count pins (the in-crate shape test and -aura-cli's cross-boundary `--vocabulary` count e2e). (2) The **injected per-project resolver** (C16) is -the genuinely new, *deferred* invariant-9 surface: nothing structurally stops a project -resolver from accepting arbitrary type-ids, or a project from layering a blueprint -*marketplace* (store + type-discovery API) **on top of** the format — one layer *above* the -engine contract. Invariant 9 holds **at the engine**; the project/ecosystem boundary (what -an injected resolver may resolve, where closed-set discipline draws the line) is a -**World/C21-layer charter point**, not an engine fix. Invariant 8 (frozen) is likewise -reframed by C24/C21: from "topology baked at build time" to "the World may *generate* -topology at runtime (research plane), the chosen blueprint is frozen into the deploy -artifact, and any run is deterministic once instantiated" (C1). - -### C25 — The role model: nine authoring roles, cut by artifact + surface + iteration cost -**Guarantee.** aura's user-facing design is organized by **roles**, cut by the -artifact a role owns, the surface/language it works in, and the iteration cost it -tolerates — never by org chart. One trader — or an LLM actor, which is the point of -the whole authoring design — must be able to fill *every* role; the separation -separates *surfaces with different iteration costs and failure modes*, not people, -and switching roles is a visible act. The nine roles (ratified 2026-07-03, #188): -**1 system programmer** (the engine; Rust, this repo; all invariants), -**2 extension dev** (native nodes + analysis blocks; Rust in project/shared crates; -C7/C8, determinism in `eval`), **3 toolchain dev** (playground/viewer/skills -pipeline; vendor-side; authoring ergonomics), **4 data curator** (recorded streams, -sidecars, derived/news recordings; CLI + recording edge; C2/C3/C6), -**5 methodology designer** (validation/eval **process documents** — "the game -rules"; closed data vocabulary; anti-false-discovery), **6a strategy designer** -(blueprint topologies; data — op-script/blueprint JSON, the graph idiom; C5/C7), -**6b campaign designer** (**campaign documents** — persisted experiment intent; -data, the pipeline/block idiom; C1), **7 player** (owns nothing — explores traces, -tunes runtime params; the playground, C22/C12; read-only by design), and -**8 operator/auditor** (frozen bots, reconciliation, money management; deploy-edge -CLI, manifests, reproduce; C13). The 6a/6b split follows invariant 12's tier -boundary (node/harness vs World); the interface between them is the id machinery -(content id = byte-exact, identity id = topological). Param gradient: 6a defines -topology + open params, 6b spans spaces over them, 7 moves inside those spaces -live. A role is **recognizable** when four things exist: a named artifact type it -owns, a surface addressed to it, an entry path, and its invariants enforced at its -boundary. **Text-first / headless-first is an invariant, not a taste** (user, -2026-07-03: editors must never be the only way; LLM operability demands it): every -artifact class has a canonical, complete, text-serialized form; every operation is -executable headless via CLI/API; visual surfaces are *stateless projections* that -read/write the same canonical form and add no semantics — read-only projections -(viewers) rank far above write editors. The **Blockly litmus test** is the -acceptance criterion for every artifact vocabulary, editor or no editor: a block -palette must be *generatable* from the vocabulary — every block with typed slots, -every valid composition snappable, every invalid one not. -**Forbids.** An artifact class whose only authoring path is a visual editor or a -Rust compile cycle when the role's iteration cost demands data (the role-6b -lesson: a tiny campaign change must never cost a compile); stringly-typed fields, -implicit inter-block coupling, or "arbitrary JSON here" holes in a role's -vocabulary (they fail the litmus test); collapsing the 6a/6b idioms into one -surface (dataflow boxes-and-wires vs sequential pipeline blocks are different -languages). -**Why.** The game-engine analogy that seeded aura (C16) extends to its people: -engines separate system programmers, toolchain devs, asset creators, level -designers, and players because their artifacts, tools, and iteration loops -differ — a design that forces one role's technique on another (Rust for campaign -tweaks, an editor as the only entry) mis-prices iteration exactly where research -throughput lives. Realization: cycle 0106 (#189) shipped roles 5/6b their -artifacts (process/campaign documents, C18 realization note); roles 4/7/8 remain -technically present but faceless (no addressed verb families yet); role homes in -the project layout and docs-by-role are open (#192 context). -**Amendment (2026-07-20, #295 — control surfaces are projections).** The -text-first invariant fixes the control-surface layering: the text artifact -vocabulary (blueprints, process/campaign documents, registry records) is the -canonical layer, and every control surface — the one-shot CLI's executor -verbs, any future long-running host, any MCP face, a World program — is a -projection/executor over those artifacts, never a second home for intent. -The owner resolved the direction 2026-07-21 (minuted on #295): **document-first -completion first** — delivered by #300: the executor verb set is settled -(`run`, the four thin per-verb generators, and the document verbs -validate/introspect/register/show/run/runs, `show` being #300's read-back -addition), the verbs' per-verb identity re-ratified (#300 F8 — reduction, not -grammar collapse), and a typed-protocol host or MCP face remains demand-driven, -unbuilt. - -### C26 — Harness input binding: role names bind archive columns -**Guarantee.** A strategy blueprint declares WHICH data it consumes as the NAMES -of its root input roles: a role whose name is in the **closed column -vocabulary** — `open`, `high`, `low`, `close`, `spread`, `volume`, plus `price` -as the backward-compatible alias of `close` — binds by default to that column of -the campaign cell's (or run invocation's) instrument. A campaign document may -**override** the name default per role via the additive `data.bindings` block -(`role name → column name`; serde `default` + skip-if-empty, so binding-less -documents keep their content ids) — the 6b rebind seam: the blueprint's content -identity never changes. Resolution (`aura-runner`'s `binding` module, -`resolve_binding` — moved out of the shell with #295) produces ONE ordered -plan consumed by BOTH halves of the old weld: the columns are opened -(`aura_ingest::open_columns`, the generalization of `open_ohlc`/#92) and the -wrapped root roles declared (`aura-runner::member`'s `wrap_r`) in the **same -canonical order** — `M1Field` declaration order (open, high, low, close, spread, -volume), filtered to the consumed set — which is the C4 merge tie-break order, -so role i receives source i by construction. A close column is always part of -the plan (shared when the strategy consumes `close`/`price`, else opened for the -broker/executor pair alone). The vocabulary is **Blockly-litmus-clean** (C25): -role-name slots draw from a closed enum, the bindings block is typed -`role → column` with no free-form holes, and validation is two-tier — values -against the column vocabulary in the intrinsic tier (`validate_campaign`), keys -against the strategies' `input_roles()` in the resolver tier -(`validate_campaign_refs`). -**Forbids.** Guessing a column for an unknown role name (refuse with the -vocabulary + the override remedy, never default); opening columns in any order -other than the canonical declaration order (the C4 tie-break contract); -free-form or logic-bearing binding values (the C17/C25 line — a binding value is -a typed column reference, nothing else); fabricating synthetic OHLC when a -multi-column strategy meets synthetic data (the walk generates a close series -only — refuse with the `--real` remedy). -**Extension point.** When recorded non-price sources land (the #71 Source seam), -the binding VALUE-space grows additively from archive columns to recorded-stream -references — a root role like `sentiment` becomes bindable to a recorded stream -then, and until then refuses with the vocabulary-naming message. The role-name -key-space and the campaign `bindings` carrier are unchanged by that growth. -**Why.** The single-price weld (`aura-runner::member`'s `wrap_r`'s hard-wired `price`←close and the six -`M1Field::Close`-only open sites) made every strategy monocular regardless of -what its blueprint declared; binding by role name keeps the blueprint the single -source of its own data needs (C24: topology-as-data carries its input contract), -lets a campaign re-aim a strategy without touching its content id (C18 -identity), and pins opening and declaring to one shared, canonically-ordered -plan so the two halves cannot drift (C1/C4 determinism). +- **[C1 — Determinism and disjoint parallelism](contracts/c01-determinism.md)** — + A backtest is a deterministic, synchronous, non-concurrent event loop — same + (inputs, seed) yields per-run bit-identical output; parallelism runs across + disjoint sims (and campaign cells), never within one. +- **[C2 — Causality / no look-ahead](contracts/c02-causality.md)** — + A node sees only the past: input windows are read-only and end at the cursor, + and a resampler emits a bar only once complete — the future is physically + absent, not merely discouraged. +- **[C3 — One merge, at ingestion only](contracts/c03-single-merge.md)** — + Heterogeneous timestamped sources are k-way-merged by timestamp into one + chronological cycle stream at the ingestion boundary (normalized to canonical + epoch-ns there); no merge or as-of join inside the graph. +- **[C4 — Cycle granularity](contracts/c04-cycle-granularity.md)** — + The clock is data-driven — one input record equals one cycle in global + timestamp order with a monotonic `cycle_id`, ties breaking by source + declaration order. +- **[C5 — Freshness-gated recompute and sample-and-hold](contracts/c05-freshness-gating.md)** — + `cycle_id` advances everywhere, but a node re-evaluates only when at least one + of its inputs is fresh this cycle; otherwise it holds its last output. +- **[C6 — Firing policy A and B, per input group](contracts/c06-firing-policy.md)** — + A node declares per input group one of two firing policies — A + (fire-on-any-fresh + hold, as-of join) or B (all-fresh barrier synchronized + on the cycle timestamp) — and may mix them freely. +- **[C7 — Four scalar base types, streamed as SoA](contracts/c07-scalar-soa.md)** — + Only `i64`/`f64`/`bool`/`timestamp` stream, as columnar SoA type-erased at the + edge (kind checked once at wiring): the hot-path carrier is the tag-free + 64-bit `Cell`; the self-describing `Scalar` serves the dynamic boundaries. +- **[C8 — The node contract](contracts/c08-node-contract.md)** — + A node's static signature (typed input ports, a one-port output record of + base-scalar columns, typed param knobs) is declared once pre-build on the + value-empty recipe; the engine provides zero-copy input windows, `eval` + returns a borrowed row or `None`, and `lookbacks`/`finalize`/`label` complete + the lifecycle. +- **[C9 — Fractal, acyclic composition](contracts/c09-fractal-composition.md)** — + A composite is an authoring-level Node (typed inputs + one output port + carrying a 1..K-field record) that inlines away at compile into the flat, + acyclic `FlatGraph`; feedback only via an explicit delay node; topology + round-trips both ways as World-owned data. +- **[C10 — Bias stream; signal quality in R; cost model; money at the deploy edge](contracts/c10-bias-r-cost.md)** — + A strategy emits an unsized signed bias `f64 ∈ [-1,+1]`; a decoupled + downstream executor (stop-rule → position-management, the stop defining the + risk unit R) turns it into a managed trade in R; signal quality is measured + in R (gross → net via a composable cost-model graph, pure feed-forward, no + Sizer); money and real-broker fill live only at the live deploy edge. +- **[C11 — Generalized sources; record-then-replay](contracts/c11-sources-record-replay.md)** — + Anything nondeterministic, external, or slow is materialized into a recorded, + timestamped stream before it enters the engine; a backtest replays the + recording and never makes a live call mid-replay. +- **[C12 — The atomic sim unit and the four orchestration axes](contracts/c12-atomic-sim-unit.md)** — + The atomic unit is `(frozen topology + param-set + data-window + RNG-seed) → + deterministic run → metrics`, orchestrated by param-sweep, optimization, + walk-forward, and Monte-Carlo (= sweep over seeds) over read-only + `Arc`-shared data. +- **[C13 — Hot-reload is authoring-only; deploy is frozen](contracts/c13-hot-reload-frozen-deploy.md)** — + Research code hot-reloads as native Rust cdylibs through the two-tier + ABI-stamped `AURA_PROJECT` descriptor; the live bot is a statically-linked, + versioned, frozen artifact — never hot-swapped. +- **[C14 — Headless core, two faces](contracts/c14-headless-two-faces.md)** — + The engine is a UI-agnostic library with a primary programmatic/CLI face (a + four-way exit-code automation contract) and a visual playground face; no UI + knowledge inside the engine. +- **[C15 — Resampling-as-node; sessions/calendars](contracts/c15-resampling-sessions.md)** — + A resampler is a clock-sensitive node emitting a completed bar only at the + boundary; calendars and instrument specs are metadata beside the hot path; + session context arrives as scalar streams via a `Session` node. +- **[C16 — Engine / project separation; node reuse](contracts/c16-engine-project-split.md)** — + aura is the reusable engine each research project depends on via cargo, with + three-tier cargo-native node reuse and a two-tier project model (data-only + `Aura.toml` project, or a native node crate attached via `[nodes]`). +- **[C17 — Authoring surface](contracts/c17-authoring-surface.md)** — + All computational logic is authored in native Rust via Claude Code + the + skills pipeline; topology is serializable World-owned data; aura ships no + embedded coding-LLM. +- **[C18 — Project management and the run registry](contracts/c18-registry.md)** — + One repo = one project (git + Gitea), plus an Aura-native run registry storing + per-run manifests + metrics with content-addressed topology, re-deriving full + results on demand; lineage lives in family/campaign stores. +- **[C19 — Bootstrap: blueprint → instance](contracts/c19-bootstrap.md)** — + Construction is a distinct, recursive bootstrap phase compiling a + param-generic named blueprint into a frozen, raw-index-wired `FlatGraph` + instance by binding params + data + seed. +- **[C20 — Strategy ↔ harness](contracts/c20-strategy-harness.md)** — + A strategy is a reusable, context-free composite blueprint; the harness is + the closed root sim graph (sources + strategy + brokers + sinks under a + clock) — C1's disjoint unit, not a node. +- **[C21 — The World: the meta-level is the product](contracts/c21-world.md)** — + The World — the program that dynamically constructs and orchestrates families + of harnesses whose topology is World-owned data (C24) — is aura's product; + the single-harness engine is its substrate. +- **[C22 — The playground is a trace explorer](contracts/c22-playground-traces.md)** — + The only durable, displayable substance is the recorded trace captured by + sinks (declared taps, C27) persisted into the registry; the playground is an + execution viewer, never a persistent scene editor. +- **[C23 — Graph compilation and behaviour-preserving optimisation](contracts/c23-graph-compilation.md)** — + The bootstrap is a compilation into a flat, index-wired `FlatGraph` that is + the target of behaviour-preserving (bit-identical, C1) optimisation — + intra-graph CSE/DCE and sweep-invariant hoisting (both still deferred). +- **[C24 — The blueprint is a serializable, World-owned data value](contracts/c24-blueprint-data.md)** — + A blueprint is a first-class, canonically serialized, versioned data value + with an engine load path (data → blueprint → `FlatGraph`), referencing nodes + by compiled-in type identity and carrying topology but no node logic. +- **[C25 — The role model: nine authoring roles](contracts/c25-role-model.md)** — + aura's user-facing design is organized by nine authoring roles cut by owned + artifact + surface + iteration cost, all fillable by one actor; text-first / + headless-first is an invariant, and every control surface is a projection + over the canonical text-artifact vocabulary. +- **[C26 — Harness input binding: role names bind archive columns](contracts/c26-input-binding.md)** — + A strategy declares its data needs as root input-role names from a closed + column vocabulary, overridable per campaign without touching content + identity, resolved into one canonically-ordered open-and-declare plan. +- **[C27 — Declared taps](contracts/c27-declared-taps.md)** — + A blueprint may declare named, pure output-side taps (the twin of + `input_roles`) that compile into `FlatGraph.taps` and bind run-mode-aware to + caller-built sinks; an unbound tap is inert, not a fault. +- **[C28 — Internal stratification: ladder, process column, shell](contracts/c28-stratification.md)** — + The workspace crates realize a domain-specificity ladder (engine → market → + measurement/strategy → backtest → execution) beside a process column, an + assembly position, and an import-only shell — the import direction enforced + structurally by the full-workspace `c28_layering` guard. --- -### C27 — Declared taps: named measurement points bind sinks run-mode-aware -**Guarantee.** A blueprint may declare **taps** — named, pure output-side -declarations `{ name, from: {node, field} }`, the output-side twin of -`input_roles` (C26). A tap names an interior producer's output field without -naming a sink, exactly as a `Role` names an abstract input without naming a -source. At compile the tap resolves — and, for an interior composite, **hoists** -to the root — through the same lowering remap edges and `OutField` re-exports -use (`resolve_tap_wire`, a `flat_taps` accumulator threaded through the lowering -recursion), landing in `FlatGraph.taps` as a `FlatTap { name, node, field }` -whose name survives compile and is load-bearing for by-name binding (like -`SourceSpec.role`, #275). Binding is run-mode-aware: the run-mode-owning layer -constructs a sink (a `Recorder`) at a bound tap via `FlatGraph::bind_tap`, which -takes a **caller-built** `Box` sink (so the engine keeps its -`aura-core`-only production dependency — it never constructs a domain sink type) -and appends it plus an edge before bootstrap. The single-run path (`run_signal_r`) -binds and records each declared tap, persisting the series through the existing -trace store (`env.trace_store()`), so the tap columns surface through the same -tooling the campaign path feeds; a sweep/reduce run leaves taps unbound. -**Forbids.** A tap carrying a channel endpoint or effect in the serialized -artefact — recording policy is run-mode authority, not fragment-embedded (a -fragment must not drag its measurement decisions into every harness that embeds -it; the tier ontology, C20/C21). The engine constructing a domain sink type -(the `aura-core`-only wall — the sink is caller-built). Order statistics -(median, etc.) inside the graph — they stay sink/analysis-side (C18); -multi-instrument study inputs stay harness/World tier. -**Non-error.** An **unbound** tap is inert, not a fault — unlike an unbound root -input role, which `check_root_roles_bound` rejects (C26): observation is optional, -a fed input is mandatory. A declared-but-unbound tap compiles and runs, its -producer evaluating and its output discarded (a no-out-edge producer is a valid -runnable sink — the Kahn sort emits it, `check_ports_connected` gates only inputs). -**Why.** Observability must be expressible in a hand-authored blueprint — the -measurement-shaped study computes in the graph and surfaces via taps, no -throwaway Rust harness — while recording stays a run-mode decision, not a -fragment-embedded effect. Taps are designed **DCE-compatible** (a bound tap is a -natural DCE root, an unbound tap a dead declaration) but this contract does -**not** depend on DCE: an unbound tap's sink is simply never constructed -(build-time elision, which the engine already tolerates). The chain-pruning -benefit — a sweep paying zero for the study wires behind an unbound tap — is -**deferred to the future DCE cycle** (C23); the mechanism ships now, verified -sound. (#282, 2026-07-18.) +## Open architectural threads -### C28 — Internal stratification: the trading ladder, the process column, the shell -**Guarantee.** The aura workspace's crates realize a layered responsibility -model along two axes. A **ladder**, ordered by domain specificity from inner to -outer: -1. **engine** — the domain-free deterministic streaming runtime and vocabulary - (cells/kinds/freshness/firing, graph build, compile, run, taps/trace — C1–C9, - C19, C23, C24), the domain-free node roster (arithmetic/logic/rolling nodes - plus the generic `Recorder`/`GatedRecorder`/`SeriesReducer` sinks), and - generic statistics (the Monte-Carlo and moving-block-bootstrap kernels). The - layer name is wider than the crate `aura-engine`: it spans `aura-core`, - `aura-engine`, the domain-free part of `aura-std`, and the generic-statistics - surface. -2. **market** — what a market *is*, carrying no analysis or strategy intent: - instruments and pip/point geometry (C15), session anchoring, bar resampling, - market-data ingestion (`aura-ingest`). -3. **measurement** — descriptive statistics over market streams (session - statistics, conditional rates, distributions). Sibling of **strategy**: both - consume *market*, neither imports the other. -4. **strategy** — the *definition* of trading signals: bias, stop rules, sizing, - cost-model nodes. -5. **backtest** — the *evaluation* of strategies: simulated execution without - money (the `SimBroker`, position-management), the R-metric reductions and the - position-event table, and the per-run scaffold. Backtest mirrors *execution* - on the research side — execution semantics in R units, money exiled. -6. **execution** — money, account-term sizing, the live broker; already exiled to - the deploy edge by C10/C13, existing today only as that ratified boundary. +Genuinely open only; resolved threads live in the contracts that absorbed them +(pre-refactor wording in [INDEX.history.md](INDEX.history.md)). -Beside the ladder stands the **process column** — the research-process machinery -(the run registry C18, the process/campaign document types, campaign execution). -It consumes run *artifacts*, not market streams, and orchestrates runs of any -ladder layer; it stands beside the ladder, not on a rung. Between ladder/column -and the shell stands the **assembly** position (`aura-runner`, #295): it -composes the rungs into the canonical member-run recipe — harness assembly, -input binding (C26), the C1-load-bearing param↔config translators, and the -shipped default `MemberRunner` — importing the ladder rungs, `aura-composites`, -`aura-ingest`, and the process column, and imported only by the shell and by -downstream World programs. It also carries a direct production dependency on -the external `data-server` tree (inherited from the shell with the recipe: the -runner constructs archive servers itself). The external tree therefore enters -the workspace at the ingestion edge (`aura-ingest`, External components), the -assembly position (`aura-runner`), and the shell (which imports everything by -definition) — and at no ladder or column crate beyond those; the -`c28_layering` guard pins the exact set. The **shell** (the `aura` CLI) imports everything, -is imported by nothing, exports nothing, and holds no domain logic: -argv/dispatch, argv→document translation (including the op-script construction -front-end, `graph_construct`), presentation, and the `aura new` project -scaffolder (authoring-tooling template emission — shell-resident, like -rendering, until a second consumer wants it). Rendering stays in the shell -until the C22 web face provides its second consumer (#295 deferral). - -**Import rule.** An outer ladder layer may import inner layers, never the -reverse; *measurement* and *strategy* are siblings (no import either way); -*backtest* may import *strategy*. The process column imports the engine layer -plus the run-artifact and metric interfaces — realized since #291/#292 as a -production dependency on `aura-backtest` (the trading instantiation -`M = RunMetrics`) — and is imported by no ladder crate; column-internal edges -are free. The assembly position imports ladder and column alike and is -imported by neither; the shell imports all; nothing imports the shell — both -now enforced by the full-workspace `c28_layering` table plus its shell-content -check (#295). **`dev-dependencies` are exempt** — tests may cross layers (the -existing `engine`/`ingest`/`registry` → `std` edges are dev-only and stay). -The rule binds **crate structure, not graph composition**: a blueprint may -feed a measurement statistic into a bias input — data flow inside a graph is -free (C24); layers govern imports, graphs compose freely. - -**Forbids.** An inner layer importing an outer one — most sharply the engine -layer importing backtest reductions, or market importing strategy. A ladder crate -importing the process column. Money (currency P&L, account sizing) inside the -strategy or backtest layers (C10/C13 — it lives only at the execution/deploy -edge). Once a second consumer warrants the split, keeping multiple layers welded -in one crate's roster. - -**Why.** The stack is a framework specialized for traders: the inner engine is a -domain-free analysis substrate (the same machinery would measure robot telemetry -or any timestamped process), the outer rungs the trading specialization, and the -two together are the "game engine for traders" (C16's game/engine split applied -*inside* the workspace, from the engine|project boundary to the intra-stack -layering). Separable responsibilities behind narrow, contract-defined seams let a -contributor focus on one rung without absorbing the rest, and make each boundary -auditable. Enforcement is the crate graph itself: once crates are cut along these -cells, a layer violation is a compile error and every new edge a visible -`Cargo.toml` diff — no linter, no discipline appeals. Five of the six seams are -already ratified contracts and this model only *names* them as layer boundaries: -the node contract (C8, engine ↔ any vocabulary), blueprint-as-data (C24, -topology ↔ engine — its shape selecting the run scaffold is the additive #286 -refinement, phase 3 below), declared taps (C27, run ↔ analysis — the only way -values leave a run), the run registry (C18, run ↔ process column), and the -deploy edge (C10/C13, everything ↔ money). Only the -process column ↔ metric interface is new (a named metric vocabulary *supplied* by -measurement and backtest instead of baked in R-only — this is #147). - -**Status (2026-07-19, milestone "Stratification — ladder, process column, -shell", #286).** This contract states the *target*. At HEAD the crates cut by -mechanical role (vocabulary / reductions / documents / orchestration / CLI), not -cleanly by layer, so the layering is realized only partially: -- The dependency *direction* now obeys the rule across the engine stack. The - `aura-engine → aura-backtest` production violation is cut (phase 2, #292): - `RunReport` is generic over its metric payload `M` (the engine names no - concrete metric type), the pip/R reductions and the MC assembly (`summarize`, - `McAggregate`/`RBootstrap`/`monte_carlo`) moved to `aura-backtest`, and the - statistics kernel (`MetricStats`/`quantile`/`resample_block`/`SplitMix64`) - moved to the `aura-analysis` foundation. The engine's `[dependencies]` are now - `aura-core` + `aura-analysis` only; the backtest layer instantiates - `M = RunMetrics` (`aura-backtest → aura-engine`, an outer→inner edge). The - ladder direction is enforced for the engine/backtest/analysis rungs by the - structural test (`c28_layering`, seven-row table). -- **#147 retired (user-ratified direction 2026-07-20; shipped the same day).** - Item 1 — the metric genericity `RunReport` — shipped as phase 2 above - (#292). Item 2 — the registry deflation vocabulary — shipped as the ratified - "A1" cut, its deferral trigger having fired when measurement supplied its - first deflatable metric (the IC, #290): the deliberately narrow - `MetricVocabulary` trait (resolve/roster/direction/value/one null draw) - lives in `aura-analysis`, re-exported through `aura-engine`; the R - vocabulary (`RunMetricKey`, `r_based`, the rosters, the centred - moving-block `null_draw`) is supplied by `aura-backtest`; the registry's - rank/optimize/deflate machinery is generic over `M: MetricVocabulary` with - refusal prose derived from the carried roster; the IC (`aura-cli`) is the - second production implementor, bringing its within-run permutation null. - The C10 wall stays monomorphic (`check_r_metric`/`generalization` accept - only the R vocabulary; `r_based` is the enforcement point). Explicitly - still deferred ("A2"): measurement runs as sweep-family citizens (report - unification, campaign engine generic-over-M) — until a concrete - family/campaign demand exists. The #136 one-implementor rationale is - superseded by the second implementor; the registry's process-column - trading-awareness holds unchanged. See #147 for the full disposition. -- The `aura-std` four-layer roster is now cut by layer (phase 4, #288): `aura-std` - holds the engine nodes only (arithmetic/logic/rolling + the generic sinks); - `aura-market` (`session`, `resample`), `aura-strategy` (`bias`/`stop_rule`/ - `sizer` + the cost nodes) and `aura-backtest` (`sim_broker`, - `position_management`) carry the outer rungs, each depending only on `aura-core`; - the closed node roster moved to `aura-vocabulary`. The `aura-analysis` - interweave is resolved (phase 5, #291): the backtest reductions live in - `aura-backtest::metrics` beside their `position_management` producer, and - `aura-analysis` is reduced to domain-free statistics + selection provenance - (`[dependencies]` = serde only). The *measurement* rung is seeded (#295): - `aura-measurement` carries the IC vocabulary + reduction (its run verb remains - the additive shape dispatch, #286). No crate exists yet for *execution* - (unbuilt — the C10/C13 edge only). Under this model the - `aura-registry → aura-research` edge is column-internal and legal. -- Phased realization (each independently shippable; behaviour byte-identical - except the purely additive shape dispatch): (1) this contract; (2) cut the - engine's backtest-metrics edge via `RunReport` — **done** (#292: - metric-generic run record, the pip/R reductions + MC assembly relocated to - `aura-backtest`, the statistics kernel to `aura-analysis`, dependency - inversion, ladder enforced by the structural test; realizes item 1 of #147); - (3) the #286 - shape dispatch plus the per-run scaffold as a library — **done** (#295: the - pure per-run scaffold pieces live in `aura-backtest::scaffold`; the composed - recipe — harness assembly, binding, translators, the default `MemberRunner` - — is the assembly crate `aura-runner`; the measurement rung is seeded as - `aura-measurement` with the IC; the shell is reduced to - argv/translation/presentation, enforced structurally. Residual: ~20 refusal - sites inside `aura-runner`'s single-run verb paths still terminate the - process; their conversion to `RunnerError` propagation is tracked as #297 — - the campaign path already refuses via `MemberFault`, never a process exit); - (4) split the `aura-std` roster along engine / market / strategy / - backtest — **done** (#288: four `aura-core`-only node crates, the closed roster - moved to `aura-vocabulary`, the ladder direction enforced by a structural test); - (5) split `aura-analysis` into generic statistics and backtest metrics — - **done** (#291: metrics moved to `aura-backtest::metrics`, `aura-analysis` - reduced to the domain-free half, engine re-export surface name-unchanged); (6) - generify the column's metric interface (demand-driven — this is #147). Crate - names now realized (`aura-market`/`aura-strategy`/`aura-backtest`/ - `aura-vocabulary`); full evidence on #288, #286 and the milestone. - -Provenance note (#295): the shell-boundary cut closes a ratified structural -debt (this contract's own phase plan) and un-blocks the future World program; -it was not a demonstrated downstream blocker — the first downstream project's -observed friction was document-vocabulary gaps, never Rust-level recipe -reachability. - ---- - -## Open architectural threads not yet resolved - -- **Playground & World UI surface** — the playground is core (C22); the surface - is a **web frontend served from disk-persisted traces** (revised 2026-06, issue - #101; not egui). Settled for the first cut: raw per-tap trace persistence to - disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static - self-contained HTML chart page (uPlot, vendored like the `render_html` - Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The - families-comparison **view** — `aura chart ` overlaying one tap across a - family's members on a shared y-scale — shipped (#107, see the C22 amendment). - Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family / - multi-strategy comparison, multi-column tap selection #47), a local server, and the - run/replay clock controls. +- **Playground & World UI surface** — the visual face is a web frontend served + from disk-persisted traces (C22); shipped: per-tap chart pages, the + families-comparison view, decimation, run-context header. Open: richer + comparison meta-views (sweep-surface heatmaps, cross-family comparison, + multi-column tap selection #47), a local server, run/replay clock controls. - **Parameter-space search strategies** (Bayesian/genetic) — pluggable policies - atop the atomic sim unit (C12), not yet designed. Search over *params* sits atop the - atomic unit; search over *structure* (graph mutation / crossover — NEAT-style) - additionally needs topology-as-data (C24). -- **`aura new` scaffolder and the experiment-builder API** — `aura new` - scaffolds a Rust project *crate* (node / strategy blueprints) against the - engine (C16/C20); the experiment-builder API surface (harness wiring, - structural axes, sweep combinators) is not yet designed. `Aura.toml`'s - schema was settled paths-only in cycle 0102 (data archive root, runs dir — - see the C13 realization note); the load boundary itself shipped there. -- **The analysis meta-level — composable orchestration + project-as-crate authoring - (tracked: #109).** aura's differentiator (C21) had two unbuilt halves: (1) the - orchestration axes (sweep / Monte-Carlo / walk-forward) becoming *composable tools* - a user wires into an analysis workflow, rather than today's hard-wired CLI verbs - (`sweep_family` / `walkforward_family` in `aura-cli`) — still open; and (2) the - project-as-crate authoring layer above — its **load boundary landed in cycle 0102** - (`Aura.toml` discovery + cdylib loading + merged vocabulary, C13 realization note) - and the **`aura new` scaffolder in cycle 0103**; of that layer only the - experiment-builder API remains open (C16/C17). - On the meta-level, authoring a strategy is *wiring - existing nodes* (C9 composition) + *defining the analysis framework* (C21), not - writing new nodes; the user wants it driven via the CLI, later the interactive - server / playground (C14/C22). **Resolved (2026-06-29) → C24.** The fork was - mis-posed as "(a) run-a-Rust-experiment-via-CLI vs (b) wire-nodes-via-CLI". The real - axis is **topology = compile-time Rust source vs runtime, serializable, World-owned - data value**, settled as the **value** (C24, the game-engine principle): node - *logic* stays Rust (C17), *topology* becomes data the World generates / serializes / - reproduces. "Wire-nodes-via-CLI" is dropped — interactive wiring through the CLI is a - non-goal; it collapses to a file + parser, which *is* C24's load path. What - **remains** under #109: the **blueprint data format** itself (C24 Status — its own - brainstorm / milestone), and the **composable-orchestration** half — the axes - (sweep / MC / walk-forward) becoming tools applied to a World-constructed harness - instead of the hard-wired `aura-cli` verbs (`sweep_family` / `walkforward_family`). - **Status (cycle 0106, #189 — the artifact half shipped v1).** The #188 role-model pass - re-cut the "experiment-builder API" reading of this thread (role-6b work mis-typed in - role-2 technique): experiment intent is now **data**, not a Rust API. Cycle 0106 shipped - both document vocabularies (process document, campaign document — `aura-research`), - two-tier validation, the op-script-precedent introspection contract - (`aura process|campaign introspect --vocabulary/--block/--unwired/--content-id`), and - content-addressed stores (C18 realization note) — authorable and checkable headless, no - compile, no run. **Status update (cycles 0107–0110).** The executor question resolved and - shipped (cycles 0107/0108, #198/#200: `aura-campaign` executes the v2 pipeline shape over - the `MemberRunner` seam); the #188 amendment package landed 2026-07-03 (C25 role-model - entry, C20/C22 refinements, invariant-10 clarification). The "once it carries" dissolution - of the hard-wired verbs (user decision 2026-07-03 on #188) is **running as the milestone - "Verb dissolution" (#210)**: the fork triage of 2026-07-04 decided its design (dissolve - the real-data blueprint branches only, built-in/synthetic branches stay verb-wired until - #159; generated documents auto-registered; full behaviour parity modulo the recorded - additive instrument stamp), and the verb set dissolved in the #210-decision-7 order: cycle 0110 took `aura sweep - --real …` (enabled by the `std::sweep` selection group becoming optional — - all-or-nothing, selection-free = terminal-stage-only; wire form and every stored content - id unchanged), then generalize, walkforward, and mc's R-bootstrap path, each now sugar over - a generated, content-addressed campaign document through the one executor, plus a structural - risk-regime axis (`CampaignDoc.risk: [RiskRegime]`, sole `vol{length,k}` variant — the stop - compared as a matrix dimension, stamped into every member manifest, never argmax-swept). - Ad-hoc verb intent no longer evaporates into shell history — the #188 diagnosis's cure - applied to the verbs themselves. **The dissolved form is per-verb, by intended scope** - (ratified at milestone close, fieldtest 2026-07-07): for sweep it is the blueprint file - (` --real`), while `aura sweep --strategy r-sma --real` stays the inline built-in - path (its member lines carry no instrument/topology_hash/selection stamp and register no - campaign document) — the built-in `--strategy` demo surface is #159's hard-wired-harness - retirement target, not the dissolution's; for generalize/walkforward/mc the dissolved - form is now the same generic grammar as sweep — an arbitrary blueprint positional plus - `--real` and repeatable `--axis =` (#220). [HISTORY — superseded in - two steps. #159 (cuts 1b-4, post-2026-07-07) retired the built-in `--strategy` - sweep/demo surface: the inline `aura sweep --strategy r-sma --real` path no longer - exists. #220 then removed the verbs' weld — at milestone close the dissolved form of - generalize/walkforward/mc was still the welded `--strategy r-sma --real`; #220 made - the three verbs blueprint-generic (arbitrary blueprint + arbitrary `--axis`; mc keeps - the synthetic `--seeds` family unchanged beside the new `--real --axis` campaign mode; - generalize takes `--real `, >=2, with one value per axis), deleted the - welded `--strategy` grammar (clap rejects it, exit 2), dissolved `RGrid`, and unified - the verbs on the #214 invocation struct. The surviving form everywhere is `aura - --real … --axis …` over examples/r_*.json.] Milestone closed 2026-07-07 on a green end-to-end fieldtest - (0 bugs; behaviour preservation, campaign-substrate reach-through, and the risk-regime axis - all confirmed); residual findings are discoverability/ergonomics follow-ups (#216 risk-axis - discoverability, #217 verb knob asymmetry, #218 no-project store litter). - **Amendment (2026-07-14, #256 fork B):** the dissolved walkforward/mc - translations' leading sweep became the enumerate-only **`std::grid`** stage — - a fieldless vocabulary block, legal only as the first stage and immediately - before `std::walk_forward`. Only the grid's parameter points ever crossed the - stage seam (the wf stage re-sweeps them per IS window itself), so the leading - stage no longer executes members or persists a `Sweep` registry family: - persisted dissolved-walkforward/mc campaign documents lose that family, while - the visible grades are byte-identical (the exact-grade E2E pins are - unchanged). The executor's inter-stage seam is now a typed two-armed flow - (points-only vs executed members); report-consuming stages are fenced by - preflight. `translate_generalize` keeps its executed `std::sweep(argmax)` - leading stage — its generalize stage consumes the argmax winner report as the - cell nominee. -- **Inferential honesty of the World — family-selection without false-discovery - control (tracked: milestone "Inferential validation (defend against false - discovery at sweep scale)", #144 / #145 / #146; adjacent #139).** The World's - massively-parallel sweep (C12 axes 1–2, C21) is aura's differentiator *and* its - most direct route to false discovery: `optimize` / `rank_by` select the single - best family member by a bare argmax, with no penalty for the number of - configurations tried, no preference for a robust parameter neighbourhood over a - sharp in-sample peak, and no cross-instrument generalization read; `mc` is a - descriptive seed-sweep, not an inferential significance test. The hygiene - invariants keep one backtest honest (C1 determinism, C2 no look-ahead); they do - **not** make a *selection across a family* honest. Recognized direction: the - selection/aggregation layer must deflate a winner for the family size (#144), - prefer a plateau over a peak (#145), and score cross-instrument generalization - (#146), with a per-candidate out-of-sample bootstrap as the adjacent - significance read (#139, landed cycle 0075). Distinct from the two threads above — - neither orchestration *composability* (#109) nor a search *policy* (Bayesian/ - genetic), but the statistical *validity* of the selection itself. Not a - C-invariant until built; recorded as the World's third half — **now structurally - complete** (all three pieces built, cycle 0078). - **Status (cycle 0076):** the trials-deflation piece (#144) landed — - `optimize_deflated` (aura-registry) wraps `optimize` and records, additively on - the winning member's manifest (C18, the new `RunManifest.selection`), a deflated - score + an overfit probability for the family size, **without changing which - member wins** (C23; `optimize` / `rank_by` stay a bare argmax — the deflation is - recorded provenance, not a re-ranking). The R arm is a centred moving-block - reality-check (reusing the `r_bootstrap` kernel); the `total_pips` arm a - closed-form expected-max-of-K dispersion floor. **Status (cycle 0077):** the - plateau-over-peak piece (#145) landed — `optimize_plateau` (aura-registry) - argmaxes the **neighbourhood-smoothed** metric surface (mean or worst-case over - each member's closed mixed-radix grid neighbourhood) instead of the bare peak, - recorded on the same `RunManifest.selection` carrier, now an orthogonal *rule × - annotation*: `SelectionMode::{Argmax, PlateauMean, PlateauWorst}` with either the - deflation annotation (#144) or the plateau annotation (`neighbourhood_score` / - `n_neighbours`). It is opt-in via a `--select` flag — default argmax stays - byte-identical (C23). The grid lattice surfaces from the engine - (`SweepBinder::sweep_with_lattice`); the policy stays in aura-registry with - `walk_forward` selection-agnostic (C9). Selection is now a *pluggable objective* - (bare argmax → trials-deflated → plateau), not a single hard-wired argmax. - **Status (cycle 0078):** the last piece — cross-instrument generalization (#146) — - landed, completing the inferential half. Unlike #144/#145 (which *select + annotate* - a within-family winner on `RunManifest.selection`), #146 is an **aggregator/ - validator**, not a selector: the `aura generalize` subcommand runs a *brought* - candidate across an instrument list and `generalization` (aura-registry) reduces the - per-instrument R-metrics to a **worst-case floor** (min over instruments — the - cross-instrument sibling of `PlateauMode::Worst`, R-only per C10) + a sign-agreement - count + the per-instrument breakdown. It is a *recomputable aggregate* over a new - `FamilyKind::CrossInstrument` family (C12's comparison axis, realized), each member - self-identifying via the new `RunManifest.instrument` lineage field (C18) — *not* - stamped on `FamilySelection` (there is no within-family winner to annotate). The - inferential half is now structurally built; the per-candidate OOS bootstrap (#139, - cycle 0075) is its adjacent significance read. -- **`aura-std` contents** — substantively populated (~30 node modules): SMA/EMA, - arithmetic (`Add`/`Sub`/`Mul`/`Sqrt`/`LinComb`), logic (`And`/`Gt`/`Latch`/`EqConst`), - the `Delay` z⁻¹ register, `Resample`, `RollingMin`/`RollingMax`, `Session`, the R chain - (`Bias`/`FixedStop`/`Sizer`/`PositionManagement`), the legacy `SimBroker` pip yardstick, - the cost-model graph (`CostNode`/`CostRunner`/`CostSum` + `ConstantCost`/`VolSlippageCost`/ - `CarryCost`), and the `Recorder`/`GatedRecorder`/`SeriesReducer` sinks. Further blocks - land demand-driven as the walking-skeleton needs them. -- **`strategies/` split** — a later split, *inside a project*, of top-level + atop the atomic sim unit (C12), not yet designed. Search over *structure* + (graph mutation/crossover) additionally rides on topology-as-data (C24). +- **Composable orchestration** (#109 remainder) — the axes (sweep / MC / + walk-forward) as composable tools a World program wires, beyond today's + document-driven executor verbs; the campaign/process document layer (C18, + C25) is the shipped substrate. +- **`strategies/` split** — a later split, inside a project, of top-level strategies from reusable building blocks in `nodes/`; not a day-1 cut. -- **Sequencing** — the runnable single-harness *substrate* comes first (walking - skeleton: a closed harness that runs deterministically and records via a sink); - the World/meta layer (C21) and the playground trace-explorer (C22) are the - differentiating layers that follow — you orchestrate and visualize a thing that - must first run once. + +> History: [INDEX.history.md](INDEX.history.md) diff --git a/docs/design/contracts/c01-determinism.history.md b/docs/design/contracts/c01-determinism.history.md new file mode 100644 index 0000000..6a6c8e9 --- /dev/null +++ b/docs/design/contracts/c01-determinism.history.md @@ -0,0 +1,25 @@ +# C1 — Determinism and disjoint parallelism: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c01-determinism.md](c01-determinism.md). + +**Realization note (2026-07-16, #277).** Cross-sim parallelism now also spans +campaign cells: the executor flattens the cell matrix, groups it by instrument +ordinal, and walks sequential chunks of K instrument groups +(`--parallel-instruments`, default 4 — a structural bound on distinct resident +instruments, the RAM lever for the external data-server's per-reference file +retention); within a chunk, cells run concurrently on the process-global rayon +pool shared with the member/window fan-out. Results are collected into +document-order slots, so outputs stay byte-identical across worker counts and +bounds. Two deliberate scheduling-dependent carve-outs, both outside this +contract's per-run bit-identity (which governs successful runs): on the +run-fatal path (non-containable faults, e.g. a dead registry store) the +propagated fault is the lowest document-order fault among the cells that +completed before the abort flag latched, and the set of per-cell family lines +already written by then is scheduling-dependent — inert, because no +campaign-run record is written on that path and store reads are name-keyed, +never line-ordered. Duplicate campaign instruments are refused at both the +validate tier and the executor's preflight: the per-cell family name embeds +the raw instrument string, so uniqueness is what keeps concurrent appends from +racing one name's run-index assignment. diff --git a/docs/design/contracts/c01-determinism.md b/docs/design/contracts/c01-determinism.md new file mode 100644 index 0000000..d55a972 --- /dev/null +++ b/docs/design/contracts/c01-determinism.md @@ -0,0 +1,69 @@ +# C1 — Determinism and disjoint parallelism + +**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event +loop that reaches a unique state after each input tick. Same input (incl. seed) +→ bit-identical run. Two backtests are fully disjoint and run concurrently +without locking. The bit-identity is *per run*: one backtest of given (inputs, +seed) reproduces byte-for-byte. It does **not** extend to a *derived metric* +recomputed for the same params by two *different command paths* (e.g. a swept +member's `sqn` vs the same cell re-run under `aura generalize`), which may differ +by floating-point reassociation (≤1 ULP) because the two paths accumulate the +same logical reduction in a different operation order (IEEE-754 non-associativity). +C1 governs the determinism of a single run, not the cross-command bit-identity of +a re-derived statistic. (fieldtest 0078, ratified.) + +**Forbids.** Concurrency *within* a single sim; any nondeterministic input that +is not captured as an explicit input (see C11, C12). + +**Why.** Real money rides on backtest results; reproducibility and an audit +trail are non-negotiable. Speed comes from parallelism *across* sims, which +disjointness makes lock-free. + +## Current state + +The per-run event loop is `Harness::run` (`crates/aura-engine/src/harness.rs`): +each cycle picks the live source head with the smallest `(timestamp, source +index)`, pops one record, increments `cycle_id`, and forwards the value — +allocating nothing per cycle beyond a reused scratch buffer. A single run is +fully sequential, so its output is bit-identical by construction; the +end-of-stream `finalize` pass runs after the loop and adds no within-sim +concurrency. + +Cross-sim parallelism spans the sweep/member/window fan-out and campaign cells +(#277). The campaign executor `execute` (`crates/aura-campaign/src/exec.rs`) +flattens the (strategy, instrument, window, regime) cell matrix into +document-order `PlannedCell`s, groups them by instrument ordinal, and walks +sequential chunks of K instrument groups — `--parallel-instruments` +(`DEFAULT_PARALLEL_INSTRUMENTS = 4`, `crates/aura-campaign/src/lib.rs`). K is a +structural bound on distinct resident instruments — the RAM lever for the +external data-server's per-reference file retention (~1.4 GB per instrument on +the reference dataset), not a pool-worker count. Within a chunk, cells run +concurrently via `rayon::par_iter` on the process-global pool shared with the +member/window fan-out, and results are written into document-order `slots`, so +outputs stay byte-identical across worker counts and bounds. + +Two scheduling-dependent carve-outs sit outside the per-run bit-identity, which +governs successful runs. On the run-fatal path (non-containable faults, e.g. a +dead registry store) a shared `AtomicBool` abort flag latches, and the propagated +fault is the lowest document-order fault among the cells that completed before +the latch; the set of per-cell family lines already written by then is +scheduling-dependent — inert, because no campaign-run record is written on that +path and store reads are name-keyed, never line-ordered. + +Duplicate campaign instruments are refused at both the validate tier +(`crates/aura-campaign/src/lib.rs`, a `seen_instruments` scan) and the executor's +`preflight`: the per-cell registry family name embeds the raw instrument string, +so name uniqueness is what keeps concurrent appends from racing one name's +run-index assignment. + +## See also + +- [C4](c04-cycle-granularity.md) — tie determinism preserves this contract; the + source-index tie-break is what keeps the merge order deterministic. +- [C11](c11-sources-record-replay.md), [C12](c12-atomic-sim-unit.md) — + recorded/pinned inputs are how nondeterministic sources become explicit inputs; + the harness is the atomic disjoint unit. +- [C23](c23-graph-compilation.md) — the raw-index compilation primitive; names + survive only as non-load-bearing debug symbols. + +> History: [c01-determinism.history.md](c01-determinism.history.md) diff --git a/docs/design/contracts/c02-causality.md b/docs/design/contracts/c02-causality.md new file mode 100644 index 0000000..3462447 --- /dev/null +++ b/docs/design/contracts/c02-causality.md @@ -0,0 +1,36 @@ +# C2 — Causality / no look-ahead + +**Guarantee.** A node sees only the past. Input history is a read-only window +that ends at the current cursor; a resampler emits a bar only once it is +complete. + +**Forbids.** Any node access to data with `timestamp > now`; emitting a partial +/ still-forming bar. + +**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that +leaks the future is worse than none. Making the future *physically absent* from +what a node receives beats merely discouraging it. + +## Current state + +A node's `eval` receives `Ctx::new(&inputs, ts)` (`crates/aura-engine/src/harness.rs`): +the input windows are read-only slices whose head (`window[0]`) is the value +pushed at or before the current `cycle_id`. The engine forwards exactly one +merged record per cycle in ascending timestamp order, so a node structurally +cannot reach a record with `timestamp > now` — the future is absent from what it +receives, not merely off-limits. + +The resampler (`crates/aura-market/src/resample.rs`) emits a completed coarse bar +only on bucket rollover; a still-forming bar is never emitted, and an incomplete +final bucket with no following rollover tick simply never emits. The completed +bar carries no timestamp of its own — it takes the rollover instant, the close +instant of the bar just emitted (C4). + +## See also + +- [C3](c03-single-merge.md) — the single ingestion merge is what makes a + read-only, monotone cursor possible. +- [C4](c04-cycle-granularity.md) — the data-driven cycle clock the cursor + advances along. +- [C5](c05-freshness-gating.md) — held (stale) values are real past values, not + a look-ahead escape hatch. diff --git a/docs/design/contracts/c03-single-merge.md b/docs/design/contracts/c03-single-merge.md new file mode 100644 index 0000000..220fa73 --- /dev/null +++ b/docs/design/contracts/c03-single-merge.md @@ -0,0 +1,36 @@ +# C3 — One merge, at ingestion only + +**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp +into one chronological cycle stream at the ingestion boundary; source-native +time units (e.g. data-server's Unix-`time_ms`) are normalized there to the +canonical epoch-ns `timestamp` of C7. + +**Forbids.** Any merge / as-of join *inside* the graph. + +**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate +sources (news daily-bias + M5 + ticks) causally combinable without leaking the +future. Keeping the merge at one boundary keeps the graph semantics simple. + +## Current state + +The k-way merge is `Harness::run`'s per-cycle pick of the live source head with +the smallest `(timestamp, source index)` (`crates/aura-engine/src/harness.rs`). +Each `Source` yields records in ascending timestamp order — the ingestion +precondition the merge relies on — and there is no merge or as-of join anywhere +inside the DAG; the engine only routes in-graph edges. + +The ms→epoch-ns normalization is a single seam, `aura_ingest::unix_ms_to_epoch_ns` +(`crates/aura-ingest/src/lib.rs`): the data-source ingestion edge transposes the +data-server's Array-of-Structs `M1Parsed` records into aura's Structure-of-Arrays +base columns (C7) and normalizes their Unix-millisecond time to the canonical +epoch-ns `Timestamp` at this one boundary. Both the eager `load_m1_window` +transpose and the lazy streaming `M1FieldSource` route through that same seam. + +## See also + +- [C4](c04-cycle-granularity.md) — the merged stream is the data-driven cycle + clock; ties break by source declaration order. +- [C2](c02-causality.md) — one ordered timeline is what makes heterogeneous-rate + sources combinable without look-ahead. +- [C7](c07-scalar-soa.md) — the canonical epoch-ns `Timestamp` and the four SoA + base columns the merge normalizes into. diff --git a/docs/design/contracts/c04-cycle-granularity.history.md b/docs/design/contracts/c04-cycle-granularity.history.md new file mode 100644 index 0000000..b250052 --- /dev/null +++ b/docs/design/contracts/c04-cycle-granularity.history.md @@ -0,0 +1,15 @@ +# C4 — Cycle granularity: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c04-cycle-granularity.md](c04-cycle-granularity.md). + +**Realization (#275, 2026-07-15).** Ingestion sources are supplied to `run` **by +role name**, not by list position. `SourceSpec` carries a `role: Option` +(the bound `Role`'s name, load-bearing for source binding), and `Harness::run_bound` +resolves a keyed supply against those roles, emitting sources in `SourceSpec` +declaration order. The C4 tie-break stays "source declaration order" — now +independent of how the caller orders the supply — and a mis-bound feed is a named +`SourceBindError` at start time rather than a silently wrong run. The raw-index +`run(Vec)` primitive (positional, C23) is unchanged; every hand-built graph keeps +`role: None`. diff --git a/docs/design/contracts/c04-cycle-granularity.md b/docs/design/contracts/c04-cycle-granularity.md new file mode 100644 index 0000000..0181259 --- /dev/null +++ b/docs/design/contracts/c04-cycle-granularity.md @@ -0,0 +1,43 @@ +# C4 — Cycle granularity + +**Guarantee.** The clock is data-driven: one input record = one cycle, advanced +in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp, +multiple sources) break by source declaration order. + +**Forbids.** A fixed time-grid clock; nondeterministic tie ordering. + +**Why.** The market *is* an irregular event sequence; a grid is arbitrary and +either wastes empty cycles or clumps ticks. Backtest and live differ only in the +origin of records, not the cycle semantics. Tie determinism preserves C1. + +## Current state + +`Harness::run` (`crates/aura-engine/src/harness.rs`) increments `cycle_id` once +per popped record and each cycle picks the smallest `(timestamp, source index)`; +the source-index tie-break is the source *declaration* order. + +Ingestion sources are supplied to the production run path by role name, not list +position (#275). `SourceSpec` carries `role: Option` — the bound `Role`'s +name, load-bearing for source binding only, with every other flat-graph name +staying a non-load-bearing debug symbol (C23). `Harness::run_bound` resolves a +keyed supply against those roles via `bind_sources`, emitting sources in +`SourceSpec` declaration order, so the tie-break is independent of how the caller +orders the supply. A mis-bound feed is a named `SourceBindError` +(`MissingFeed` / `ExtraFeed` / `DuplicateFeed` / `UnnamedSource`) at start time, +not a silently wrong run. The raw-index `run(Vec)` primitive (positional, C23) is +unchanged: `SourceSpec::raw` carries `role: None` and every hand-built graph keeps +`role: None`; a raw-index arity mismatch stays an engine-wiring panic, while a +keyed role mismatch is a user data-binding `Result`. + +## See also + +- [C1](c01-determinism.md) — tie determinism is a precondition of the per-run + bit-identity. +- [C3](c03-single-merge.md) — the single ingestion merge is the clock's source + stream. +- [C23](c23-graph-compilation.md) — the raw-index compilation primitive; names + non-load-bearing except the binding role. +- [C5](c05-freshness-gating.md) — `cycle_id` advances every cycle, but recompute + is gated. + +> History: [c04-cycle-granularity.history.md](c04-cycle-granularity.history.md) diff --git a/docs/design/contracts/c05-freshness-gating.md b/docs/design/contracts/c05-freshness-gating.md new file mode 100644 index 0000000..3dd06c2 --- /dev/null +++ b/docs/design/contracts/c05-freshness-gating.md @@ -0,0 +1,35 @@ +# C5 — Freshness-gated recompute and sample-and-hold + +**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node +re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by +run-count); otherwise it holds its last output. Stale inputs contribute their +last (held) value. + +**Forbids.** Recomputing every node every cycle ("push all" is true for the +*clock*, not for *recompute*); treating a held value as missing. + +**Why.** Total recompute does not scale to many sparse high-frequency sources; +freshness-gating is the performance discipline that keeps the synchronous model +fast. + +## Current state + +Freshness is a run-count epoch. Each input slot carries `SlotState { fresh_at, +last_ts }` (`crates/aura-engine/src/harness.rs`); a slot is fresh this cycle iff +`fresh_at == cycle_id`. The firing predicate `fires` re-evaluates a node when any +`Firing::Any` input is fresh this cycle, or when a barrier group completes (≥1 +member fresh this cycle and every member carrying `last_ts == ts`) — the barrier +/ co-freshness machinery itself is C6. + +A node that does not fire pushes nothing, so downstream consumers keep reading the +last value via `window[0]`: sample-and-hold falls out of the push model rather +than being a separate mechanism. A held value is a real past value, never treated +as missing. + +## See also + +- [C6](c06-firing-policy.md) — the firing predicate and the barrier / + co-freshness rule this gating builds on. +- [C1](c01-determinism.md) — freshness is a deterministic function of `cycle_id`, + so gating never perturbs the bit-identical run. +- [C2](c02-causality.md) — a held value is past data, not look-ahead. diff --git a/docs/design/contracts/c06-firing-policy.history.md b/docs/design/contracts/c06-firing-policy.history.md new file mode 100644 index 0000000..284eff2 --- /dev/null +++ b/docs/design/contracts/c06-firing-policy.history.md @@ -0,0 +1,14 @@ +# C6 — Firing policy A and B, per input group: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c06-firing-policy.md](c06-firing-policy.md). + +**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any, +Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a +mode-A input is its own trivial group, so "per input group" and the per-input tag +coincide. The barrier's synchronization token is the cycle **timestamp**, not the +`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so +RustAst's `cycle_id`-equality barrier could never fire across them. A group fires +when every member's last push carries the current cycle's timestamp, guarded by +"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This +fires both the multi-source bar and the within-source diamond rejoin (every push +in a cycle carries that cycle's timestamp). diff --git a/docs/design/contracts/c06-firing-policy.md b/docs/design/contracts/c06-firing-policy.md new file mode 100644 index 0000000..2ba7501 --- /dev/null +++ b/docs/design/contracts/c06-firing-policy.md @@ -0,0 +1,42 @@ +# C6 — Firing policy A and B, per input group + +**Guarantee.** A node declares, per input group, one of two firing policies: +**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held +daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from +four separate 15m sources: the candle is complete only when all four are fresh). +A single node may mix an A input and a B group, and it fires when **any** of its +input groups fires (OR over groups). + +**Forbids.** A single global firing mode; forcing per-node-only granularity. + +**Why.** Both are genuinely needed; RustAst, the retired predecessor, implemented +only B. Per-input-group granularity is required by the OHLC-plus-bias case where +one node needs both an A input and a B group at once. + +## Current state + +Firing is tagged per input — `Firing::{Any, Barrier(u8)}` +(`crates/aura-core/src/node.rs`) carried on each input's `PortSpec.firing`, not a +separate group object. Inputs sharing a `Barrier` id form a group; a mode-A input +is its own trivial group, so "per input group" and the per-input tag coincide. + +The barrier's synchronization token is the cycle **timestamp**, not the +`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so a +`cycle_id`-equality barrier could never fire across them — the token has to be the +data timestamp of the push. + +Firing is evaluated by `fires()` in `crates/aura-engine/src/harness.rs`: a mode-A +input fires the node when it is fresh this cycle (`fresh_at == cycle_id`), a stale +input contributing its held value; a barrier group fires when ≥1 member is fresh +this cycle **and** every member's last push carries the current cycle's timestamp. +The "≥1 fresh" guard is what stops a group completed in a prior cycle from +re-firing. This one rule fires both the multi-source bar and the within-source +diamond rejoin (C3), since every push in a cycle carries that cycle's timestamp. + +## See also +- [C3](c03-single-merge.md) — the one-merge boundary; the within-source diamond rejoin the barrier also completes +- [C4](c04-cycle-granularity.md) — cycle/timestamp granularity that forces the barrier token to be a timestamp +- [C5](c05-freshness-gating.md) — freshness-gated recompute and sample-and-hold (mode A's hold semantics) +- [C8](c08-node-contract.md) — the node signature declares each input's firing group + +> History: [c06-firing-policy.history.md](c06-firing-policy.history.md) diff --git a/docs/design/contracts/c07-scalar-soa.history.md b/docs/design/contracts/c07-scalar-soa.history.md new file mode 100644 index 0000000..d2b069b --- /dev/null +++ b/docs/design/contracts/c07-scalar-soa.history.md @@ -0,0 +1,32 @@ +# C7 — Four scalar base types, streamed as SoA: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c07-scalar-soa.md](c07-scalar-soa.md). + +**Realization (`Cell` carrier split, 2026-06).** The streamed value is now split +into a tag-free 64-bit word and its kind. `Cell` (`crates/aura-core/src/cell.rs`) +is a type-erased `u64`: constructed per base type (`from_i64/f64/bool/ts`) and +read only by naming the type at the call site (`i64()/f64()/bool()/ts()` — +branch-free bit-casts). The kind is therefore resolved once at the boundary and +the value itself carries no tag — C7's "the type lives at the column/edge, not in +the value" made explicit on the single-value carrier (and a single 8-byte word +vs. the 16-byte tagged enum). `Scalar` becomes `{ kind: ScalarKind, cell: Cell }` +— the self-describing form for the *dynamic* boundaries (builder binding, serde, +rendering) — with `debug_assert`-guarded native accessors (caller asserts the +kind; free in release) and a hand-written **value** `PartialEq` that preserves +the former enum's IEEE-754 semantics (`NaN != NaN`, `+0.0 == -0.0`), pinned by +the `scalar_eq_is_value_not_bitwise` fixture. Behaviour-preserving. + +**Realization (`Cell` becomes the hot-path carrier, 2026-06, #74).** The carrier +swap deferred above has landed: `Node::eval` now returns `Option<&[Cell]>` and +every node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free +8-byte words. The kind lives only at the schema/column: the harness forwards each +field via the new branch-free `AnyColumn::push_cell` (infallible — the edge +kind match is verified once at bootstrap, the surviving guard +`bootstrap_rejects_*_kind_mismatch`). `Scalar` remains on the self-describing +dynamic boundaries: the param plane (`build`/`bind`/`compile_with_params`/sweep +points/`RunManifest`), `AnyColumn::get` (the type-erased read for sinks/serde), +and source ingestion (`Source::next`, the heterogeneous C3 merge). The removed +per-value runtime kind check on node output is the same authoring-bug class C8 +already leaves to a `debug_assert` (output width); node-output-kind correctness +is the node's declared `FieldSpec` contract, caught by each node's own test. +Behaviour-preserving (C1). diff --git a/docs/design/contracts/c07-scalar-soa.md b/docs/design/contracts/c07-scalar-soa.md new file mode 100644 index 0000000..483c003 --- /dev/null +++ b/docs/design/contracts/c07-scalar-soa.md @@ -0,0 +1,64 @@ +# C7 — Four scalar base types, streamed as SoA + +**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64, +epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams +(OHLCV) are bundles of base columns — this is the **node-output model** too: a +node emits a record of 1..K base columns (C8), each forwarded field-wise to a +consumer slot; the bundle is structural, never a fifth scalar type. Edges are +type-erased to these four kinds; the type check is paid once at wiring/sim-start, +then the topology is frozen per sim → direct dispatch, no per-event allocation. + +**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those +live as metadata beside the hot path; `dyn Any` payloads; per-event heap +allocation; topology mutation mid-sim. + +**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar +set and SoA. The open set is composites (schemas of columns), not scalar types. +Type-erasure at the edge is also forced by the cdylib boundary (C13). + +## Current state + +The streamed value is split into a tag-free 64-bit word and its kind. `Cell` +(`crates/aura-core/src/cell.rs`) is the type-erased `u64` hot-path carrier: +constructed per base type (`from_i64/from_f64/from_bool/from_ts`) and read only by +naming the type at the call site (`i64()/f64()/bool()/ts()` — branch-free +bit-casts, no tag to check). The kind lives once, at the column/edge/port, never +in the value — C7's principle made explicit on the single-value carrier (one +8-byte word, not a 16-byte tagged enum). `Cell` deliberately has no tagged +constructors, so a value can never re-acquire a per-value tag once it crosses onto +the hot path. + +`Node::eval` returns `Option<&[Cell]>` (`crates/aura-core/src/node.rs`) and every +node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free 8-byte +words. The harness forwards each field via `AnyColumn::push_cell` +(`crates/aura-core/src/any.rs`) — branch-free and infallible, because the edge's +`from_field`→slot kind match is verified once at bootstrap, not per value. There +is deliberately no per-value runtime kind check on node output: node-output-kind +correctness is the node's declared `FieldSpec` contract (C8), caught by each +node's own test — the same authoring-bug class C8 already leaves to a +`debug_assert` for output width. + +`Scalar` (`crates/aura-core/src/scalar.rs`) is the self-describing, kind-tagged +carrier for the *dynamic* boundaries, where a value travels detached from any +co-present schema and so must remember its own type: the param plane (builder +binding via `bind`, sweep points, `RunManifest`), `AnyColumn::get` (the +type-erased read for sinks/serde), and source ingestion (`Source::next → +Option<(Timestamp, Scalar)>`, the heterogeneous C3 merge). `Scalar` and `Cell` +are **disjoint**, bridged by `Scalar::from_cell(kind, cell)` (decode, the kind +supplied by the co-present schema — e.g. a validated param point carries bare +cells, its kind living once in the co-indexed `ParamSpec`) and `Scalar::cell()` +(encode, kind dropped once checked against the slot). Reading a `Scalar` is +caller-asserted: `as_i64/as_f64/as_bool/as_ts` panic on a kind mismatch (a caller +bug). `Scalar`'s value equality preserves IEEE-754 semantics (`NaN != NaN`, +`+0.0 == -0.0`) and never equates across kinds (`i64(0) != f64(0.0)`), pinned by +the `scalar_eq_is_value_not_bitwise` fixture; `Cell`'s own `Eq`/`Hash` are +bit-exact (the semantics of a raw word, not of a number). + +## See also +- [C1](c01-determinism.md) — behaviour-preservation / bit-identity across the carrier split +- [C3](c03-single-merge.md) — heterogeneous ingestion, where `Scalar` is the ingest carrier +- [C8](c08-node-contract.md) — the node-output record and `eval`'s Cell buffer; `FieldSpec` output-kind contract +- [C13](c13-hot-reload-frozen-deploy.md) — the cdylib boundary that also forces edge type-erasure +- [C23](c23-graph-compilation.md) — bootstrap compilation; the once-at-wiring edge kind check; names non-load-bearing + +> History: [c07-scalar-soa.history.md](c07-scalar-soa.history.md) diff --git a/docs/design/contracts/c08-node-contract.history.md b/docs/design/contracts/c08-node-contract.history.md new file mode 100644 index 0000000..cbd0d42 --- /dev/null +++ b/docs/design/contracts/c08-node-contract.history.md @@ -0,0 +1,159 @@ +# C8 — The node contract: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c08-node-contract.md](c08-node-contract.md). + +**Realization (cycle 0005).** `NodeSchema.output` is a `Vec` (named base +columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field` +selects one producer column per edge; consuming a whole record is N edges (no +"bind whole record" mechanism). The K fields of one record are **co-fresh by +construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns +`Option<&[Cell]>` — a borrowed row into a node-owned buffer — so the forward +path allocates nothing per cycle (C7) (the carrier is now a tag-free `Cell` — see +the C7 carrier note). + +**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is +now realized at the substrate: **recording is a node role, not a type.** A +recording node reads its typed input windows + `ctx.now()` in `eval` and pushes +the record to a destination it holds as a field (a channel, a chart handle) — an +**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a +node that only records returns `None` (pure consumer), and a node may record +**and** return an output the engine forwards in the same `eval` (the "both" +case). **Encoding & return contract.** A pure consumer declares `output: vec![]` +— the empty record *is* the sink declaration; there is no separate type, trait, +or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run +loop debug-asserts the returned row's width equals the declared output width +(`row.len() == schema.output.len()`). Field-wise wiring resolves +`Edge::from_field` against the producer's `output` at bootstrap, so no edge can +bind a field of a zero-output node — it fails with `BadIndex` — making a sink +structurally unwireable as an in-graph producer; its only output is the +out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out +of the graph is the node's own side effect — and that boundary is the +determinism / graph-as-data boundary (C1/C7). + +**Realization (cycle 0015 — param declaration).** The tunable-parameter half of +this contract is now realized: a node declares its knobs in `schema()` as +`params: Vec` (`ParamSpec { name: String, kind: ScalarKind }`), and +`Composite::param_space()` (cycle 0024; was `Blueprint::param_space()`) aggregates +them into one flat, path-qualified list — a +read-only projection of the graph-as-data (C9), mirroring the inline order +(C19/C23) so a param's slot matches the later flat-node order. Two refinements to +the guarantee's "typed, with ranges": (1) the declaration carries `name` + `kind` +only — the **search-range is the run's, not the node's** (which subset / grid a +sweep covers is an experiment axis, #32/C20; the node declares the knob's existence +and type, never its search interval). (2) Identity is **positional** (the slot, +C23 "by index, not name"); the path-qualified `name` is a non-load-bearing debug +symbol (like `FieldSpec.name`) — uniqueness is at the slot. **Refinement (cycle +0032):** identity in the **flat graph** stays positional (C23, unchanged), but the +`param_space()` **name projection** — the authoring / by-name address space (the +surface a sweep axis or single-run binding addresses) — must be **injective** for a +blueprint to compile (a duplicated path is unaddressable; see C9). Two layers: +positional wiring below (the flat graph), an injective name address space above (the +authoring boundary). So same-type siblings in one composite no longer silently +share a name — they must be `.named(...)` apart, which is also what disambiguates a +same-type fan-in (C9). A vector knob (`LinComb.weights`) expands to `N` flat +`weights[i]` entries, `N` topology-fixed (C19). Permitted kinds are `i64`/`f64`/ +`bool` (a `timestamp` knob is a structural axis, C20, never a numeric sweep param). +Binding a value to a slot landed in cycle 0016 (#31, see C19/C23); enumerating a +sweep (#32) is the deferred next layer. The 0016 binding also moved the param +declaration's *authoring* home into the value-empty recipe, whose `params()` is read +pre-build. In 0016–0023 the built node's `schema().params` still reported the same +slots, kept in lockstep by a per-node test (a duplication filed as debt, #36); **cycle +0024 dissolved this** — the signature is declared *once* on the recipe and the built +node no longer carries `schema()` (#36 closed). See the C8 cycle-0024 realization. + +**Realization (cycle 0024 — the signature lives in the blueprint, #43/#36).** The +node's whole declared interface — its `NodeSchema` (input scalar types + firing, +output record, params) — now lives **once**, on the value-empty recipe +`PrimitiveBuilder` (ex-`LeafFactory`), read pre-build by `param_space()`, the +compile-time structural validation, and the render. This closes two debts: **#43** +(a value-empty recipe used to declare *no* input/output interface pre-build — only +params) and **#36** (params declared twice, recipe vs built node, kept in lockstep +by a per-node test — those 8 tests are deleted, the duplication structurally gone). +The split that makes this work: a node's signature is **fully static** per blueprint +(input kinds/firing, output fields, params — verified across the roster; a variable- +arity node like `LinComb` takes its arity as a *recipe argument*, not an injected +param), so it can be declared without building; the **one** param-dependent quantity, +an input's buffer **lookback** depth, is no longer in the signature but answered by +`Node::lookbacks() -> Vec`, read only by bootstrap to size the windows. +`Node::schema()` is therefore **removed** — the signature is pre-build data, not a +built-node method. `BlueprintNode::signature()` answers uniformly for both arms (a +primitive returns its recipe's schema; a composite *derives* it from the interior: +role kinds in, re-exported field kinds out, aggregated params), so "every node has a +signature in the blueprint" holds for composites too. + +**Realization (cycle 0027 — name input ports, refs #21/#51).** `PortSpec` now +carries a `name: String` (it drops `Copy`, like `ParamSpec`), so an input port is +named just as `FieldSpec.name` (output) and `ParamSpec.name` (param) already are. +The name is **non-load-bearing** (C23): wiring stays positional by slot, bootstrap +and the run loop never read it; it exists for tracing / graph rendering (#13). Leaf +primitives declare their slot names (`SimBroker`'s `exposure`/`price`, etc.); +`derive_signature` carries a composite's `Role.name` into the derived input port, +so the graph model (`model_to_json`) is homogeneously named across inputs, outputs, +and params and across both graph levels. This does **not** close #21 (a swapped +same-kind wiring is still only kind-checked) — it makes those slots +self-documenting; a name-consuming validation is its own future cycle. + +**Realization (cycle 0040 — wiring totality: every input slot connected exactly +once, #65).** The node contract's "the engine provides a window into each input" +presupposes each input is actually fed; until now an unwired interior input slot was +accepted and bootstrapped to a silent empty column (`harness.rs`), and two producers +into one slot — ill-formed, since a slot holds one column — was likewise +uncompiled-against. Cycle 0040 makes a valid graph **total and single-valued** over +its interior input slots: `check_ports_connected` (in `validate_wiring`, run at every +nesting level) requires every interior node's every declared input slot to be covered +by **exactly one** wiring act — one interior `Edge { to, slot }` or one role +`Target { node, slot }`, edges and role targets counted uniformly. Zero coverage is +`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. A +composite's **own** input roles (`source: None`) are coverage *providers* (the +wired-by-enclosing boundary, the root case already guarded by `UnboundRootRole`) — +never consumers, so only interior input slots are subject to the rule. There is **no +optional-input concept**: every declared input port is required (no shipped node runs +meaningfully without one; an unwarmed mode-A input is *wired-but-not-yet-valued*, not +unwired). The check is **index-based and name-free** — it touches no name machinery +and emits nothing into the flat graph, so C23 is untouched (it proves the existing +raw-index wiring is total and single-valued). Inherited identically by the raw +`Composite::new` path and the `GraphBuilder::build()` path (both compile via +`compile_with_params`). + +**Refinement (Construction-layer milestone — render labels, 2026-06-05).** A node +additionally exposes `label() -> String`, a **single-line, non-load-bearing** +render symbol: a default trait method the run loop never calls (wiring is by +index, C23). Overrides carry the node's identifying params (`SMA(2)` vs `SMA(4)`) +so a graph render (C9 graph-as-data, #13) disambiguates identical node types and +surfaces a mis-wiring. Like `FieldSpec.name`, it is an informative debug symbol, +not part of the C8 dataflow contract — adding it changes no run behaviour. + +**Refinement (cycle 0070 — end-of-stream `finalize` lifecycle, 2026-06-25).** The +node contract gains a second lifecycle phase beside `eval`: `finalize(&mut self)`, +a **default-no-op** trait method the engine calls **once per node, in topological +order, after the source loop drains** (the `Harness::run` epilogue). It lets a sink +**fold online** — accumulate per `eval` into O(trades)/O(1) owned state and flush +one compact summary at stream end — instead of pushing a record every fired cycle +into an unbounded channel that buffers O(cycles) rows until the run ends (the +retention BLOCKER #138 hit: a full-history Stage-1 sweep held ~2 GiB/member over +~5.5M one-minute bars). C1/C7/C8 are preserved: `finalize` runs once *after* the +deterministic event loop, adding no within-sim concurrency (C1); a folding sink +holds only owned accumulator state, no interior mutability (C7); it still declares +`output: vec![]` and its flush is the same out-of-graph side effect (C8) — one +summary row, not a per-cycle stream. Realized by `GatedRecorder` (emits only gated +rows + the genuine final row) and `SeriesReducer` (folds one column, emits one +summary row), aura-std siblings of the per-cycle `Recorder`, which survives for the +live / `--trace` / test-tap path. The recording sink's *own* per-cycle allocation +(the `Recorder`→`Probe` rename + its accumulate-vs-stream choice, #77) stays open; +`finalize` is the "non-channel exit from the graph" that issue's +accumulate-then-read option named as missing. + +**Realization (2026-07-11 — composite `doc`, #125).** `Composite` carries an +optional authored rationale `doc: Option` — the prose twin of its `name` +and the same C23 category: a non-load-bearing debug symbol. Authored via +`Composite::with_doc` / `GraphBuilder::doc`; persisted as a Tier-1 +additive-optional `CompositeData` field (no format-version bump; absent-field +documents keep their exact bytes, so existing content ids are untouched); +**blanked in the identity projection** (`strip_debug_symbols`) while staying +canonical-byte-bearing; dissolved at inline like the name (never reaches +`FlatGraph`). Surfaced read-only (C22): the graph model emits an optional +trailing `"doc"` fragment per scope (`json_str` hardened for multi-line free +text — `\n`/`\t`/`\r` named, other control chars as `\u00XX`), the viewer shows +it in both composite view states (collapsed tooltip, expanded cluster frame) and +the root's as a muted header line. The construction op-script vocabulary +deliberately has no doc-carrying surface yet (scope cut recorded on #125). diff --git a/docs/design/contracts/c08-node-contract.md b/docs/design/contracts/c08-node-contract.md new file mode 100644 index 0000000..5817054 --- /dev/null +++ b/docs/design/contracts/c08-node-contract.md @@ -0,0 +1,146 @@ +# C8 — The node contract + +**Guarantee.** A node is the universal composable dataflow unit — a **producer, a +consumer, or both**. It has a **signature**, its `NodeSchema`: each input port +(scalar kind + firing group), its output record, and its own **tunable +parameters** — each a **typed knob** (`name` + scalar kind). The node declares a +knob's *existence and type*, never its search interval: which subset/grid a sweep +covers is the run's concern (an experiment axis, C20), not the node's. The knobs +aggregate into the blueprint's flat, path-qualified param-space the optimizer +sweeps (C12/C19/C20). The whole signature is **declared once, pre-build, on the +value-empty recipe** (C19), so the entire interface is legible without building; +the built node carries no `schema()`. The one param-dependent quantity — an +input's buffer **lookback** depth (e.g. an `Sma`'s window = its `length`) — is not +in the signature but answered by `lookbacks()`, read once at bootstrap to size the +windows. The built node implements `eval(ctx) -> Option<&[Cell]>`: the engine +provides read-only, zero-copy windows into each input's SoA ring buffer +(`ctx.f64_in(i)[k]`, sized at wiring), and `eval` returns a borrowed row — one +`Cell` per declared output column — into a node-owned buffer, so the forward path +allocates nothing per cycle (C7). A node may *additionally* keep its own mutable +series for derived/intermediate state. Binding is **field-wise**: one edge selects +one producer column, so consuming a whole record is N edges — there is no +"bind whole record" mechanism. The K columns of one record are **co-fresh by +construction** (one `eval`, one timestamp), so C6 is untouched. + +A **producer/transformer** exposes **one output port**, whose payload is a +**record of 1..K base-scalar columns** (a scalar is the degenerate K=1 record); a +**pure consumer (sink)** — chart, equity, logger — declares **no output** and +records via an out-of-graph side effect. **Recording is a node role, not a type:** +a node declaring `output: vec![]` *is* a sink (the empty record is the whole +declaration — no `Sink` type, trait, or engine flag), its `eval` returns `None` or +a zero-width `Some(&[])`, and it pushes its record to a destination it holds as a +field (a channel, a chart handle). A node may record **and** return a forwarded +output in the same `eval` (the "both" case). `None` = filter / not-yet-warmed-up / +pure sink. Sources are pure producers; sinks are pure consumers. Beside `eval`, +the contract has a second lifecycle phase, `finalize()` — a default-no-op hook the +engine calls once per node, in topological order, after the source loop drains — +letting a sink fold online and flush one compact summary at stream end instead of +streaming a row every fired cycle. A node also exposes `label()`, a single-line, +non-load-bearing render symbol (C23). + +**Forbids.** A node sizing/growing its input lookback at runtime; more than one +output **port** per node; a fifth scalar type or a heterogeneous output payload (a +record is a bundle of base columns, C7); copy-on-read of input history; a `Sink` +type/trait/engine flag or a "bind whole record" mechanism (both are structurally +absent by design). + +**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage lookback +bookkeeping, and history passes through zero-copy. Fixed, pre-sized buffers suit +deterministic, pre-dimensioned sims (no realloc in the hot loop). Declaring the +signature once, pre-build on the recipe, keeps the whole interface legible without +building and dissolves the drift of declaring params twice (recipe vs built node). +The empty-output sink keeps in-graph routing engine-owned data (the edge table) +and pushes every escape *out* of the graph onto the node's own side effect — that +boundary is the determinism / graph-as-data boundary (C1/C7). `finalize` gives a +folding sink O(trades)/O(1) owned accumulator state and one summary row, instead of +an unbounded channel that buffers O(cycles) rows until the run ends; it runs once +*after* the deterministic event loop, adding no within-sim concurrency (C1), and +holds only owned state, no interior mutability (C7). + +## Current state + +The signature types and the `Node` trait live in `aura-core/src/node.rs`. +`NodeSchema { inputs: Vec, output: Vec, params: Vec }`; +`PortSpec { kind, firing, name }`, `FieldSpec { name, kind }`, and +`ParamSpec { name, kind }` each carry a `name` that is a **non-load-bearing** debug +symbol (C23) — wiring is positional by slot; bootstrap and the run loop never read +it. Permitted param kinds are `i64`/`f64`/`bool` (a `timestamp` knob is a +structural axis, C20, never a numeric sweep param). + +`trait Node` declares `lookbacks() -> Vec`, `eval(&mut self, ctx) -> +Option<&[Cell]>`, `label() -> String` (default a placeholder every shipped node +overrides; overrides carry identifying params so `SMA(2)` vs `SMA(4)` disambiguate +in a graph render, #13), and `finalize(&mut self)` (default no-op). There is **no** +`Node::schema()` — the signature is pre-build data, not a built-node method. The +signature is declared once on the value-empty recipe `PrimitiveBuilder` +(`aura-core/src/node.rs`), whose `schema()` / `params()` are read pre-build by the +param-space aggregation, the structural validation, and the render; `named(...)` +sets a node instance's authoring name and `bind(slot, value)` fixes a knob to a +structural constant, removing it from the param surface. + +Field-wise binding is `Edge::from_field` (`aura-engine/src/blueprint.rs`), resolved +against the producer's `output` at bootstrap. A sink's empty `output` therefore +makes it structurally unwireable as an in-graph producer — no edge can bind a field +of a zero-output node. The run loop (`aura-engine/src/harness.rs`) debug-asserts +`row.len()` equals the declared output width, and that `lookbacks()` arity equals +`signature().inputs.len()`. The end-of-stream `finalize` epilogue is realized in +`Harness::run` (`aura-engine/src/harness.rs`), which calls `finalize()` once per +node in topological order after the source loop drains; `GatedRecorder` and +`SeriesReducer` (`aura-std`) are the folding siblings of the per-cycle `Recorder` +(`aura-std/src/recorder.rs`), which survives for the live / `--trace` / test-tap +path. + +`BlueprintNode::signature()` (`aura-engine/src/blueprint.rs`) answers uniformly for +both arms: a primitive returns its recipe's schema, a composite *derives* it via +`derive_signature` (input-port kinds from interior target slots, output-field kinds +from re-exported `OutField`s, aggregated params). `Composite::param_space()` +(same file) aggregates every node's declared params into one flat, path-qualified +list mirroring the inline order (C19/C23) — a read-only projection of the +graph-as-data (C9). Identity in the flat graph is positional; the `param_space()` +**name projection** (the by-name authoring/sweep address space) must be +**injective** for a blueprint to compile, enforced by +`check_param_namespace_injective` → `CompileError::DuplicateParamPath`, so +same-type siblings in one composite must be `.named(...)` apart (also what +disambiguates a same-type fan-in, C9). A vector knob (`LinComb.weights`) expands to +`N` flat `weights[i]` entries, `N` topology-fixed (C19). + +Wiring is **total and single-valued** over interior input slots: +`check_ports_connected`, run inside `validate_wiring` at every nesting level, +requires every interior node's every declared input slot to be covered by +**exactly one** wiring act (one interior `Edge { to, slot }` or one role +`Target { node, slot }`, counted uniformly). Zero coverage is +`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. +A composite's **own** input roles (`source: None`) are coverage *providers* (the +wired-by-enclosing boundary, the root case guarded by `UnboundRootRole`), never +consumers. There is no optional-input concept — every declared input port is +required. The check is index-based and name-free, so C23 is untouched. + +A `Composite` carries an optional authored rationale `doc: Option` +(`aura-engine/src/builder.rs`: `GraphBuilder::doc` / `Composite::with_doc`) — the +prose twin of its `name`, the same C23 non-load-bearing category. It is persisted +as an additive-optional `CompositeData` field (`aura-engine/src/blueprint_serde.rs`; +absent-field documents keep their exact bytes, so existing content ids are +untouched), blanked in the identity projection by `strip_debug_symbols`, dissolved +at inline like the name, and surfaced read-only in the graph model and viewer +(C22). + +Deferred: the recording sink's own per-cycle allocation — the `Recorder`→`Probe` +rename and its accumulate-vs-stream choice — stays open (#77). Naming input ports +(`PortSpec.name`) makes swap-prone same-kind slots self-documenting but does not add +a name-consuming wiring validation; a swapped same-kind wiring is still only +kind-checked (#21). The construction op-script vocabulary has no doc-carrying +surface yet (scope cut on #125). + +## See also +- [C1](c01-determinism.md) — the deterministic, non-concurrent event loop; `finalize` runs once after it; the determinism / graph-as-data boundary. +- [C6](c06-firing-policy.md) — the co-freshness of a record's K columns. +- [C7](c07-scalar-soa.md) — the four base scalar types, the tag-free `Cell` carrier, records as bundles of base columns, zero per-cycle allocation. +- [C9](c09-fractal-composition.md) — graph-as-data; `param_space` as a read-only projection; same-type fan-in disambiguation by name. +- [C12](c12-atomic-sim-unit.md) — the aggregated param-space the optimizer sweeps. +- [C18](c18-registry.md) — `FieldSpec.name` as metadata for sinks and the playground. +- [C19](c19-bootstrap.md) — the value-empty recipe, the pre-build signature, and inline order. +- [C20](c20-strategy-harness.md) — the sweep axis; the search-range is the run's; a `timestamp` axis is structural. +- [C22](c22-playground-traces.md) — read-only surfacing of the composite `doc` and the blueprint view. +- [C23](c23-graph-compilation.md) — names as non-load-bearing debug symbols; positional (by-index) identity. + +> History: [c08-node-contract.history.md](c08-node-contract.history.md) diff --git a/docs/design/contracts/c09-fractal-composition.history.md b/docs/design/contracts/c09-fractal-composition.history.md new file mode 100644 index 0000000..e0e43fd --- /dev/null +++ b/docs/design/contracts/c09-fractal-composition.history.md @@ -0,0 +1,85 @@ +# C9 — Fractal, acyclic composition: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c09-fractal-composition.md](c09-fractal-composition.md). + +**Refinement (Construction-layer milestone, 2026-06-05).** "A composite is itself +a `Node`" is an **authoring-level** identity: a composite declares the same +interface (typed inputs + ≤1 output, C8) and is wireable wherever a node is, but it +is **not** a runtime object. The bootstrap **compiles it away by inlining** its +sub-graph into the one flat instance (C19/C23): the composite *boundary* dissolves +into the raw index wiring the run loop already consumes, and names stay +**non-load-bearing** (informative debug symbols only, as `FieldSpec.name` already +is). So "nestable arbitrarily" and "graph-as-data" hold at the **blueprint +(source)** level; the running graph is the flat, index-wired **`FlatGraph`**. The earlier reading — *a composite survives as a `Box` +driving a nested sub-engine* — is **explicitly rejected**: it would keep the +interior opaque to the cross-graph optimiser (C23) and add a runtime sub-loop the +flat model does not need. Inlining is what makes the composite boundary free. + +**Realization (cycle 0018 — composite multi-output record, #40).** "Exposes one +output" is one output **port** carrying a **record of 1..K re-exported fields**, not +one field: `Composite.output` is a `Vec` (was a +single `OutPort`). Each entry is a **named projection** of one interior +`(node, output-field)`; a consumer selects which re-exported field it reads via the +same `Edge::from_field` that already binds leaf record columns (C8 realization, +cycle 0005). This is the **same arity** C8 already grants a leaf (OHLCV = one port, +5 columns): a multi-line indicator (MACD = macd/signal/histogram) is **one** record +of K fields, **one** port, **one** row per `eval` — C8/C7/C4 untouched, a boundary +completion, not a contract change. A strategy composite is simply the K=1 case +(one bias field, C10). The re-export **names** are **non-load-bearing** (C23): +they live at the blueprint boundary and in render (cycle 0022/#46 folds each onto +its producing node as a `name := …` binding; originally `[out:]` markers, #13) +but are dropped at lowering — `ItemLowering::Composite.output` is `Vec<(usize, +usize)>`, raw index pairs only, so the flat graph is name-free (verified: the +compiled-view render stayed bit-identical across this change). + +**Realization (cycle 0019 — name the composite boundary, #41; param-overlay retired, +cycle 0031).** The named-projection shape covers the surviving boundary edge-kinds: +`input_roles` is a `Vec` (was a bare `Vec>`), +alongside `output: Vec`. Each is an ordered, positionally-indexed +**named projection** of interior handles. **Param projection is no longer a +composite overlay** (the index-addressed `ParamAlias` was retired in cycle 0031): +a node's surface param name flows from its own **instance name** — every node +carries a name (default = its lowercased type label, override via `.named()`), and +`param_space()` is uniformly `.` at every level including the root. A +same-type fan-in is distinguished by naming the colliding legs, the same single act +that qualifies their param paths. Like the output and role names, **node names are +non-load-bearing** (C23): they live at the blueprint boundary and in render but are +dropped at lowering — the flat graph is wired by raw index. The full composite +boundary signature (named inputs, multi-outputs) and the per-node param path are +legible without changing the flat graph. + +**Refinement (param-namespace injectivity, cycle 0032; supersedes the fan-in +distinguishability check).** A blueprint compiles only if its `param_space()` name +projection is **injective** — every path-qualified knob name is unique. A duplicated +path is a knob no binding can select alone (it has no distinct by-name address, +C12/C19), so it is a `CompileError::DuplicateParamPath` carrying the offending path; +the cure is to give the colliding same-type sibling nodes distinct names with +`.named(...)`, the same single act that qualifies their param paths. The +param-bearing indistinguishable fan-in is *one instance* of a duplicated path (two +default-named same-type legs share a leaf path). `signature_of`, `leaf_has_param`, +and the fan-in-specific check (`check_fan_in_distinguishability` / +`check_composite_fan_in`) are **retired** (cycle 0032), replaced by one structural +`check_param_namespace_injective` over the `param_space()` names, run before name +resolution in `compile_with_params` and both binders — so the canonical by-name +author sees the structural cure rather than a downstream `AmbiguousKnob` symptom +(that `BindError` arm is retired too: an injective space can never multi-match). +Paramless interchangeable same-name sources stay legal (no path, no duplicate). +The old signature-collision predicate had extra breadth — it also rejected an +**asymmetric param/paramless** collision and a **role-vs-leg** collision, *neither* +a path duplicate (each colliding configuration keeps a unique param path, or none). +That breadth guarded **render identity**, dead since the renderer was retired in +0026; both extra rejections are **intentionally dropped**. A future node-/wiring-name +distinguishability check, if ever wanted (e.g. for the WASM graph view's #21 +thread), is decoupled from param-space injectivity. Construction-phase only; the +flat graph stays name-free (C23). + +**Refinement (2026-06-29 — graph-as-data round-trips both ways, C24).** "The built +graph is introspectable runtime data" is now contract-level **bidirectional**: a +blueprint is not only *emitted* as data (`model_to_json`, the render half) but is +itself a **serializable data value with a load path** (data → blueprint → +`FlatGraph`), so topology is a value the World generates, stores, and reproduces — +see C24. The Rust builder API stays the primary *human / LLM* authoring surface +(C17); the data form is the *machine* surface the World owns. (Load path **realised** +cycle 0087 / #155, `d5602ec`: `aura-engine::blueprint_from_json`.) diff --git a/docs/design/contracts/c09-fractal-composition.md b/docs/design/contracts/c09-fractal-composition.md new file mode 100644 index 0000000..ce34686 --- /dev/null +++ b/docs/design/contracts/c09-fractal-composition.md @@ -0,0 +1,88 @@ +# C9 — Fractal, acyclic composition + +**Guarantee.** A composite is an **authoring-level** `Node`: it declares the same +interface (typed inputs + ≤1 output, C8), is wireable wherever a node is, wires an +interior sub-graph, and exposes one output **port** carrying a **record of 1..K +re-exported fields** — signal, combined signal, and (with execution) strategy are +the same abstraction, nestable arbitrarily. A composite is **not a runtime +object**: the bootstrap compiles it away by **inlining** its sub-graph into the one +flat instance (C19/C23), so "nestable arbitrarily" and "graph-as-data" hold at the +**blueprint (source)** level while the running graph is the flat, index-wired +`FlatGraph`. The dataflow graph is a DAG; the only feedback path is an explicit +delay/state node (the RTL "register", C1). Wiring is written in Rust (builder API, +C17); the built graph is introspectable runtime data and, per C24, +**bidirectional** — a blueprint is not only *emitted* as data but is itself a +serializable data value with a load path (data → blueprint → `FlatGraph`), so +topology is a value the World generates, stores, and reproduces. The Rust builder +stays the primary human/LLM authoring surface; the data form is the machine surface +the World owns. + +**Forbids.** Implicit dataflow cycles (combinational loops); special-casing +"signal-of-signals" as separate mechanics; a composite surviving as a runtime +sub-engine (a `Box` driving a nested sub-loop); a blueprint whose +`param_space()` name projection is non-injective — a path-qualified knob name that +repeats, since it then has no distinct by-name address (C12/C19). + +**Why.** Self-application of one contract gives unlimited composition with no +adapter zoo. Acyclicity keeps the synchronous reactive model well-defined; forcing +feedback through a visible delay node keeps per-cycle determinism intact (C1) and +the one legitimate feedback path explicit. Graph-as-data enables visualization, +freezing, and re-parameterization for sweeps, and — round-tripping both ways (C24) +— lets the World own topology as content it serializes, loads, and generates rather +than as Rust baked into the binary. Inlining, not a runtime sub-object, is what +makes the composite boundary free: it keeps the interior transparent to the +cross-graph optimiser (C23) and adds no runtime sub-loop the flat model does not +need; the boundary dissolves into the raw index wiring the run loop already +consumes, and names stay non-load-bearing (C23). + +## Current state + +A composite is `Composite` (`aura-engine::blueprint`), a reusable sub-graph fragment +that is never `eval`'d. It holds interior items, interior edges (local indices), +`input_roles: Vec`, and `output: Vec`. + +- **Multi-output record.** `output` is `Vec` — each + entry a **named projection** of one interior `(node, output-field)`. A consumer + selects which re-exported field it reads via `Edge::from_field`, the same binding + that already selects a leaf record's columns (C8). This is the same arity C8 + grants a leaf (OHLCV = one port, 5 columns): a multi-line indicator (MACD = + macd/signal/histogram) is one record of K fields, one port, one row per `eval`. A + strategy composite is the K=1 case (one bias field, C10). +- **Named boundary.** `input_roles` is `Vec` — role `r` + fans its source value into the interior `targets`. Inputs and outputs are ordered, + positionally-indexed named projections of interior handles. +- **Per-node param path.** A node's surface param name flows from its own **instance + name**: every node carries a name (default = its lowercased type label, override + via `Node::named`, `aura-core::node`), and `param_space()` + (`aura-engine::blueprint`) is uniformly `.` at every level including + the root. A same-type fan-in is distinguished by naming the colliding legs — the + same single act that qualifies their param paths. +- **Param-namespace injectivity.** A blueprint compiles only if its `param_space()` + name projection is injective. A duplicated path is a knob no binding can select + alone (C12/C19), rejected as `CompileError::DuplicateParamPath` carrying the + offending path; the cure is `.named(...)` on the colliding same-type siblings. One + structural check, `check_param_namespace_injective` (`aura-engine::blueprint`), + runs before name resolution in `compile_with_params` and both binders. Paramless + interchangeable same-name sources stay legal (no path, no duplicate). +- **Names dropped at lowering.** Output, role, and node names are non-load-bearing + (C23): they live at the blueprint boundary and in render, and are dropped at + inline — `ItemLowering::Composite` carries `output: Vec<(usize, usize)>` (raw index + pairs) and `roles: Vec>`, so the flat graph is name-free and wired by + raw index. +- **Bidirectional data form.** The emit half is `model_to_json` + (`aura-engine::graph_model`); the load path is realised — + `aura-engine::blueprint_from_json` (`aura-engine::blueprint_serde`, #155) parses + the data form back to a blueprint that compiles to a `FlatGraph` (C24). + +## See also +- [C1](c01-determinism.md) — determinism, acyclic dataflow, the delay/state register. +- [C8](c08-node-contract.md) — the node contract: ≤1 output, record columns, `Edge::from_field`. +- [C10](c10-bias-r-cost.md) — the strategy as a K=1 bias composite; the cost-model graph. +- [C12](c12-atomic-sim-unit.md) — binding by name; why a duplicated knob path is unbindable. +- [C17](c17-authoring-surface.md) — the Rust builder as the human/LLM authoring surface. +- [C19](c19-bootstrap.md) — bootstrap: blueprint → instance; construction-phase checks. +- [C23](c23-graph-compilation.md) — names non-load-bearing; the cross-graph optimiser; inlining. +- [C24](c24-blueprint-data.md) — the blueprint as a serializable, World-owned data value. +- [C28](c28-stratification.md) — the layer ladder: construction phase vs the run loop. + +> History: [c09-fractal-composition.history.md](c09-fractal-composition.history.md) diff --git a/docs/design/contracts/c10-bias-r-cost.history.md b/docs/design/contracts/c10-bias-r-cost.history.md new file mode 100644 index 0000000..4a7519b --- /dev/null +++ b/docs/design/contracts/c10-bias-r-cost.history.md @@ -0,0 +1,386 @@ +# C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c10-bias-r-cost.md](c10-bias-r-cost.md). + +**Realization (cycle 0007).** [HISTORY — pre-reframe; `SimBroker` is now legacy per +the 2026-06-28 reframe.] The signal-quality half was realized at the substrate as +two `aura-std` nodes on the unchanged engine (the engine stays domain-free — it +routes only `f64` records). The **exposure stream** was realized as `Exposure { +scale }`: `clamp(signal / scale, -1, +1)`, one `f64` per fired cycle. The +**sim-optimal broker** was realized as `SimBroker { pip_size }`: a two-input node +(exposure, price) accumulating `prev_exposure · (price − prev_price) / pip_size` +and emitting cumulative pip equity — the exposure held *into* a cycle (decided at +t-1) earns that cycle's return (causal, C2); `pip_size` is held reference metadata +(C7/C15). An end-to-end harness (SMA-cross → `Exposure` → `SimBroker` → recording +sink) produced a recorded pip-equity curve, bit-identical across runs (C1). + +**Realization (per-instrument pip channel, 2026-06, #22).** [HISTORY — pertains to +the legacy `SimBroker` pip channel.] `SimBroker`'s `pip_size` is sourced **per +instrument** from the recorded geometry sidecar (`instrument_geometry`, over +data-server's `symbol_meta`), at the ingestion / source edge (never `Aura.toml`, +never `Ctx`); the engine stays domain-free. (The original cycle-0022 form was a +Rust-authored vetted floor `InstrumentSpec { pip_size }` + `instrument_spec(symbol)`; +cycle 0074 removed that floor — see the C15 note — once the sidecar geometry made it +redundant for the real path. Refuse-don't-guess on absent geometry.) The honesty +rule is **refuse, don't guess**: a real-data run for a symbol with no vetted spec +is a usage error (`exit 2`). Threaded through the CLI `aura run --real` path; the +manifest broker label records the looked-up pip. + +**Realization (position-event schema, cycle 0063, #114).** [Survives as the +**deploy / reconciliation** schema per the 2026-06-28 reframe — no longer a +broker-input research artifact.] A closed `PositionAction { Buy, Sell, Close }` +enum + the `PositionEvent` row (`event_ts`, `action`, `position_id`, +`instrument_id`, unsigned `volume`; no `open_ts`; direction *is* the action) live +beside `RunMetrics` as a post-run value type (not a per-`eval` node — C8) — both in +the `aura-analysis` crate since cycle 0079 (#136); originally `aura-engine`, see the +C16 cycle-0079 note. `action` serde-encodes as a bare `i64` (Buy=0, Sell=1, +Close=2), the C7 scalar column form, with an out-of-range code rejected on read. +The table stays broker-independent. + +**Reframe (2026-06-23, #117 — exposure → bias, R as the signal-quality unit).** +[HISTORY — its R spine survives into the 2026-06-28 contract; its Stage-2 currency / +realistic-broker / register / flat-1R-vs-compounding portions are SUPERSEDED by that +reframe.] The contract was reframed from **exposure** (a signed fractional +position) to an **unsized bias** plus **R** as the signal-quality unit. The +realization notes below describe the **pre-reframe code** (`Exposure`, `SimBroker`, +pip-equity), retained as history and as the **ancestor** of the current chain: +`Exposure { scale }` is the ancestor of the unsized `bias` node, and `SimBroker`'s +pip integral is the ancestor of the R-evaluator (which additionally requires a +stop, since R is stop-defined). The `exposure → bias` rename, the RiskExecutor / +Sizer / Veto nodes, and the R-evaluator **landed in cycle 0065**; the +position-event schema (0063, #114) survives as the audit layer. Industry grounding +for this reframe: LEAN / nautilus_trader / backtrader / QSTrader / vectorbt / +zipline (see #117 decision log). + +**Realization (cycle 0065 — Stage-1 R signal quality, #119/#126/#127/#128/#129).** +[The R spine here is the live model; the *Stage-2 deferral* clause at its end is +SUPERSEDED by the 2026-06-28 reframe — there is no Stage-2 currency / compounding +layer; cost is now the cost-model graph and money lives only at the live deploy +edge. "Stage-1" reads as a historical identifier, not a gate half.] `Exposure → +Bias` renamed the unsized strategy output (node + output field); the persisted +`exposure_sign_flips` metric key (serde alias), the `SimBroker` `exposure` input +slot, and the on-disk `exposure` **tap** label retain the old name. The +strategy-output **param namespace** (the `Bias` instance, its `bias.scale` knob, +the `bias_scale` manifest param) was completed in #134. A **stop-rule** defines 1R: +`FixedStop` (a triggered-constant primitive) and a `vol_stop(length, k)` +**composition** `k·√EMA(Δ²)` (a composition of `Mul` / `Sqrt` primitives). +**`PositionManagement`** (`aura-std`) is the stateful heart: it latches the +entry-cycle stop distance as the immutable R-denominator, marks against the +one-cycle-lagged fill (no look-ahead, C2), and emits a **dense 14-column per-cycle +R-record** (one row per eval, C8; the trade ledger is the `closed_this_cycle` +subset, the R-equity is `cum_realized_r + unrealized_r`). The **`Sizer`** (`size = +risk_budget / stop_distance`, flat-1R) was the feed-forward sizing seam, and **R is +size-invariant** — scaling `risk_budget` leaves every `realized_r` unchanged +(pinned by a RED test); per the 2026-06-28 reframe the Sizer and `size` / `volume` +are removed from research (size is a deploy concept). **`summarize_r`** is a +post-run fold (sibling of `summarize`, **not** an in-graph node) → `RMetrics` (E[R], +SQN, win-rate, profit-factor, max-R-drawdown, conviction terciles, net-of-cost); +per the 2026-06-28 reframe it folds the cost-model's net-R rather than recomputing +a scalar cost. The **RiskExecutor** ships as a public `aura-composites` +composite-builder (`risk_executor(StopRule, risk_budget)`) with a +`StopRule{Fixed,Vol}` **structural axis** (C20); per the 2026-06-28 reframe its +Sizer interior and `risk_budget` arg are dropped / vestigial, the **Veto** stays a +documented seam, not a runtime node. The layer is operable from the CLI: `aura run +--harness ` — a compile-time selector over Rust-authored +harnesses (C9/C17) — folds `summarize_r` into `RunMetrics.r`, the `r-sma` +harness fanning one bias into both `SimBroker` (legacy pip) and the RiskExecutor +(R); an `r_equity` tap charts the by-trade R-equity. Composites live in the +dedicated `aura-composites` crate, so `aura-engine`'s runtime dependency stays +`aura-core`-only and `aura-std` is an `aura-engine` `[dev-dependencies]` (the graph +stays acyclic). +[HISTORY — the built-in `--harness` selector was retired with the demos → +blueprint-data (#159, cuts 1b-4); `run` is now blueprint-driven — `aura run +` over examples/r_*.json.] + +**Realization (cycle 0066).** SQN is the operational single-number objective for +ranking an r-sma sweep family by signal quality — C12 **axis-2 (argmax-metric)** +over the C18 family store. `metric_cmp` (`aura-registry`) learns the +higher-is-better R metrics `sqn`, `expectancy_r`, `net_expectancy_r`; a member +with no `r` block sorts last (`NEG_INFINITY`). `aura sweep --strategy r-sma` +produces the rankable R family, each member folding `summarize_r` into +`RunMetrics.r`. The default grid varies **only the signal** (`fast` / `slow` SMA +lengths), holding the stop and sizing fixed: `risk_budget` is R-invariant and +`bias.scale` is sign-only under flat-1R, and — load-bearing — the **stop defines +1R**, so varying it would change what R *means* per member and break cross-member +SQN comparability. Each swept member's manifest records the fixed R-defining params +beside the floated knobs (reproducible from its own manifest, C18). +[HISTORY — the built-in `--strategy` sweep surface was retired with the demos → +blueprint-data (#159, cuts 1b-4); the rankable R sweep now runs as `aura sweep + --axis …` over r_sma_open.json (an example then; relocated to +`crates/aura-cli/tests/fixtures/` by #248).] + +**Realization (cycle 0067, #130 + #135).** **#130 (SQN100):** `RMetrics` gains +`sqn_normalized = (mean_R/stdev_R)·√(min(n, 100))` — the n-normalized "SQN score" +(`SQN_CAP = 100`), turnover-robust where raw `sqn` rewards trade count. It is an +**opt-in** rank key (`Metric::SqnNormalized`); raw `sqn` and the default ranker +stay byte-unchanged, and the field carries `#[serde(default)]` (C18). Below the cap +(`n ≤ 100`) it equals raw `sqn` exactly. **#135 (r-sma `--trace`):** +`r_sma_sweep_family` persists each member's equity / exposure / r_equity under +`runs/traces///` via the same `persist_traces_r` the single run uses; +per-member `--trace` is symmetric across all three sweep strategies. +[HISTORY — the built-in `--strategy` sweep triple (sma / momentum / r-sma) was +retired with the demos → blueprint-data (#159, cuts 1b-4). The per-member +`--trace` symmetry was NOT carried to the `aura sweep ` path — it +was silently dropped at #159/#220 (`run_blueprint_sweep` never wired `persist`; +`let _ = persist`), and #168 makes the surface honest: `sweep`/`walkforward` (like +the pre-existing `run`/`mc`) now refuse `--trace` outright. See the CLI-`--trace` +retirement amendment below.] + +**Realization (cycle 0068, #115 — position-event derive).** [The derivation +survives as the **deploy / reconciliation** layer per the 2026-06-28 reframe — its +"realistic brokers consuming it" goal is retired; money is the live-edge concern.] +`derive_position_events(record, instrument_id) -> Vec` (in +`aura-analysis` since cycle 0079, #136; originally `aura-engine`, beside +`summarize_r`) is the **first difference of the executed book**: a pure post-run +reduction over the `PositionManagement` dense record (read positionally as +type-erased `Scalar`s, C7 SoA — no in-graph node, so the hot path stays +domain-free, C14), emitting a `Buy` / `Sell` at each open and a `Close` at each +exit, a reversal (or stop-then-same-cycle reopen) emitting **Close then the +opposite open at one `event_ts`** (close first — the C8 ">1 event per instant" case +that forces a *derived* table, not a per-`eval` output). The close sizes the +**actual book** (the closed position's stored volume), never an exposure delta. +`instrument_id` is a caller-supplied scalar (`aura-analysis` depends only on +`aura-core`). A position open at window end emits its open with **no synthetic +`Close`** (the table records actual executed events; `summarize_r`'s force-close is +for the R metric only). The `r_col` ⟷ PM-record lockstep is guard-pinned for +`direction` too. + +**Reframe (2026-06-28, #116 — realistic broker retired; cost-model graph in R; money to the live edge).** +This is the live contract above. Ratified in an in-context design discussion +(reference issue #116). It **preserves** the durable spine — unsized bias stream +(sign = direction, magnitude = conviction), signal quality in R, the stop defining +R, and the decoupling of direction from sizing from fill — and the shipped Stage-1 +realizations (the `Bias` node, `FixedStop` / `vol_stop`, `PositionManagement`, +`summarize_r` / `RMetrics` / `sqn_normalized`, SQN as ranking objective). The +**RiskExecutor composite survives in shape** (bias + price → stop-rule → +position-management) but with its **Sizer interior removed** and its `risk_budget` +argument dropped / vestigial (it sized the now-removed Sizer); the Veto remains an +optional documented seam. The contract **supersedes** the following, which were +**design intent, largely unbuilt** (#116 — the realistic broker and the whole +Stage-2 currency layer were never implemented; the Stage-1 R chain was) and are +retired: +- the **"realistic broker"** concept — an authored-friction historical broker — + rejected as "horseshoe-throwing" (real friction is not historically knowable), + replaced by the **cost model**: a composable C9 graph of cost nodes (in + `aura-std` / `aura-composites`), approximating not claiming realism, + generalizing / subsuming `round_trip_cost` into `net_expectancy_r`; +- the **"Currency P&L is Stage 2"** paragraph in full, the **Stage-1-vs-Stage-2 + hard sequencing gate**, and **currency / fixed-fractional / compounding** in + research — compounding is now a post-hoc money-management transform of the + net-R sequence at the deploy / account layer; +- the **`z⁻¹` register on the fill edge** and the **flat-1R-vs-compounding + structural axis** as research mechanism — there is no equity → size edge in + research, so no register and no such axis in the loop; +- the **Sizer in research** and currency **size / `volume`** — size is a deploy + concept; the research executor is stop + position-management in R; + `PositionManagement`'s `size` port is dropped or constant-driven (no dangling + port, C8), its `size` field and the event table's `volume` are vestigial in + research; +- the **"a broker is an ordinary in-graph node / no special external broker + subsystem"** forbid, **for the live edge only**: in-graph brokers are retired + outright, and the live broker is now an explicit **I/O adapter** at the C11 + recording / C13 deploy edge — not an in-graph node and not part of the research + graph (the no-in-graph-broker-subsystem prohibition still holds inside the + graph); +- the **position-event table as the realistic-broker input** and its + first-difference-of-the-book (`deal = target − book − in_flight`) execution + framing — the table (schema 0063 #114, derive 0068 #115) **survives** as the + **deploy / reconciliation** artifact (real volume), not a research artifact and + not "fed to a broker" in research. +Money, a real broker, and cTrader Open API are a **separate, later live / +deploy-edge** concern (C11 record-then-replay; C13 frozen-deploy invariant) — the +only `belastbare` ground truth, measured, never modelled. The honesty principle is +explicit: the net-R curve is a research / ranking hypothesis; the forward / live +run is ground truth. `SimBroker` is downgraded to a legacy / optional pip +yardstick, not to be expanded. (Terminology note: with Stage 2 gone, the +**"Stage-1"** label below is no longer one half of a two-stage gate — it survives +only as a historical cycle name, like the `exposure` → `bias` on-disk alias, +denoting the shipped feed-forward R chain; the identifier family that carried it +was renamed to the r-family — `r-sma` / `r-breakout` / `r-meanrev` — in cycle +0100, #174.) + +**Realization (cycle 0081 — cost-model graph, cycle 1, #148).** The first concrete +cost node + the net-R seam shipped. `ConstantCost` (`aura-std`) is an ordinary +downstream node (C9) emitting a 3-field cost-in-R record `{cost_in_r, cum_cost_in_r, +open_cost_in_r}` isomorphic to `PositionManagement`'s R-triple; R-pure +(`cost_per_trade / |entry − stop|`, notional cancels, no equity held). The scalar +`round_trip_cost` argument of `summarize_r` is **retired**: `summarize_r` folds a +co-temporal cost stream (positional 1:1 join — `cost[i]` is `record[i]`'s cycle) into +`net_expectancy_r`, one home for cost / no double-count, byte-identical to the old +cost = 0 baseline on an empty stream. The headline sink is **`net_r_equity`** = +`LinComb(4)[cum_realized_r, unrealized_r, −cum_cost_in_r, −open_cost_in_r]` → Recorder +(C8/C18), a sibling of `r_equity`, emitted only when a cost is authored. Wired on the +**run path** via `--cost-per-trade` (`r_sma_graph`); sweep / walk-forward / mc pass +`None` (cost on the reduce-mode sweep path and cost in OOS pooling deferred — the +positional join holds only while one cost node fires in lockstep with PM). +[Superseded 2026-07-11 (#234): the `--cost-*` run-path flags are gone (#221 removed +that surface); cost is authored as the campaign document's `cost` block and reaches +every family/campaign member — see the cycle-net-r realization below.] Deferred to +later milestone cycles: the general `CostNode` trait + multi-node cost-graph +composite-builder, data-grounded factors (realized-vol → slippage, recorded-rate → +swap), per-cycle-held accrual (carry / funding), and the conviction-weighting +R-aggregation axis. Decision log: #148. + +**Realization (cycle 0082 — cost-graph composition, cycle 2, #148).** The cost graph +**composes**: a second, *state-dependent* cost node plus an aggregator prove that two +cost nodes sum into one net-R curve with `summarize_r` and the `net_r_equity` tap +structurally unchanged. `VolSlippageCost` (`aura-std`) charges `slip_vol_mult · vol / +|entry − stop|` in R, reading an **independent short-horizon realized-range** vol +(`RollingMax − RollingMin`, window distinct from the stop's own vol — scaling slippage +by the stop's vol would collapse cost-in-R to a constant, indistinguishable from +`ConstantCost`). `CostSum` (`aura-std`) is the cost-graph **output node**: it sums `N` +cost nodes' 3-field cost-in-R records **per-field** into one aggregate (`n = 1` is the +identity), so the seam consumes a single cost stream regardless of node count — one +home for cost, the positional join unchanged. **Co-temporality contract (load-bearing, +generalizes to all future factors):** since `summarize_r` positionally joins `cost[i] +↔ record[i]`, a cost node is gated **only by the PM trade-geometry**; any not-yet-warm +state input (the vol proxy warms later than PM) contributes **0 cost** that cycle +rather than withholding — the node still emits its row, so the cost stream stays +co-temporal 1:1 with the PM record. This makes co-temporality structural and +warm-up-independent, preserves the C18 golden, and is honest (no slippage estimate yet +→ no charge). `ConstantCost` satisfies it trivially; only state-dependent nodes need +the missing-factor → 0 rule. Wired on the **run path** via `--slip-vol-mult`, +composable with `--cost-per-trade` (their costs sum); sweep / walk-forward / mc pass +`None`. The 3-field cost triple is now restated by-convention across the two producers ++ `CostSum` (a compiler-unlinked lockstep) — to be unified by the still-deferred +general `CostNode` trait (now justified by two concrete nodes). Decision log: #148. +[Superseded 2026-07-11 (#234): the `--slip-vol-mult`/`--cost-per-trade` flags are gone; +authoring moved to the campaign `cost` block — see the cycle-net-r realization below.] + +**Realization (cycle 0083 — CostNode trait + shared cost-record contract, cycle 3, +#148).** The deferred unifier ships. A new `aura-std/src/cost.rs` owns the cost-model +node abstraction: the 3-field cost triple is now **one source of truth** +(`COST_FIELD_NAMES` / `COST_WIDTH`, mirroring `position_management::{FIELD_NAMES, +WIDTH}`), read by both producers + `CostSum` + the CLI wiring — the cycle-0082 +by-convention lockstep is **structurally gone** (four restatements collapsed to one). +The `CostNode` **factor trait** carries a cost node's only per-node difference — the +price-unit **cost numerator** (`cost_numerator(&mut self, &Ctx) -> f64`), plus +`extra_inputs` (default none) and `label`; everything else is the generic +`CostRunner` **adapter**, which holds the shared `cum`/`out` state and +implements `Node`, writing the co-temporality skeleton (geometry-only gating, +`numerator / latched` R-normalization, the closed/open charge, the running `cum`, the +3-field emit) **once**. `ConstantCost` and `VolSlippageCost` are now thin factors whose +`new()` returns `CostRunner`. Honours C9 (a `CostRunner` is a plain downstream +`Node`, no runtime sub-object) and C23 (`name()` dropped as dead surface — `label()` +remains the non-load-bearing symbol). **Behaviour-preserving**: the builders emit +unchanged schemas, so the wiring / `net_r_equity` seam / `summarize_r` are untouched and +the `numerator / latched` token form is byte-identical; the existing suite passes +verbatim and two new CLI characterization goldens pin the exact flat/composed +`net_expectancy_r` (the prior tests only asserted `net < gross`). Decision log: #148. + +**Realization (cycle 0084 — cost-graph composite-builder, cycle 4, #148).** Decision E +ships. A new `cost_graph(Vec) -> Composite` in `aura-composites` (the +C16 layer that couples the engine builder + `aura-std` nodes) is the cost-model graph's +**authoring primitive**: it fans the 4 PM-geometry inputs to `N` cost nodes, surfaces +each node's extra inputs (discovered via `schema().inputs[GEOMETRY_WIDTH..]`, +`GEOMETRY_WIDTH` now re-exported from `aura-std`) as `cost[k].` composite roles, +sums them through `CostSum`, and exposes the 3-field aggregate. The CLI's manual +slot-indexed cost-wiring + the hardcoded `MAX_RUN_COST_NODES = 2` cap are deleted — the +composite handles arbitrary arity. **Behaviour-preserving** (C11): the composite inlines +at bootstrap to the same flat fan-in, so the cycle-0083 `net_expectancy_r` goldens are +byte-identical (four `aura-composites` unit tests pin the exposed role-set + output +triple, incl. arbitrary-arity per-node namespacing). Honours C9 (ordinary downstream +nodes), C16 (wiring stays out of `aura-engine`), C23 (role/port names are +non-load-bearing). **Carried debt (#152, for the deferred sweep-cost cycle):** the +`cost[k].` index-namespacing is restated across `CostSum` / `cost_graph` / the CLI +(a build-validated lockstep, not the silent positional kind 0083 collapsed), and +`cost_graph` `.leak()`s runtime port names per build — fine for one-shot run-path +construction, but to be interned (the `COL_PORTS` production pattern, not the test-only +`.leak()`) before cost reaches the per-member sweep path. Decision log: #148. +[Discharged 2026-07-11 (#152/#234): `cost_port`/`intern_port` in `aura-std/src/cost.rs` +are the process-global interned single source; both `cost_graph` `.leak()`s are gone.] + +**Realization (cycle 0085 — per-cycle-held accrual, cycle 5, #148).** C10's +"per-trade factors deduct at close; **per-cycle-held factors accrue over the hold**" +clause is first realized. A cost factor now declares *when* it charges via +`ChargeMode { AtClose, PerHeldCycle }` (a defaulted `CostNode::charge_mode()`, default +`AtClose`), read by the **one shared `CostRunner`** — not a second runner type (a +commission is intrinsically at-close, a carry intrinsically per-held-cycle; the timing +belongs to the factor, preserving the cycle-0083 "skeleton written once" win). The +`PerHeldCycle` arm accrues `per` into a per-position `acc` every held cycle, dumps the +accrued total into `cum` at close (resetting `acc`), and marks the open position via a +**growing `open_cost_in_r`**. The first accrual node, **`CarryCost`** (`aura-std`, +a `ConstantCost` twin differing only in `charge_mode()`), is a labelled stress +parameter — the flat base of the accrual family; a run-path `--carry-per-cycle` flag +pushes it into the existing `cost_graph`/`CostSum` aggregation (no new wiring; a +per-trade and a per-held-cycle node compose, the costs summing per-field). **Approach B +(honest bleed), achieved without a `summarize_r` fold change:** the headline +`net_r_equity` curve bleeds continuously over the hold because the bleed lives in +`open_cost_in_r`, which the `net_r_equity` tap already subtracts — so `summarize_r` and +the CLI `net_eq` wiring are **untouched**, and the cycle-0083/0084 `net_expectancy_r` +goldens stay byte-identical (the `AtClose` arm is the pre-cycle-5 eval tokens verbatim). +The 3-field cost record's semantics generalize cleanly: `cost_in_r` = cost realized this +cycle (into `cum`); `open_cost_in_r` = the open position's cost marked-to-market +as-of-now (would-be-close for `AtClose`, accrued-so-far for `PerHeldCycle`) — singly +counted, no `cum`/`open` double-count (the two are mutually exclusive per cycle). Honours +C9 (a `CostRunner` is an ordinary downstream `Node`), C11 (byte-identity), C16 +(factor in `aura-std`, `aura-engine` domain-free), C23 (`label()` carries the rate). The +B-vs-A discriminator (a growing intra-hold `open_cost_in_r`, invisible to the scalar +`net_expectancy_r`) is pinned by unit B-proofs + a `net_r_equity`-bleeds-over-the-hold +integration test. **Deferred (each its own #148 cycle):** a notional-based carry +(`price × rate`, reads the price tap), the calendar-aware **overnight swap** proper +(rollover-boundary timing, 3× Wednesday, long/short asymmetry — deploy-edge realism), and +sweep-path cost. Decision log: #148. + +**Realization (cost-flag harness scoping, #153).** Cost flags are defined only +against an R-evaluator harness (the gross-R → net-R chain). On a non-R harness +(`--harness sma`/`macd`, which produce no R) a cost flag is a **usage error +(exit 2)**, not a silent no-op — refuse-don't-guess. Negative cost rates are +likewise rejected (exit 2) with a named diagnostic that identifies the offending +flag. CLI ergonomics only; the cost-model graph and the R math are untouched. +[HISTORY — the built-in `--harness` selector (its `sma`/`macd` non-R examples) +was retired with the demos → blueprint-data (#159, cuts 1b-4); the cost-flag +scoping rule survives on the `aura ` r-sma run path over +examples/r_*.json.] + +**Realization (2026-07-06 — the risk regime as a structural campaign axis, #210).** +The `StopRule{Fixed,Vol}` structural axis is realized at the campaign-document +level as `CampaignDoc.risk: [RiskRegime]` (a serializable, content-addressable +mirror of the runtime enum — `aura-research`, variants `Vol{length,k}` and +`VolTf{period_minutes,length,k}` (#262), the fixed-stop rule additive when +needed). It is a **kept-separate** matrix axis, a +peer of instruments and windows: the executor keys the nominee map by +`(strategy, window, regime)`, so generalize aggregates across instruments +*within* a regime, never across regimes. Regimes are therefore **compared** at +presentation, never argmax-**selected** across — a cross-regime E[R] argmax would +compare R-multiples in different R units (the stop defines 1R), so the legacy +verbs' `--stop-length`/`--stop-k` joint stop grid is retired as a campaign +methodology (finding the best regime = the same walk-forward / worst-case-R +validation, run once per regime, and picking the most robust — a comparison, not +a search). Each member manifest stamps its resolved stop (default included), +closing the C18 gap; absent/empty `risk` = one implicit default regime, +absent-serializing for content-id parity. Two default *representations* +coexist by design (#217): a dissolved sweep binds no regime at all +(`risk: []`), late-resolved per member by `stop_rule_for_regime` at run +time, while walkforward/mc/generalize — whose `--stop-length`/`--stop-k` +became optional — bind the default regime *eagerly* into the campaign +document (`risk: [Vol{length:3,k:2.0}]`). Same R behaviour either way, but +deliberately different document content ids: a stop-less verb invocation's +document equals its explicit `--stop-length 3 --stop-k 2.0` spelling, not +a stop-less sweep's document. Deferred (documented): regime-aware +**trace** persistence — the trace re-run and cell-key dir naming still assume the +default stop, since `CellRealization` carries no regime (`#212`); the core +run/stamp/generalize path is unaffected. + +**Realization (cycle net-r — cost reaches the family/campaign path, #234/#152, +2026-07-11).** The deferred sweep-path cost ships, delivered where C24 put experiment +intent: the campaign document gains an additive `cost: Vec` block — a closed, +externally tagged vocabulary over the three shipped nodes (`constant` / `vol_slippage` +/ `carry`, field names = the builders' own `ParamSpec` names), mirroring the `risk` +block (serde `default` + skip-if-empty: cost-less docs hash byte-identically, C18). +Net is the **default**: an absent block is the explicit zero-cost model and +`summarize_r`'s net family equals gross under it — every result is net, no second +"gross-labelled" result kind. `cost_nodes_for` (beside `stop_rule_for_regime`) is the +one doc→builder binding, every component fully bound (the wrapped param space stays +cost-invariant); `wrap_r` carries an optional cost leg (the #221-deleted wiring +rebuilt: cost_graph off the executor's four geometry outputs, the vol proxy back in +production, a gated cost recorder in reduce mode as the `summarize_r` join input, the +`LinComb(4)` `net_r_equity` curve in trace mode). Member manifests stamp the +components; both re-run sides re-derive them (reproduce via `cost_specs_from_params` — +the #233 stop pattern — and the persist re-run binds the same model, so the C1 drift +alarm compares like with like; costed families reproduce bit-identically, pinned incl. +`Carry`). `TapChannel::Net` routes `net_r_equity` to persisted curves; a cost-less doc +requesting it keeps a remedy-naming skip notice. The `--cost-*` run-path flags do not +return; the verb sugar passes no cost (docs carry it) — the same lean-flag call as the +C26 bindings block. #152's interning ships with it (`cost_port`/`intern_port`, both +`cost_graph` `.leak()`s gone). Decision log: #234. diff --git a/docs/design/contracts/c10-bias-r-cost.md b/docs/design/contracts/c10-bias-r-cost.md new file mode 100644 index 0000000..1eabfdd --- /dev/null +++ b/docs/design/contracts/c10-bias-r-cost.md @@ -0,0 +1,361 @@ +# C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge + +**Guarantee.** A strategy's primary, backtestable output is **not** an equity +curve, **nor a position-event table**, **nor a position size**, but a **bias +stream**: the DAG expresses exactly one *state* at time t (C8 — a node emits at +most one record per `eval`), so a strategy emits one **signed, bounded bias** +`f64 ∈ [-1, +1]` per cycle (per instrument) — the **sign is direction, the +magnitude is conviction**, and conviction is optional (a bare ±1 / 0 is the modal +case). Bias is **unsized**: position sizing and the protective stop do **not** +live in the strategy. The chain is `signals (scores) → decision node → bias +stream`. + +**Risk-based execution is a decoupled downstream layer; in research it is Stop + +position-management in R, no Sizer.** Turning a bias into a tracked trade is the +job of a downstream **execution** chain, never the strategy's. The **stop-rule** +sets a protective stop, which **defines the risk unit R** (1R = the loss taken if +stopped). In the research loop the executor is **stop-rule → position-management**, +operating **directly in R**, with the **Veto** an optional documented +pre-trade-gate seam (a pass-through identity DCE'd away under C19/C23 when +absent): there is **no Sizer**. Sizing in *currency* (`size` / `volume`) is a +**deploy** concept. C8's wiring totality (no optional-input concept — every +declared port is covered by exactly one wiring act) forbids a *dangling* `size` +port, so the resolution is concrete: research position-management either **drops +its `size` input port**, or has that port **driven by a constant unit node** (the +flat-1R degenerate) — never an unwired "vestigial" port; the record's `size` +field is held at unit and carries no research information, and the +position-event table's `volume` column likewise. Consequently the +**position-event table is demoted to a deploy / reconciliation artifact** (real +volume lives there), no longer a research artifact and no longer "fed to a +broker". The three-way decoupling of **direction (bias) / sizing / fill** stands +as a *structure*; sizing and fill are simply pushed entirely to the deploy edge, +out of the research loop. + +**Signal quality is measured in R — gross R and net R.** A downstream +**R-evaluator** consumes the executor run and integrates the per-trade R-outcomes +into an **R-expectancy** / R-curve: the account- and instrument-agnostic yardstick +for *"how much R out per 1R risked?"*. R, not pips, is the unit (pips are not +risk-normalized). Two readings of the same unit: **gross R** (signal only) vs +**net R** (after the cost model), with `net R = gross R − cost-in-R`. The headline +artifact is the **net-R equity curve** — the cost-drag drawn onto the R curve, +recorded through a named **`net_r_equity`** tap/sink (sibling of the existing +`r_equity` tap; a sink is the only thing the registry can display, C8/C18/C22). No +new unit is invented: it is R, gross and net, **continuous with the existing +`net_expectancy_r`**. A bare **gross-R run with no cost model attached is valid**: +the cost layer is optional and additively composed-on (the zero-cost baseline is +the "default simple" floor). + +**Cost is a COST MODEL — a composable C9 graph of cost nodes, in R, that +approximates (never claims) realism.** The realistic broker is *retired*: real +friction — slippage (live liquidity / order-size / volatility at fill), swaps +(broker-set, time-varying), even recorded feed spread (often a fake constant) — is +**not historically knowable**, so an authored-friction historical broker is +"horseshoe-throwing". Its replacement is a **cost model**: an ordinary downstream +**C9 graph of cost nodes** that *approximates* the cost side a broker would +produce, explicitly as an approximation. The cost nodes live in `aura-strategy`, +the cost-graph composite-builder (`cost_graph`) in `aura-composites` (C16), never +in the domain-free `aura-engine` (C14/C16). They are **not** "additive on the R +stream" in isolation: a cost node **reads the state it depends on** — the price +stream, a realized-volatility tap, a C11-recorded interest-rate source, and the +executor's per-cycle R-record / trade events — and emits a **cost-in-R** stream +that is subtracted from gross R to yield net R. Cost attaches at the structurally +correct grain: **per-trade** factors (commission, a flat cost-per-trade) deduct +from `realized_r` at close; **per-cycle-held** factors (carry / funding / swap) +accrue over the holding duration. The model **generalizes** the scalar +`round_trip_cost` / `net_expectancy_r` into a possibly **state-dependent graph**: +the scalar `round_trip_cost` is the degenerate constant-per-trade special case, +**subsumed** by the cost graph — the post-run `summarize_r` fold no longer +recomputes cost independently but folds the cost-model's net-R stream into +`net_expectancy_r` (one home for cost, no double-count). Discipline: every cost +factor is **either** a clearly-labelled **stress-parameter** (e.g. a flat +cost-per-trade is a breakeven-threshold probe) **or** **data-grounded / +falsifiable** (e.g. realized-volatility → slippage; recorded interest-rate data +via C11 → funding / swap). **Default simple; complexity is earned per grounded +factor.** Stacking unfalsifiable guesses (over-modelling) is the anti-pattern. + +**Co-temporality contract (load-bearing, generalizes to all cost factors).** +Because `summarize_r` positionally joins `cost[i] ↔ record[i]`, a cost node is +gated **only by the position-management trade-geometry**; any not-yet-warm state +input (a vol proxy warms later than the executor) contributes **0 cost** that +cycle rather than withholding — the node still emits its row, so the cost stream +stays co-temporal 1:1 with the executor's R-record. Co-temporality is therefore +structural and warm-up-independent, preserves the C18 golden, and is honest (no +slippage estimate yet → no charge). A constant-per-trade factor satisfies it +trivially; only state-dependent factors need the missing-factor → 0 rule. + +**The research loop is pure feed-forward — compounding is removed.** Flat-1R-style +R accounting needs no equity, and with the Sizer gone there is **no equity → size +edge at all** in research: the loop is **feed-forward, maximally parallel and +deterministic (C1)**, the cost model a feed-forward subtraction on the R stream. +**Compounding is removed from research**: it is a **post-strategy money-management +transform** — a pure function of the per-trade **net-R sequence** and a +bet-fraction *f*, multiplicative and path / order-dependent, the **sole** source +of feedback — so compounding, Kelly-*f*, and drawdown-under-compounding are +derived **post-hoc, analytically, at the deploy / account layer** from the net-R +distribution. Consequently there is no `z⁻¹` fill-edge register and no +flat-1R-vs-compounding structural axis in the research loop (with no in-loop +feedback there is nothing for a register to cut; this is strictly C9 / C23-cleaner). + +**Conviction-based risk allocation survives — as an R-aggregation axis, not a +Sizer.** Scaling risk by bias strength is **signal-side and R-denominated**. +Because per-trade R is **size-invariant**, conviction cannot be expressed by +scaling position size (that is invisible to R); it is expressed by **weighting the +per-trade R-contribution** in the R-equity: **flat** (sum of `realized_r`, sign +only) vs **conviction-weighted** (sum of `|bias| · realized_r`, sign + magnitude). +This is a **feed-forward, additive, order-independent research axis**, distinct +from the removed money-Sizer; it is tested via the `conviction_at_entry` record +field and `conviction_terciles_r` metric, and may sit in-graph or as a post-hoc +fold. + +**Money / real broker / cTrader Open API = a separate, later live / deploy-edge +concern — the only reliable (`belastbare`) ground truth, measured never +modelled.** Reliable friction statistics require **forward-trading against a real +broker** (e.g. cTrader Open API), and are non-stationary even then. This fits C11 +(record-then-replay: real fills are recorded live, then replayable) and the +frozen-deploy invariant (C13: deploy = frozen bot + broker connection); +reconciliation with the real account is an **external I/O adapter** at the +recording / deploy edge, not an in-graph node. **This is the only place account +money appears.** Currency-denominated *reference geometry* — pip value, +stop-distance-in-currency from the C15 `instrument_geometry` sidecar — is still +read at the **ingestion** edge to normalize a currency / pip cost factor into R; +notional size cancels in `cost_in_R = cost_in_currency / (size · stop_dist)`, so +the cost model is R-pure without ever holding *equity*. It is equity / account +money, not reference geometry, that lives only at the deploy edge. The +broker-independent **position-event table** (`event_ts, action[buy/sell/close], +position_id, instrument_id, volume`) is the **deploy / reconciliation** record at +this edge (real volume), not a research artifact. + +**Honesty principle.** The net-R curve under the cost model is a **research / +ranking tool and a hypothesis**; the **forward / live run against a real broker is +the ground truth**. The cost model *approximates*; it **never claims realism**. +`SimBroker` (the legacy pip-equity, unsized-exposure node) is **redundant as a +quality measure** with the net-R cost model in place (R displaces pips). It is retained as a **legacy / simple optional pip yardstick** +(still wired alongside the R executor in the r-family member for an honest dual +readout), **not part of the new model and not to be expanded**. + +**Forbids.** Putting **sizing or the stop in the strategy** (bias is unsized; the +stop-rule owns the stop, R is the unit); treating an **equity curve** (R or +currency) as the strategy's direct output; **putting a Sizer / currency size / +`volume` into the research loop** (size is a deploy concept; research is in R); +leaving a **dangling `size` input port** on the research executor (C8 wiring +totality forbids it — drop the port or drive it with a constant unit); **any +equity → size / equity → anything feedback in research** (the research loop is +pure feed-forward; compounding is a post-hoc money-management transform, not an +in-loop edge); modelling an **authored-friction "realistic broker" over historical +data** (real friction is not historically knowable — use the approximating cost +model, and treat the real broker as the live-edge ground truth only); **claiming +the cost model is realism** (it is an explicit approximation); **stacking +unfalsifiable cost guesses** (each cost factor is a labelled stress-parameter *or* +data-grounded — over-modelling is the anti-pattern); **computing cost in two +homes** (the cost graph owns cost; the post-run fold subsumes the old scalar +`round_trip_cost`, never double-counts it); expressing **conviction by scaling +position size** (size-invisible to R; conviction is an R-aggregation weight); +making the **position-event table the strategy's direct DAG output** (it is +derived, not emitted per `eval` — a decision instant may need >1 event, which C8 +forbids) or a **research** measure of signal quality (it is a deploy / +reconciliation artifact); **measuring signal quality in currency / account money** +in the research loop at all (account money lives only at the live deploy edge; +reference geometry at ingestion is not account money); baking a broker into the +strategy or an **in-graph broker subsystem** (the in-graph realistic broker is +retired; the only broker is the live-edge I/O adapter, C11 / C13); a signed-volume +direction trick in the event table (use `action`); storing `open_ts` (derive it +from the opening event). + +**Why.** A strategy's edge is a *procedure*, not a currency outcome ("focus on the +procedure, not the money"): the right primary question is *"how much R out per 1R +risked?"*, and R — defined by the stop — is the only account- and +instrument-agnostic, risk-normalized unit (pips are not). Separating **direction +(bias) from sizing from fill** is the decomposition every mature system converges +on (LEAN's Alpha → Portfolio-Construction → Execution is a near isomorphism; +backtrader, QSTrader, zipline all emit an unsized directional signal sized +downstream) — and aura pushes *sizing* and *fill* off the research loop for two +**distinct** reasons: **sizing** is off because per-trade R is **size-invariant** +(size carries no information in R — flat-1R is perfectly knowable, it just does not +matter), and **fill / friction** is off because real friction is **not +historically knowable** and therefore not honest over history. Keeping research +**pure feed-forward** leaves the signal-quality layer parallel and deterministic +(C1); the **only** real feedback (equity → bet-fraction) is **compounding**, a +closed-form, path-dependent transform of the net-R sequence, and therefore belongs +**after** the strategy, at the deploy / account layer, not as an in-loop register. +The **cost model as a C9 graph** keeps cost within the one Node / graph +abstraction (C9) and generalizes the scalar `net_expectancy_r` continuously, while +the **gross-R / net-R** split states the cost-drag honestly without inventing a +unit. Refusing the historical realistic broker is an **honesty** stance: the only +reliable friction is **measured forward** against a real broker (cTrader Open API) +— the live deploy edge, the sole place account money and ground truth appear. The +DAG holds exactly one state at t and a node emits ≤1 record per `eval` (C8), so the +faithful per-cycle output is the **bias** (one value); position *events* are a +derived, deploy-side consequence. Industry grounding for the bias/R spine: LEAN / +nautilus_trader / backtrader / QSTrader / vectorbt / zipline (#117 decision log). + +## Current state + +**Bias and executor.** The unsized strategy output is the `Bias` node +(`aura-strategy/src/bias.rs`). The per-symbol **RiskExecutor** is the +`risk_executor(StopRule, risk_budget)` composite-builder +(`aura-composites/src/lib.rs`) with a `StopRule{Fixed, Vol, VolTf}` structural +axis (C20); `risk_executor_vol_open` is the gridding sibling with the two vol-stop +knobs left open as sweep axes. Stop rules: `FixedStop` (a triggered constant) and +the `vol_stop(length, k)` composition `k·√EMA(Δ²)` built from `Mul`/`Sqrt` +primitives; `VolTfStop` for a resampled-timeframe vol stop (`FixedStop` in +`aura-strategy/src/stop_rule.rs`, `vol_stop` composite in `aura-composites`). The +**Veto** is a documented seam, not a runtime node. + +**The Sizer is a vestige, not yet removed.** The ratified design (`#116`) has *no* +Sizer in research, but the composite still physically wires +`stop → Sizer → PositionManagement`, with the `Sizer` bound to a constant +`risk_budget` (`risk_budget` is a formal argument; the runner calls +`risk_executor(stop, 1.0)` — `aura-runner/src/member.rs`). This is the +constant-unit-driven-`size`-port resolution of C8 wiring totality (not the +drop-the-port resolution): `risk_budget` is a constant, never equity-fed, so no +`equity → size` edge exists and the pure-feed-forward invariant holds; under R +size-invariance the `Sizer`'s `size = 1.0 / stop_distance` carries no research +information. Physically deleting the `Sizer` node and its `risk_budget` argument is +outstanding cleanup toward the ratified shape. + +**Position management and R metrics.** `PositionManagement` +(`aura-backtest/src/position_management.rs`) is the stateful heart: it latches the +entry-cycle stop distance as the immutable R-denominator, marks against the +one-cycle-lagged fill (no look-ahead, C2), and emits a dense per-cycle R-record +(one row per eval, C8; the trade ledger is the `closed_this_cycle` subset, the +R-equity is `cum_realized_r + unrealized_r`). `summarize_r` +(`aura-backtest/src/metrics.rs`) is a post-run fold (sibling of `summarize`, **not** +an in-graph node) → `RMetrics` (E[R], SQN, `sqn_normalized`, win-rate, +profit-factor, max-R-drawdown, `conviction_terciles_r`, `net_expectancy_r`). It +folds a co-temporal cost stream (positional 1:1 join) into `net_expectancy_r` — +one home for cost, byte-identical to the cost = 0 baseline on an empty stream. +`sqn_normalized = (mean_R / stdev_R)·√(min(n, 100))` (`SQN_CAP = 100`) is an opt-in +turnover-robust rank key; below the cap it equals raw `sqn` exactly. SQN is the +operational single-number ranking objective; a sweep's default grid varies **only +the signal**, holding the stop fixed — the **stop defines 1R**, so varying it +across members would change what R *means* per member and break cross-member SQN +comparability. + +**Cost model.** `aura-strategy/src/cost.rs` owns the abstraction: the 3-field cost +triple `{cost_in_r, cum_cost_in_r, open_cost_in_r}` is one source of truth +(`COST_FIELD_NAMES` / `COST_WIDTH = 3`, mirroring the position-management record), +prefixed by the 4-wide `GEOMETRY_WIDTH` geometry inputs. The `CostNode` **factor +trait** carries a node's only per-node difference — the price-unit cost numerator +(`cost_numerator`), plus `extra_inputs`, `label`, and a defaulted `charge_mode()`; +everything else is the generic `CostRunner` **adapter** (a plain +downstream `Node`, C9; no runtime sub-object), which writes the co-temporality +skeleton (geometry-only gating, `numerator / latched` R-normalization, the +closed/open charge, the running `cum`, the 3-field emit) **once**. `ChargeMode +{AtClose, PerHeldCycle}` selects timing per factor (a commission is intrinsically +at-close, a carry per-held-cycle); the single `CostRunner` handles both arms, the +`PerHeldCycle` arm accruing `per` into a per-position `acc` each held cycle and +dumping it into `cum` at close. Three shipped factors, all `aura-strategy`: +`ConstantCost` (a labelled stress-parameter, `cost_per_trade / |entry − stop|`), +`VolSlippageCost` (`slip_vol_mult · vol / |entry − stop|`, reading an independent +short-horizon realized-range vol proxy distinct from the stop's own vol), and +`CarryCost` (a `ConstantCost` twin with `charge_mode() = PerHeldCycle`, the flat +base of the accrual family). `CostSum` (`aura-strategy/src/cost_sum.rs`) is the +cost-graph **output node**, summing `N` nodes' 3-field records per-field into one +aggregate (`n = 1` is the identity), so the seam consumes a single cost stream +regardless of node count. `cost_port` / `intern_port` (`aura-strategy/src/cost.rs`) +intern runtime port names process-globally (the `COL_PORTS` production pattern, +#152), reused across per-member rebuilds. `cost_graph(Vec)` +(`aura-composites/src/lib.rs`) is the authoring primitive: it fans the 4 geometry +inputs to `N` cost nodes, surfaces each node's extras +(`schema().inputs[GEOMETRY_WIDTH..]`) as `cost[k].` roles, sums through +`CostSum`, and exposes the 3-field aggregate at arbitrary arity. The headline sink +is **`net_r_equity`** = `LinComb(4)[cum_realized_r, unrealized_r, −cum_cost_in_r, +−open_cost_in_r]` → Recorder (C8/C18), a sibling of `r_equity`, emitted only when a +cost is authored; a per-held-cycle factor bleeds continuously over the hold because +the bleed lives in `open_cost_in_r`, which this tap already subtracts (Approach B, +no `summarize_r` fold change). + +**Cost on the campaign / sweep path (net is the default).** Cost is authored as +the campaign document's additive `cost: Vec` block +(`aura-research/src/lib.rs`, `CampaignDoc`) — a closed, externally-tagged +vocabulary over the three shipped nodes (`constant` / `vol_slippage` / `carry`, +field names = the builders' `ParamSpec` names), mirroring the `risk` block (serde +`default` + skip-if-empty: cost-less docs hash byte-identically, C18). An absent +block is the explicit zero-cost model and `summarize_r`'s net family equals gross +under it — every result is net, no second gross-labelled result kind. +`cost_nodes_for` (beside `stop_rule_for_regime`, `aura-runner/src/translate.rs`) is +the one doc→builder binding; `wrap_r` (`aura-runner/src/member.rs`) carries the +optional cost leg (cost_graph off the executor's four geometry outputs, the vol +proxy in production, a gated cost recorder in reduce mode as the `summarize_r` join +input, the `LinComb(4)` `net_r_equity` curve in trace mode). Both re-run sides +re-derive the model (`cost_specs_from_params`), so the C1 drift alarm compares like +with like; costed families reproduce bit-identically (incl. `Carry`). +`TapChannel::Net` (`aura-runner/src/runner.rs`) routes `net_r_equity` to persisted +curves; a cost-less doc requesting it keeps a remedy-naming skip notice. There are +no `--cost-*` run-path flags (removed #221/#234); cost travels in the document. + +**Risk regime as a structural campaign axis.** The `StopRule{Fixed, Vol}` axis is +realized at the campaign-document level as `CampaignDoc.risk: [RiskRegime]` +(`aura-research`, variants `Vol{length, k}` and `VolTf{period_minutes, length, k}` +(#262), the fixed-stop rule additive when needed) — a kept-separate matrix axis, +peer of instruments and windows. The executor keys the nominee map by `(strategy, +window, regime)`, so `generalize` aggregates *within* a regime, never across. +Regimes are **compared** at presentation, never argmax-**selected** across (a +cross-regime E[R] argmax would compare R-multiples in different R units). Each +member manifest stamps its resolved stop (default included). Two default +representations coexist by design (#217): a dissolved sweep binds no regime +(`risk: []`, late-resolved per member by `stop_rule_for_regime`), while +`walkforward`/`mc`/`generalize` bind the default regime eagerly +(`risk: [Vol{length:3, k:2.0}]`) — same R behaviour, deliberately different +document content-ids. Deferred: regime-aware **trace** persistence — the trace +re-run and cell-key dir naming still assume the default stop, since +`CellRealization` carries no regime (#212). + +**Legacy pip yardstick.** `SimBroker` (`aura-backtest/src/sim_broker.rs`, node kind +`simbroker`) is a legacy pip-equity node, retained as an optional pip yardstick +and still wired alongside the R executor in the r-family member for an honest dual +pip/R readout; not part of the cost model, not to be expanded. + +**Position-event table (deploy / reconciliation).** A closed `PositionAction {Buy, +Sell, Close}` enum + the `PositionEvent` row (`event_ts`, `action`, `position_id`, +`instrument_id`, unsigned `volume`; no `open_ts`; direction *is* the action) and +`derive_position_events(record, instrument_id) -> Vec` live in +`aura-backtest/src/metrics.rs` as post-run value types (not per-`eval` nodes, C8). +`action` serde-encodes as a bare `i64` (Buy=0, Sell=1, Close=2), the C7 scalar +column form, out-of-range rejected on read. `derive_position_events` is the **first +difference of the executed book** — a pure reduction over the position-management +dense record (read positionally as type-erased `Scalar`s, C7 SoA; no in-graph node, +so the hot path stays domain-free, C14) — emitting a `Buy`/`Sell` at each open and +a `Close` at each exit; a reversal (or stop-then-same-cycle reopen) emits **Close +then the opposite open at one `event_ts`** (the C8 ">1 event per instant" case that +forces a *derived* table). The close sizes the **actual book** (the closed +position's stored volume), never an exposure delta. A position open at window end +emits its open with **no synthetic `Close`** (the table records actual executed +events; `summarize_r`'s force-close is for the R metric only). The table stays +broker-independent. + +**CLI.** The run/sweep surface is blueprint-driven — `aura ` +over `examples/r_*.json` / `crates/aura-cli/tests/fixtures/`; the built-in +`--harness` / `--strategy` selectors were retired with the demos → blueprint-data +cut (#159). The strategy identifier family is the **r-family** — `r-sma` / +`r-breakout` / `r-meanrev` (renamed from the Stage-1 family in #174). `metric_cmp` +(`aura-registry`) ranks the higher-is-better R metrics `sqn`, `expectancy_r`, +`net_expectancy_r`, `sqn_normalized`; a member with no `r` block sorts last. + +**Deferred work.** Physical removal of the vestigial `Sizer` and its `risk_budget` +argument (ratified shape, #116). Data-grounded cost factors beyond the shipped +stress parameters: a notional-based carry (`price × rate`, reads the price tap) and +the calendar-aware **overnight swap** proper (rollover-boundary timing, 3× Wednesday, +long/short asymmetry — deploy-edge realism) (#148). The **conviction-weighting** +R-aggregation axis (flat vs `|bias|·realized_r`), tested via `conviction_at_entry` / +`conviction_terciles_r` (#148). Regime-aware trace persistence (#212). Money, a +real broker, and cTrader Open API remain the separate live / deploy-edge concern +(C11 record-then-replay, C13 frozen deploy). + +## See also +- [C1](c01-determinism.md) — determinism / bit-identity (costed families reproduce byte-for-byte) +- [C2](c02-causality.md) — no look-ahead (one-cycle-lagged fill; the co-temporality 0-cost warm-up rule) +- [C8](c08-node-contract.md) — ≤1 record per `eval`; sinks are the only displayable surface +- [C9](c09-fractal-composition.md) — cost nodes are ordinary downstream nodes; the cost model is a C9 graph +- [C11](c11-sources-record-replay.md) — record-then-replay; the live broker as recorded I/O; bit-identity of re-runs +- [C13](c13-hot-reload-frozen-deploy.md) — the frozen-deploy invariant (deploy = frozen bot + broker connection) +- [C14](c14-headless-two-faces.md) — the domain-free engine (no cost / broker logic in `aura-engine`) +- [C15](c15-resampling-sessions.md) — the `instrument_geometry` sidecar (pip value / stop-distance-in-currency) +- [C16](c16-engine-project-split.md) — the composites layer (`cost_graph`, `risk_executor`) +- [C18](c18-registry.md) — the registry / golden (byte-identity, content-addressed docs) +- [C20](c20-strategy-harness.md) — structural axes (`StopRule`, the risk regime) +- [C23](c23-graph-compilation.md) — names non-load-bearing; DCE of the absent Veto +- [C24](c24-blueprint-data.md) / [C25](c25-role-model.md) — experiment intent in the campaign document +- [C28](c28-stratification.md) — the crate-layer ladder + +> History: [c10-bias-r-cost.history.md](c10-bias-r-cost.history.md) diff --git a/docs/design/contracts/c11-sources-record-replay.md b/docs/design/contracts/c11-sources-record-replay.md new file mode 100644 index 0000000..0b3e0c8 --- /dev/null +++ b/docs/design/contracts/c11-sources-record-replay.md @@ -0,0 +1,39 @@ +# C11 — Generalized sources; record-then-replay determinism boundary + +**Guarantee.** A source is anything that produces timestamped scalar streams — +market data (the data-server) and non-financial sources (e.g. a news-agent node +emitting a bias) are treated identically. Anything nondeterministic, external, +or slow (LLM/news/web) is materialized into a recorded, timestamped stream +*before* it enters the engine; a backtest replays the recording, live computes +fresh in real time and records it for future backtests. A bias enters as a value +held until the next event (firing policy A, C6). + +**Forbids.** Any live external call *inside* a backtest replay. + +**Why.** It is the only model compatible with reproducible backtests (C1) — LLM +calls are nondeterministic and far too slow per-cycle. External LLM (IONOS) calls +happen only at the recording/live-source edge, with explicit per-session consent, +never inside a sim (the IONOS consent rule lives in `~/.claude/CLAUDE.md`). + +## Current state +The producer seam that makes "a source is anything" concrete is the object-safe +`Source` trait (`crates/aura-engine/src/harness.rs`), driven by the single k-way +ingestion merge (C3); the seam itself is detailed in C12. The realized side is the +market-data replay path: `DataServer` (the external `data-server` git-dependency +crate, re-exported through `crates/aura-ingest`) +parses an archived M1 window and streaming `M1FieldSource` producers +(`crates/aura-ingest/src/lib.rs`) replay it deterministically into the merge. The +recording *direction* — a live source materialized into a timestamped stream for +later replay — and generalized non-price recorded sources (a news-agent bias) are +design law that rides the same `Source` seam (the #71 source seam) but has not yet +landed. The held-until-next-event semantics of a sparse recorded source are the +firing-policy-A / sample-and-hold mechanism already realized under C5/C6. At the +opposite edge the live broker is an I/O adapter at the recording/deploy boundary, +never part of the strategy (C13; domain invariant 7). + +## See also +- [C1](c01-determinism.md) +- [C3](c03-single-merge.md) +- [C6](c06-firing-policy.md) +- [C12](c12-atomic-sim-unit.md) +- [C13](c13-hot-reload-frozen-deploy.md) diff --git a/docs/design/contracts/c12-atomic-sim-unit.history.md b/docs/design/contracts/c12-atomic-sim-unit.history.md new file mode 100644 index 0000000..e8b98d9 --- /dev/null +++ b/docs/design/contracts/c12-atomic-sim-unit.history.md @@ -0,0 +1,36 @@ +# C12 — The atomic sim unit and the four orchestration axes: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c12-atomic-sim-unit.md](c12-atomic-sim-unit.md). + +**Realization (cycles 0028, 0049).** Axis 1 (param-sweep) is built. `GridSpace` +(0028) enumerates a cartesian lattice over discrete per-slot value-lists; +`RandomSpace` (0049) draws `N` seeded points over typed continuous `ParamRange`s +(I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), validated against the +param-space before any run. Both implement the `Space` trait the disjoint +`sweep` / `run_indexed` core is generic over, so either enumeration runs through +one execution path (C1: results in enumeration order, not completion order). The +seeded sampler reuses the bit-stable `SplitMix64` as a **code-path-disjoint** +instance from the data-edge seed RNG (the source-seam firewall, #52/#71: they +share only the `u64` type, never a path). + +**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the +only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait +(`peek`/`next`, object-safe) the k-way merge drives — and a streaming +**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing +**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and +decoding each `Scalar` on demand: the **source ring** is resident O(one chunk), +not O(window length) — the measured `resident_records()` predicate, a *per-source* +bound, not whole-process RSS. (Data-server's `FileCache` retains each window's +parsed chunks read-only for the pass — ~56 B/record — so process residency is +O(records-touched); that is the **replay-many** sharing C12 wants — one window +parsed once across a sweep family — not a leak. The single-pass cost is tracked as +#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the +gap closes by a streaming path *existing*, not by deleting the eager one). **Realized +(2026-06-29, family cycles shipped).** cross-*sim* `Arc<[T]>` sharing — one window shared +zero-copy across many disjoint sweep sims — is now in force: the sweep / Monte-Carlo / +walk-forward families (axes 2–4 above) build their members over **one** shared +`Arc` (one `FileCache`), so a window is parsed once and every member's +`M1FieldSource` borrows the same cached `Arc<[M1Parsed]>` chunks +(`crates/aura-ingest/src/lib.rs:316`). The single-pass *parse* cost stays tracked as #95. diff --git a/docs/design/contracts/c12-atomic-sim-unit.md b/docs/design/contracts/c12-atomic-sim-unit.md new file mode 100644 index 0000000..1ced0f0 --- /dev/null +++ b/docs/design/contracts/c12-atomic-sim-unit.md @@ -0,0 +1,82 @@ +# C12 — The atomic sim unit and the four orchestration axes + +**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window + +RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime +values injected at graph build — no recompile per param-set; the optimizer sees a +generic vector of typed ranges. A **bound blueprint param is that param's +default**: "open" means *must be bound by an axis*, "bound" means *default, +overridable by an axis*, so axis 1 (param-sweep) may name a bound param — the +family boundary re-opens it on the probe and on every member reload, and the axis +binds it per cell; `run`/`mc` still require every param resolved (a truly open +param refuses). Identity is untouched: `content_id_of` +(`crates/aura-research/src/lib.rs`) and `topology_hash` +(`crates/aura-cli/src/main.rs`) read the authored document, never a re-opened +probe, and each member manifest records its per-cell bindings (#246, ratified +2026-07-12; the restriction this amends — axes bind only open knobs — was an +implementation consequence of `bind()` shrinking the param surface, not a recorded +decision). Raw data is shared read-only across sims via `Arc<[T]>` (the +data-server is built for this). Four axes orchestrate the atomic unit: +(1) param-sweep (grid/random), (2) optimization (argmax metric), (3) walk-forward +(rolling in-sample optimize + out-of-sample test), (4) Monte-Carlo (N seeded +realizations perturbing input). **MC = sweep over seeds**; each realization is +itself deterministic given its seed. + +**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the +primitive — those are pluggable policies atop the atomic unit; recompiling on a +param change. + +**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell" +(embarrassingly parallel across the unit) cleanly separated from search policy. +Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the atomic +unit is one harness instance, selected by the harness's **structural axes** (C20); +the structural experiment matrix is the outer orchestration over this dimension, +the tuning sweep the inner (C19/C20). + +## Current state +`Harness::run` is typed as a **producer seam** — the object-safe `Source` trait +(`peek`/`next`, plus `bounds` and a `resident_records()` residency probe; +`crates/aura-engine/src/harness.rs`) that the k-way merge drives (C3). Two source +shapes back it. The eager `load_m1_window`/`close_stream` path +(`crates/aura-ingest/src/lib.rs`), wrapped as `VecSource`, materializes a whole +stream and is retained for bounded loads. The streaming `M1FieldSource` +(`crates/aura-ingest/src/lib.rs`) pulls a data-server window lazily, borrowing one +`Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and decoding each +`Scalar` on demand, so its **source ring** is resident O(one chunk), not +O(window length) — `resident_records()` is a *per-source* bound, not +whole-process RSS. + +Raw-data sharing is realized both within and across sims. The data-server's +`FileCache` retains each window's parsed chunks read-only for the pass, so process +residency is O(records-touched) — the **replay-many** sharing model, not a leak. +Across sims, the sweep / Monte-Carlo / walk-forward families (axes 2–4) build their +members over **one** shared `Arc` (one `FileCache`; +`crates/aura-ingest/src/lib.rs`), so a window is parsed once and every member's +`M1FieldSource` borrows the same cached `Arc<[M1Parsed]>` chunks. The single-pass +*parse* cost of a window is tracked as #95. + +Axis 1 (param-sweep) is built. `GridSpace` enumerates a cartesian lattice over +discrete per-slot value-lists; `RandomSpace` draws `N` seeded points over typed +continuous `ParamRange`s (I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), +validated against the param-space before any run; a `ListSpace` enumerates an +explicit point list. All implement the `Space` trait the disjoint `sweep` / +`run_indexed` core is generic over (`crates/aura-engine/src/sweep.rs`), so every +enumeration runs through one execution path (C1: results in enumeration order, not +completion order). The other axes are realized as families over that core: +`walk_forward` + `WindowRoller` (`crates/aura-engine/src/walkforward.rs`) for the +rolling in-sample/out-of-sample split, `monte_carlo` +(`crates/aura-backtest/src/mc.rs`) for the seed sweep; optimization is argmax over +a sweep's metrics, a policy atop the core rather than a baked-in strategy. The +seeded sampler reuses the bit-stable `SplitMix64` +(`crates/aura-analysis/src/lib.rs`) as a **code-path-disjoint** instance from the +data-edge seed RNG (the source-seam firewall, #52/#71: they share only the `u64` +type, never a path). + +## See also +- [C1](c01-determinism.md) +- [C11](c11-sources-record-replay.md) +- [C18](c18-registry.md) +- [C19](c19-bootstrap.md) +- [C20](c20-strategy-harness.md) +- [C23](c23-graph-compilation.md) + +> History: [c12-atomic-sim-unit.history.md](c12-atomic-sim-unit.history.md) diff --git a/docs/design/contracts/c13-hot-reload-frozen-deploy.history.md b/docs/design/contracts/c13-hot-reload-frozen-deploy.history.md new file mode 100644 index 0000000..00cc4cd --- /dev/null +++ b/docs/design/contracts/c13-hot-reload-frozen-deploy.history.md @@ -0,0 +1,25 @@ +# C13 — Hot-reload is authoring-only; deploy is frozen: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: +> [c13-hot-reload-frozen-deploy.md](c13-hot-reload-frozen-deploy.md). + +**Realization (cycle 0102 — the load boundary; per-invocation reload).** The +authoring-loop half is realized: a project is an external cdylib crate loaded +per invocation through a two-tier `#[repr(C)]` descriptor (`AURA_PROJECT`, +`aura-core::project`) — a C-ABI stamp prefix (rustc version + aura-core +version, baked per consuming build) validated **before** any Rust-ABI field is +touched, then the vocabulary resolver + enumerable type-id list behind the +stamp gate. "Hot-reload" reads, in v1, as **per-invocation load of the +freshest build**: the author (Claude) runs `cargo build`, the next `aura` +invocation locates the artifact via `cargo metadata` (debug default, +`--release` opt-in) and loads it **load-and-hold** (leaked, never unloaded). +Scope boundary: load-and-hold is trivially sound only because the CLI is a +one-shot process — a future long-running host (the open local-server thread, +C22) must re-solve reload (host restart or subprocess isolation), never +in-process unload. Mismatch of either stamp refuses (exit 1) naming both +sides; the project vocabulary is charter-checked at load (`::`-namespaced ids, +no duplicates against std, list↔resolver cross-check) — the invariant-9 +data-plane discipline of the C24 enforcement-shift note, now enforced at the +one seam. diff --git a/docs/design/contracts/c13-hot-reload-frozen-deploy.md b/docs/design/contracts/c13-hot-reload-frozen-deploy.md new file mode 100644 index 0000000..b2be7f8 --- /dev/null +++ b/docs/design/contracts/c13-hot-reload-frozen-deploy.md @@ -0,0 +1,63 @@ +# C13 — Hot-reload is authoring-only; deploy is frozen + +**Guarantee.** A node/strategy is authored as a native Rust `cdylib`, +hot-reloaded during the authoring loop (Rust-ABI; host and node built with the +same toolchain). The live/deploy bot is a statically-linked, versioned, frozen +artifact. + +**Forbids.** Hot-swapping a running live bot; loading third-party / foreign- +toolchain plugins. + +**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen +and reproducible (audit trail: this bot = this commit). A sweep pays no +hot-reload tax — params are runtime data ([C12](c12-atomic-sim-unit.md)), so +the cdylib loads once. + +## Current state + +The authoring-loop half is realized; the frozen live/deploy artifact is not yet +built. + +A research project compiles to an external cdylib exporting one symbol, +`AURA_PROJECT`, a two-tier `#[repr(C)]` `ProjectDescriptor` +(`aura-core::project`, `crates/aura-core/src/project.rs`, emitted by the +`aura_project!` macro). A **C tier** — `magic` (`AURAPROJ`), +`descriptor_version`, the rustc-version and aura-core-version stamps, the +namespace, all C-compatible field types — is validated **before** any Rust-ABI +field is touched; a **Rust tier** — the vocabulary resolver +`fn(&str) -> Option` and the enumerable type-id list — is read +only once both stamps match. The stamp cannot certify the channel it rides on, +so it does not depend on the Rust ABI it certifies; both stamps are baked at +*aura-core* compile time (build.rs, per consuming build), so host and dylib each +carry their own side's truth. + +"Hot-reload" reads, in v1, as **per-invocation load of the freshest build**: the +author runs `cargo build`, and the next `aura` invocation locates the artifact +via `cargo metadata` (debug default, `--release` opt-in) and loads it +**load-and-hold** — the `Library` is leaked, never unloaded, so its `'static` +strings and fn-pointers stay valid (`aura-runner::project::load`, +`crates/aura-runner/src/project.rs`). `validate_c_tier` checks the four stamps +front-to-back; a mismatch of either version stamp refuses with exit 1 naming +both sides (`ProjectError::Incompatible`). The project vocabulary is +charter-checked at load (`check_charter`): non-empty namespace, every id +`::`-prefixed, no duplicate, no collision against the std vocabulary, +list↔resolver cross-check — the invariant-9 data-plane discipline (the +[C24](c24-blueprint-data.md) enforcement-shift note) enforced at this one +seam. + +**Scope boundary.** Load-and-hold is trivially sound only because the CLI is a +one-shot process. A future long-running host — the local-server thread +([C22](c22-playground-traces.md)) — must re-solve reload by host restart or subprocess +isolation, never in-process unload. + +## See also + +- [C12](c12-atomic-sim-unit.md) — params are runtime data, so the cdylib + loads once across a sweep +- [C16](c16-engine-project-split.md) — engine/project separation; the + per-case dependency policy the cdylib boundary serves +- [C22](c22-playground-traces.md) — the long-running host that must re-solve reload +- [C24](c24-blueprint-data.md) — the enforcement-shift note whose + data-plane discipline the charter check applies + +> History: [c13-hot-reload-frozen-deploy.history.md](c13-hot-reload-frozen-deploy.history.md) diff --git a/docs/design/contracts/c14-headless-two-faces.history.md b/docs/design/contracts/c14-headless-two-faces.history.md new file mode 100644 index 0000000..9cb847b --- /dev/null +++ b/docs/design/contracts/c14-headless-two-faces.history.md @@ -0,0 +1,29 @@ +# C14 — Headless core, two faces: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: +> [c14-headless-two-faces.md](c14-headless-two-faces.md). + +**Realization (cycles 0098/0099, #175 — the CLI meets GNU/clig.dev conventions +via clap).** The programmatic/CLI face's argument surface moved from a hand-rolled +argv parser to a `clap` derive parser (admitted under the C16 per-case dependency +policy — research-side, a dev-loop/compile tax, never a frozen-artifact tax; +invariant 8 untouched, the change is confined to `aura-cli`, a leaf binary the +frozen deploy artifact cannot pick up). One declarative source now yields scoped +`aura --help`, `--version`/`-V`, a per-flag Options section, and GNU +`--flag=value` / `--` / long-option abbreviation. The **exit-code partition** is a +durable part of the automation contract, so a caller can branch on the failure +class without parsing stderr: **exit 0 = success**; **exit 2 = usage error** — a +command-line fault (clap parse errors + aura's post-parse argument-structure +validations, *including* the content of an argv-named blueprint file, which is +itself an argument); **exit 1 = runtime failure** — a well-formed command whose +needed environment / recorded state is missing, or bad piped stdin data. The four +dual-grammar subcommands (run/sweep/walkforward/mc) keep both grammars under one +token via an optional `[blueprint]` positional + a post-parse `is_file()` +dispatch; the execution layer is unchanged (arg-plumbing via thin `*_from` +adapters). Error-message casing is normalized to the clap house style — +every hand-rolled usage line reads `Usage: aura …` (#179, cycle 0101); +refusal diagnostics stay unprefixed (diagnostics are not usage lines). The +machine-first help surface (JSON/manifest help, stdin op-scripts) stays on the +#157/C21 track, not this cycle (settled as human/GNU convention compliance). diff --git a/docs/design/contracts/c14-headless-two-faces.md b/docs/design/contracts/c14-headless-two-faces.md new file mode 100644 index 0000000..985f772 --- /dev/null +++ b/docs/design/contracts/c14-headless-two-faces.md @@ -0,0 +1,89 @@ +# C14 — Headless core, two faces + +**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a +**programmatic/CLI** face (the primary surface for the LLM and automation — +author a node, run a sim/sweep, emit structured metrics) and a **visual** face +for human exploration. Visualization is only a downstream consumer node +([C8](c08-node-contract.md)) on the streams. + +**Forbids.** Any UI/pixel knowledge inside the engine. + +**Why.** The LLM drives programmatically, the human visually; a headless core +serves both. The visual face is the **playground** ([C22](c22-playground-traces.md)) — +a **web frontend served from disk-persisted traces** (#101, ratified 2026-06): +the engine writes recorded traces to disk and a browser charts them. It is +staged after the runnable substrate but is core to aura's identity, not +optional. Reading a serialized trace file is a stricter form of "no UI knowledge +in the engine" than an in-process native UI reaching zero-copy into the live SoA +columns — the pivot to the disk-trace frontend keeps this contract's core intact +and arguably tightens it. + +## Current state + +**The CLI conventions.** The programmatic/CLI face is a `clap` derive parser +(`aura-cli`, `crates/aura-cli/src/main.rs`) that meets GNU / clig.dev +conventions: one declarative source yields scoped `aura --help`, +`--version`/`-V`, per-flag Options sections, and GNU `--flag=value` / `--` / +long-option abbreviation. `clap` is admitted under the +[C16](c16-engine-project-split.md) per-case dependency policy — a +research-side, dev-loop/compile tax confined to `aura-cli`, a leaf binary the +frozen deploy artifact ([invariant 8](c13-hot-reload-frozen-deploy.md)) cannot +pick up. Usage lines follow the clap house style (`Usage: aura …`); +refusal diagnostics stay unprefixed, since a diagnostic is not a usage line. + +**The exit-code partition** is a durable part of the automation contract: a +caller branches on the failure class without parsing stderr. + +- **0 — success.** +- **2 — usage error:** a command-line fault (clap parse errors + aura's + post-parse argument-structure validations, *including* the content of an + argv-named blueprint file, which is itself an argument). +- **1 — runtime failure:** a well-formed command whose needed environment / + recorded state is missing, or bad piped stdin data. +- **3 — campaign completed with failed cells:** a well-formed campaign that ran + to completion but with at least one failed cell (#272; `exit_on_campaign_result` + in `crates/aura-cli/src/main.rs`, threaded from the run registry — + [C18](c18-registry.md)). + +**Dual grammar.** The four dual-grammar subcommands (run/sweep/walkforward/mc) +keep both grammars under one token via an optional `[blueprint]` positional plus +a post-parse `is_file()` dispatch; the execution layer is unchanged (arg-plumbing +via thin `*_from` adapters). The machine-first help surface (JSON/manifest help, +stdin op-scripts) is deferred to the #157 / [C21](c21-world.md) track, distinct +from this human/GNU-convention compliance. + +**Two artifact classes, two redundancy budgets** (#249, ratified 2026-07-13). +The data layer the programmatic face emits is a public interface read raw (by +humans and LLMs), and its records split into two classes with opposite +redundancy budgets. **Generated outcome records** (manifests, metrics, family +reports) have a single writer at run time — redundancy there cannot drift and is +deliberately spent on direct readability. **Authored intent artifacts** +(blueprints, op-scripts, campaign documents) are author-maintained — every +redundancy is a drift site and stays out. Consequence in the run manifest +(`RunManifest`, `crates/aura-engine/src/report.rs`): the untouched bound +`defaults` are stamped one-directionally beside `params` (which keeps its "what +varied" reproduce semantics, disjoint by construction from `defaults`, "what was +held"), so a raw reader of a fully-bound run no longer sees a misleading +`"params": []`; the blueprint itself carries no such duplication. + +**The programmatic face is an executor** (#295, ratified 2026-07-20). The +member-run recipe — the definition of what a standard aura backtest is — is +library content (`aura-runner` / `aura-backtest` / `aura-measurement`, +[C28](c28-stratification.md) assembly position), not CLI content: the binary is +argv → document → executor → presentation, and a downstream World program +reaches the same recipe with no binary involved. See +[C25](c25-role-model.md)'s control-surface amendment for the projection rule. + +## See also + +- [C8](c08-node-contract.md) — visualization is a downstream sink/consumer node +- [C16](c16-engine-project-split.md) — the per-case dependency policy under + which `clap` is admitted +- [C18](c18-registry.md) — the run registry that yields the exit-3 + completed-with-failures class +- [C21](c21-world.md) — the #157 machine-first help track (deferred) +- [C22](c22-playground-traces.md) — the visual face / playground +- [C25](c25-role-model.md) — the control-surface amendment (executor projection) +- [C28](c28-stratification.md) — the assembly position of the member-run recipe + +> History: [c14-headless-two-faces.history.md](c14-headless-two-faces.history.md) diff --git a/docs/design/contracts/c15-resampling-sessions.history.md b/docs/design/contracts/c15-resampling-sessions.history.md new file mode 100644 index 0000000..6246c61 --- /dev/null +++ b/docs/design/contracts/c15-resampling-sessions.history.md @@ -0,0 +1,38 @@ +# C15 — Resampling-as-node; sessions/calendars: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date +> stamp and may be superseded; this file is NOT current truth and NOT a +> grounding surface. Current contract: +> [c15-resampling-sessions.md](c15-resampling-sessions.md). + +**Realization (instrument specs, 2026-06, #22 → #124).** The "instrument specs are +metadata" half of this contract is realized by the **recorded geometry sidecar**: +`instrument_geometry(server, symbol)`, over data-server's `symbol_meta`, returns +neutral broker-agnostic `InstrumentGeometry` (the raw provider JSON never enters this +repo) — non-scalar reference data held beside the hot path, keyed by symbol, feeding +the sim-optimal broker's pip divisor (C10). The real-path pip is sourced from +`InstrumentGeometry.pip_size`; refuse-don't-guess on absent geometry. **History:** +cycles 0022/0063 first carried a Rust-authored vetted floor (`InstrumentSpec` — a +single `pip_size`, later a six-field deploy-grade row `instrument_id`, +`contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`, `quote_currency`, plus +`tick_size`/`digits`), cross-checked against the sidecar geometry. **Cycle 0074 +removed that floor**: with the sidecar geometry supplying the real-path pip, the +authored table was redundant, so `InstrumentSpec` / `instrument_spec` / the vetted +list were deleted. The speculative deploy-grade fields and the override/floor tiers +of the #124 hierarchy (tier 1 authored override, tier 3 authored floor) are +**deferred** until a consumer — the cost model's currency→R normalization (C10), or +the live deploy edge (real broker) — needs them, read from the sidecar geometry then. The `quote_currency` runtime-type transition is likewise +deferred to that consumer; #124 collapses to **recorded sidecar geometry → refuse** +in the meantime. + +**Realization (SessionNode — `bars_since_open` only, by design; #154).** The +session-context half of this contract ships **one** scalar stream, not the three the +Guarantee lists: `Session` (`crates/aura-std/src/session.rs`) emits only +`bars_since_open: i64` (tz-aware, DST-correct, baked Frankfurt open). This is a deliberate +narrowing pinned in the node's own contract — *"there is no separate in-session bool gate, +`bars_since_open` alone is the contract"*: a downstream `EqConst(== N)` gate subsumes the +`in_session: bool` stream (pre-open instants give `<= 0`, which never match), and +`session_open_ts: timestamp` has no consumer. The two streams the Guarantee names remain +the original design intent, **deferred until a consumer needs them** (default-simple; +forward-queued as #154). The stream model is untouched — session context is still exposed +as scalar streams fed from beside the hot path. diff --git a/docs/design/contracts/c15-resampling-sessions.md b/docs/design/contracts/c15-resampling-sessions.md new file mode 100644 index 0000000..41e07b7 --- /dev/null +++ b/docs/design/contracts/c15-resampling-sessions.md @@ -0,0 +1,55 @@ +# C15 — Resampling-as-node; sessions/calendars + +**Guarantee.** A resampler is a node (finer stream → coarser bar stream), +clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars +and instrument specs are **metadata** — non-scalar reference data held beside +the hot path, never streamed. Session *context* is exposed as scalar streams +via a `SessionNode`, so "the 3rd 15m candle after session open" is a plain node +checking `bars_since_open == 3`, not special-cased session logic. + +**Forbids.** Streaming the calendar; special-casing session logic outside the +stream model. + +**Why.** Keeps the line consistent — everything a signal needs arrives as a +stream; reference data feeds source/session nodes from beside the hot path. + +## Current state + +**Session context — `bars_since_open` is the shipped contract (#154).** The +session-context node `Session` (`crates/aura-market/src/session.rs`) emits +**one** scalar stream, `bars_since_open: i64` (tz-aware, DST-correct, over a +baked Frankfurt open), and takes no scalar params; the zero-arg +`SessionFrankfurt` roster preset (#261) exposes it with a single +`period_minutes` knob. This is a deliberate narrowing pinned in the node's own +contract: `bars_since_open` alone is the gate — a downstream `EqConst(== N)` +subsumes an `in_session: bool` stream (pre-open instants read `<= 0`, which +never match), and nothing consumes a `session_open_ts: timestamp`. The two +further session-context streams the design intent names — `in_session: bool` +and `session_open_ts: timestamp` — remain **deferred** until a consumer needs +them (#154); the stream model is untouched, session context is still exposed as +scalar streams fed from beside the hot path. + +**Instrument geometry — recorded sidecar, refuse-don't-guess.** The "instrument +specs are metadata" half is realized by the **recorded geometry sidecar**: +`aura_ingest::instrument_geometry(server, symbol)` returns the neutral, +broker-agnostic `InstrumentGeometry` (re-exported from `data_server::meta`; the +raw provider JSON never enters this repo) — non-scalar reference data keyed by +symbol, held beside the hot path. It feeds the sim-optimal broker's pip divisor +(C10; consumed in `aura-runner`): the real-path pip is sourced from +`InstrumentGeometry.pip_size`, and geometry that is absent is **refused, not +guessed**. There is no Rust-authored instrument floor — `InstrumentSpec` / +`instrument_spec` and the vetted table were removed once the sidecar supplied +the real-path pip. The speculative deploy-grade fields and the authored +override/floor tiers of the #124 hierarchy (tier 1 override, tier 3 floor), +together with the `quote_currency` runtime-type transition, are **deferred** +until a consumer — the cost model's currency→R normalization (C10) or the live +deploy edge (real broker) — needs them, read from the sidecar geometry then; +#124 collapses to *recorded sidecar geometry → refuse* in the meantime. + +## See also +- [C2](c02-causality.md) — the resampler emits a bar only once complete +- [C8](c08-node-contract.md) — the node contract the resampler and session node obey +- [C10](c10-bias-r-cost.md) — the pip divisor and currency→R normalization the geometry feeds +- [C11](c11-sources-record-replay.md) — reference data fed from beside the hot path + +> History: [c15-resampling-sessions.history.md](c15-resampling-sessions.history.md) diff --git a/docs/design/contracts/c16-engine-project-split.history.md b/docs/design/contracts/c16-engine-project-split.history.md new file mode 100644 index 0000000..d91c8b0 --- /dev/null +++ b/docs/design/contracts/c16-engine-project-split.history.md @@ -0,0 +1,60 @@ +# C16 — Engine / project separation; three-tier node reuse: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date +> stamp and may be superseded; this file is NOT current truth and NOT a +> grounding surface. Current contract: +> [c16-engine-project-split.md](c16-engine-project-split.md). + +**Realization (cycle 0079, #136 — the analysis leaf becomes its own crate).** The +trading-domain **analysis** layer — the post-run reductions that are pure functions of a +run's recorded data (`RunMetrics`, `RMetrics`, the R reduction `summarize_r` / +`r_metrics_from_rs`, the position-event table `PositionEvent` / `PositionAction` / +`derive_position_events`, and the multiple-comparison hurdle math `inv_norm_cdf` / +`expected_max_of_normals`) — is extracted out of `aura-engine`'s `report` module into a +dedicated **`aura-analysis`** crate (deps: `aura-core` + `serde` only; `serde_json` is a +dev-dependency, test-only). This sharpens the engine↔domain seam: the run loop's crate no +longer *defines* the trading metrics — it `pub use`-re-exports them for source +compatibility this cycle, and still hosts the trace-coupled `summarize`, `RunReport`, +`RunManifest`, and the columnar trace utils. The non-node engine-workspace crates are now +`aura-engine` (run loop), `aura-cli` (`aura` binary), `aura-ingest` (ingestion edge), +`aura-composites` (composite-builder convenience layer), and `aura-analysis` (post-run +domain reductions + selection provenance). Behaviour-preserving (C1): every serde shape +is byte-identical (C18 goldens green). + +**Realization (cycle 0080, #136 — engine-side extraction complete).** The last +trading-domain types still *defined* in `aura-engine`'s `report` module — the +selection-provenance types `FamilySelection` / `SelectionMode` — also move to +`aura-analysis` (the only non-cyclic home: `RunManifest.selection` needs the type and the +engine already depends on `aura-analysis`; the registry/CLI reach them unchanged via the +re-export). Behaviour-preserving (C1; suite 665/0). **Settled (2026-06-27, user-ratified +in the #136 thread):** the `aura-registry` `Metric` / `metric_cmp` / deflation vocabulary +**stays in the registry by design** — the registry is the trading *selection* layer, not +the domain-free engine, and its deflation statistics branch irreducibly on metric +identity (R-bootstrap vs. `total_pips` dispersion floor); forcing genericity would be a +one-implementor abstraction. *(Superseded 2026-07-20: measurement's IC became the second +implementor — #290 — and the vocabulary moved behind the `MetricVocabulary` trait +supplied by the outer rungs; see the C28 #147 disposition.)* **Deferred to the World / +C21 layer (explicitly *not* #136):** +making `RunReport` generic over a metric type and turning the orchestration/registry layer +into a reusable domain-agnostic substrate — until then `RunReport` / `RunManifest` / +`summarize` stay trace-coupled in the engine. + +**Realization (cycle 0107, #198 — the campaign-execution leaf).** The non-node engine +workspace gains **`aura-campaign`**: campaign-execution *semantics* (cell enumeration, +preflight, gate evaluation, winner selection, walk-forward rolling, realization assembly — +see C18) as a leaf library crate whose deps deliberately exclude `aura-ingest` / +`aura-std` / `aura-composites`; harness construction and data binding enter only through +the one-method `MemberRunner` seam, so consumers (the `aura` CLI today; the playground and +tests tomorrow) bind their own runners while the semantics live here once. It is +explicitly NOT C21's World (a project-side program): it realizes one campaign document +and owns no topology, no data sources, no UI. + +**Realization (2026-07-12 — wiring-only tier, #241).** The smallest project +is now data-only: `Aura.toml` + `blueprints/` + `runs/`, no crate. The load +boundary tier-selects (a `[nodes]` pointer list in `Aura.toml` → load that +crate; a root `Cargo.toml` → the pre-#241 native project, unchanged; neither +→ std-vocabulary-only). `aura new` scaffolds the data-only tier; +`aura nodes new` scaffolds a node crate **beside** the project and attaches +it — the visible role-2 switch (C25 role model). Provenance widened +additively: a data-only run stamps commit-only. Invariant 9's "a project is +always a Rust crate" was deliberately amended (user decision, 2026-07-12). diff --git a/docs/design/contracts/c16-engine-project-split.md b/docs/design/contracts/c16-engine-project-split.md new file mode 100644 index 0000000..0d46146 --- /dev/null +++ b/docs/design/contracts/c16-engine-project-split.md @@ -0,0 +1,100 @@ +# C16 — Engine / project separation; three-tier node reuse + +**Guarantee.** aura is the reusable **engine**; each research project is a +separate external repo that depends on aura via cargo (the game-engine / game +split). **Node reuse is cargo-native, in three tiers:** `aura-std` (universal +blocks, shipped with the engine) / **shared node crates** (cross-project-reusable, +their own repos, pulled as cargo git deps) / **project-local node crates** +(experimental, project-specific). A reusable node is an `rlib` dependency; the +hot-reload unit is the project-side `cdylib` that composes it (C13). + +**A project is a directory anchored by a static `Aura.toml`** (project context, +paths only: data archive root, runs dir), in two tiers (#241, ratified +2026-07-12): +- **Data-only** — the default, smallest project: `Aura.toml` + `blueprints/` + + `runs/`, blueprints and research documents over the std vocabulary, **no + crate, no build step**. +- **Native** — node logic lives in **node crates**: separate `cdylib` crates + referenced via `Aura.toml [nodes]`, scaffolded and attached by a dedicated + verb (the visible role-2 switch, C25). + +The load boundary **tier-selects**: a `[nodes]` pointer list → load that crate; +a root `Cargo.toml` → the pre-#241 native project, unchanged; neither → +std-vocabulary-only. `aura new` scaffolds the data-only tier; `aura nodes new` +scaffolds a node crate beside the project and attaches it. During research the +`aura` host loads and runs the project (C13 hot-reload); for deploy the chosen +strategy + broker freeze into a standalone binary. + +**Dependency policy (amended 2026-06-10).** Dependencies are admitted by +deliberate, **per-case review** — what a crate pulls in weighed against what it +buys — with **particular scrutiny for anything that enters the frozen deploy +artifact** (C13: this bot = this commit). Well-established standard crates +(`serde`, `rayon`, …) pass that review and are used wherever they do the job, +including in the bot. There is **no blanket zero-dependency commitment** and +**no blanket admission**; hand-rolling what a vetted standard crate already does +is the anti-pattern, not the dependency. `aura-ingest` is the data-source +ingestion edge, **not** a dependency firewall. + +**Forbids.** Project-specific signals in the aura repo (it keeps at most +example/fixture nodes under `examples/` for its own tests); a multi-project +manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the +package mechanism). Dependency admission is governed by the per-case policy +above, not a blanket ban. + +**Why.** The engine/game split keeps the engine sharp and reusable while each +project versions its own research with its own forward-queue. Promotion +(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism. +The original "a project is always a Rust crate" invariant was deliberately +amended (#241) once the role model diagnosed that the smallest project carries +no native logic at all. + +## Current state + +The engine workspace carries the non-node crates beyond `aura-std`; the full +roster and its layering are C28's ladder — this contract does not re-enumerate +it. `aura-ingest` is the data-source ingestion edge, where the `data-server` +external tree enters. + +**Engine ↔ domain seam — the metric-generic engine** (decision log: +#136 → #147/#290/#291/#292). `aura-engine` is the **domain-free, metric-generic** +run loop: `RunReport` (`crates/aura-engine/src/report.rs`) is parametrized +over a metric type. The concrete trading instantiation `M = RunMetrics` and the +backtest metric reductions (`RunMetrics`, `RMetrics`, `summarize_r`, +`PositionEvent` / `PositionAction` / `derive_position_events`) live in the +outer-rung crate **`aura-backtest`** (`crates/aura-backtest/src/metrics.rs`). +The domain-neutral statistics substrate and selection provenance — the +multiple-comparison hurdle math (`inv_norm_cdf`, `expected_max_of_normals`), the +bootstrap/permutation kit (`MetricStats`, `resample_block`, `permute`), the +`MetricVocabulary` trait, and `FamilySelection` / `SelectionMode` — live in +**`aura-analysis`** (`crates/aura-analysis/src/lib.rs`). `aura-registry`'s +selection vocabulary and `aura-measurement`'s IC (`IcMetrics`) are the trait's +two production implementors. This is the realized form of the once-deferred +"make `RunReport` generic / turn the orchestration layer into a domain-agnostic +substrate" work. + +**Campaign-execution leaf.** `aura-campaign` owns campaign-execution +*semantics* — cell enumeration, preflight, gate evaluation, winner selection, +walk-forward rolling, realization assembly (C18) — as a leaf library whose deps +deliberately exclude `aura-ingest` / `aura-std` / `aura-composites`; harness +construction and data binding enter only through the one-method `MemberRunner` +seam (`&dyn MemberRunner`), so each consumer binds its own runner while the +semantics live here once. It is explicitly **not** C21's World: it realizes one +campaign document and owns no topology, data sources, or UI. + +**Two-tier scaffolding.** `aura new` emits the data-only project +(`crates/aura-cli/src/scaffold.rs`: `Aura.toml` + `blueprints/*.json` + `runs/`, +no crate); `aura nodes new` emits a node crate beside the project and appends +the `[nodes]` pointer (`crates/aura-cli/src/main.rs`; the load-boundary tier +hints live in `graph_construct.rs::tier_hint_for_type_id`). Provenance widened +additively: a data-only run stamps **commit-only** +(`crates/aura-runner/src/project.rs::Env::provenance`). + +## See also +- [C1](c01-determinism.md) — the behaviour-preserving invariant the crate extractions kept +- [C13](c13-hot-reload-frozen-deploy.md) — hot-reload authoring loop / frozen deploy artifact +- [C18](c18-registry.md) — the run registry and campaign realization assembly +- [C21](c21-world.md) — the World the campaign leaf is deliberately not +- [C25](c25-role-model.md) — the role-2 switch `aura nodes new` makes visible +- [C28](c28-stratification.md) — the full crate roster and the layer ladder + +> History: [c16-engine-project-split.history.md](c16-engine-project-split.history.md) diff --git a/docs/design/contracts/c17-authoring-surface.md b/docs/design/contracts/c17-authoring-surface.md new file mode 100644 index 0000000..210f798 --- /dev/null +++ b/docs/design/contracts/c17-authoring-surface.md @@ -0,0 +1,52 @@ +# C17 — Authoring surface + +**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** — +is authored in native Rust through **Claude Code + the skills pipeline**: the +human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI, +and reports metrics. This binds **computation** — a node's math, a +meta-program's control flow — to native Rust (the RustAst lesson, sharpened: a +small LLM cannot reliably *apply* a computational DSL, and a strong one does not +need one). It does **not** bind **topology** — which node feeds which, the +structural axes, the param-space — to Rust *source*: topology is a +**serializable data value** the World owns (C24), a non-Turing static DAG over +the closed, compiled-in node vocabulary (C8/C16) that carries no computation +(#109, resolved 2026-06-29). The no-DSL guard therefore forbids a +*computational* experiment / strategy language, not a static topology-data +format — the experiment-matrix *generator* may stay Rust (C20); its *output* is +topology-data. Declarative config (`Aura.toml`) carries only **static project +context** (paths only: data archive root, runs dir), never logic. aura ships +**no embedded coding-LLM**; IONOS LLMs are used only as a *runtime data source* +(news-agent bias, C11), gated by per-session consent, never in the code path. + +**Forbids.** An in-app LLM chat that generates node code inside aura; using +IONOS (weaker models) as the authoring brain; a *computational* experiment / +strategy DSL the author must apply (the RustAst trap) — distinct from a static +topology-data format, which is permitted. + +**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's +failure; making weaker models the coding brain reintroduces the very problem. +Keeps aura's scope an engine + playground, not an LLM-IDE. Topology-as-data is +*not* the RustAst trap re-opened: that trap is a DSL the author must *apply*; a +topology-data blueprint is machine-generated / owned and applied by no one, so +the no-DSL guard stands unbroken — a *closed-vocabulary* data artifact is not +the forbidden *open, logic-bearing* language (C25). + +## Current state + +The authoring loop is the skills pipeline over native Rust: node / strategy / +harness logic compiles into the closed node vocabulary, and the `aura` CLI runs +it. Topology-as-data is realized as **blueprints** — a data-only project's +`blueprints/*.json` are serialized static DAGs over the std vocabulary (the +`aura new` scaffold writes a starter `blueprints/signal.json`; +`crates/aura-cli/src/scaffold.rs`); the blueprint data format itself is C24's +contract. `Aura.toml` carries paths-only project context (C16). The IONOS +runtime-source path (news-agent bias) is the only external-LLM use — C11's +record-then-replay edge, consent-gated, never in the sim code path. + +## See also +- [C8](c08-node-contract.md) — the closed, compiled-in node vocabulary topology draws on +- [C11](c11-sources-record-replay.md) — IONOS news bias as a recorded runtime source +- [C16](c16-engine-project-split.md) — the engine/project split and `Aura.toml` project context +- [C20](c20-strategy-harness.md) — the experiment-matrix generator that stays Rust +- [C24](c24-blueprint-data.md) — the topology / blueprint data format the World owns +- [C25](c25-role-model.md) — a closed-vocabulary artifact vs the forbidden open DSL diff --git a/docs/design/contracts/c18-registry.history.md b/docs/design/contracts/c18-registry.history.md new file mode 100644 index 0000000..5fcb804 --- /dev/null +++ b/docs/design/contracts/c18-registry.history.md @@ -0,0 +1,273 @@ +# C18 — Project management: one repo = one project, plus a run registry: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c18-registry.md](c18-registry.md). + +**Realization (cycle 0029 — the flat run registry).** The experiments-&-results +plane shipped as `aura-registry`: an append-only JSONL store (`runs/runs.jsonl`), +one serde_json `RunReport` (`RunManifest{commit, params, window, seed, broker}` + +`RunMetrics`) per line, with a typed read-path (`load`) and best-first ranking +(`rank_by`/`optimize`). C9 holds — the registry depends on `aura-engine`, never +the reverse. + +**Realization (cycle 0045 — lineage as related records, #70).** The *lineage* +depth is now realized as a **family store**: a sweep / Monte-Carlo / walk-forward +run (the C12 axes) is persisted as a *set of related records* — each a +`FamilyRunRecord` (a `RunReport` stamped with its `family` + `run` + `kind` + +`ordinal`) — in a sibling JSONL (`families.jsonl`), leaving the flat `runs.jsonl` +path and its `append`/`load`/`rank_by`/`optimize` API byte-for-byte unchanged. +the user-facing `family_id = "{family}-{run}"` handle is **derived** from the +stored `family` name plus a per-name `run` index (assigned as a numeric max+1 — +not a content hash; re-running the same family mints a fresh id). `group_families` is the round-trip that re-derives a family +from the stored links (re-listable / rankable as a unit — C21). The manifest *is* +the re-derivation recipe (#71): no input-stream blob / path / payload enters a +record, and a member's window is **producer-supplied** via +`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage), +never a materialized-`Vec` scan at the call site. CLI surface: `aura mc`, +`aura runs families`, `aura runs family [rank ]`; `aura sweep` / +`walkforward` / `mc` persist via `append_family` with an optional `--name`. + +**Realization (cycle 0078 — cross-instrument family + instrument lineage, #146).** +The comparison axis (C12) is realized as a `FamilyKind::CrossInstrument` family: +`aura generalize` runs one candidate across an instrument list and persists the M +per-instrument runs via `append_family`, each member self-identifying through a new +first-class `RunManifest.instrument` lineage field (serde-widened with +`skip_serializing_if`, so legacy lines and every non-cross-instrument path stay +byte-identical — C14/C23). The cross-instrument *generalization score* (worst-case R +floor + sign-agreement + per-instrument breakdown) is a **recomputable aggregate** +over those members, not a persisted family-level record — distinct from #144/#145's +per-winner selection annotation on `RunManifest.selection`. +Deferred (Non-goals): replay-dedup (content-addressed *identity* shipped — #158, +cycle 0094, Realization below); the "run-diff" +depth and ranking families against each other (cross-family, vs. within-family); +and a live producer for the flat `runs.jsonl` standalone-run path — no CLI command +writes it (sweep/walkforward persist to the family store; `aura run` does not +persist). **Resolved (#73, 2026-06): retired** — `aura runs list` / `rank` dropped; +families (C21) subsume standalone over-time comparison. The `aura-registry` flat +lib API is retained: `rank_by`/`optimize` keep live consumers (`optimize` backs +walk-forward's in-sample step, `rank_by` backs `runs family … rank`); `append`/ +`load` (the flat-store half) remain public API with **no in-tree caller** after +this retire — tested, available to external consumers, a latent dead-code surface +a later sweep may revisit. **Unknown-id contract (ratified, Runway fieldtest +2026-06).** `aura runs family ` treats an unknown-but-well-formed id as an +*empty family* (prints nothing, exit 0) — the same treat-as-empty discipline as +`Registry::load` reading a missing store as `Ok(empty)`, and deliberately +distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection +*before* any store access, not a found-nothing lookup. Tightening to a non-zero +`no such family ` exit (typo-safety) is an available future UX choice, not a +current contract. + +**Refinement (2026-06-29 — a generated run's topology lives in / is addressed by +its manifest, C24).** The manifest identifies a run's topology today only via +`commit` (this engine + a hand-coded harness) — sufficient while harnesses are a +finite hand-coded menu. Once the World **generates or structurally searches** +topologies (C21/C24), `commit` no longer identifies the graph, so the manifest must +**carry or content-address the topology-data** to stay the re-derivation recipe +(C18's "reproducible from a recorded manifest"). This pulls the previously-deferred +**content-addressed identity** non-goal forward as the natural home for a generated +topology's identity. The format and the carrier are C24's design; recorded here as +the reproduction requirement it must meet. + +**Realization (2026-07-01, cycle 0094 — content-addressed reproduction of a generated +run, #158).** A blueprint sweep's topology is now **content-addressed**: the canonical +`blueprint_to_json` bytes are stored once, keyed by the `topology_hash` the manifest +already carries, in a **dumb bytes-by-key store** beside the run registry +(`runs/blueprints/.json` — `Registry::put_blueprint`/`get_blueprint`, aura-registry; +no `sha2`, no parse — the caller owns the hash, and reproduction's bit-identical compare is +the integrity check). `aura reproduce ` re-derives every persisted member: load +the member's blueprint by its `topology_hash`, reconstruct the sweep point from the +recorded params, re-run through the **same** `run_blueprint_member` the live sweep uses +(so bit-identity is by construction, C1), and compare metrics — refuse-don't-guess on an +unknown id / missing stored blueprint (exit 1 — recorded state is missing, C14's +runtime-failure class; this paragraph over-claimed exit 2 until #298 recorded the code's +actual, C14-consistent behaviour), DIVERGED → exit 1. The id resolves first as the +derived `{family}-{run}` handle; a bare enumeration name naming exactly one stored run +resolves as fallback, an ambiguous name refuses listing the candidate handles (#298 — +the list-then-reproduce seam). The manifest + the +content-addressed store + the commit are the complete re-derivation recipe (C18). One +blueprint is stored per family (all members share the signal `topology_hash` — C11/C12 +dedup). `aura graph introspect --content-id` exposes the same id for an op-script, via the +one `content_id` primitive `topology_hash` also uses (acc 1); a Tier-1 optional the +blueprint does not use leaves the id byte-stable (acc 3, composing #156/#164). +`serde_json/float_roundtrip` is enabled so stored f64 metrics round-trip exactly through +`families.jsonl` — the precondition for a bit-identical compare (C1). **Signal-only this +cycle**: the id covers the signal blueprint; the fixed r-sma scaffolding stays +commit-identified (it is not yet blueprint-data, C24). Whole-harness / structural-axis +content-addressing remains deferred. The debug-name-in-id question was settled additively +(cycle 0104, #171): the **identity id** — the canonical form with every non-load-bearing +debug symbol blanked (invariant 11), hashed through the same `content_id` primitive and +surfaced as `aura graph introspect --identity-id` beside `--content-id` (combinable) — +makes same-topology blueprints comparable across authoring paths, while the byte-exact +`topology_hash` keeps every role untouched (introspection-only: no manifest field, no +store key until a dedup consumer exists). Reproduction is proven on +synthetic (deterministic) data; recorded-dataset reproduction rides the DataServer seam +(#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` — +branching on the family kind, it reconstructs each member's seed-driven synthetic walk from +the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives +bit-identically through the same `run_blueprint_member` path. **Cycle 0097 (#173):** reproduce +spans the third variant, `FamilyKind::WalkForward` — the same branch rebuilds each OOS member's +windowed slice from `manifest.window` (winner params via the shared `manifest→cells` recovery), +so an `aura walkforward` family re-derives bit-identically too. All three family kinds now +persist *and* reproduce through the one shared `topology_hash`+`put_blueprint` hook. + +**Realization (cycle 0106, #189 — research-artifact document stores).** The registry's +content-addressed store family grew two siblings beside `blueprints/`: `processes/` and +`campaigns/` hold the two research-artifact document types shipped by the #188 role-model +pass — the **process document** (role 5: a named validation/eval methodology, a closed std +stage vocabulary wrapping shipped primitives) and the **campaign document** (role 6b: +persisted experiment intent — instruments × windows × strategy refs by content/identity id × +param axes × process ref (content-id-only) × data-level presentation). Documents are +canonical JSON (`format_version` envelope, omit-defaults, no trailing newline) keyed by the +**shared content-id primitive, now library-hosted** (`aura_research::content_id_of`; +`aura-cli`'s `content_id`/`topology_hash` delegate byte-identically — the id goldens pin the +move). Unlike `put_blueprint` (caller owns the hash), the document puts self-key from +canonical bytes; gets are `Ok(None)` treat-as-empty. The **referential tier** +(`validate_campaign_refs`) resolves process/strategy refs against the stores (identity refs +by store scan in this cycle; index-first since #191, below) and checks each campaign axis — name AND declared `ScalarKind`, the axis +carries its kind once — against the referenced blueprint's `param_space`. Campaign P1 +control constructs (axes/gates/ladders per #188) carried intent only in this cycle: no +executor existed yet — executor need was re-tested against the intent-persistence +diagnosis (#189 triage, decided item 6; the cycle-0106 fieldtest F7 verdict was that +evidence), and the v1 executor shipped the next cycle (below). + +**Realization (cycle 0107, #198/#196 — the campaign executor and its realization store).** +`aura campaign run ` executes a campaign (a file is register-then-run +sugar; the content id is canonical): zero-fault referential gate, then the process +pipeline — v1 executable shape `std::sweep [std::gate]* [std::walk_forward]?` +(`std::monte_carlo`/`std::generalize` refused loudly at preflight in this cycle; they +execute since cycle 0108, below) — runs once per (strategy, instrument, window) cell in +doc order. Execution +*semantics* live in the **`aura-campaign` library crate** (#198 home decision: reachable +beyond the CLI; NOT C21's project-side World): grid odometer over the campaign axes, +members through the engine `sweep` over a **`ListSpace`** (explicit point set beside +`GridSpace`/`RandomSpace` — a gate's survivor subset has no cartesian structure), +per-member gates via the 14-name `member_metric` roster (an R-predicate over a missing R +block fails conservatively), walk-forward re-rolled in the doc's epoch-ms unit +(`WindowRoller`; IS windows search ONLY the survivor points; OOS winner reports carry +`manifest.selection`), deflation nulls seeded from the doc's `seed` (C1: realization is a +pure function of doc + stores + data). Harness/data binding stays consumer-side behind the +one-method **`MemberRunner`** seam — the CLI binds the shipped loaded-blueprint reduce +convention with unique suffix-join of raw axis names onto the wrapped `param_space`. The +registry grew the **`campaign_runs.jsonl`** JSONL sibling (beside `runs.jsonl` / +`families.jsonl`): one thin `CampaignRunRecord` per run — campaign/process ids, seed, and +per-cell realized stage prefixes linking family ids, gate survivor ordinals, and sweep +selections — over untouched family records, run-counted per campaign id. Zero survivors +truncate a cell's realized prefix and exit 0 (a null result is a valid research result); +`emit` is honored (`family_table`/`selection_report` lines); `persist_taps` was deferred +LOUDLY on stderr in this cycle (the wiring shipped in cycle 0109, below). The **blueprint on-ramp** (#196) closes +the F5 authoring gap: `aura graph register` (store put keyed by content id == topology +hash), `aura graph introspect --params` (the RAW `param_space` namespace campaign axes +are validated against), and a blueprint-file mode on `--content-id`. The +`std::walk_forward` vocabulary was corrected to machinery-true fields +(`in_sample_ms`/`out_of_sample_ms`/`step_ms`/`mode` — `WindowRoller`'s three lengths and +both `RollMode`s; the shipped `folds` slot mapped to nothing the machinery accepts), with +a new `ZeroWalkForwardLength` intrinsic fault; wf-bearing process docs get new content ids +by design (the 0106 fieldtest corpus stays as the historical record). Known debt: +metric-roster triplication (still hand-maintained, but drift from the shipped +`aura-analysis` types is now test-caught by a cross-crate guard, #190; +single-source removal waits on #147), deflation-constant duplication (#199). + +#300 (2026-07-21) closed the store's read loop: `aura process|campaign show +` prints a registered document's canonical bytes — the +generate → retrieve → hand-extend → re-register cycle needs no direct store +filesystem access. + +**Realization (cycle 0108, #200 — the annotator stages execute).** The v2 executable +shape is `std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? +[std::generalize]?` — an ordered optional annotator suffix, each at most once, +`std::generalize` strictly last; the executor preflight refuses what the intrinsic tier +deliberately admits (`[sweep, mc, walk_forward]` is intrinsically valid — the tier +boundary is test-pinned on both sides), plus three new static guards +(single-instrument generalize, non-R generalize metric via the registry's +`check_r_metric`, zero mc `resamples`/`block_len`). **Both annotators are terminal** +(unanimous #200 triage): nothing flows out of them; filtering stays the gate's monopoly. +`std::monte_carlo` bootstraps the stage's *incoming R-evidence* with one semantics, +input-shaped by position — after a walk_forward, ONE `r_bootstrap` over the wf family's +pooled per-window OOS `net_trade_rs` in roll order (`PooledOos`; #259 materialized this +conduit as the cost-netted per-trade series `r − cost_in_r`, equal to the gross series +bit-for-bit when no cost model is bound); after sweep/gates, one `r_bootstrap` per +surviving member's fresh in-memory series (`PerSurvivor`, ordinals into the population +family; a zero-trade member records the engine's defined all-zero degenerate) — seeded +from the campaign doc's `seed` (C1; `net_trade_rs` is `#[serde(skip)]`, so annotators run +in-executor or not at all). `std::generalize` executes at **campaign +scope**: after all cells, per (strategy, window) the per-cell *nominees* (last wf +window's OOS report, else the sweep winner; none on gate truncation) across instruments +feed the shipped `generalization()` when ≥ 2 exist — divergent per-instrument winners +are exposed via their params, never averaged away; a shortfall is recorded, not computed +around. Realization: `StageRealization.bootstrap` (`StageBootstrap::PerSurvivor | +PooledOos`) and `CampaignRunRecord.generalizations` (`CampaignGeneralization` keyed +strategy × window with `winners`/`missing`), both serde-default sparse — pre-0108 +`campaign_runs.jsonl` lines parse unchanged (C14/C23). No document content id moved +(introspection doc-strings dropped "in v1" only). Noted debt: the mc arm detects the wf +family by the stringly `block == "std::walk_forward"` literal (a pre-existing lockstep +pattern with the realization's block strings). + +**Realization (cycle 0109, #201 — persist_taps wired).** Campaign presentation persists +traces: the tap namespace is a **closed vocabulary** of the wrap convention's four sink +names (`equity`/`exposure`/`r_equity`/`net_r_equity`; `aura_research::tap_vocabulary`, +intrinsic `DocFault::UnknownTap` — the escalation for a new observable is a new +vocabulary entry or an authored blueprint sink, never an open node-path namespace). +Scope is the per-cell **nominee only**: after the pipeline settles, the CLI re-runs each +nominee once in non-reduce mode (all four channels drained; windowed to the nominee +manifest's own ns bounds) and **asserts metrics equality** against the recorded nominee — +the C1 drift alarm, hard refusal on divergence (the reproduce precedent, enforced). +Traces land in the existing TraceStore as +`traces/{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json` +(`ensure_name_free(Family)` once; window ordinal doc-derived), chartable by the unchanged +viewer. The record carries ONE sparse pointer, `CampaignRunRecord.trace_name` +(`Some("{campaign8}-{run}")` iff the doc requests taps — the claim-sentinel contract: +`execute` claims, `append_campaign_run` composes the name via the single-sourced +`derive_trace_name`, `execute` mirrors it onto the returned copy). Loud lines replace +the retired deferral: per-cell no-nominee skip, per-run unproducible-tap skip +(`net_r_equity` needs a cost leg; the campaign runner wires none), one summary. +`aura-campaign` stays trace-agnostic (the MemberRunner seam is unchanged; the stamp is a +pure name derivation). Noted debt: `aura chart` over the campaign family ROOT (cells +spanning instruments) is untested/semantically undefined — only per-cell read-back is +pinned. + +**Realization (#272, 2026-07-14 — per-cell fault isolation).** A member +fault (no-data, bind, run, or a caught panic) is a recorded per-cell outcome, +never a global abort: `run_cell` returns a fault-annotated `CellRealization` +(`fault: Option`, closed `CellFaultKind`) instead of `Err`, so +`execute`'s existing accumulate-then-append-once tail persists every healthy +cell and the one run record. Containment granularity is the cell for a sweep +stage (a grid hole compromises selection) and the fold for walk_forward +(surviving folds pool; failed folds recorded as `StageRealization. +window_faults`, the summary naming the ratio). `ExecFault::Registry` and +doc-shape preflight faults stay global. The CLI declares holes (per-cell +notes + a completion summary) and a run with ≥1 failed cell exits **3** +("completed with failed cells" — distinct from 0/1/2). A partially-covered +window carries a `CellCoverage` annotation (effective bounds + interior gap +months, from the #264 archive primitives). Generalize already treats a +no-nominee cell as `missing`, so a failed cell surfaces there unchanged. +Member panics are caught with `catch_unwind`(`AssertUnwindSafe`) at the three +member-run sites and recorded as `MemberFault::Panic`; a ref-counted +`SilencedPanic` guard (a process-global panic-hook save/no-op/restore behind a +`static Mutex`, held only around each `catch_unwind`) suppresses the default +crash backtrace so "recorded, campaign continues" is observably true on stderr, +not merely true in the record. The guard's mutex serialises only the +ref-count/hook-swap (O(1)), never the member computation, so C1 disjoint-parallel +execution and determinism are preserved; ref-counting (save on 0→1, restore on +1→0) keeps concurrent sweep/walk-forward threads and any caller-installed hook +correct. + +**Realization (#191, 2026-07-17 — identity-ref resolution is index-first).** +`find_blueprint_by_identity` no longer re-loads the whole blueprint store per +reference: a fourth persistent JSONL sidecar, `blueprint_identity_index.jsonl` +(identity id → content id; fixed-name sibling of the runs store, appends under +the #276 lock), is consulted first, and every hit is **verified** by loading +that one blueprint under the current resolver and recomputing its identity id — +the index is a cache, never an oracle, so resolution results stay +scan-identical under roster drift, store surgery, or index corruption (the one +unspecified corner is unchanged in kind: which same-identity twin answers was +`read_dir`-order-dependent before and is index-history-dependent now). Any miss +or failed verification runs the old scan as a **full-store repair pass**, +collect-then-diff-append: the walk's last-wins mapping is diffed against the +pre-walk snapshot after the walk, so a converged index — twin stores included — +appends nothing (the twin-convergence pin). Index reads never fail a lookup +(missing/unreadable file → empty, unparseable lines skipped), repair appends +are best-effort, and a pre-index store backfills itself on its first miss, no +migration; a read-only store keeps scanning as before. Write paths, the engine, +and both callers are untouched; maintenance is lazy-only — put-time indexing +was rejected because it would need a roster-free doc-level identity function +whose equivalence to the loaded-composite path no green test ratifies (decision +log: #191 comments). diff --git a/docs/design/contracts/c18-registry.md b/docs/design/contracts/c18-registry.md new file mode 100644 index 0000000..8113274 --- /dev/null +++ b/docs/design/contracts/c18-registry.md @@ -0,0 +1,287 @@ +# C18 — Project management: one repo = one project, plus a run registry + +**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git +(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as +the forward-queue, a research thrust = a milestone, the +`idea → experimental → validated → deployed` label gradient). (2) **Experiments +& results:** an Aura-native **run registry** — one record per run = a *manifest* +(node-commit + params + data-window + seed + broker profile + instrument + +`topology_hash` + selection) + *metrics*, queryable, with *lineage* (composite ← +signals; run ← inputs). Determinism (C1/C12) makes a run reproducible from its +tiny manifest, so the registry stores manifests + metrics and re-derives full +results on demand. Because the World may **generate or structurally search** +topologies, `commit` alone no longer identifies the graph: the manifest carries +a `topology_hash` and the registry keeps the canonical topology bytes in a +content-addressed store beside it, so the **manifest + the content-addressed +store + the commit** are the complete re-derivation recipe (C24). Depth: +**structured** (promotion/status, lineage, run-diff). + +**Forbids.** Storing results not reproducible from a recorded manifest; +duplicating git/Gitea inside aura; a multi-project workspace manager. + +**Why.** Comparing experiments over time is the heart of the research loop and +has no home in git/Gitea; determinism makes a structured registry cheap. Every +run emits a manifest + metrics, and the registry/index is built over manifests +that already exist. Content-addressed identity is the natural home for a +generated topology's identity: once a graph is no longer fully named by its +`commit`, the re-derivation recipe must carry or content-address the +topology-data to remain "reproducible from a recorded manifest" — the format and +carrier are C24's design, the reproduction requirement is C18's. + +## Current state + +**Two planes, one crate.** The code/forward-queue plane is git + Gitea, unchanged +by aura. The experiments-&-results plane is the run registry in `aura-registry` +(C9: it depends on `aura-engine`, never the reverse). + +**The store family.** All members are fixed-name siblings of the runs-store path +(`Registry::path.with_file_name(...)`, `aura-registry`): + +- `runs.jsonl` — the append-only flat store, one `RunReport` per line + (`RunManifest` + `RunMetrics`), with a typed read-path (`load`) and best-first + ranking (`rank_by`/`optimize`). No live producer writes it today — sweep / + walk-forward / mc persist to the family store and `aura run` does not persist — + so the flat lib API is retained but selectively live: `rank_by` backs + `aura runs family … rank`, `optimize` backs walk-forward's in-sample step + (`aura-campaign` and the registry's own selection helper both call it), while + `append`/`load` remain public API with no in-tree caller (a latent surface for + external consumers). +- `families.jsonl` — the family store. A sweep / Monte-Carlo / walk-forward / + cross-instrument run persists as a *set of related records*, each a + `FamilyRunRecord` (a `RunReport` stamped with `family` + `run` + `kind` + + `ordinal`; `FamilyKind ∈ {Sweep, MonteCarlo, WalkForward, CrossInstrument}`). + `group_families` re-derives a family from the stored links (re-listable / + rankable as a unit — C21). The user-facing `family_id = "{family}-{run}"` handle + is **derived** from the stored `family` name plus a per-name `run` index + (numeric max+1 — not a content hash, so re-running the same family mints a fresh + id). CLI: `aura runs families`, `aura runs family [rank ]`; + `aura sweep`/`walkforward`/`mc` persist via `append_family` with an optional + `--name`. +- `campaign_runs.jsonl` — one thin `CampaignRunRecord` per campaign run (below), + over untouched family records. +- `blueprints/.json`, `processes/`, `campaigns/` — the content-addressed + document stores (below). +- `blueprint_identity_index.jsonl` — the identity-ref resolution cache (below). + +**The manifest is the re-derivation recipe.** No input-stream blob / path / +payload enters a record; a member's data window is **producer-supplied** via +`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage), never +a materialized-`Vec` scan at the call site. New manifest fields (e.g. the +first-class `instrument` lineage field) are serde-widened with +`skip_serializing_if`, so legacy lines and every path that does not set them stay +byte-identical (C14/C23). `serde_json/float_roundtrip` is enabled so stored f64 +metrics round-trip exactly through `families.jsonl` — the precondition for a +bit-identical compare (C1). + +**Content-addressed reproduction.** A run's topology is content-addressed: the +canonical `blueprint_to_json` bytes are stored once, keyed by the `topology_hash` +the manifest carries, in a **dumb bytes-by-key store** +(`Registry::put_blueprint`/`get_blueprint` → `blueprints/.json`; no `sha2`, +no parse — the caller owns the hash and reproduction's bit-identical compare is +the integrity check). One blueprint is stored per family (all members share the +signal `topology_hash` — C11/C12 dedup). `aura reproduce ` re-derives **every** +persisted member: load its blueprint by `topology_hash`, reconstruct the member +(a `Sweep` point from the recorded params; a `MonteCarlo` member's seed-driven +synthetic walk from `manifest.seed`; a `WalkForward` OOS member's windowed slice +from `manifest.window`, winner params via the shared manifest→cells recovery), +re-run through the **same** `run_blueprint_member` the live path uses (bit-identity +by construction — C1), and compare metrics. All three family kinds persist *and* +reproduce through the one shared `topology_hash`+`put_blueprint` hook. Refuse- +don't-guess (`aura_runner::reproduce`): an unknown id, a missing stored blueprint, +or a DIVERGED compare each exit **1** — recorded state missing or mismatched is +C14's runtime-failure class. Id resolution: first as the derived `{family}-{run}` +handle; a bare enumeration name naming exactly one stored run resolves as fallback; +an ambiguous name refuses, listing the candidate handles (the list-then-reproduce +seam, #298). The recipe's scope is **signal-only** today: content-addressing covers +the signal blueprint, while the fixed scaffolding stays commit-identified (not yet +blueprint-data, C24); whole-harness / structural-axis content-addressing is +deferred. Reproduction is proven on synthetic (deterministic) data; +recorded-dataset reproduction rides the DataServer seam (#124). + +**The one `content_id` primitive.** `topology_hash`, the document store keys, and +`aura graph introspect --content-id`/`--identity-id` all route through +`aura_research::content_id_of` (aura-cli's `content_id`/`topology_hash` delegate +byte-identically). The **identity id** is the canonical form with every +non-load-bearing debug symbol blanked (invariant 11 / C23), hashed through the same +primitive: same-topology blueprints become comparable across authoring paths, while +the byte-exact `topology_hash` keeps every debug role untouched (introspection-only +— no manifest field, no store key, until a dedup consumer exists). A Tier-1 optional +the blueprint does not use leaves the id byte-stable. `--content-id` and +`--identity-id` are combinable. + +**Cross-instrument generalization.** `FamilyKind::CrossInstrument`: `aura generalize` +runs one candidate across an instrument list and persists the M per-instrument runs +via `append_family`, each member self-identifying through `RunManifest.instrument`. +The generalization score (worst-case R floor + sign-agreement + per-instrument +breakdown) is a **recomputable aggregate** over those members, not a persisted +family-level record. + +**Research-artifact document stores.** `processes/` and `campaigns/` hold two +document types (C25 roles 5 / 6b): the **process document** (a named +validation/eval methodology — a closed std stage vocabulary wrapping shipped +primitives) and the **campaign document** (persisted experiment intent — +instruments × windows × strategy refs by content/identity id × param axes × process +ref (content-id-only) × data-level presentation). Documents are canonical JSON +(`format_version` envelope, omit-defaults, no trailing newline) keyed by the shared +content-id primitive; unlike `put_blueprint` (caller owns the hash) the document +puts **self-key** from their canonical bytes, and gets are `Ok(None)` +treat-as-empty. The **referential tier** (`validate_campaign_refs`) resolves +process/strategy refs against the stores (identity refs index-first, below) and +checks each campaign axis — name AND declared `ScalarKind`, the axis carrying its +kind once — against the referenced blueprint's `param_space`. `aura process|campaign +show ` prints a registered document's canonical bytes (#300), so the +generate → retrieve → hand-extend → re-register loop needs no direct store +filesystem access. + +**The campaign executor.** `aura campaign run ` executes a +campaign (a file is register-then-run sugar; the content id is canonical): a +zero-fault referential gate, then the process pipeline. The executable shape is +`std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? [std::generalize]?` +— an ordered optional annotator suffix, each at most once, `std::generalize` +strictly last. The executor preflight is deliberately stricter than the intrinsic +tier (`[sweep, mc, walk_forward]` is intrinsically valid — the tier boundary is +test-pinned on both sides), plus static guards: single-instrument generalize, a +non-R generalize metric (via the registry's `check_r_metric`), zero mc +`resamples`/`block_len`, and `ZeroWalkForwardLength`. Execution *semantics* live in +the **`aura-campaign`** library crate (reachable beyond the CLI; NOT C21's +project-side World): a grid odometer over the campaign axes, members through the +engine `sweep` over a **`ListSpace`** (an explicit point set beside +`GridSpace`/`RandomSpace` — a gate's survivor subset has no cartesian structure), +per-member gates via the 14-name `member_metric` roster (an R-predicate over a +missing R block fails conservatively), walk-forward re-rolled in the doc's epoch-ms +unit (`WindowRoller`; IS windows search only the survivor points; OOS winner reports +carry `manifest.selection`), deflation nulls seeded from the doc's `seed` — the whole +realization is a pure function of doc + stores + data (C1). Harness/data binding +stays consumer-side behind the one-method **`MemberRunner`** seam (the shipped +implementation is `aura_runner::DefaultMemberRunner`; the CLI binds the loaded- +blueprint reduce convention with a unique suffix-join of raw axis names onto the +wrapped `param_space`). The `campaign_runs.jsonl` sibling records one thin +`CampaignRunRecord` per run — campaign/process ids, seed, and per-cell realized +stage prefixes linking family ids, gate survivor ordinals, and sweep selections — +run-counted per campaign id. Zero survivors truncate a cell's realized prefix and +exit 0 (a null result is a valid research result); `emit` is honored +(`family_table`/`selection_report` lines). The **blueprint on-ramp** (#196): +`aura graph register` (store put keyed by content id == topology hash), +`aura graph introspect --params` (the raw `param_space` namespace axes validate +against), and a blueprint-file mode on `--content-id`. `std::walk_forward`'s +machinery-true fields are `in_sample_ms`/`out_of_sample_ms`/`step_ms`/`mode` +(`WindowRoller`'s three lengths + both `RollMode`s). + +**Annotators are terminal.** Nothing flows out of an annotator; filtering stays the +gate's monopoly. `std::monte_carlo` bootstraps the stage's *incoming* R-evidence +with one semantics, input-shaped by position: after a walk_forward, one +`r_bootstrap` over the wf family's pooled per-window OOS `net_trade_rs` in roll order +(`StageBootstrap::PooledOos`; the conduit is the cost-netted per-trade series +`r − cost_in_r`, equal to the gross series bit-for-bit when no cost model is bound, +#259); after sweep/gates, one `r_bootstrap` per surviving member's fresh in-memory +series (`StageBootstrap::PerSurvivor`, ordinals into the population family; a +zero-trade member records the engine's defined all-zero degenerate) — seeded from the +doc's `seed` (`net_trade_rs` is `#[serde(skip)]`, so annotators run in-executor or +not at all). `std::generalize` executes at **campaign scope**: after all cells, per +(strategy, window) the per-cell *nominees* (last wf window's OOS report, else the +sweep winner; none on gate truncation) across instruments feed the shipped +`generalization()` when ≥ 2 exist — divergent per-instrument winners are exposed via +their params, never averaged away; a shortfall is recorded, not computed around. +`StageRealization.bootstrap` and `CampaignRunRecord.generalizations` +(`CampaignGeneralization` keyed strategy × window with `winners`/`missing`) are +serde-default sparse (C14/C23). Known debt: the mc arm detects the wf family by the +stringly `block == "std::walk_forward"` literal. + +**Per-cell fault isolation (#272).** A member fault (no-data, bind, run, or a caught +panic) is a recorded per-cell outcome, never a global abort: `run_cell` returns a +fault-annotated `CellRealization` (`fault: Option`, closed +`CellFaultKind`) instead of `Err`, so `execute`'s accumulate-then-append-once tail +persists every healthy cell and the one run record. Containment granularity is the +cell for a sweep stage (a grid hole compromises selection) and the fold for +walk_forward (surviving folds pool; failed folds recorded as +`StageRealization.window_faults`, the summary naming the ratio). `ExecFault::Registry` +and doc-shape preflight faults stay global. The CLI declares holes (per-cell notes + +a completion summary) and a run with ≥ 1 failed cell exits **3** ("completed with +failed cells" — distinct from 0/1/2). A partially-covered window carries a +`CellCoverage` annotation (effective bounds + interior gap months, #264). Generalize +already treats a no-nominee cell as `missing`, so a failed cell surfaces there +unchanged. Member panics are caught with `catch_unwind(AssertUnwindSafe)` at the +three member-run sites and recorded as `MemberFault::Panic`; a ref-counted +`SilencedPanic` guard (a process-global panic-hook save/no-op/restore behind a +`static Mutex`, held only around each `catch_unwind`) suppresses the default crash +backtrace so "recorded, campaign continues" is observably true on stderr. The guard's +mutex serialises only the O(1) ref-count/hook-swap, never the member computation, so +C1 disjoint-parallel execution and determinism hold; ref-counting (save on 0→1, +restore on 1→0) keeps concurrent threads and any caller-installed hook correct. + +**Persisted taps (#201).** Campaign presentation persists traces. The tap namespace +is a **closed vocabulary** of the wrap convention's four sink names +(`equity`/`exposure`/`r_equity`/`net_r_equity`; `aura_research::tap_vocabulary`, +intrinsic `DocFault::UnknownTap` — the escalation for a new observable is a new +vocabulary entry or an authored blueprint sink, never an open node-path namespace). +Scope is the per-cell **nominee only**: after the pipeline settles the CLI re-runs +each nominee once in non-reduce mode (all four channels drained, windowed to the +nominee manifest's own ns bounds) and **asserts metrics equality** against the +recorded nominee — the C1 drift alarm, a hard refusal on divergence (the reproduce +precedent, enforced). Traces land in the existing `TraceStore` as +`traces/{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json`, chartable by the +unchanged viewer. The record carries one sparse pointer, `CampaignRunRecord.trace_name` +(`Some("{campaign8}-{run}")` iff the doc requests taps — the claim-sentinel contract: +`execute` claims, `append_campaign_run` composes the name via the single-sourced +`derive_trace_name`, `execute` mirrors it onto the returned copy). `aura-campaign` +stays trace-agnostic (the `MemberRunner` seam is unchanged; the stamp is a pure name +derivation). Loud stderr lines cover a per-cell no-nominee skip and a per-run +unproducible-tap skip (`net_r_equity` needs a cost leg the campaign runner wires none +of). Known debt: `aura chart` over the campaign family ROOT (cells spanning +instruments) is untested / semantically undefined — only per-cell read-back is pinned. + +**Identity-ref resolution is index-first (#191).** `find_blueprint_by_identity` +consults the persistent `blueprint_identity_index.jsonl` sidecar (identity id → +content id; a fixed-name sibling of the runs store, appended under the #276 lock) +first, and **verifies every hit** by loading that one blueprint under the current +resolver and recomputing its identity id — the index is a cache, never an oracle, so +resolution stays scan-identical under roster drift, store surgery, or index corruption +(the one unspecified corner — which same-identity twin answers — is unchanged in kind: +`read_dir`-order-dependent before, index-history-dependent now). Any miss or failed +verification runs the old full-store scan as a **repair pass**, collect-then-diff- +append: the walk's last-wins mapping is diffed against the pre-walk snapshot, so a +converged index — twin stores included — appends nothing (the twin-convergence pin). +Index reads never fail a lookup (missing/unreadable → empty, unparseable lines +skipped); repair appends are best-effort; a pre-index store backfills on its first +miss (no migration); a read-only store keeps scanning. Write paths, the engine, and +both callers are untouched; maintenance is lazy-only — put-time indexing was rejected +because it would need a roster-free doc-level identity function whose equivalence to +the loaded-composite path no green test ratifies (decision log: #191). + +**Retired verbs and the unknown-id contract.** Standalone `aura runs list` / `rank` +are retired (#73): families (C21) subsume standalone over-time comparison. The +unknown-id contract (ratified, Runway fieldtest 2026-06) is live law: `aura runs +family ` treats an unknown-but-well-formed id as an **empty family** (prints +nothing, exit 0) — the same treat-as-empty discipline as `Registry::load` reading a +missing store as `Ok(empty)`. This is deliberately distinct from `aura reproduce +`, which refuses an unknown id with exit 1 (reproduction of a named-but-absent +family is missing recorded state, not a found-nothing lookup). Tightening `runs +family` to a non-zero `no such family ` exit (typo-safety) is an available future +UX choice, not a current contract. + +**Deferred.** The **run-diff** depth and **cross-family** ranking (families against +each other, vs. within-family) remain deferred; whole-harness / structural-axis +content-addressing remains deferred (C24). Known debt across the campaign stack: +metric-roster triplication (test-caught by the #190 cross-crate guard; single-source +removal waits on #147) and deflation-constant duplication (#199). + +## See also +- [C1](c01-determinism.md) — determinism / bit-identity, the reproduction and + drift-alarm correctness invariant +- [C8](c08-node-contract.md) — sinks are the recording mechanism into the registry +- [C9](c09-fractal-composition.md) — the registry depends on the engine, never the + reverse +- [C11](c11-sources-record-replay.md) — record-then-replay; producer-supplied + windows; C11/C12 dedup +- [C12](c12-atomic-sim-unit.md) — the four orchestration axes (sweep / MC / + walk-forward / comparison) the family store persists +- [C14](c14-headless-two-faces.md) — additive serde back-compat and the + runtime-failure exit class +- [C21](c21-world.md) — the World; families as the re-listable unit +- [C23](c23-graph-compilation.md) — names non-load-bearing; the identity-id blanking +- [C24](c24-blueprint-data.md) — the blueprint as serializable data; the topology + data format the manifest content-addresses +- [C25](c25-role-model.md) — the role model; process/campaign documents as + closed-vocabulary artifacts + +> History: [c18-registry.history.md](c18-registry.history.md) diff --git a/docs/design/contracts/c19-bootstrap.history.md b/docs/design/contracts/c19-bootstrap.history.md new file mode 100644 index 0000000..33a08ac --- /dev/null +++ b/docs/design/contracts/c19-bootstrap.history.md @@ -0,0 +1,118 @@ +# C19 — Bootstrap: blueprint → instance (recursive): history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c19-bootstrap.md](c19-bootstrap.md). + +**Realization (cycle 0016 — param-set injection).** The bootstrap now binds an +injected param-set, realizing C12's "params injected at graph build (the optimizer +sees a generic vector of typed ranges)" and C19's "factory `params → sized node`" +*literally*: a blueprint leaf is **value-empty** — `BlueprintNode::Leaf` holds a +`LeafFactory { name, params, build }` recipe, not a built node — and the value lives +only in the injected vector (no baked default), so the blueprint stays a pure +param-generic recipe. (Renamed in cycle 0024: `BlueprintNode::Primitive` holds a +`PrimitiveBuilder { name, schema, build }` — the recipe now carries the full +signature, see the C8 0024 realization; `bootstrap_with_params`/`compile_with_params` +moved onto `Composite` when `struct Blueprint` collapsed into it, see the C19 0024 +realization.) `bootstrap_with_params(Vec)` / +`compile_with_params` **build each leaf through its own constructor** (the single +sizing/validation gate) from its kind-checked slice **while lowering** (build-then- +wire), consuming the vector slot-by-slot in the *same* depth-first walk +`param_space()` projects — so the two share one traversal (subsuming the #34 dual- +traversal hazard) and the value reaches the node at the slot the sweep enumerates. +Arity is checked up front (`param_space().len()`); a wrong-kind or wrong-length +vector is a typed `CompileError::{ParamKindMismatch, ParamArity}` (the typed-value +check C8 deferred). The lowering/edge/source rewrite is structurally unchanged, so +the **flat graph stays bit-identical** for a given point (C23, the correctness +invariant). The value *domain* (e.g. `length ≥ 1`) stays the constructor's own +`assert`; the search-range is still the run's (#32/C20). One value-empty leaf +detail for C22: the blueprint view (pre-run, param-generic) labels a leaf by **bare +type** (`PrimitiveBuilder::label`, was `LeafFactory::label`, → `[SMA]`) — the +value-bearing `SMA(2)` of C8's render- +label refinement now appears only in the *compiled* view (built nodes, `Node::label`). + +**Realization (cycle 0017 — blueprint render = main graph + definitions).** The +`aura graph` blueprint view (C9 graph-as-data, #13) renders the **authored +structure** as a *program with subroutines*: a flat **main graph** wiring the +harness with each composite shown as a **single opaque node** `[name]`, plus a +`where:` section that defines each **distinct** composite type **once** (its +interior with named input-entry nodes and outputs folded onto their producers as +`name := …` bindings (render refined through cycles 0019–0022; originally +`[in:k]`/`[out]` port markers); deduped by `name()`, collected +recursively so nested composites are opaque nodes with their own definitions). This +supersedes #13's original cluster-box model and is the durable split this view +realizes: **blueprint = source** (composites as named subroutines, body once) vs. +**compiled = inlined machine form** (C23, boundary dissolved). The substantive cause +for retiring cluster boxes is a real renderer defect — `ascii-dag` 0.9.1's subgraph +level-centering rounds sibling x-positions with `/2`, overlapping wide sibling +labels (width/parity-sensitive, no config/padding dodge surviving unequal-width +siblings); the **flat** layout is collision-free, and both views now build flat +graphs only. The model also scales (blueprint size tracks top-level wiring, not +inlined node count) and removes #13's nested-composite `unimplemented!` (the +definitions pass recurses). The interactive *enter/focus* counterpart (a composite +collapsed to a navigable node) is the playground's, parked as a separate concern +(#37); the static CLI keeps the all-at-once definitions form (#38). + +**Realization (cycle 0024 — the root is the fully-bound composite; the flat graph is +a named type).** `struct Blueprint` is **deleted**: the root graph IS a `Composite`, +and `compile_with_params`/`bootstrap_with_params`/`param_space` are its methods. +What distinguished the root — its bound data sources — is now a property of its input +roles: `Role` carries `source: Option` (`None` = an open interior port, +wired by the enclosing graph; `Some(kind)` = a bound ingestion feed). A composite is +**runnable iff every root role is bound** (C3: sources bind at ingestion only); an +open root role is a compile-time `CompileError::UnboundRootRole`. So the "main graph" +is no longer a separate kind — only the composite all of whose roles are +source-bound, governed by the same conditions as any other node. Compilation now +targets a **named type**: `compile` validates structurally **pre-build** (via +`signature()`, no node constructed — an ill-typed wiring is caught before any build +closure fires) and emits `FlatGraph { nodes, signatures, sources, edges }` — the +C23 flat graph, now first-class — which `Harness::bootstrap` consumes (kinds/firing +from the carried signatures, buffer depth from `lookbacks()`). The per-flat-node +signature travels beside the node, so `bootstrap` reads it without a built-node +`schema()` call (which no longer exists, see the C8 0024 realization). + +**Realization (cycle 0026 — graph render redesign: model + WASM-Graphviz viewer, +#51).** `aura graph` no longer renders ASCII. The render path is now two pieces: +a read-only **model serializer** (`aura_engine::model_to_json`, iteration 1) that +walks the root composite + every distinct composite type into a deterministic, +hand-rolled JSON model (C14, golden-tested; the engine's last hand-rolled JSON +writer after `RunReport::to_json` moved to serde in cycle 0033; the +swapped-param mis-wire property moved here from the old compiled-view test), and a +**self-contained HTML viewer** (`aura-cli::render::render_html`, iteration 2) that +inlines that model, the ported prototype viewer JS, and a vendored Graphviz-WASM +blob into one page emitted to stdout. Layout/SVG happen in the browser via +WebAssembly — aura ships no layout engine and stays a serializer (C9: graph-as-data, +no `eval`/build on the path). The viewer is a render asset (C10 — no node/strategy +logic, no DSL); it labels every input pin from the model's now-real names (C23 +debug symbols, named in cycle 0027) and colours wires by the four scalar base +types (C4). This **retires `ascii-dag`** and its adapter (`graph.rs`), the +`--compiled`/`--macd` flag plumbing, and the invented `#Sf`/`:=`/`histogram →` +notation — superseding the cycle-0017 flat-ascii model and its renderer-defect +workaround. The DOT/SVG are Graphviz-version-dependent and not golden-tested; the +deterministic JSON model is the asserted contract. + +**Realization (cycle 0034 — structural-constant bind: a knob *removed* from +param_space, #55).** `PrimitiveBuilder::bind(slot, value)` adds the **third** param +category beside the topology factory-arg (C7/C19) and the tuning param (the +cycle-0016 value-pin): a **structural constant**. The cycle-0016 binding *pins* a +value in the injected vector while the knob **stays** in `param_space` (a tuning +param the sweep varies); `bind` instead **removes** the slot from `param_space` +entirely — the knob is gone, not fixed. The discriminator is the #55 +deform-vs-tune test: a value whose variation yields another valid point of the +*same* strategy is a **tuning param** (stays in `param_space`); a value whose +variation *deforms* the strategy into a different one (e.g. the `2` of an +"SMA2-entry" bound to its two-candle construction) is a **structural constant** +(bound out), so a sweep never enumerates deformed strategies as valid family +members. Mechanically `bind` shrinks the builder's declared param surface +(`schema.params`) and wraps its build closure to re-splice the constant at its +original positional slot; the construction layer +(`collect_params`/`lower_items`/`param_space`/`compile_with_params`) is +**byte-unchanged** — both dock sites already key off `builder.params()`, so the +shrink propagates for free, and chained binds reconstruct the correct positional +vector because each layer computes its slot index relative to the param list it +sees. C23 is unaffected: `bind` resolves the param **name** to a position at +**authoring** time (the by-name authoring address space, the 0032 amendment) and +the flat graph stays wired by raw index — the name never reaches it. The +complementary question — exporting a **named frozen** strategy (all/most knobs +bound) as a reusable blueprint *value* — is deferred (#60); `bind` ships only the +per-knob overlay, no registry (C9/C10 intact). diff --git a/docs/design/contracts/c19-bootstrap.md b/docs/design/contracts/c19-bootstrap.md new file mode 100644 index 0000000..1a8aa0f --- /dev/null +++ b/docs/design/contracts/c19-bootstrap.md @@ -0,0 +1,118 @@ +# C19 — Bootstrap: blueprint → instance (recursive) + +**Guarantee.** Construction is a distinct phase, recursive at every level. Each +node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)` +sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic +graph-as-data produced by running a Rust builder (C9); it carries *free* numeric +params (declared ranges) and *free* input roles. The **bootstrap** binds +`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen +instance** — buffers sized, topology fixed. This is precisely the "wiring / graph +build" that C7 ("sized at wiring", "topology frozen per sim") and C12 ("params +injected at graph build") reference; the same machinery applies recursively up to +the harness (C20). A sweep builds many instances from one blueprint; instances are +disjoint (C1). + +This binding is a **compilation**: the param-generic, named blueprint (the source) +is lowered to a flat, type-erased **`FlatGraph`** (C23) **wired by raw index, not +by name** (`Edge { from, to, slot, from_field }`) — composite boundaries dissolve +entirely and field / role names are demoted to **non-load-bearing** debug symbols +(exactly as `FieldSpec.name` already is, C8/C23). One narrow exception (#275): a +`SourceSpec.role` — the lowered bound-`Role` name — is **load-bearing for source +binding**, the key `Harness::run_bound` / `bind_sources` resolve a keyed source +supply against; every other flat-graph name (edges, ports, composite boundaries) +stays a non-load-bearing debug symbol, and the raw-index positional `run` path +carries no role. "No recompile" means no **Rust / cdylib** rebuild (C12/C13: the +cdylib loads once); re-deriving an instance per param-set is a cheap **graph +re-compilation**, not a code recompile. + +**Forbids.** Params that change topology (a topology change is a *different* +blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a sim +against an un-bootstrapped blueprint. + +**Why.** Separating the param-generic blueprint from the param-bound instance is +what makes one strategy reusable across a whole sweep and lets the optimizer mutate +"the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead of +rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12 +explicit — and naming it a *compilation* makes its successor explicit: the flat +graph is the target of behaviour-preserving optimisation (C23). + +## Current state + +**Value-empty recipe.** A blueprint leaf is **value-empty** — `BlueprintNode::Primitive` +holds a `PrimitiveBuilder { name, schema, build }` recipe +(`crates/aura-core/src/node.rs`), not a built node — and the tuning value lives only +in the injected param vector (no baked default), so the blueprint stays a pure +param-generic recipe. This realizes C12's "params injected at graph build" and C19's +factory `params → sized node` literally. `compile_with_params(&[Scalar])` / +`bootstrap_with_params(Vec)` on `Composite` +(`crates/aura-engine/src/blueprint.rs`) **build each leaf through its own +constructor** — the single sizing/validation gate — from its kind-checked slice +*while lowering* (build-then-wire), consuming the vector slot-by-slot in the *same* +depth-first walk `param_space()` projects, so the two share one traversal and the +value reaches the node at the slot the sweep enumerates. Arity is checked up front +(`param_space().len()`); a wrong-kind or wrong-length vector is a typed +`CompileError::{ParamKindMismatch, ParamArity}`. The value *domain* (e.g. +`length ≥ 1`) stays the constructor's own `assert`; the search-range is the run's +(#32/C20, still pending). The lowering is structurally invariant under the injected +point, so the **flat graph stays bit-identical** for a given point (C23, the +correctness invariant). + +**The root is the fully-bound composite.** There is no separate `Blueprint` type: +the root graph **is** a `Composite`, and `compile` / `compile_with_params` / +`bootstrap_with_params` / `param_space` are its methods. What once distinguished the +root — its bound data sources — is a property of its input roles: `Role` carries +`source: Option` (`crates/aura-engine/src/blueprint.rs`) — `None` = an +open interior port wired by the enclosing graph, `Some(kind)` = a bound ingestion +feed (C3: sources bind at ingestion only). A composite is **runnable iff every root +role is bound**; an open root role is a compile-time `CompileError::UnboundRootRole`. +`compile` validates structurally **pre-build** (via `signature()`, no node +constructed — an ill-typed wiring is caught before any build closure fires) and +emits the `FlatGraph`, which `Harness::bootstrap` consumes (kinds/firing from the +carried signatures, buffer depth from `lookbacks()`); the per-flat-node signature +travels beside the node, so `bootstrap` needs no built-node `schema()` call. + +**`FlatGraph` shape.** `FlatGraph { nodes, signatures, sources, edges, taps }` +(`crates/aura-engine/src/harness.rs`): `signatures[i]` is the static `NodeSchema` of +`nodes[i]` gathered at lowering; `sources` are the lowered bound roles in +role-declaration order; `taps` are the declared measurement points (C27), with +interior-composite taps hoisted to the root list. + +**Structural-constant bind (#55).** `PrimitiveBuilder::bind(slot, value)` +(`crates/aura-core/src/node.rs`) adds the **third** param category beside the +topology factory-arg (C7/C19) and the tuning param (the value-pin): a **structural +constant**. Pinning a value in the injected vector leaves the knob **in** +`param_space` (a tuning param the sweep varies); `bind` instead **removes** the slot +from `param_space` entirely — the knob is gone, not fixed. The discriminator is the +deform-vs-tune test: a value whose variation yields another valid point of the +*same* strategy is a **tuning param** (stays in `param_space`); a value whose +variation *deforms* the strategy into a different one (e.g. the `2` of an +"SMA2-entry" bound to its two-candle construction) is a **structural constant** +(bound out), so a sweep never enumerates deformed strategies as valid family +members. Mechanically `bind` shrinks the builder's declared param surface +(`schema.params`) and wraps its build closure to re-splice the constant at its +original positional slot; both dock sites already key off `builder.params()`, so the +construction layer is byte-unchanged and chained binds reconstruct the correct +positional vector (each layer computes its slot index relative to the param list it +sees). `bind` resolves the param **name** to a position at **authoring** time (the +by-name authoring address space) and the flat graph stays wired by raw index — the +name never reaches it (C23 unaffected). The complementary export of a **named +frozen** strategy (all/most knobs bound) as a reusable blueprint *value* is deferred +(#60); `bind` ships only the per-knob overlay, no registry (C9/C10 intact). + +**Graph render.** `aura graph` renders no ASCII: the render path is a deterministic +JSON **model serializer** (`aura_engine::model_to_json`, +`crates/aura-engine/src/graph_model.rs`) plus a self-contained WASM-Graphviz **HTML +viewer** (`aura-cli::render::render_html`) that inlines the model and lays it out in +the browser — aura ships no layout engine (C9: graph-as-data). This render surface, +and the debug-symbol labels it draws, belongs to C9/C22. + +## See also +- [C7](c07-scalar-soa.md), [C9](c09-fractal-composition.md), [C12](c12-atomic-sim-unit.md) — sized-at-wiring, graph-as-data, params-injected-at-build that this phase names. +- [C8](c08-node-contract.md) — the `PrimitiveBuilder` / `NodeSchema` recipe and `FieldSpec.name` as the non-load-bearing precedent. +- [C13](c13-hot-reload-frozen-deploy.md) — "no recompile" means no cdylib rebuild. +- [C20](c20-strategy-harness.md) — the harness is the recursive top of the bootstrap. +- [C22](c22-playground-traces.md) — the graph render / viewer surface. +- [C23](c23-graph-compilation.md) — the flat graph as the optimisation target. +- [C24](c24-blueprint-data.md) — the deferred named-frozen-blueprint value (#60). + +> History: [c19-bootstrap.history.md](c19-bootstrap.history.md) diff --git a/docs/design/contracts/c20-strategy-harness.history.md b/docs/design/contracts/c20-strategy-harness.history.md new file mode 100644 index 0000000..f3668aa --- /dev/null +++ b/docs/design/contracts/c20-strategy-harness.history.md @@ -0,0 +1,36 @@ +# C20 — Strategy ↔ harness; the harness is the root sim graph: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c20-strategy-harness.md](c20-strategy-harness.md). + +**Realization (GER40 session-breakout blueprint milestone, 2026-06-17 — refs +#94/#96/#97, spec 0051).** Made concrete on the first real-data strategy: a +**hand-wired `FlatGraph` is not a shippable strategy** — it carries no +`param_space()`, so the World families (sweep / walk_forward / compare, C21) +cannot consume it without a hand re-author (the friction the GER40 deep-dive +fieldtest surfaced). The **canonical shippable form is the `Composite` +blueprint** (the authoring/source level, C9/C19); the `FlatGraph` is only its +compiled substrate (C23). The breakout now ships as +`ger40_breakout_blueprint(bar_period, …)` whose `param_space()` is exactly its +tuning knobs (`{entry_bar.target, exit_bar.target}`); the **bar period is a +construction argument** binding Resample + Session together — a structural +matrix axis (C12: a different period is a *different strategy*, not a sweep +point), never a `param_space` entry, so a sweep cannot desync the two clocks. +This is the concrete instance of the structural-axis-vs-tuning-param split +above (and `delay.lag`, a C8 structural constant, is bound out of the space). + +**Refinement (2026-06-29 — experiment-matrix *members* are topology-data, C24).** +"The experiment matrix is ordinary Rust control flow, not a config schema" holds for +the **generator** — the loop / conditional that *enumerates* structural variants may +stay Rust (C17). But each **member** it yields is a **topology-data value** (C24), +not a hand-coded blueprint nor engine-baked source: a structural axis selects among +*data* blueprints the World constructs, mutates, and serializes. This is what makes +structural variation **first-class and searchable** rather than a hand-enumerated +menu — `HarnessKind` and the per-strategy `*_sweep_family` functions in `aura-cli` +are the pre-C24 scaffolding (topology-as-engine-source, a C16 tension), retired as +C24 + the project-as-crate layer land. [C26 realization, 2026-07-10 (#231): the +scaffolding's last data weld — `wrap_r`'s hard-wired `price`←close role and the +`M1Field::Close`-only open sites — is retired; a strategy's input roles now bind +archive columns by name (C26). `wrap_r`'s remaining R-scaffolding retirement +stays #159.] diff --git a/docs/design/contracts/c20-strategy-harness.md b/docs/design/contracts/c20-strategy-harness.md new file mode 100644 index 0000000..2fdb2ea --- /dev/null +++ b/docs/design/contracts/c20-strategy-harness.md @@ -0,0 +1,95 @@ +# C20 — Strategy ↔ harness; the harness is the root sim graph + +**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9): broker-, +data-, and viz-independent, with inputs declared as named **roles** (symbol-agnostic +where possible) and the **bias** stream (C10) as output. A **harness** (the +experimental setup) is the **root sim graph** — sources bound to the strategy's +input roles + the strategy + attached broker node(s) + sinks — and is itself +produced by the bootstrap (C19). A harness *instance* is C1's disjoint unit (the +root scope). The harness has **two kinds of parameterization**: **structural axes** +(which strategy, which instrument(s), which broker(s), which window, which **risk +regime**) whose variation selects *different* instances — the **experiment matrix** +(the risk regime is the fourth axis, #210, carried as `CampaignDoc.risk`; +kept-separate, compared-not-selected, see C10) — and **tuning params** (the +strategy's numeric params) swept *within* a fixed structure (Fork A). The same +strategy blueprint is reused across backtest, sweep, visual workspaces, and the +frozen live bot — each a different harness. **Both strategy and harness/experiment +are authored in Rust** via builder APIs (C17); the experiment matrix is ordinary +Rust control flow (loops/conditionals), not a config schema. Ontologically, a node +(incl. a strategy composite) is an **open** fragment — free input roles + ≤1 output +(C8/C9) — that does not run alone; a **harness is the closed root graph**: a +strategy with its input roles bound to sources and its output terminated in +broker/sink nodes, under a clock. A harness is therefore **not a node** (no free +inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's +disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21) +orchestrates them as objects. + +**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a +declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as a +node (it is the closed root scope, not an open composable node). The mini-DSL +prohibition is scoped by the #188 role-model pass: it forbids an **open, +logic-bearing** experiment language (the RustAst trap), **not** the closed-vocabulary +**campaign document** (C25/C18) that carries persisted experiment intent — data +windows × strategy ids × param axes × process ref, under the total, declarative P1 +construct tier (bounded axes, gates, ladders; no variables, no general recursion, no +unbounded iteration — invariant 5's no-free-feedback move applied one level up; +#188/#189). A *generator* that enumerates structural variants may still be +plain Rust; what it yields, and what a campaign declares, is data. Genuinely new +logic (metrics, analysis blocks) escalates to a new Rust block, never to a freetext +hole in the artifact. + +**Why.** Reusability needs the strategy to be a context-free blueprint that many +harnesses embed. Modelling the harness as a root graph keeps it within the one +Node/graph abstraction (C9) and makes "10 strategies in one environment" and "one +strategy × N instruments" plain nested loops over the structural axes. Rust +authoring (not config) preserves full programmatic power — conditional/adaptive +matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap +C17 rejects. + +## Current state + +**The shippable form is the `Composite` blueprint.** A **hand-wired `FlatGraph` is +not a shippable strategy** — it carries no `param_space()`, so the World families +(sweep / walk_forward / compare, C21) cannot consume it without a hand re-author. +The **canonical shippable form is the `Composite` blueprint** (the authoring source +level, C9/C19); the `FlatGraph` is only its compiled substrate (C23). This is +concrete in the GER40 session-breakout example (`ger40_breakout_blueprint`, +`crates/aura-ingest/examples/shared/breakout_real.rs`), whose `param_space()` is +exactly its tuning knobs and whose **bar period is a construction argument** binding +Resample + Session together — a structural matrix axis (C12: a different period is a +*different strategy*, not a sweep point), never a `param_space` entry, so a sweep +cannot desync the two clocks. Structural constants (e.g. a `delay.lag`, C8) are +bound out of the space. + +**Experiment-matrix members are topology-data.** The "ordinary Rust control flow" +clause holds for the **generator** — the loop / conditional that *enumerates* +structural variants may stay Rust (C17). But each **member** it yields is a +**topology-data value** (C24), not a hand-coded blueprint nor engine-baked source: a +structural axis selects among *data* blueprints the World constructs, mutates, and +serializes, which is what makes structural variation first-class and searchable +rather than a hand-enumerated menu. Persisted experiment *intent* is the campaign +document (C25/C18). + +**Pre-C24 scaffolding retired.** The pre-C24 scaffolding — `HarnessKind` and the +per-strategy `*_sweep_family` functions — is **gone**. The family builders are now +generic and blueprint-driven: `blueprint_sweep_family`, `blueprint_walkforward_family`, +`blueprint_mc_family` (and `blueprint_sweep_over`) in +`crates/aura-runner/src/family.rs`. The R-evaluator scaffold `wrap_r` +(defined in `crates/aura-runner/src/member.rs`; imported and called from +`runner.rs`) survives; its last hard data weld — the +hard-wired `price`←close role and the `M1Field::Close`-only open sites — is retired +(a strategy's input roles now bind archive columns by name, C26), and `wrap_r`'s +remaining R-scaffolding retirement is tracked at #159. + +## See also +- [C9](c09-fractal-composition.md) — the strategy as a composite blueprint. +- [C10](c10-bias-r-cost.md) — the bias output and the risk-regime axis. +- [C17](c17-authoring-surface.md) — Rust authoring and the DSL prohibition. +- [C19](c19-bootstrap.md) — the bootstrap that produces the harness. +- [C21](c21-world.md) — the World orchestrates harnesses as objects. +- [C23](c23-graph-compilation.md) — the `FlatGraph` compiled substrate. +- [C24](c24-blueprint-data.md) — experiment-matrix members as topology data. +- [C25](c25-role-model.md), [C18](c18-registry.md) — the campaign document / run registry. +- [C26](c26-input-binding.md) — roles bind archive columns by name. + +> History: [c20-strategy-harness.history.md](c20-strategy-harness.history.md) diff --git a/docs/design/contracts/c21-world.md b/docs/design/contracts/c21-world.md new file mode 100644 index 0000000..d40118f --- /dev/null +++ b/docs/design/contracts/c21-world.md @@ -0,0 +1,59 @@ +# C21 — The World: the meta-level is the product + +**Guarantee.** Above the harness (C12's disjoint sim unit) sits the **World** — the +project's program / "game" (a Rust crate, C16). Within it, **harnesses are +dynamically constructible, first-class objects**: meta-programs (walk-forward, +sweep, optimize, Monte-Carlo — C12's orchestration axes) construct harness +instances at runtime via the bootstrap (C19), run them disjointly in parallel +(C1), aggregate / compare their results, and discard the transient instances. A +walk-forward rolls windows → bootstraps a harness per window → stitches +out-of-sample equity + parameter stability into one meta-result. Orchestrating +*families* of harnesses is **first-class**, not a headless afterthought; the run +registry (C18) is the World's memory. A harness's **topology is a serializable +data value the World owns** (C24), not Rust source — the precondition for the +World's full ambition: it can **generate**, **mutate**, **structurally search** +(genetic / NEAT-style graph operators), **compare**, and +**serialize-for-reproduction** (C18) *families that vary structure*, not only +numeric params. A param sweep over a fixed skeleton is the degenerate case; +varying the skeleton itself needs topology-as-value. (#109, resolved 2026-06-29.) + +**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep / +comparison) to second-class headless-only status; treating the single backtest as +the product. + +**Why.** What happens *within* one harness — backtest a strategy → equity — is +commodity; every quant system covers it. aura exists for the meta-level: a +programmable space where harnesses are dynamically built and families of them +orchestrated and explored. The deterministic single-harness engine (C1–C20) is +the **substrate**; the World is the **product**. The game-engine principle (C16) +made literal: the engine owns the scene / blueprint graph as content, instantiates +and runs it, the way aura owns harness topology as data (C24). + +## Current state + +The four orchestration axes (sweep, walk-forward, Monte-Carlo, generalize) are +realized as families the World constructs, runs disjointly (C1), and aggregates. +The dissolved orchestration verbs translate each invocation into a +content-addressed process + campaign document, run through a single campaign +executor (`aura-campaign`, driven by `aura-runner`), so every ad-hoc family run is +durable, diffable, reproducible intent rather than a bespoke code path per verb. +Harness topology is a World-owned serializable data value (C24): blueprints are +JSON that the bootstrap (C19) compiles to a flat instance (C23); the run registry +(C18) persists each run's manifest as the World's memory, and any family member is +re-derivable from that manifest. + +The **programmable analysis meta-level** is only partly realized. Families are +*produced* and *compared* (the trace explorer's family view, C22) but are not yet +themselves **composable** — sweep / MC / walk-forward as first-class meta-level +operators one can nest and pipe — and the structural-search ambition (generate / +mutate / genetic / NEAT-style graph operators over topology-as-value) is not built. +That is the open search-policy direction; today the World constructs fixed family +shapes rather than searching over the space of shapes. + +## See also +- [C24](c24-blueprint-data.md) — harness topology as a World-owned data value +- [C16](c16-engine-project-split.md) — the engine / project split and the game-engine principle +- [C12](c12-atomic-sim-unit.md) — the atomic sim unit and the four orchestration axes +- [C19](c19-bootstrap.md) — blueprint → instance bootstrap +- [C18](c18-registry.md) — the run registry, the World's memory +- [C22](c22-playground-traces.md) — the trace explorer that plays these families diff --git a/docs/design/contracts/c22-playground-traces.history.md b/docs/design/contracts/c22-playground-traces.history.md new file mode 100644 index 0000000..0a3816b --- /dev/null +++ b/docs/design/contracts/c22-playground-traces.history.md @@ -0,0 +1,137 @@ +# C22 — The playground is a trace explorer; sinks are the recording mechanism: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c22-playground-traces.md](c22-playground-traces.md). + +**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the +substrate level: a recorded trace is exactly what a recording node pushed out of +the graph (no engine recording registry; the constructing World holds each +recording node's destination). The engine's single `observe: usize` affordance is +removed — `Harness::run` returns `()` and recording is a node-side concern, so one +run records *many* streams (one per recording node) instead of exactly one row. +Recorded streams are sparse and timestamped (a record per fired cycle, tagged +`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the +`Harness` API change (observe removed, `run -> ()`) is recorded here. + +**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped +(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA, +C7) `ColumnarTrace` to `runs/traces//` (`aura-registry::TraceStore`, beside the +run registry's `runs.jsonl`), reachable via `aura run [--real …] +--trace `; `aura chart [--panels]` reads them back, aligns all taps on a +synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph; +C1 — pure, no live external call), and emits a static self-contained uPlot page +(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14): +encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in +`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`). +First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served +page injected the chart *series* but no run-context header (the run manifest is +persisted on disk in `index.json`, deliberately not surfaced in that first cut — a +ratified scope call). The header (#102) and serve-time decimation (#108) landed in +cut 2 (see the served-page-hardening amendment below); the families-comparison view +landed too (#107). A local server and replay-clock controls remain open (see "Open +architectural threads"). + +**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends +to family runs: `aura sweep|mc|walkforward --trace ` persists *each member* as +a nested standalone run-dir `runs/traces///`, reusing +`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a +side-effect inside each per-member closure). The `member_key` is content-derived and +deterministic — sweep: the *varying* axes rendered as a filesystem-portable +directory component (`-` tokens joined by `_`, charset +`[A-Za-z0-9._-]`, case-less values, length-capped with an FNV fallback), MC +`seed{N}`, walk-forward `oos{ns}` — never +a runtime ordinal, because members run in parallel under the engine's `Fn + Sync` +HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write` +targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`, +stdout/registry are byte-unchanged. Because `TraceStore` resolves `/` +as a subpath, `aura chart /` charts any single member with no +view-side change — the write-side precondition for the family-comparison **view** +(overlay / small-multiples across members) — now realized (#107; see the +families-comparison amendment below). + +**Amendment (CLI `--trace` retired, #168 / #224).** The CLI-verb `--trace` story above (single-run `run --trace`, amendment 3b56efb; per-member `sweep|mc|walkforward --trace`, #104) describes capabilities that no longer reach the code. `run` and `mc` refuse `--trace` (structurally parseable, refused at dispatch); the per-member family `--trace` was never wired to the blueprint sweep (`run_blueprint_sweep`'s `let _ = persist`) and was silently dropped when the welded `--strategy` triple retired (#159/#220). #168 makes the surface honest — `sweep`/`walkforward --trace` now refuse up front (exit 2, forward-pointer to #224). The single live `TraceStore::write` site is the campaign `presentation.persist_taps` → `runs/traces//` (`persist_campaign_traces`); `aura chart ` reads that back. Restoring CLI-side per-member trace-writing is deferred to #224. **[Delivered 2026-07-11 (#224 + fieldtest B1 fix): `sweep`/`walkforward --trace` now WRITE per-member traces on the real-data campaign path (the depth-2 fan-out `runs/traces////`), refusing only on the synthetic path; `aura chart ` resolves both the depth-1 campaign layout and the depth-2 fan-out (members keyed `/`, C1-sorted). The refusal story in this amendment is historical.]** + +**Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an +**opt-in real-data source axis**: `aura sweep|walkforward --real [--from +] [--to ]` streams real M1 close bars from the data-server archive (the #71 +`M1FieldSource` seam — C6 record-then-replay: a pre-recorded archive, never a live +call mid-sim) instead of the built-in synthetic stream, per family member. A +CLI-side `DataSource` provider (synthetic | real) supplies each member's source, +pip (resolved from the recorded geometry sidecar (`instrument_geometry`) — C10 +refuse-don't-guess; a symbol with no recorded geometry exits 2 before any data +access), window, and — for walk-forward — its `WindowRoller` sizes +(synthetic: bar-index 24/12/12; real: fixed calendar-time 90d/30d/30d in ns; CLI +flags for these are deferred). **[Amended 2026-07-11 (#239): on the dissolved +`walkforward --real`/`mc --real` sugar path the fixed 90/30/30 sizes are a +*ceiling*, not a constant — `fit_wf_ms_sizes` (aura-cli) passes them through +byte-identically whenever they fit the resolved campaign window, and scales them +down preserving the 3:1 IS:OOS ratio (step = OOS, one roll at the minimum) when +the window is shorter; authored process documents still validate their declared +sizes as-is.]** The engine, ingest, and registry are untouched (C9 — +the whole change is in `aura-cli`); the synthetic path is byte-unchanged. +**MC is excluded** (#106 Fork A): its seed varies a *synthetic* price-walk +realization (C12 axis 4), which is undefined over real data's single realization — +`aura mc --real` refuses with exit 2; a real MC needs a bootstrap-resampling axis +(its own cycle). The real walk-forward member key `oos{from.0}` widens from a small +synthetic bar index to an epoch-ns integer (still portable, collision-free). The +built-in demo strategy's **length grid is likewise data-kind-dependent** +(`DataSource::strategy_lengths`): synthetic keeps the short lengths that fit the +18/60-bar demo streams (trend SMA `{2,3}×{4,5}`, MACD `2/4/3`), real uses realistic +M1 lengths (trend `{50,100}×{200,400}`, MACD `12/26/9`) so the cross is a genuine +trend signal over tens of thousands of bars, not noise. This — like the roller +sizes — is a *demo-strategy calibration* patch; the real answer is project-authored +strategies (C9: a project crate owns its own grid), deferred to the project-env +work. + +**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison +**view** the #104 amendment left open is now realized. `aura chart ` +classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a +single run; else member subdirs with `index.json` are a family; else not-found) and, +for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every +member** in one self-contained uPlot page: `build_comparison_chart_data` emits one +labelled series per member, all on a **single shared y-scale** — members measure one +identical quantity, so a shared scale is what makes them comparable, unlike the +single-run overlay of *different* taps (per-series scales). The series align on the +same union-ts spine via `join_on_ts`, so **one mechanism yields three correct +readings**: sweep/MC members share the data window (a true overlay), walk-forward +members are disjoint OOS windows (null-complementary → the stitched curve). The view +consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted +by key for determinism, C1); a family is its *first producer* — the deliberate first +step toward the programmable analysis meta-level (C21) **without** rebuilding the +orchestration axes (that is its own later cycle). Name resolution is made a **total +function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing +command (`run`/`run `/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family) +to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by +both a run and a family) is unreachable. Every error path exits 2, never panics +(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is +`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder + +`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when +no `--tap` is given. **Still deferred:** multi-column tap selection +(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are +single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as +composable meta-level tools). + +**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred +follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine ++ registry untouched, C14). **Run-context header (#102):** a `ChartMeta` +(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params +for a single run) is built from the `RunManifest` in +`build_chart_data`/`build_comparison_chart_data`, injected into +`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js` +(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across +members `(min from, max to)` — the only reading that labels a disjoint walk-forward +family's true OOS coverage (it collapses to the shared window for sweep/MC). +**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max +transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots +(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a +multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full +recorded data stays on disk (view-only), and the walk-forward null-fill page collapses +for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under +budget. **Refinement (#111, 2957561):** the per-bucket reduction is **tap-aware** — an +*envelope* series (equity) keeps min/max, but a *bounded level* series (the C10 +bias level — pre-reframe: exposure — ∈[-1,+1]) reduces by per-bucket **mean** (a `ReduceKind` on each `Series`, +set by `reduce_for_tap`). Without it min/max made every multi-year exposure a solid +-1..+1 band (each bucket straddles many sign flips), reading as per-point oscillation; +the mean shows the net/duty-cycle level. Deferred refinements: rendering the min/max +envelope honestly as range bars / OHLC rather than a polyline (#112); a `--width` +budget flag and true intra-bucket min/max ordering for the non-default continuous +x-mode (#110). diff --git a/docs/design/contracts/c22-playground-traces.md b/docs/design/contracts/c22-playground-traces.md new file mode 100644 index 0000000..0b22117 --- /dev/null +++ b/docs/design/contracts/c22-playground-traces.md @@ -0,0 +1,144 @@ +# C22 — The playground is a trace explorer; sinks are the recording mechanism + +**Guarantee.** The World is a *program* (C21): nothing is displayable until it +runs, and its harnesses are transient machinery (built, run, discarded — C19). The +only durable, displayable substance is the **recorded trace**, captured by +**sinks** — pure consumer nodes (C8) that persist a stream (equity, a bias / +exposure level, a node's output) into the run registry (C18). **Displayable = +exactly what a sink recorded**; with no sink, only the input params + summary +metrics remain. The **playground** is therefore an **execution viewer / trace +explorer**, and it plays *any* harness (the harness *player*, not a harness, never +bound to a default one): before a run it shows the program *structure* +(graph-as-data, C9) + param knobs; during a run, live sink streams; after a run, +recorded traces + metrics from the registry — including meta-views (stitched +walk-forward, sweep surfaces, multi-strategy / instrument comparison). +Observability is **explicit and selective** — you instrument what you want to see; +the choice of sinks is part of the experiment. The engine ships sample blueprints +and a scaffolded starter so a newcomer has something to run at once; a fresh +project is empty until built, run, and instrumented. The shipped visual face is +**web-from-disk** — disk-persisted traces rendered as self-contained HTML chart +pages (not egui). + +**Forbids.** A scene-editor model that assumes a persistent, populated world; +constructing or wiring topology in the UI (topology is Rust + hot-reload, C9/C17 — +the UI reflects the live graph-as-data and tunes runtime params via sliders, C12); +retaining un-sinked data; binding the playground to a single / default harness. The +"never a scene editor" forbid is precisely **no persistent scene-at-rest** (a World +is a program) — it does *not* forbid a future blueprint *editor* over the +World-owned topology data (C24); that data format would be the substrate such an +editor round-trips, not a new mechanism. + +**Why.** A program has no scene-at-rest to inspect; what persists is what it +records. Making sinks the one recording-and-observability mechanism keeps "what can +I see?" answerable by "what did I instrument?", and keeps the engine UI-agnostic +(C14). Live param tuning (runtime values, no topology change — C12 / C19) gives the +interactive feel without a wiring DSL. The visual face is **orthogonal and +optional**: the blueprint data format (C24) is required by C21 (generation / +structural search) and C18 (reproduction) *at the CLI level, autonomously*, +independent of any visual surface — so the playground's form (web or egui, viewer +or editor, or not built at all) is downstream, and topology authoring does not live +in it. Generalized by the role model (C25): every artifact's canonical, complete +form is **text**, every operation is executable headless, and any visual surface is +a **stateless projection** that reads/writes that canonical form and adds no +semantics; read-only projections (this trace explorer) rank far above write +editors, because the Blockly-litmus discipline (C25) already keeps each vocabulary +palette-generatable without one. + +## Current state + +**Sinks at the substrate.** A recorded trace is exactly what a recording (tap) node +pushed out of the graph — there is no engine recording registry; the constructing +World holds each recording node's destination. `Harness::run` returns `()` and +recording is a node-side concern, so one run records *many* streams (one per tap) +rather than exactly one row. Recorded streams are sparse and timestamped (a record +per fired cycle, tagged `ctx.now()`), matching a trace of timestamped events (C18). + +**Encoding / storage / rendering split (C14).** A drained tap is encoded as a +columnar (SoA, C7) `ColumnarTrace` — struct→JSON only — in `aura-engine` +(`report.rs`); file I/O is `aura-registry::TraceStore`, persisting each run under +`runs/traces//` beside the run registry's `runs.jsonl`; rendering is +`aura-cli` (`render_chart_html` + the vendored `chart-viewer.js`). The engine stays +headless. + +**Single run.** `aura run ` persists every tap the blueprint +declares to the trace store under the run's own name, on both shapes: a +`bias`-output strategy (`aura-runner::member::run_signal_r`, R-wrapped) and a bare +measurement blueprint with ≥1 declared tap but no `bias` +(`aura-runner::measure::run_measurement`). A tap-free run writes nothing to `runs/` +and its stdout stays byte-identical. The CLI `--trace ` flag is **retired** on +`run` (and on `mc`): it parses but is refused at dispatch (exit 2) — naming a trace +is the family / campaign path's job, not a per-run flag. + +**Families.** `aura sweep|walkforward --real … --trace ` persists *each +member* via the campaign path: the dissolved verb is translated into a +content-addressed process + campaign document whose `presentation.persist_taps` +requests the tap vocabulary, run through the one campaign executor; +`aura-runner::runner::persist_campaign_traces` writes each member under a depth-2 +fan-out `runs/traces////`. Every written member is +independently re-run once in non-reduce trace mode over its own recorded window, +and its re-derived metrics are asserted equal to the recorded member's — a **C1 +drift alarm** that refuses (exit 1) rather than persist a silently-wrong trace. The +**synthetic** sweep / walk-forward path refuses `--trace` (exit 2); +**Monte-Carlo is excluded from trace persistence** (`mc --real` refuses +`--name`/`--trace`, exit 2 — the real-data R-bootstrap campaign itself runs, but +records no per-member family traces; the synthetic `--seeds` family's realization +argument, C12, does not carry over to one real series). +`TraceStore::ensure_name_free` makes name resolution a total function, refusing +cross-kind reuse of one name by both a run and a family. + +**Viewer.** `aura chart [--tap ] [--panels]` classifies the name on disk +(`TraceStore::name_kind`: top-level `index.json` → a single Run; member subdirs → a +Family; else NotFound, which then tries campaign-name resolution), reads the traces +back, and aligns all taps on a synthetic union-timestamp spine via the post-run +`join_on_ts` (C3 — a post-run join, not an in-graph merge; C1 — pure, no live +external call), emitting a self-contained uPlot page (vendored, like the graph +render). A **single run** overlays all its taps, each on its own y-scale (or the +one `--tap` selects); a **family** overlays **one tap** (default `equity`, `--tap` +to pick) of **every member** on a **single shared** y-scale — members measure one +identical quantity, so a shared scale is what makes them comparable. One mechanism +yields three correct readings: sweep / MC members share the data window (a true +overlay), walk-forward members are disjoint OOS windows (null-complementary → the +stitched curve). A **run-context header** (`ChartMeta` → `window.AURA_TRACES.meta` → +the pure `buildHeader`) renders chips for kind / name / members / tap / broker / +window / seed / commit / bound-params; a family's window is the **span** +`(min from, max to)` across members. Serve-time **min-max decimation** (`decimate`, +`CHART_DECIMATE_BUCKETS = 2000`) thins the page to a few-thousand spine slots (full +recorded data stays on disk, view-only), and the reduction is **tap-aware** +(`reduce_for_tap` / `ReduceKind`): an unbounded cumulative curve (`equity`) keeps +the per-bucket min/max **envelope** so drawdowns survive, while a bounded level +series (`exposure` / the C10 bias level, ∈ [-1,+1]) reduces by per-bucket **mean** +so its net / duty-cycle level shows instead of collapsing to a solid ±1 band. +`--panels` renders stacked panels instead of the overlay. + +**Newcomer.** `aura new ` scaffolds a **data-only** project — a paths-only +`Aura.toml`, a runnable starter blueprint `blueprints/signal.json` over the std +vocabulary (an SMA-cross → `Bias` strategy), `.gitignore`, and a project `CLAUDE.md` +— no crate, no build step (`aura nodes new` attaches a native node crate, C16/C17). +The engine also ships example blueprints under +`crates/aura-cli/examples/r_{sma,breakout,channel,meanrev}.json`. Honest gap against +the Guarantee's "populated trace immediately": these shipped blueprints are +*strategies* (a `bias` output, no declared taps), so `aura run` prints summary +R-metrics to stdout but writes no on-disk trace — a **chartable** trace today comes +from a blueprint that declares taps (single run) or from a `--real … --trace` family +campaign. + +**Deferred.** Live sink streams *during* a run are not built — taps are +buffer-then-drain (collected, then written after the run), and there is no local +replay server or replay clock; the shipped face is post-run static HTML. Reworking +tap draining into live subscribers (stream-to-disk / fold / live consumer) is #283. +Embeddable chart / graph fragments for hosting in external pages are unbuilt (#150). +The programmable analysis meta-level — families as composable orchestration +operators — is C21's open direction, not this contract's. + +> History: [c22-playground-traces.history.md](c22-playground-traces.history.md) + +## See also +- [C27](c27-declared-taps.md) — declared taps: the named sinks a run persists +- [C8](c08-node-contract.md) — sinks are pure consumer nodes +- [C21](c21-world.md) — the World is the program the playground plays +- [C24](c24-blueprint-data.md) — the topology data a future editor would round-trip +- [C25](c25-role-model.md) — visual surfaces as stateless projections +- [C14](c14-headless-two-faces.md) — headless core, two faces (the encode / store / render split) +- [C18](c18-registry.md) — the run registry the trace store sits beside +- [C9](c09-fractal-composition.md) — topology grown in Rust; the structure the viewer reflects +- [C10](c10-bias-r-cost.md) — the bounded bias / exposure level the viewer reduces by mean diff --git a/docs/design/contracts/c23-graph-compilation.md b/docs/design/contracts/c23-graph-compilation.md new file mode 100644 index 0000000..2cf2fb3 --- /dev/null +++ b/docs/design/contracts/c23-graph-compilation.md @@ -0,0 +1,77 @@ +# C23 — Graph compilation and behaviour-preserving optimisation + +**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic, +named **blueprint** (the authoring source — nodes, composites, strategy, harness; +C8/C9/C20) into a flat, type-erased **`FlatGraph`** — the frozen runnable instance +(C7) — **wired by raw index, not by name** (`Edge { from, to, slot, from_field }`). +Composites are **inlined** at this step (C9): the composite *boundary* dissolves +entirely (there is no composite in the flat graph). The blueprint's field / +input-role *names* are **non-load-bearing** — the wiring resolves by index, and names +survive at most as **informative debug symbols** (exactly as `FieldSpec.name` already +is, C8), kept for tracing / rendering (C9 graph-as-data) but carrying no run +semantics. The flat graph is then the target of **behaviour-preserving optimisation** +— any transform that leaves every observable sink trace bit-identical (C1 is the +correctness invariant) — on two levels: + +- **intra-graph** (within one graph): **common-subexpression elimination** — two + identical nodes (same type, same bound params, same input source) merge to one with + output fan-out, so fractal composition (C9) costs no redundant compute — and + **dead-node elimination** — a node on no path to a sink is dropped. Pure-consumer + sinks (C8, no output) are never CSE candidates, so their out-of-graph side effects + are preserved by construction. +- **across the sweep family** (over the family of flat graphs one blueprint yields + under a param sweep, C12): the sub-graph whose nodes carry **no swept param** and + whose inputs are **all sweep-invariant** (transitively from the sources — same data + window) is **loop-invariant** w.r.t. the sweep and is computed **once**; its output + is materialised as a recorded stream (C11) shared read-only across the sweep + instances (`Arc<[T]>`, C12). Only the parameter-dependent suffix re-runs per sweep + point — loop-invariant code motion over the sweep loop. + +The compilation also carries one structural gate that **is** load-bearing: +**param-namespace injectivity**. `check_param_namespace_injective` runs over the +`param_space()` name projection before name resolution, reads the **boundary** name +projection — the by-name authoring address space — and rejects a duplicated path +(`CompileError::DuplicateParamPath`). This makes the authoring address space +injective **without** making names load-bearing in the flat graph: node names qualify +the param path at construction and are dropped at lowering (the flat graph stays +wired by raw index). + +**Forbids.** Any "optimisation" that changes an observable trace (it breaks C1); +optimising over a representation that is not graph-as-data — an opaque trait-object +interior (the rejected nested-composite reading of C9) cannot be analysed or +rewritten across its boundary, so it forecloses both levels. + +**Why.** Determinism (C1) is not merely an audit property; it is the **licence** for +these rewrites — a behaviour-preserving transform is only meaningful because "same +input → bit-identical run" makes "same result" decidable, and a sweep-invariant +sub-graph is reproducible enough to compute once and share. The flat graph +representation (C19) is the necessary condition: only a flat, inspectable +graph-as-data — with each node's declared param-ranges (C8) — lets the compiler +identify identical sub-expressions, dead nodes, and the sweep-invariant frontier. +This is precisely why a composite is an inlining (C9) and not a runtime sub-engine: +the flat graph is what makes the whole graph optimisable. + +## Current state + +The flat-graph **representation** — blueprint → inline / compile → flat instance — is +the load-bearing substrate, built **first**, and is realized: `Composite::compile` +emits the first-class `FlatGraph` (C19). The per-node param declarations the +sweep-level pass presupposes carry `name` + `kind` (C8); the swept *range* is still +pending (#32/C20), as is the sweep orchestration itself (C12/C21). + +The **optimisation passes** — intra-graph CSE/DCE and sweep-invariant hoisting — are +**deferred, behaviour-preserving follow-on work**; no pass exists yet +(`Composite::compile` in `crates/aura-engine/src/blueprint.rs` adds none). Each is +gated by a "flat graph with pass ≡ flat graph without pass, bit-identical run" test +(C1). The one compilation-time structural check that is **live** today is the +param-namespace injectivity gate above — `check_param_namespace_injective`, defined in +`crates/aura-engine/src/blueprint.rs` and applied at that file's own compile entry +points (`compile`, `compile_with_params`, and the `bind` paths) as well as from +`construction.rs`'s `finish`, raising `CompileError::DuplicateParamPath`. + +## See also +- [C1](c01-determinism.md) — the bit-identity correctness invariant that licences every rewrite. +- [C8](c08-node-contract.md) — `FieldSpec.name` as the non-load-bearing precedent and the per-node param declarations. +- [C9](c09-fractal-composition.md) — inlining vs runtime sub-engine. +- [C11](c11-sources-record-replay.md), [C12](c12-atomic-sim-unit.md) — recorded-stream sharing and the sweep family. +- [C19](c19-bootstrap.md) — the flat-graph representation this optimises. diff --git a/docs/design/contracts/c24-blueprint-data.history.md b/docs/design/contracts/c24-blueprint-data.history.md new file mode 100644 index 0000000..f7144b2 --- /dev/null +++ b/docs/design/contracts/c24-blueprint-data.history.md @@ -0,0 +1,160 @@ +# C24 — The blueprint is a serializable, World-owned data value (the topology data format): history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c24-blueprint-data.md](c24-blueprint-data.md). + +**Status (2026-06-29; first cut shipped — cycle 0087 / #155, `d5602ec`; +construction service shipped — cycle 0088 / #157).** The +**principle** is settled (this contract, ratified in an in-context design discussion +— the #109 resolution). The **first cut now ships**: a `Composite` blueprint +serializes to a **canonical, versioned** data value (`format_version` envelope, +omit-defaults JSON) and **loads back** (data → blueprint → `FlatGraph`) via +`blueprint_to_json` / `blueprint_from_json` (`aura-engine::blueprint_serde`), +referencing nodes by **compiled-in type identity** through an **injected resolver** +whose concrete closed `match` over the `aura-std` vocabulary lives outside the engine +(`aura-std::std_vocabulary`) — the engine stays domain-free, no node registry +(invariant 9). Acceptance met: a serialized blueprint runs **bit-identical** (C1) to +its Rust-built twin. `model_to_json` (C9) remains the render half; this closes the +loop for the round-trippable vocabulary. **Cycle 0088 (#157) adds the +introspectable construction service**: a declarative, replayable by-identifier +op-script (`aura graph build` / `introspect` over a JSON op-list, the engine +`GraphSession` / `replay`) builds a runnable blueprint through validated ops — the +engine's construction gates split into *eager* per-op checks (name resolution, edge +kind-match, the double-wire arm, param bind, and acyclicity — a `connect` that would close a +cycle is rejected eagerly, #161) and *holistic* finalize checks (wiring +totality, param-namespace injectivity, root-role boundness), **both cadences calling +the same extracted predicates** (`edge_kind_check`, the shared resolution helpers, +`check_root_roles_bound` — no second validator) — plus build-free introspection over +the closed vocabulary (types, a node's ports/kinds, a partial document's unwired +slots). It **emits** the #155 blueprint; acceptance met: a graph built purely through +the ops compiles identical (C1) to its Rust-built twin, an invalid op is rejected at +the op naming the cause, introspection answers without a build. The engine `Op` stays +serde-free (the wire DTO is CLI-side); the vocabulary's enumerable companion +(`std_vocabulary_types`) lives in `aura-std` (no registry, invariant 9). **Cycle +0089 follow-up (#161 / #162, after the cycle-0088 fieldtest).** The eager acyclicity +gate above (`GraphSession::closes_cycle`, a reachability check at the closing +`connect`) closed a fieldtest-found hole where `graph build` accepted a non-DAG +op-list (invariant 5 / C9). **Lockstep:** it is a *second* home of invariant-5 +alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two +must co-evolve when the explicit delay/register node — invariant-5's sole legal +feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be +rejected at construction. The holistic *finalize* faults now also read +**by-identifier** (#162): `finish()` translates the index-carrying `CompileError`s +(`UnconnectedPort` / `RoleKindMismatch` / `UnboundRootRole`) into by-identifier +`OpError` variants — still *calling* the unchanged holistic gates (the +no-second-validator lockstep preserved), only translating their result. +**Cycle 0090 (#156) codifies the forward-compat two-tier discipline.** Tier-1 +(additive-optional) is serde-default silent-ignore with no `format_version` bump +(a new optional field defaults to prior behaviour, C1) — now proven by +`unknown_optional_field_is_tolerated_byte_identically` (an unknown optional key +loads byte-identically and runs bit-identically). Tier-2 (must-understand: a new +node type, edge semantics, or structural-axis kind) bumps `format_version` so an +old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`, +already green). The per-section required-flag scheme is deferred (no current +Tier-2 section to validate it; recorded on #156). +Pre-ship dormancy (#61, 2026-07-10): until the first external ship there are +no out-of-repo readers — reader and writer change atomically in one commit — +so the Tier-2 bump discipline is dormant and structurally-semantic additions +(the `gangs` section) land as additive-optional fields of v1; the first ship +consciously freezes v1, gangs included, and activates the bump discipline. + +**Milestone delivered — 2026-06-30, cycle 0090 — the serializable format + loader + construction service.** A green end-to-end milestone fieldtest (`fieldtests/milestone-topology-as-data/`) proves the full author → serialize → load → construct → introspect → reproduce story from the public surface alone, 0 behavioural bugs; the round-trippable format (#155), the construction service (#157), and the forward-compat two-tier discipline (#156) all shipped. Polish filed forward: op-script grammar docs + a stale example (#163), the canonical trailing-newline / `Composite` value ergonomics (#164), CLI discoverability of `build`/`introspect` (#159). + +**Realization (2026-06-30, cycle 0092 — runs are built FROM blueprint-data, #165).** +`aura run ` now loads a serialized **signal** blueprint +(`blueprint_from_json` → the closed `std_vocabulary`; an unknown type fails clean as +`UnknownNodeType`, the data-plane face of invariant 9), wraps it in the r-sma run +scaffolding (sinks / broker / data supplied **at run**, not serialized — C24's deferred +set), runs it, and emits a `RunReport` **bit-identical** (C1) to its Rust-built twin — +proven by `loaded_signal_runs_bit_identical_to_rust_built`. The `RunManifest` now carries +a **`topology_hash`**: SHA256 of the canonical (#164) `blueprint_to_json`, the #158 +reproducibility anchor, a Tier-1 optional field (#156). The hash + helper live +research-side (`aura-cli` + `sha2`), off the frozen engine (invariant 8). This is the +**keystone of the World/C21 milestone**: topology-as-data is now *runnable*, not only +serializable. The harness wrap was made shareable — `r_sma_graph()` = +`wrap_r(sma_signal(...))`, so the Rust path and the data path are the same seam +over the same signal (a behaviour-preserving C19/C23 restructure). Scope note: the hash +covers the **signal** only (the fixed scaffolding is identified by `commit`); a content-id +distinguishing harness-structural variants is a #158/#166 concern once scaffolding varies. + +**Realization (2026-07-01, cycle 0093 — the World orchestrates FAMILIES from blueprint-data, +#166).** `aura sweep --axis =` loads an **open** signal blueprint, +grids the by-name axes against its `param_space()`, builds each member through cycle-1's +`wrap_r` seam, and aggregates to a `FamilyKind::Sweep` family — byte-identical in shape +to the hard-wired sweep (`append_family` / `sweep_member_reports` reused verbatim). This is the +C21 step beyond a single run: the World now constructs and orchestrates **families** of +harnesses from topology-data, not just one. Each member manifest carries the **shared** +`topology_hash` (one signal topology, only params vary; `member_key` distinguishes members) — +reproducible per C18. Two design facts: a sweep needs an **open** blueprint (a fully-bound one +has an empty `param_space`, distinct from cycle-1's bound run fixture), and the signal is +**re-loaded per member** from its serialized doc (the `Composite` is `!Clone` — its +`PrimitiveBuilder`s hold `Box` build closures, #164 — and `bootstrap_with_cells` +consumes the graph). **Monte-Carlo over a loaded blueprint shipped (cycle 0095, #170).** `aura mc + --seeds N` runs a **closed** blueprint across N seeds — each seed a distinct +synthetic price walk (`synthetic_walk_sources`, the `mc_family` `SyntheticSpec` pattern), the +draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and aggregates to a +`FamilyKind::MonteCarlo` family with one stored blueprint per family (the C18 hook), so it +`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint +(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered +exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the +pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward + --axis = [--select argmax|plateau:mean|plateau:worst]` re-optimizes +the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each +24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric + +`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a +`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical +(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One +substitution deep from the hard-wired r-sma WF arm (the loaded blueprint via +`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure +exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form +honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the +meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP +is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the +`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint +warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle +0096, #169):** `aura sweep --list-axes` lists a loaded blueprint's open +sweepable knobs (one `:` per line, `param_space()` order), then exits — the names it +prints are exactly what `--axis` binds. Every listed name is **mandatory** on a `sweep` / +`walkforward`: the blueprint must be fully bound before it runs, so a subset grid is refused with +the missing knob named (`BindError::MissingKnob`) and there is no default — pin a knob you do not +want to vary with a single-value axis (`--axis =`). A single `blueprint_axis_probe` helper now single-sources +the wrapped probe (`wrap_r(loaded_signal).param_space()`) for the sweep terminal, the MC +closed-check, AND the listing (three former inline copies → one), so **listed == swept by +construction** (and stays so across #159's harness retirement — the listing tracks whatever the +sweep actually resolves, never a second source of truth). The names are prefixed by the current +r-sma wrapping (`sma_signal.fast.length`, the nested-composite prefix), not the raw +`param_space` — which is why the discovery lives on the sweep verb (it owns the wrapping), not +`graph introspect`. CLI `--trace` is refused on all verbs (#168 for sweep/walkforward; run/mc already refused); the live trace-writer is the campaign `presentation.persist_taps` (`persist_campaign_traces`), and restoring per-member CLI traces is tracked by #224. **[Delivered 2026-07-11 (#224): `sweep`/`walkforward --trace` write per-member traces on the real-data campaign path (depth-2 fan-out, chartable by the printed family handle); only the synthetic path still refuses.]** + +**Content-addressed reproduction shipped (cycle 0094, #158, C18)**: `topology_hash` +landed cycle 0092; re-deriving a member's FlatGraph from a stored manifest + the +content-addressed blueprint store (`aura reproduce`) shipped cycle 0094 (see C18 +Realization). What **remains** is a content-id that covers **structural-axis / whole-harness +variants** (the scaffolding is not yet blueprint-data); the debug-name-in-id question is +settled by cycle 0104's **identity id** (#171: `blueprint_identity_json` + `graph +introspect --identity-id`, an additive sibling — the byte-exact content id keeps the +store/reproduce roles; introspection-only until a dedup consumer exists). Still deferred: retiring the pre-C24 +hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `*_sweep_family`) once +the project-as-crate layer lands (#159, paired with #157's data-authoring surface). **Out of the first cut's round-trippable set** +(deliberate; fails clean as `UnknownNodeType`, never a silent wrong graph): recording +sinks (capture an `mpsc::Sender` — runtime identity, not param-generic data, C19) and +construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / `Session` — +structural-axis args, a C20 concern), additively addable later (#156). The +**project-as-crate load boundary landed in cycle 0102** (`Aura.toml` discovery + +`cdylib` loading + merged project ∪ std vocabulary — see the C13 realization +note; the `aura new` scaffolder followed in cycle 0103 — one command emits a +buildable project whose blueprint runs through the merged vocabulary); of the +two layers this paragraph used to name as sequencing-coupled, what remains +open is the **composable-orchestration** thread (#109): topology-as-data is +the substrate it stands on. **Canonical project shape (#181, resolved +2026-07-02):** the `aura new` templates (`scaffold.rs`) are the canonical +authoring shape and evolve with the engine; the cycle-0102 `demo-project` +fixture is an intentionally frozen known-good twin for the load-boundary +tests. The two are deliberately **not** lockstep-guarded: no consumer requires +them to match, each is e2e-guarded on its fitness for purpose (build → +descriptor load → charter check → deterministic run — blueprint *wiring* +content is pinned in neither, stated honestly), and an equality guard would +convert every deliberate template improvement into forced churn of the frozen +fixture — the same cross-purpose coupling that rules out regenerating the +fixture from the scaffolder. + +**[C26, 2026-07-10 (#231): the single-price data weld inside the surviving `wrap_r` scaffolding is retired — input roles bind archive columns by name; the wrapper's remaining R-scaffolding retirement stays #159.]** (From the C24 Forbids clause as of that date; the single-price weld was retired at #231/C26, but `wrap_r` itself survives — its full R-scaffolding retirement stays deferred, #159.) diff --git a/docs/design/contracts/c24-blueprint-data.md b/docs/design/contracts/c24-blueprint-data.md new file mode 100644 index 0000000..d377c92 --- /dev/null +++ b/docs/design/contracts/c24-blueprint-data.md @@ -0,0 +1,300 @@ +# C24 — The blueprint is a serializable, World-owned data value (the topology data format) + +**Guarantee.** A **blueprint** — the param-generic, named graph-as-data a Rust +builder produces (C9/C19) — is a **first-class serializable data value** with a +stable, versioned format, and the engine has a **load path** (data → blueprint → +`FlatGraph`, C23), not only the Rust-builder construction path. Topology is +therefore a **value the World owns** (C21): constructed, **generated**, mutated, +**structurally searched**, serialized for **reproduction** (C18), and (optionally) +rendered or edited by a visual face (C22). `model_to_json` (C9) is the **existing +reverse half** (blueprint → data, for render); this contract closes the loop. The +format carries **topology + structural axes + the param-space**, referencing nodes +by their **compiled-in type identity** — the closed, typed node vocabulary of the +std set plus the project `cdylib` (C8/C16) — and carries **no node logic** +(computation is compiled Rust, C17). It is **non-Turing-complete by construction** +(a static DAG, no control flow), so it structurally cannot become a second RustAst. + +**Forbids.** Putting **node logic / computation** in the format (the RustAst trap — +logic is Rust, C17); a **Turing-complete or control-flow-bearing** blueprint format +(it is static data; the *generator* that emits it may be Rust, C20, but the artifact +is not a language); a **by-name node marketplace / registry** (the vocabulary is the +compiled-in closed set referenced by type identity — the format is not a distribution +mechanism, C9/C16); treating the **visual face (C22) as the motivation or the +authoring brain** (the format is required by C21/C18 at the CLI level, independent of +any UI); a **generated / searched run whose topology is not recoverable from its +manifest** (C18 reproduction); baking topology as **engine source** compiled into the +binary. The hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `r_sma_graph`, +`sample_blueprint_with_sinks`, the built-in `*_sweep_family`) were pre-C24 scaffolding, +a C16 tension, and are now **retired** (#159/#220): topology reaches the engine only as +data, resolved by the injected vocabulary. The `wrap_r` R-scaffolding wrap +(`aura-runner`) **survives** — it is the r-sma wrap every `run` / `sweep` / `mc` / +`walkforward` path applies to the loaded signal at run (supplied at run, not serialized); +its full R-scaffolding retirement stays deferred (#159). Harness input roles bind archive +columns **by name**, not by a single baked-in price weld (C26). + +**Why.** The World (C21) is the product, and it cannot orchestrate, **structurally +search**, or **reproduce** *families that vary topology* while topology stays opaque +Rust source baked into the engine. The **game-engine principle** (C16's engine / game +split) makes it concrete: the engine is compiled native systems; the "game" is +**content** — and topology *is* content, a data value the engine owns, serializes, +instantiates, and mutates, exactly as a game engine owns its scene / prefab / +blueprint graphs. The #109 fork was never "*who types the wiring*" — a small LLM +cannot reliably *apply* a DSL and a strong one does not need one, so a +manifest-as-authoring-surface dies on both horns (C17). It is "**is topology a +compile-time *source* artifact or a runtime, serializable, World-owned *value*?**", +and determinism (C1) + reproduction (C18) + structural search (the open search-policy +thread) force the **value**. The engine already treats topology as a runtime value +(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the +flat graph as the optimisation target); C24 only adds that the value **serializes out +of Rust and loads back in**. + +## Current state + +The **principle** is settled — the #109 resolution, ratified in an in-context design +discussion. The format, its loader, the construction service, and the family +orchestration all ship. The acceptance criterion is green: a serialized blueprint +runs **bit-identical** (C1) to its Rust-built twin. + +### The format and load path + +A `Composite` serializes to a **canonical, versioned** data value — a +`format_version` envelope (`BlueprintDoc`), omit-defaults JSON — and **loads back** +(data → blueprint → `FlatGraph`) via `blueprint_to_json` / `blueprint_from_json` +(`aura-engine::blueprint_serde`). Nodes are referenced by **compiled-in type +identity** through an **injected resolver** whose concrete closed `match` over the +std vocabulary lives outside the engine (`aura-vocabulary::std_vocabulary`), so the +engine stays domain-free, with no node registry (invariant 9). An unknown type fails +clean as `LoadError::UnknownNodeType` — the data-plane face of invariant 9, never a +silent wrong graph. `model_to_json` (C9) remains the render half; this closes the +loop for the round-trippable vocabulary. + +**Out of the round-trippable set** (deliberate; fails clean as `UnknownNodeType`): +recording sinks (they capture an `mpsc::Sender` — runtime identity, not param-generic +data, C19) and construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / +`Session` — structural-axis args, a C20 concern), additively addable later (#156). + +### Runs and families are built FROM blueprint-data + +`aura run ` loads a serialized **signal** blueprint and emits a +`RunReport` **bit-identical** (C1) to its Rust-built twin; the run scaffolding (sinks +/ broker / data) is supplied **at run**, not serialized. Beyond a single run, the +World constructs and orchestrates **families** of harnesses from topology-data: + +- `aura sweep --axis =` → `FamilyKind::Sweep`. A sweep + needs an **open** blueprint (a fully-bound one has an empty `param_space`). +- `aura mc --seeds N` → `FamilyKind::MonteCarlo`, each seed a + distinct synthetic price walk drawn disjoint-parallel through the engine + `monte_carlo` seam (invariant 1). MC binds no axis, so it needs a **closed** + blueprint (the sweep's distinction inverted); an open one returns a named `Err`, + rendered exit-2 at the builder boundary — no hidden exit in the pure builder. +- `aura walkforward --axis = [--select + argmax|plateau:mean|plateau:worst]` → `FamilyKind::WalkForward`: re-optimizes the + loaded blueprint's params over the `--axis` grid on each 24/12/12 IS window, + selects the winner by `sqn_normalized`, runs it out-of-sample. Reduce-mode members + are R-measured (`oos_r` the meaningful summary; stitched pip-equity empty, C10). + +The family builders live in `aura-runner::family` (`blueprint_sweep_family` / +`blueprint_mc_family` / `blueprint_walkforward_family`). Every member manifest carries +the **shared** `topology_hash` (one signal topology, only params vary; `member_key` +distinguishes members), and each family stores its blueprint(s) content-addressed so +`aura reproduce` re-derives every member bit-identically (`aura-runner::reproduce`, +C18). The synthetic-walk DGP is the MC machinery, not trader-grade statistics; a +real-data block-bootstrap — and retiring the `synthetic_walk_sources` `len:60`↔warm-up +coupling — rides #172. + +### Reproduction identity + +The `RunManifest` carries a **`topology_hash`**: SHA256 of the canonical +`blueprint_to_json` (see *Canonical form*), the #158 reproducibility anchor, a Tier-1 +optional field (#156). The hash and its helper live research-side (`aura-cli` + +`sha2`), off the frozen engine (invariant 8). The hash covers the **signal** only — +the fixed scaffolding is identified by `commit`; a content-id distinguishing +structural-axis / whole-harness variants is a #158/#166 concern once scaffolding is +itself blueprint-data. The **identity id** (#171) is an additive sibling: +`blueprint_identity_json` (`aura-engine::blueprint_serde`) + `graph introspect +--identity-id`. The byte-exact content id keeps the store/reproduce roles; the +identity id is introspection-only until a dedup consumer exists. + +### Axis discovery + +`aura sweep --list-axes` lists a loaded blueprint's open sweepable +knobs (one `:` per line, `param_space()` order) and exits; the printed +names are exactly what `--axis` binds. Every listed name is **mandatory** on `sweep` / +`walkforward` — the blueprint must be fully bound before it runs — so a subset grid is +refused with the missing knob named (`BindError::MissingKnob`, `aura-engine`) and +there is no default; pin a knob you do not want to vary with a single-value axis. A +single `blueprint_axis_probe` (`aura-runner`) single-sources the wrapped probe for the +sweep terminal, the MC closed-check, and the listing, so **listed == swept by +construction** (and stays so across the harness retirement — the listing tracks +whatever the sweep actually resolves). The names are prefixed by the current wrapping +(`sma_signal.fast.length`, the nested-composite prefix), which is why discovery lives +on the sweep verb (it owns the wrapping), not `graph introspect`. `--trace` on +`sweep` / `walkforward` writes per-member traces on the real-data campaign path +(depth-2 fan-out, chartable by the printed family handle, #224); the synthetic path +still refuses (`run` / `mc` refuse `--trace` outright). The live trace-writer is the +campaign `presentation.persist_taps` (`persist_campaign_traces`, `aura-runner::runner`). + +### The construction service (op-script) + +`aura graph build` / `introspect` over a JSON op-list (#157) is the introspectable, +replayable, by-identifier construction surface; it **emits** the serializable +blueprint. The engine `GraphSession` / `replay` (`aura-engine::construction`) build a +runnable blueprint through validated ops. Construction gates split into **eager** +per-op checks — name resolution, edge kind-match (`edge_kind_check`), the double-wire +arm, param bind, and acyclicity (a `connect` that would close a cycle is rejected +eagerly at the closing op, `GraphSession::closes_cycle`, #161) — and **holistic** +finalize checks — wiring totality (`validate_wiring`), param-namespace injectivity +(`check_param_namespace_injective`), root-role boundness (`check_root_roles_bound`). +Both cadences call the **same extracted predicates** — no second validator. The +holistic faults read **by-identifier** (#162): `finish()` translates the +index-carrying `CompileError`s (`UnconnectedPort` / `RoleKindMismatch` / +`UnboundRootRole`) into by-identifier `OpError` variants while still *calling* the +unchanged holistic gates. Build-free introspection answers over the closed vocabulary: +`graph introspect --vocabulary | --node | --unwired`. The engine `Op` stays +serde-free; the wire DTO (`OpDoc`) is CLI-side (`aura-cli::graph_construct`). +Acceptance: a graph built purely through the ops compiles identical (C1) to its +Rust-built twin, an invalid op is rejected at the op naming the cause, introspection +answers without a build. + +**Invariant-5 lockstep.** The eager acyclicity gate is a *second* home of invariant 5 +alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two +must co-evolve when the explicit delay/register node — invariant 5's sole legal +feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be +rejected at construction. + +### Op-script grammar + +The op-script is a JSON **array of ops**, each object internally tagged by `"op"`, +replayed in order; nodes are referenced **by identifier**, ports as dotted +`.`. The eight verbs: + +- `source` — `{"op":"source","role":,"kind":}` — declare a root + source role producing a base column of `kind`. +- `input` — `{"op":"input","role":}` — declare a root input role (kind inferred + from the slots it feeds). +- `add` — `{"op":"add","type":,"name":?,"bind":{:}?}` — + add a node of compiled-in type identity `type`; **`name` is its identifier** + (mirrors the builder's `.named(...)`; defaults to the lowercased type label, so + two unnamed nodes of one type collide); `bind` sets params. +- `feed` — `{"op":"feed","role":,"into":[,…]}` — fan a root role into + interior input slots. +- `connect` — `{"op":"connect","from":,"to":}` — wire an interior output + field to an input slot; a `connect` closing a cycle is rejected eagerly + (invariant 5). +- `expose` — `{"op":"expose","from":,"as":}` — promote an interior output + field to a boundary output, aliased by `as` (a real alias, not a naming; contrast + `add`'s `name`) — one of the two verbs that keep `as` (with `tap`). +- `tap` — `{"op":"tap","from":,"as":}` — declare a measurement **tap** on + an interior output field under `as` (#284) — the output-side twin of `expose`: a + recorded observation point (a `Composite.taps` entry, C27), not a boundary output. + Name-addressed like every other op (no raw index); tap names are their own + namespace (a duplicate refuses). The `finish` gate threads op-declared taps into + the built `Composite` (`.with_taps`); a single `aura run` records each, a sweep + leaves them inert. +- `gang` — `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}` + — fuse two or more sibling params into ONE public knob: the member addresses + leave the sweepable param space and `as` replaces them; the bound or swept + value fans out to every member at bootstrap. Members must share one scalar + kind and stay open (un-bound). + +Value forms are the serialized representations: a `Scalar` bind value is the typed tag +form `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` / `{"Timestamp":}`, and `kind` +is the capitalized `ScalarKind` (`"F64"`). Worked example: +`fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json` (an +SMA-crossover bias). + +### Canonical form (#164) + +The canonical blueprint artifact is exactly the bytes `blueprint_to_json` returns — a +JSON value with **no trailing newline** (`serde_json::to_string`, not framed with a +display `println!`), so the CLI and library emit paths are **byte-identical** — a +prerequisite for content-addressed topology identity (#158, a hash over the canonical +form). The canonical JSON is also the blueprint's **equality / identity surface**: +`Composite` (`aura-engine::blueprint`) and `PrimitiveBuilder` (`aura-core::node`) +deliberately carry **no in-memory `PartialEq` / `Debug`** — the recipe holds a +`build: Box` closure (non-comparable, non-printable), so equality could only be +defined over the serialized *data*, of which `blueprint_to_json` is already the single +source; a second in-memory notion would be a drift hazard against the very form #158 +content-addresses. The loader stays lenient (a trailing newline on input still parses, +Tier-1 robustness). + +### Forward-compat: the two-tier discipline (#156) + +Tier-1 (additive-optional) is serde-default silent-ignore with **no** `format_version` +bump — a new optional field defaults to prior behaviour (C1). Tier-2 (must-understand: +a new node type, edge semantics, or structural-axis kind) **bumps** `format_version` so +an old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`). +**Pre-ship dormancy (#61):** until the first external ship there are no out-of-repo +readers — reader and writer change atomically in one commit — so the Tier-2 bump +discipline is dormant and structurally-semantic additions (the `gangs` section) land as +additive-optional fields of v1; the first ship consciously freezes v1, gangs included, +and activates the bump discipline. The per-section required-flag scheme is deferred (no +current Tier-2 section to validate it; recorded on #156). + +### Enforcement shift — invariant 9 on the data plane + +Lifting topology onto the data plane relocates *where* the engine/project boundary +(invariant 9, no bespoke node registry/marketplace) is enforced, without changing what +it forbids. **Pre-C24** a blueprint referencing another project's node was a **compile +error** — topology was in-process Rust, so cross-boundary node resolution was +*structurally impossible*, compiler-guaranteed for free. **Post-C24** that same +reference is a **type-id string in a serialized blueprint**, refused only by the +injected resolver's closed set (`UnknownNodeType`). The anti-NIH rationale survives +intact (node *logic* stays cargo+Gitea-only; the format distributes +**topology-as-content**, never logic — C17), and `std_vocabulary` stays a compiled-in +dispatch table, not a registry — but its enforcement **degrades from +compiler-guaranteed to resolver-seam discipline**. Two consequences: + +1. **#160** (the guard that the closed vocabulary stays honest) is no longer droppable + hygiene — it guards the data-plane face of the boundary (its own drift direction + still fails *safe*, `UnknownNodeType`). The `std_vocabulary_roster!` macro + (`aura-vocabulary`) expands the resolver `match` and the enumerable + `std_vocabulary_types()` list from **one roster**, closing the resolver-vs-list + drift mode by construction; the residual (a new zero-arg node never rostered at all) + stays fail-safe, guarded by the one-line maintainer surface plus the count pins (the + in-crate shape test and aura-cli's cross-boundary `--vocabulary` count e2e). The + per-project **merged resolver** (project ∪ std, project first) lives in + `aura-runner::project`. +2. The **injected per-project resolver** (C16) is the genuinely new, *deferred* + invariant-9 surface: nothing structurally stops a project resolver from accepting + arbitrary type-ids, or a project from layering a blueprint *marketplace* (store + + type-discovery API) **on top of** the format — one layer *above* the engine + contract. Invariant 9 holds **at the engine**; the project/ecosystem boundary (what + an injected resolver may resolve, where closed-set discipline draws the line) is a + **World/C21-layer charter point**, not an engine fix. + +Invariant 8 (frozen) is likewise reframed by C24/C21: from "topology baked at build +time" to "the World may *generate* topology at runtime (research plane), the chosen +blueprint is frozen into the deploy artifact, and any run is deterministic once +instantiated" (C1). + +### Adjacent / deferred + +The **project-as-crate load boundary** has landed (`Aura.toml` discovery + `cdylib` +loading + merged project ∪ std vocabulary; the `aura new` scaffolder emits a buildable +project whose blueprint runs through the merged vocabulary) — see C13/C16. **Canonical +project shape (#181):** the `aura new` templates (`scaffold.rs`) are the canonical +authoring shape and evolve with the engine; the `demo-project` fixture is an +intentionally frozen known-good twin. The two are deliberately **not** lockstep-guarded +— no consumer requires them to match, each is e2e-guarded on its fitness for purpose +(build → descriptor load → charter check → deterministic run; blueprint *wiring* content +is pinned in neither, stated honestly), and an equality guard would convert every +deliberate template improvement into forced churn of the frozen fixture (the same +cross-purpose coupling that rules out regenerating the fixture from the scaffolder). +What remains open is the **composable-orchestration** thread (#109): topology-as-data is +the substrate it stands on. + +## See also + +- [C9](c09-fractal-composition.md) — graph-as-data; `model_to_json` render half +- [C16](c16-engine-project-split.md) — engine / project split; the injected resolver +- [C17](c17-authoring-surface.md) — logic is Rust; the closed-vocabulary-data-artifact clarification +- [C18](c18-registry.md) — reproduction, the run registry, content-addressed store +- [C19](c19-bootstrap.md) — bootstrap: blueprint → frozen instance +- [C21](c21-world.md) — the World owns and orchestrates topology +- [C23](c23-graph-compilation.md) — bootstrap-as-compilation to `FlatGraph`; the optimisation target +- [C26](c26-input-binding.md) — role names bind archive columns +- [C27](c27-declared-taps.md) — declared taps as named measurement points +- [C1](c01-determinism.md) — determinism / bit-identity; [C10](c10-bias-r-cost.md), [C13](c13-hot-reload-frozen-deploy.md), [C20](c20-strategy-harness.md), [C22](c22-playground-traces.md), [C25](c25-role-model.md), [C28](c28-stratification.md) + +> History: [c24-blueprint-data.history.md](c24-blueprint-data.history.md) diff --git a/docs/design/contracts/c25-role-model.md b/docs/design/contracts/c25-role-model.md new file mode 100644 index 0000000..a9e4e70 --- /dev/null +++ b/docs/design/contracts/c25-role-model.md @@ -0,0 +1,90 @@ +# C25 — The role model: nine authoring roles, cut by artifact + surface + iteration cost + +**Guarantee.** aura's user-facing design is organized by **roles**, cut by the +artifact a role owns, the surface/language it works in, and the iteration cost it +tolerates — never by org chart. One trader — or an LLM actor, which is the whole +point of the authoring design — must be able to fill *every* role; the separation +separates *surfaces with different iteration costs and failure modes*, not people, +and switching roles is a visible act. The nine roles (ratified 2026-07-03, #188): + +1. **system programmer** — the engine; Rust, this repo; all invariants. +2. **extension dev** — native nodes + analysis blocks; Rust in project/shared + crates; [C7](c07-scalar-soa.md)/[C8](c08-node-contract.md), determinism in + `eval`. +3. **toolchain dev** — playground/viewer/skills pipeline; vendor-side; authoring + ergonomics. +4. **data curator** — recorded streams, sidecars, derived/news recordings; CLI + + recording edge; [C2](c02-causality.md)/[C3](c03-single-merge.md)/[C6](c06-firing-policy.md). +5. **methodology designer** — validation/eval **process documents** ("the game + rules"); closed data vocabulary; anti-false-discovery. +6. **6a strategy designer** — blueprint topologies; data (op-script/blueprint + JSON, the graph idiom); [C5](c05-freshness-gating.md)/[C7](c07-scalar-soa.md). +7. **6b campaign designer** — **campaign documents** (persisted experiment + intent); data, the pipeline/block idiom; [C1](c01-determinism.md). +8. **player** — owns nothing; explores traces, tunes runtime params; the + playground, [C22](c22-playground-traces.md)/[C12](c12-atomic-sim-unit.md); + read-only by design. +9. **operator/auditor** — frozen bots, reconciliation, money management; + deploy-edge CLI, manifests, reproduce; [C13](c13-hot-reload-frozen-deploy.md). + +The 6a/6b split follows invariant 12's tier boundary (node/harness vs World); the +interface between them is the id machinery (content id = byte-exact, identity id = +topological). The param gradient runs 6a defines topology + open params, 6b spans +spaces over them, 7 moves inside those spaces live. A role is **recognizable** when +four things exist: a named artifact type it owns, a surface addressed to it, an +entry path, and its invariants enforced at its boundary. + +**Text-first / headless-first is an invariant, not a taste** (owner-ratified +2026-07-03: editors must never be the only way; LLM operability demands it): every artifact class has a canonical, +complete, text-serialized form; every operation is executable headless via +CLI/API; visual surfaces are *stateless projections* that read/write the same +canonical form and add no semantics — read-only projections (viewers) rank far +above write editors. The **Blockly litmus test** is the acceptance criterion for +every artifact vocabulary, editor or no editor: a block palette must be +*generatable* from the vocabulary — every block with typed slots, every valid +composition snappable, every invalid one not. + +**Control surfaces are projections** (#295, ratified 2026-07-20). The text-first +invariant fixes the control-surface layering: the text artifact vocabulary +(blueprints, process/campaign documents, registry records) is the canonical layer, +and every control surface — the one-shot CLI's executor verbs, any future +long-running host, any MCP face, a World program — is a projection/executor over +those artifacts, never a second home for intent. + +**Forbids.** An artifact class whose only authoring path is a visual editor or a +Rust compile cycle when the role's iteration cost demands data (the role-6b +lesson: a tiny campaign change must never cost a compile); stringly-typed fields, +implicit inter-block coupling, or "arbitrary JSON here" holes in a role's +vocabulary (they fail the litmus test); collapsing the 6a/6b idioms into one +surface (dataflow boxes-and-wires vs sequential pipeline blocks are different +languages). + +**Why.** The game-engine analogy that seeded aura ([C16](c16-engine-project-split.md)) +extends to its people: engines separate system programmers, toolchain devs, asset +creators, level designers, and players because their artifacts, tools, and +iteration loops differ — a design that forces one role's technique on another +(Rust for campaign tweaks, an editor as the only entry) mis-prices iteration +exactly where research throughput lives. + +## Current state + +Roles 5 and 6b have their artifacts — process and campaign documents +([C18](c18-registry.md); #189). Roles 4, 7, and 8 remain technically +present but faceless: no addressed verb families yet. Role homes in the project +layout and docs-by-role are open (#192). + +**Document-first completion** is the resolved direction on the control-surface +amendment (owner-minuted 2026-07-21 on #295), delivered by #300. The executor verb +set is settled: `run`, the four thin per-verb generators, and the document verbs +validate/introspect/register/show/run/runs — `show` being #300's read-back +addition. The verbs' per-verb identity is re-ratified (#300 F8 — reduction, not +grammar collapse). A typed-protocol host or MCP face remains demand-driven and +unbuilt. + +## See also +- [C16](c16-engine-project-split.md) — the game-engine analogy this extends to people +- [C17](c17-authoring-surface.md) — all logic is Rust; closed-vocabulary data artifacts are not the forbidden DSL +- [C18](c18-registry.md) — the process/campaign document artifacts and the run registry +- [C20](c20-strategy-harness.md), [C21](c21-world.md) — the node/harness/World tiers behind the 6a/6b split +- [C24](c24-blueprint-data.md) — topology-as-data, the strategy designer's artifact +- [C26](c26-input-binding.md) — a role's input contract carried in blueprint data diff --git a/docs/design/contracts/c26-input-binding.md b/docs/design/contracts/c26-input-binding.md new file mode 100644 index 0000000..23f2a8e --- /dev/null +++ b/docs/design/contracts/c26-input-binding.md @@ -0,0 +1,73 @@ +# C26 — Harness input binding: role names bind archive columns + +**Guarantee.** A strategy blueprint declares WHICH data it consumes as the NAMES +of its root input roles: a role whose name is in the **closed column vocabulary** — +`open`, `high`, `low`, `close`, `spread`, `volume`, plus `price` as the +backward-compatible alias of `close` — binds by default to that column of the +campaign cell's (or run invocation's) instrument. A campaign document may +**override** the name default per role via the additive `data.bindings` block +(`role name → column name`; serde `default` + skip-if-empty, so binding-less +documents keep their content ids) — the 6b rebind seam: the blueprint's content +identity never changes. + +Resolution (`aura-runner`'s `binding` module, `resolve_binding`) produces ONE +ordered plan consumed by BOTH halves of the old weld: the columns are opened +(`aura_ingest::open_columns`, the generalization of `open_ohlc`/#92) and the +wrapped root roles declared (`aura-runner::member`'s `wrap_r`) in the **same +canonical order** — `M1Field` declaration order (open, high, low, close, spread, +volume), filtered to the consumed set — which is the [C4](c04-cycle-granularity.md) +merge tie-break order, so role *i* receives source *i* by construction. A close +column is always part of the plan (shared when the strategy consumes +`close`/`price`, else opened for the broker/executor pair alone). + +The vocabulary is **Blockly-litmus-clean** ([C25](c25-role-model.md)): role-name +slots draw from a closed enum, the bindings block is typed `role → column` with no +free-form holes, and validation is two-tier — values against the column vocabulary +in the intrinsic tier (`validate_campaign`, `aura-research`), keys against the +strategies' `input_roles()` in the resolver tier (`validate_campaign_refs`, +`aura-registry`). + +**Forbids.** Guessing a column for an unknown role name (refuse with the vocabulary ++ the override remedy, never default); opening columns in any order other than the +canonical declaration order (the [C4](c04-cycle-granularity.md) tie-break contract); +free-form or logic-bearing binding values (the [C17](c17-authoring-surface.md)/[C25](c25-role-model.md) +line — a binding value is a typed column reference, nothing else); fabricating +synthetic OHLC when a multi-column strategy meets synthetic data (the walk +generates a close series only — refuse with the `--real` remedy). + +**Why.** The single-price weld — `wrap_r`'s hard-wired `price`←close and the six +`M1Field::Close`-only open sites — made every strategy monocular regardless of what +its blueprint declared. Binding by role name keeps the blueprint the single source +of its own data needs ([C24](c24-blueprint-data.md): topology-as-data carries its +input contract), lets a campaign re-aim a strategy without touching its content id +([C18](c18-registry.md) identity), and pins opening and declaring to one +shared, canonically-ordered plan so the two halves cannot drift +([C1](c01-determinism.md)/[C4](c04-cycle-granularity.md) determinism). + +## Current state + +`resolve_binding` lives in `aura-runner`'s `binding` module +(`crates/aura-runner/src/binding.rs`), moved out of the shell with #295; it takes +the strategy name, its `input_roles()`, and the campaign override map and returns +one `ResolvedBinding` plan. Column opening is `aura_ingest::open_columns` +(`crates/aura-ingest`; the `M1Field` enum fixes the declaration order +open/high/low/close/spread/volume); role declaration is `wrap_r` in +`aura-runner::member`. Both validation tiers are realized: `validate_campaign` +(`crates/aura-research`) checks binding values against the column vocabulary; +`validate_campaign_refs` (`crates/aura-registry`) checks binding keys against the +strategy's declared roles. + +**Extension point (deferred).** When recorded non-price sources land (the #71 +Source seam), the binding VALUE-space grows additively from archive columns to +recorded-stream references — a root role like `sentiment` becomes bindable to a +recorded stream then, and until then refuses with the vocabulary-naming message. +The role-name key-space and the campaign `bindings` carrier are unchanged by that +growth. + +## See also +- [C4](c04-cycle-granularity.md) — the merge tie-break order that is the canonical open/declare order +- [C24](c24-blueprint-data.md) — topology-as-data carrying its own input contract +- [C18](c18-registry.md) — the content-id identity a rebind must not touch +- [C25](c25-role-model.md) — the Blockly litmus the vocabulary satisfies; the 6b rebind seam +- [C1](c01-determinism.md) — the determinism the shared plan protects +- [C27](c27-declared-taps.md) — the output-side twin (`taps` to `input_roles`) diff --git a/docs/design/contracts/c27-declared-taps.md b/docs/design/contracts/c27-declared-taps.md new file mode 100644 index 0000000..b8ffa1a --- /dev/null +++ b/docs/design/contracts/c27-declared-taps.md @@ -0,0 +1,69 @@ +# C27 — Declared taps: named measurement points bind sinks run-mode-aware + +**Guarantee.** A blueprint may declare **taps** — named, pure output-side +declarations `{ name, from: {node, field} }`, the output-side twin of `input_roles` +([C26](c26-input-binding.md)). A tap names an interior producer's output field +without naming a sink, exactly as a `Role` names an abstract input without naming a +source. At compile the tap resolves — and, for an interior composite, **hoists** to +the root — through the same lowering remap edges and `OutField` re-exports use +(`resolve_tap_wire`, a `flat_taps` accumulator threaded through the lowering +recursion), landing in `FlatGraph.taps` as a `FlatTap { name, node, field }` whose +name survives compile and is load-bearing for by-name binding (like +`SourceSpec.role`, #275). + +Binding is **run-mode-aware**: the run-mode-owning layer constructs a sink (a +`Recorder`) at a bound tap via `FlatGraph::bind_tap`, which takes a **caller-built** +`Box` sink (so the engine keeps its `aura-core`-only production +dependency — it never constructs a domain sink type) and appends it plus an edge +before bootstrap. The single-run path (`run_signal_r`) binds and records each +declared tap, persisting the series through the existing trace store +(`env.trace_store()`), so the tap columns surface through the same tooling the +campaign path feeds; a sweep/reduce run leaves taps unbound. + +**Forbids.** A tap carrying a channel endpoint or effect in the serialized artefact +— recording policy is run-mode authority, not fragment-embedded (a fragment must +not drag its measurement decisions into every harness that embeds it; the tier +ontology, [C20](c20-strategy-harness.md)/[C21](c21-world.md)). The engine +constructing a domain sink type (the `aura-core`-only wall — the sink is +caller-built). Order statistics (median, etc.) inside the graph — they stay +sink/analysis-side ([C18](c18-registry.md)); multi-instrument study +inputs stay harness/World tier. + +**Non-error.** An **unbound** tap is inert, not a fault — unlike an unbound root +input role, which `check_root_roles_bound` rejects ([C26](c26-input-binding.md)): +observation is optional, a fed input is mandatory. A declared-but-unbound tap +compiles and runs, its producer evaluating and its output discarded (a no-out-edge +producer is a valid runnable sink — the Kahn sort emits it, +`check_ports_connected` gates only inputs). + +**Why.** Observability must be expressible in a hand-authored blueprint — the +measurement-shaped study computes in the graph and surfaces via taps, no throwaway +Rust harness — while recording stays a run-mode decision, not a fragment-embedded +effect. Taps are designed **DCE-compatible** (a bound tap is a natural DCE root, an +unbound tap a dead declaration) but this contract does **not** depend on DCE: an +unbound tap's sink is simply never constructed (build-time elision, which the +engine already tolerates). (#282, 2026-07-18.) + +## Current state + +The tap types are realized in `aura-engine`: the authoring-level `Tap` +(`crates/aura-engine/src/blueprint.rs`, the output-side twin of `Role`), the +compiled `FlatTap { name, node, field }` and the `FlatGraph.taps` field +(`crates/aura-engine/src/harness.rs`), and `FlatGraph::bind_tap`, which appends a +caller-built `Box` sink plus an edge and raises a typed `UndeclaredTap` +on a tap the graph does not declare (duplicate detection across binds is the +caller's — the method keeps no cross-call state). Lowering resolves and hoists taps +via `resolve_tap_wire` and the `flat_taps` accumulator (`blueprint.rs`). The +single-run bind/record path is `run_signal_r` in `aura-runner::member`; the +sweep/reduce path never calls `bind_tap`. + +The chain-pruning benefit — a sweep paying zero for the study wires behind an +unbound tap — is **deferred to the future DCE cycle** ([C23](c23-graph-compilation.md)); +the mechanism ships now, verified sound. + +## See also +- [C26](c26-input-binding.md) — the input-side twin (`input_roles`); `check_root_roles_bound`, the mandatory-input counterpart +- [C23](c23-graph-compilation.md) — compilation/lowering and the deferred DCE cycle the tap design anticipates +- [C18](c18-registry.md) — the trace store and registry the tap series feed; order statistics live sink-side +- [C20](c20-strategy-harness.md), [C21](c21-world.md) — the tier ontology behind run-mode recording authority +- [C8](c08-node-contract.md) — the node/sink contract (a no-out-edge producer as a valid runnable sink) diff --git a/docs/design/contracts/c28-stratification.history.md b/docs/design/contracts/c28-stratification.history.md new file mode 100644 index 0000000..c49fe73 --- /dev/null +++ b/docs/design/contracts/c28-stratification.history.md @@ -0,0 +1,85 @@ +# C28 — Internal stratification: the trading ladder, the process column, the shell: history + +> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp +> and may be superseded; this file is NOT current truth and NOT a grounding +> surface. Current contract: [c28-stratification.md](c28-stratification.md). + +**Status (2026-07-19, milestone "Stratification — ladder, process column, +shell", #286).** This contract states the *target*. At HEAD the crates cut by +mechanical role (vocabulary / reductions / documents / orchestration / CLI), not +cleanly by layer, so the layering is realized only partially: +- The dependency *direction* now obeys the rule across the engine stack. The + `aura-engine → aura-backtest` production violation is cut (phase 2, #292): + `RunReport` is generic over its metric payload `M` (the engine names no + concrete metric type), the pip/R reductions and the MC assembly (`summarize`, + `McAggregate`/`RBootstrap`/`monte_carlo`) moved to `aura-backtest`, and the + statistics kernel (`MetricStats`/`quantile`/`resample_block`/`SplitMix64`) + moved to the `aura-analysis` foundation. The engine's `[dependencies]` are now + `aura-core` + `aura-analysis` only; the backtest layer instantiates + `M = RunMetrics` (`aura-backtest → aura-engine`, an outer→inner edge). The + ladder direction is enforced for the engine/backtest/analysis rungs by the + structural test (`c28_layering`, seven-row table). +- **#147 retired (user-ratified direction 2026-07-20; shipped the same day).** + Item 1 — the metric genericity `RunReport` — shipped as phase 2 above + (#292). Item 2 — the registry deflation vocabulary — shipped as the ratified + "A1" cut, its deferral trigger having fired when measurement supplied its + first deflatable metric (the IC, #290): the deliberately narrow + `MetricVocabulary` trait (resolve/roster/direction/value/one null draw) + lives in `aura-analysis`, re-exported through `aura-engine`; the R + vocabulary (`RunMetricKey`, `r_based`, the rosters, the centred + moving-block `null_draw`) is supplied by `aura-backtest`; the registry's + rank/optimize/deflate machinery is generic over `M: MetricVocabulary` with + refusal prose derived from the carried roster; the IC (`aura-cli`) is the + second production implementor, bringing its within-run permutation null. + The C10 wall stays monomorphic (`check_r_metric`/`generalization` accept + only the R vocabulary; `r_based` is the enforcement point). Explicitly + still deferred ("A2"): measurement runs as sweep-family citizens (report + unification, campaign engine generic-over-M) — until a concrete + family/campaign demand exists. The #136 one-implementor rationale is + superseded by the second implementor; the registry's process-column + trading-awareness holds unchanged. See #147 for the full disposition. +- The `aura-std` four-layer roster is now cut by layer (phase 4, #288): `aura-std` + holds the engine nodes only (arithmetic/logic/rolling + the generic sinks); + `aura-market` (`session`, `resample`), `aura-strategy` (`bias`/`stop_rule`/ + `sizer` + the cost nodes) and `aura-backtest` (`sim_broker`, + `position_management`) carry the outer rungs, each depending only on `aura-core`; + the closed node roster moved to `aura-vocabulary`. The `aura-analysis` + interweave is resolved (phase 5, #291): the backtest reductions live in + `aura-backtest::metrics` beside their `position_management` producer, and + `aura-analysis` is reduced to domain-free statistics + selection provenance + (`[dependencies]` = serde only). The *measurement* rung is seeded (#295): + `aura-measurement` carries the IC vocabulary + reduction (its run verb remains + the additive shape dispatch, #286). No crate exists yet for *execution* + (unbuilt — the C10/C13 edge only). Under this model the + `aura-registry → aura-research` edge is column-internal and legal. +- Phased realization (each independently shippable; behaviour byte-identical + except the purely additive shape dispatch): (1) this contract; (2) cut the + engine's backtest-metrics edge via `RunReport` — **done** (#292: + metric-generic run record, the pip/R reductions + MC assembly relocated to + `aura-backtest`, the statistics kernel to `aura-analysis`, dependency + inversion, ladder enforced by the structural test; realizes item 1 of #147); + (3) the #286 + shape dispatch plus the per-run scaffold as a library — **done** (#295: the + pure per-run scaffold pieces live in `aura-backtest::scaffold`; the composed + recipe — harness assembly, binding, translators, the default `MemberRunner` + — is the assembly crate `aura-runner`; the measurement rung is seeded as + `aura-measurement` with the IC; the shell is reduced to + argv/translation/presentation, enforced structurally. Residual: ~20 refusal + sites inside `aura-runner`'s single-run verb paths still terminate the + process; their conversion to `RunnerError` propagation is tracked as #297 — + the campaign path already refuses via `MemberFault`, never a process exit); + (4) split the `aura-std` roster along engine / market / strategy / + backtest — **done** (#288: four `aura-core`-only node crates, the closed roster + moved to `aura-vocabulary`, the ladder direction enforced by a structural test); + (5) split `aura-analysis` into generic statistics and backtest metrics — + **done** (#291: metrics moved to `aura-backtest::metrics`, `aura-analysis` + reduced to the domain-free half, engine re-export surface name-unchanged); (6) + generify the column's metric interface (demand-driven — this is #147). Crate + names now realized (`aura-market`/`aura-strategy`/`aura-backtest`/ + `aura-vocabulary`); full evidence on #288, #286 and the milestone. + +Provenance note (#295): the shell-boundary cut closes a ratified structural +debt (this contract's own phase plan) and un-blocks the future World program; +it was not a demonstrated downstream blocker — the first downstream project's +observed friction was document-vocabulary gaps, never Rust-level recipe +reachability. diff --git a/docs/design/contracts/c28-stratification.md b/docs/design/contracts/c28-stratification.md new file mode 100644 index 0000000..2b4d2d8 --- /dev/null +++ b/docs/design/contracts/c28-stratification.md @@ -0,0 +1,179 @@ +# C28 — Internal stratification: the trading ladder, the process column, the shell + +**Guarantee.** The aura workspace's crates realize a layered responsibility +model along two axes. A **ladder**, ordered by domain specificity from inner to +outer: + +1. **engine** — the domain-free deterministic streaming runtime and vocabulary + (cells/kinds/freshness/firing, graph build, compile, run, taps/trace — C1–C9, + C19, C23, C24), the domain-free node roster (arithmetic/logic/rolling nodes + plus the generic `Recorder`/`GatedRecorder`/`SeriesReducer` sinks), and + generic statistics (the Monte-Carlo and moving-block-bootstrap kernels). The + layer name is wider than the crate `aura-engine`: it spans `aura-core`, + `aura-engine`, the domain-free node roster (`aura-std`), and the + generic-statistics surface (`aura-analysis`). +2. **market** — what a market *is*, carrying no analysis or strategy intent: + instruments and pip/point geometry (C15), session anchoring, bar resampling, + market-data ingestion (`aura-ingest`). +3. **measurement** — descriptive statistics over market streams (session + statistics, conditional rates, distributions). Sibling of **strategy**: both + consume *market*, neither imports the other. +4. **strategy** — the *definition* of trading signals: bias, stop rules, sizing, + cost-model nodes. +5. **backtest** — the *evaluation* of strategies: simulated execution without + money (the `SimBroker`, position-management), the R-metric reductions and the + position-event table, and the per-run scaffold. Backtest mirrors *execution* + on the research side — execution semantics in R units, money exiled. +6. **execution** — money, account-term sizing, the live broker; already exiled to + the deploy edge by C10/C13, existing today only as that ratified boundary. + +Beside the ladder stands the **process column** — the research-process machinery +(the run registry C18, the process/campaign document types, campaign execution). +It consumes run *artifacts*, not market streams, and orchestrates runs of any +ladder layer; it stands beside the ladder, not on a rung. Between ladder/column +and the shell stands the **assembly** position (`aura-runner`): it composes the +rungs into the canonical member-run recipe — harness assembly, input binding +(C26), the C1-load-bearing param↔config translators, and the shipped default +`MemberRunner` (`DefaultMemberRunner`) — importing the ladder rungs, +`aura-composites`, `aura-ingest`, and the process column, and imported only by +the shell and by downstream World programs. It also carries a direct production +dependency on the external `data-server` tree (inherited from the shell with the +recipe: the runner constructs archive servers itself). The external tree +therefore enters the workspace at exactly three points — the ingestion edge +(`aura-ingest`), the assembly position (`aura-runner`), and the shell +(`aura-cli`, which imports everything by definition) — and at no other ladder or +column crate; the `c28_layering` guard pins the exact set. The **shell** (the +`aura` CLI, `aura-cli`) imports everything, is imported by nothing, exports +nothing, and holds no domain logic: argv/dispatch, argv→document translation +(including the op-script construction front-end, `graph_construct`), +presentation, and the `aura new` project scaffolder (authoring-tooling template +emission — shell-resident, like rendering, until a second consumer wants it). +Rendering stays in the shell until the C22 web face provides its second consumer. + +**Import rule.** An outer ladder layer may import inner layers, never the +reverse; *measurement* and *strategy* are siblings (no import either way); +*backtest* may import *strategy*. The process column imports the engine layer +plus the run-artifact and metric interfaces — a production dependency on +`aura-backtest` for the trading instantiation `M = RunMetrics` (#291/#292) — and +is imported by no ladder crate; column-internal edges are free (the +`aura-registry → aura-research` edge is legal). The assembly position imports +ladder and column alike and is imported by neither; the shell imports all; +nothing imports the shell. **`dev-dependencies` are exempt** — tests may cross +layers (the existing `engine`/`ingest`/`registry` → `std` edges are dev-only and +stay). The rule binds **crate structure, not graph composition**: a blueprint may +feed a measurement statistic into a bias input — data flow inside a graph is free +(C24); layers govern imports, graphs compose freely. + +**Forbids.** An inner layer importing an outer one — most sharply the engine +layer importing backtest reductions, or market importing strategy. A ladder crate +importing the process column. Money (currency P&L, account sizing) inside the +strategy or backtest layers (C10/C13 — it lives only at the execution/deploy +edge). Once a second consumer warrants the split, keeping multiple layers welded +in one crate's roster. + +**Why.** The stack is a framework specialized for traders: the inner engine is a +domain-free analysis substrate (the same machinery would measure robot telemetry +or any timestamped process), the outer rungs the trading specialization, and the +two together are the "game engine for traders" (C16's game/engine split applied +*inside* the workspace, from the engine|project boundary to the intra-stack +layering). Separable responsibilities behind narrow, contract-defined seams let a +contributor focus on one rung without absorbing the rest, and make each boundary +auditable. Enforcement is the crate graph itself: once crates are cut along these +cells, a layer violation is a compile error and every new edge a visible +`Cargo.toml` diff — no linter, no discipline appeals. Five of the six seams are +pre-existing ratified contracts and this model only *names* them as layer +boundaries: the node contract (C8, engine ↔ any vocabulary), blueprint-as-data +(C24, topology ↔ engine — its shape selecting the run scaffold is the additive +#286 refinement), declared taps (C27, run ↔ analysis — the only way values leave +a run), the run registry (C18, run ↔ process column), and the deploy edge +(C10/C13, everything ↔ money). The sixth — the process column ↔ metric interface +— is the one this contract introduced: a named metric vocabulary *supplied* by +measurement and backtest instead of baked in R-only (#147, now realized). + +## Current state + +The workspace realizes the full ladder today, cut by layer, with only the +*execution* rung unbuilt. The rung→crate map: + +- **engine** — `aura-core` + `aura-engine` + `aura-std` (the engine node roster: + arithmetic/logic/rolling + the generic sinks) + `aura-analysis` (domain-free + statistics + selection provenance). +- **market** — `aura-market` (`session`, `resample`) + `aura-ingest`. +- **measurement** — `aura-measurement` (the IC vocabulary + reduction). +- **strategy** — `aura-strategy` (`bias`/`stop_rule`/`sizer` + the cost nodes). +- **backtest** — `aura-backtest` (`sim_broker`, `position_management`, the + R-metric reductions, and the per-run scaffold `aura-backtest::scaffold`). +- **execution** — no crate exists; it is the C10/C13 deploy edge only. + +The closed node roster lives in `aura-vocabulary`. The process column is +`aura-registry` (C18) + `aura-research` + `aura-campaign`; the assembly position +is `aura-runner`; the shell is `aura-cli`. + +The dependency direction obeys the rule across the whole stack. The engine names +no concrete metric type: `aura_engine::RunReport` is generic over its metric +payload `M` (`crates/aura-engine/src/report.rs`), and `aura-backtest` supplies the +trading instantiation `M = RunMetrics` (`pub type RunReport = +aura_engine::RunReport`, `crates/aura-backtest/src/lib.rs`) — the +`aura-backtest → aura-engine` edge is outer→inner. The engine's production +`[dependencies]` are `aura-core` + `aura-analysis` only; `aura-analysis` is +reduced to the domain-free half (`[dependencies]` = serde only), holding the +statistics kernel (`MetricStats`/`quantile`/`resample_block`/`SplitMix64`) and the +selection-provenance types, while the pip/R reductions and the Monte-Carlo +assembly live in `aura-backtest`. The former monolithic `aura-std` node roster is +split by layer: `aura-std` (engine nodes), `aura-market`, and `aura-strategy` are +each `aura-core`-only node crates, while `aura-backtest` carries both the backtest +node modules (`sim_broker`, `position_management`) and the outer-rung metric +layer, so it depends on `aura-engine` as well as `aura-core`. The additive #286 +shape dispatch (blueprint shape selecting the run scaffold) is realized; the pure +per-run scaffold pieces live in `aura-backtest::scaffold`, and `aura-runner` +composes them with harness assembly, input binding, the param↔config translators, +and the shipped `DefaultMemberRunner`. The shell is reduced to argv/dispatch, +argv→document translation, and presentation. + +The direction is enforced structurally, not by discipline. The Rust compiler only +rejects import *cycles*; an acyclic-but-outward edge (e.g. `aura-std → aura-market`) +would compile silently, so `crates/aura-vocabulary/tests/c28_layering.rs` reads +every crate's production `[dependencies]` and asserts each intra-workspace edge +stays within the inner set C28 permits for that layer. The `allowed` table is the +**full workspace** — all 17 crates under `crates/` — and the test asserts the +table enumerates exactly the crates found on disk (`table_names == disk_names`), +so a newly added crate cannot escape the guard silently: even a foundation crate +with no `aura-*` deps must be listed. `[dev-dependencies]` are not inspected +(C28-exempt). Two further checks ride the same file: the shell-content check +(`aura-cli` is a pure binary with no `lib` target and a fixed module set — +argv/translation/presentation only), and the external-`data-server` pin (the tree +may be a production dependency of exactly `aura-ingest`, `aura-runner`, and +`aura-cli`). + +**#147 disposition (retired; user-ratified direction 2026-07-20, shipped the same +day).** The process-column metric interface is generic. The deliberately narrow +`MetricVocabulary` trait (resolve/roster/direction/value/one null draw) lives in +`aura-analysis` (`crates/aura-analysis/src/lib.rs`) and is re-exported through +`aura-engine`; the R vocabulary (`RunMetricKey`, `r_based`, the rosters, the +centred moving-block `null_draw`) is supplied by `aura-backtest`; the registry's +rank/optimize/deflate machinery is generic over `M: MetricVocabulary`, with +refusal prose derived from the carried roster. The IC (`aura-measurement`) is the +second production implementor, bringing its within-run permutation null. The C10 +money wall stays monomorphic — `check_r_metric`/`generalization` accept only the R +vocabulary (`r_based` is the enforcement point). Still deferred ("A2", +demand-gated, no tracking issue): measurement runs as sweep-family citizens +(report unification, campaign engine generic-over-`M`), until a concrete +family/campaign demand exists. + +**Deferred.** ~20 refusal sites inside `aura-runner`'s single-run verb paths still +terminate the process (`std::process::exit`); their conversion to `RunnerError` +propagation is tracked as **#297** (the campaign path already refuses via +`MemberFault`, never a process exit). + +## See also +- [C1](c01-determinism.md) — determinism / bit-identity, the correctness invariant the layer cuts preserve +- [C8](c08-node-contract.md) — the node contract (engine ↔ vocabulary seam) +- [C10](c10-bias-r-cost.md) / [C13](c13-hot-reload-frozen-deploy.md) — the deploy edge (everything ↔ money) +- [C16](c16-engine-project-split.md) — game/engine split, applied here inside the workspace +- [C18](c18-registry.md) — the run registry (run ↔ process column seam) +- [C23](c23-graph-compilation.md) — names are non-load-bearing +- [C24](c24-blueprint-data.md) — blueprint-as-data (topology ↔ engine seam) +- [C26](c26-input-binding.md) — input binding +- [C27](c27-declared-taps.md) — declared taps (run ↔ analysis seam) + +> History: [c28-stratification.history.md](c28-stratification.history.md)