docs(ledger): split the design ledger into an INDEX map, per-contract live files, and history sidecars
The single-file ledger had grown to 2968 lines / ~42k tokens, mixing current design law with accreted history: 59 cycle-stamped realization blocks, 18 [HISTORY] passages, 22 supersession markers, and the C10 / C22 / C24 reframe sagas layered several supersessions deep. A code-grounding audit (31 agents, adversarially verified) confirmed 11 defects stated as current truth: stale crate homes from the C28 #288 roster split (cost nodes, PositionManagement, PositionEvent, Session), the renamed InputSpec->PortSpec, the pre-#241 project model in C16 and the open-threads section, a stale HarnessKind retirement deferral in C24, and three C28-internal inconsistencies. New shape, per the ailang precedent: - INDEX.md stays the sole addressable entry point: foundation, external components, a C-id-keyed contract map (one line per contract), and only the genuinely open architectural threads. - contracts/cNN-<slug>.md carries each contract's current truth only: Guarantee / Forbids / Why with ratified refinements integrated, plus a code-anchored Current state. All confirmed defects are fixed here; crate anchors were re-verified against the tree. - contracts/cNN-<slug>.history.md (18 sidecars) and INDEX.history.md preserve every superseded block verbatim, stamps and issue refs intact, under a frozen-record banner. Nothing was deleted: superseded design intent remains an addressable working-tree artifact, off the per-cycle audit walk. - Ledger discipline is now stated in INDEX.md: live files are edited in place at cycle close, superseded text moves verbatim to the sidecar, and a supersession marker in a live file is itself an audit finding. Every contract file was verified against its old text by an independent zero-loss pass (statement-by-statement) plus a code-accuracy spot check; C-ids and contract titles are unchanged, so existing C-id citations in code, tests, and issues resolve as before.
This commit was merged in pull request #306.
This commit is contained in:
@@ -0,0 +1,233 @@
|
||||
# aura design ledger — INDEX history (frozen record)
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its stamp and may
|
||||
> be superseded; this file is NOT current truth and NOT a grounding surface.
|
||||
> Current ledger: [INDEX.md](INDEX.md). Per-contract history lives in
|
||||
> `contracts/*.history.md`.
|
||||
|
||||
## Header provenance and vocabulary gloss (pre-refactor wording)
|
||||
|
||||
Moved out of the INDEX header in the 2026-07-21 ledger refactor; the
|
||||
walking-skeleton milestone and the C1–C18 interview are completed history.
|
||||
|
||||
Provenance: contracts C1–C18 were settled in the initial rough-sketch design
|
||||
interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the
|
||||
C10 refinement to a broker-independent position table came in follow-up turns;
|
||||
C10 was reframed in cycle 0007 to an exposure stream, then again 2026-06-23
|
||||
(#117) to a **bias** stream with signal quality in **R**, the position table a
|
||||
derived layer — see C10). C19–C22 were added as the
|
||||
construction / World / playground layer; C23 and the C9/C19 compilation
|
||||
refinements were settled 2026-06-05 for the **Construction-layer** milestone (the
|
||||
blueprint→flat-graph reading of composites and graph optimisation). C24 (the
|
||||
blueprint as a serializable, World-owned data value) was settled 2026-06-29,
|
||||
resolving the long-deferred #109 fork in favour of **topology-as-data** (the
|
||||
game-engine principle, C16): the engine owns topology as content it serializes /
|
||||
loads / generates, not as Rust source baked into the binary.
|
||||
|
||||
Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a
|
||||
*milestone* is a tracker container spanning many cycles (the first milestone is
|
||||
the **walking skeleton**: ingest → one signal → exposure → sim-optimal broker →
|
||||
synthetic pip-equity signal-quality metric — the *pre-reframe* substrate, now
|
||||
reframed to bias → R-evaluator → R-expectancy, see C10).
|
||||
|
||||
## External components — the RustAst pre-authoring reading list
|
||||
|
||||
Written before `aura-core` existed as a guide for authoring it; `aura-core`
|
||||
has long been built, so the file-by-file reading list is a historical aid.
|
||||
RustAst's living role (conceptual reference, never a dependency) stays in
|
||||
[INDEX.md](INDEX.md).
|
||||
|
||||
But its **`src/ast/rtl/` layer is a working reference implementation of the very
|
||||
streaming substrate aura rebuilds**, and is worth reading before authoring
|
||||
`aura-core` — these are not just concepts, they exist as code:
|
||||
- `rtl/series/data.rs` — `RingBuffer<T>` with financial-style indexing (index 0
|
||||
= newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's
|
||||
pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no
|
||||
String/Record" = C7's closed scalar set); `ScalarSeries<f64|i64|bool>` and the
|
||||
**SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns").
|
||||
- `rtl/series/mod.rs` — `create_typed_series`, dispatching element type → storage
|
||||
backend (the type-specialized **factory** of C19).
|
||||
- `rtl/streams/mod.rs` — `Signal { cycle_id, value }` (C4's cycle clock) and the
|
||||
`Stream` / `Observer` / `ObservableStream` push traits (the reactive model of
|
||||
C4/C5).
|
||||
- `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one
|
||||
explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator
|
||||
(C12's seed-as-input).
|
||||
|
||||
aura reimplements these natively and **sharpens** them: types are monomorphized
|
||||
and edges type-erased to the four scalar kinds with direct dispatch (C7) instead
|
||||
of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>`
|
||||
(C12) instead of a `VecDeque<Value>`. RustAst shows the *shape*; aura makes it
|
||||
fast and deterministic.
|
||||
|
||||
## Open threads — resolved or superseded entries (pre-refactor wording)
|
||||
|
||||
Each entry below is the pre-refactor text. Where a thread stays genuinely
|
||||
open, INDEX.md carries a terse live successor; resolved threads live only
|
||||
here (their outcomes are recorded in the named contracts).
|
||||
|
||||
- **Playground & World UI surface** — the playground is core (C22); the surface
|
||||
is a **web frontend served from disk-persisted traces** (revised 2026-06, issue
|
||||
#101; not egui). Settled for the first cut: raw per-tap trace persistence to
|
||||
disk (columnar/SoA form, C7) + serve-time `join_on_ts` alignment + a static
|
||||
self-contained HTML chart page (uPlot, vendored like the `render_html`
|
||||
Graphviz-WASM blob), feeds overlaid or timestamp-aligned in panels. The
|
||||
families-comparison **view** — `aura chart <family>` overlaying one tap across a
|
||||
family's members on a shared y-scale — shipped (#107, see the C22 amendment).
|
||||
Still open: richer comparison meta-views (sweep-surface heatmaps, cross-family /
|
||||
multi-strategy comparison, multi-column tap selection #47), a local server, and the
|
||||
run/replay clock controls.
|
||||
|
||||
- **`aura new` scaffolder and the experiment-builder API** — `aura new`
|
||||
scaffolds a Rust project *crate* (node / strategy blueprints) against the
|
||||
engine (C16/C20); the experiment-builder API surface (harness wiring,
|
||||
structural axes, sweep combinators) is not yet designed. `Aura.toml`'s
|
||||
schema was settled paths-only in cycle 0102 (data archive root, runs dir —
|
||||
see the C13 realization note); the load boundary itself shipped there.
|
||||
|
||||
- **The analysis meta-level — composable orchestration + project-as-crate authoring
|
||||
(tracked: #109).** aura's differentiator (C21) had two unbuilt halves: (1) the
|
||||
orchestration axes (sweep / Monte-Carlo / walk-forward) becoming *composable tools*
|
||||
a user wires into an analysis workflow, rather than today's hard-wired CLI verbs
|
||||
(`sweep_family` / `walkforward_family` in `aura-cli`) — still open; and (2) the
|
||||
project-as-crate authoring layer above — its **load boundary landed in cycle 0102**
|
||||
(`Aura.toml` discovery + cdylib loading + merged vocabulary, C13 realization note)
|
||||
and the **`aura new` scaffolder in cycle 0103**; of that layer only the
|
||||
experiment-builder API remains open (C16/C17).
|
||||
On the meta-level, authoring a strategy is *wiring
|
||||
existing nodes* (C9 composition) + *defining the analysis framework* (C21), not
|
||||
writing new nodes; the user wants it driven via the CLI, later the interactive
|
||||
server / playground (C14/C22). **Resolved (2026-06-29) → C24.** The fork was
|
||||
mis-posed as "(a) run-a-Rust-experiment-via-CLI vs (b) wire-nodes-via-CLI". The real
|
||||
axis is **topology = compile-time Rust source vs runtime, serializable, World-owned
|
||||
data value**, settled as the **value** (C24, the game-engine principle): node
|
||||
*logic* stays Rust (C17), *topology* becomes data the World generates / serializes /
|
||||
reproduces. "Wire-nodes-via-CLI" is dropped — interactive wiring through the CLI is a
|
||||
non-goal; it collapses to a file + parser, which *is* C24's load path. What
|
||||
**remains** under #109: the **blueprint data format** itself (C24 Status — its own
|
||||
brainstorm / milestone), and the **composable-orchestration** half — the axes
|
||||
(sweep / MC / walk-forward) becoming tools applied to a World-constructed harness
|
||||
instead of the hard-wired `aura-cli` verbs (`sweep_family` / `walkforward_family`).
|
||||
**Status (cycle 0106, #189 — the artifact half shipped v1).** The #188 role-model pass
|
||||
re-cut the "experiment-builder API" reading of this thread (role-6b work mis-typed in
|
||||
role-2 technique): experiment intent is now **data**, not a Rust API. Cycle 0106 shipped
|
||||
both document vocabularies (process document, campaign document — `aura-research`),
|
||||
two-tier validation, the op-script-precedent introspection contract
|
||||
(`aura process|campaign introspect --vocabulary/--block/--unwired/--content-id`), and
|
||||
content-addressed stores (C18 realization note) — authorable and checkable headless, no
|
||||
compile, no run. **Status update (cycles 0107–0110).** The executor question resolved and
|
||||
shipped (cycles 0107/0108, #198/#200: `aura-campaign` executes the v2 pipeline shape over
|
||||
the `MemberRunner` seam); the #188 amendment package landed 2026-07-03 (C25 role-model
|
||||
entry, C20/C22 refinements, invariant-10 clarification). The "once it carries" dissolution
|
||||
of the hard-wired verbs (user decision 2026-07-03 on #188) is **running as the milestone
|
||||
"Verb dissolution" (#210)**: the fork triage of 2026-07-04 decided its design (dissolve
|
||||
the real-data blueprint branches only, built-in/synthetic branches stay verb-wired until
|
||||
#159; generated documents auto-registered; full behaviour parity modulo the recorded
|
||||
additive instrument stamp), and the verb set dissolved in the #210-decision-7 order: cycle 0110 took `aura sweep
|
||||
<bp.json> --real …` (enabled by the `std::sweep` selection group becoming optional —
|
||||
all-or-nothing, selection-free = terminal-stage-only; wire form and every stored content
|
||||
id unchanged), then generalize, walkforward, and mc's R-bootstrap path, each now sugar over
|
||||
a generated, content-addressed campaign document through the one executor, plus a structural
|
||||
risk-regime axis (`CampaignDoc.risk: [RiskRegime]`, sole `vol{length,k}` variant — the stop
|
||||
compared as a matrix dimension, stamped into every member manifest, never argmax-swept).
|
||||
Ad-hoc verb intent no longer evaporates into shell history — the #188 diagnosis's cure
|
||||
applied to the verbs themselves. **The dissolved form is per-verb, by intended scope**
|
||||
(ratified at milestone close, fieldtest 2026-07-07): for sweep it is the blueprint file
|
||||
(`<bp.json> --real`), while `aura sweep --strategy r-sma --real` stays the inline built-in
|
||||
path (its member lines carry no instrument/topology_hash/selection stamp and register no
|
||||
campaign document) — the built-in `--strategy` demo surface is #159's hard-wired-harness
|
||||
retirement target, not the dissolution's; for generalize/walkforward/mc the dissolved
|
||||
form is now the same generic grammar as sweep — an arbitrary blueprint positional plus
|
||||
`--real` and repeatable `--axis <wrapped-name>=<csv>` (#220). [HISTORY — superseded in
|
||||
two steps. #159 (cuts 1b-4, post-2026-07-07) retired the built-in `--strategy`
|
||||
sweep/demo surface: the inline `aura sweep --strategy r-sma --real` path no longer
|
||||
exists. #220 then removed the verbs' weld — at milestone close the dissolved form of
|
||||
generalize/walkforward/mc was still the welded `--strategy r-sma --real`; #220 made
|
||||
the three verbs blueprint-generic (arbitrary blueprint + arbitrary `--axis`; mc keeps
|
||||
the synthetic `--seeds` family unchanged beside the new `--real --axis` campaign mode;
|
||||
generalize takes `--real <SYM1,SYM2,…>`, >=2, with one value per axis), deleted the
|
||||
welded `--strategy` grammar (clap rejects it, exit 2), dissolved `RGrid`, and unified
|
||||
the verbs on the #214 invocation struct. The surviving form everywhere is `aura <verb>
|
||||
<blueprint.json> --real … --axis …` over examples/r_*.json.] Milestone closed 2026-07-07 on a green end-to-end fieldtest
|
||||
(0 bugs; behaviour preservation, campaign-substrate reach-through, and the risk-regime axis
|
||||
all confirmed); residual findings are discoverability/ergonomics follow-ups (#216 risk-axis
|
||||
discoverability, #217 verb knob asymmetry, #218 no-project store litter).
|
||||
**Amendment (2026-07-14, #256 fork B):** the dissolved walkforward/mc
|
||||
translations' leading sweep became the enumerate-only **`std::grid`** stage —
|
||||
a fieldless vocabulary block, legal only as the first stage and immediately
|
||||
before `std::walk_forward`. Only the grid's parameter points ever crossed the
|
||||
stage seam (the wf stage re-sweeps them per IS window itself), so the leading
|
||||
stage no longer executes members or persists a `Sweep` registry family:
|
||||
persisted dissolved-walkforward/mc campaign documents lose that family, while
|
||||
the visible grades are byte-identical (the exact-grade E2E pins are
|
||||
unchanged). The executor's inter-stage seam is now a typed two-armed flow
|
||||
(points-only vs executed members); report-consuming stages are fenced by
|
||||
preflight. `translate_generalize` keeps its executed `std::sweep(argmax)`
|
||||
leading stage — its generalize stage consumes the argmax winner report as the
|
||||
cell nominee.
|
||||
|
||||
- **Inferential honesty of the World — family-selection without false-discovery
|
||||
control (tracked: milestone "Inferential validation (defend against false
|
||||
discovery at sweep scale)", #144 / #145 / #146; adjacent #139).** The World's
|
||||
massively-parallel sweep (C12 axes 1–2, C21) is aura's differentiator *and* its
|
||||
most direct route to false discovery: `optimize` / `rank_by` select the single
|
||||
best family member by a bare argmax, with no penalty for the number of
|
||||
configurations tried, no preference for a robust parameter neighbourhood over a
|
||||
sharp in-sample peak, and no cross-instrument generalization read; `mc` is a
|
||||
descriptive seed-sweep, not an inferential significance test. The hygiene
|
||||
invariants keep one backtest honest (C1 determinism, C2 no look-ahead); they do
|
||||
**not** make a *selection across a family* honest. Recognized direction: the
|
||||
selection/aggregation layer must deflate a winner for the family size (#144),
|
||||
prefer a plateau over a peak (#145), and score cross-instrument generalization
|
||||
(#146), with a per-candidate out-of-sample bootstrap as the adjacent
|
||||
significance read (#139, landed cycle 0075). Distinct from the two threads above —
|
||||
neither orchestration *composability* (#109) nor a search *policy* (Bayesian/
|
||||
genetic), but the statistical *validity* of the selection itself. Not a
|
||||
C-invariant until built; recorded as the World's third half — **now structurally
|
||||
complete** (all three pieces built, cycle 0078).
|
||||
**Status (cycle 0076):** the trials-deflation piece (#144) landed —
|
||||
`optimize_deflated` (aura-registry) wraps `optimize` and records, additively on
|
||||
the winning member's manifest (C18, the new `RunManifest.selection`), a deflated
|
||||
score + an overfit probability for the family size, **without changing which
|
||||
member wins** (C23; `optimize` / `rank_by` stay a bare argmax — the deflation is
|
||||
recorded provenance, not a re-ranking). The R arm is a centred moving-block
|
||||
reality-check (reusing the `r_bootstrap` kernel); the `total_pips` arm a
|
||||
closed-form expected-max-of-K dispersion floor. **Status (cycle 0077):** the
|
||||
plateau-over-peak piece (#145) landed — `optimize_plateau` (aura-registry)
|
||||
argmaxes the **neighbourhood-smoothed** metric surface (mean or worst-case over
|
||||
each member's closed mixed-radix grid neighbourhood) instead of the bare peak,
|
||||
recorded on the same `RunManifest.selection` carrier, now an orthogonal *rule ×
|
||||
annotation*: `SelectionMode::{Argmax, PlateauMean, PlateauWorst}` with either the
|
||||
deflation annotation (#144) or the plateau annotation (`neighbourhood_score` /
|
||||
`n_neighbours`). It is opt-in via a `--select` flag — default argmax stays
|
||||
byte-identical (C23). The grid lattice surfaces from the engine
|
||||
(`SweepBinder::sweep_with_lattice`); the policy stays in aura-registry with
|
||||
`walk_forward` selection-agnostic (C9). Selection is now a *pluggable objective*
|
||||
(bare argmax → trials-deflated → plateau), not a single hard-wired argmax.
|
||||
**Status (cycle 0078):** the last piece — cross-instrument generalization (#146) —
|
||||
landed, completing the inferential half. Unlike #144/#145 (which *select + annotate*
|
||||
a within-family winner on `RunManifest.selection`), #146 is an **aggregator/
|
||||
validator**, not a selector: the `aura generalize` subcommand runs a *brought*
|
||||
candidate across an instrument list and `generalization` (aura-registry) reduces the
|
||||
per-instrument R-metrics to a **worst-case floor** (min over instruments — the
|
||||
cross-instrument sibling of `PlateauMode::Worst`, R-only per C10) + a sign-agreement
|
||||
count + the per-instrument breakdown. It is a *recomputable aggregate* over a new
|
||||
`FamilyKind::CrossInstrument` family (C12's comparison axis, realized), each member
|
||||
self-identifying via the new `RunManifest.instrument` lineage field (C18) — *not*
|
||||
stamped on `FamilySelection` (there is no within-family winner to annotate). The
|
||||
inferential half is now structurally built; the per-candidate OOS bootstrap (#139,
|
||||
cycle 0075) is its adjacent significance read.
|
||||
|
||||
- **`aura-std` contents** — substantively populated (~30 node modules): SMA/EMA,
|
||||
arithmetic (`Add`/`Sub`/`Mul`/`Sqrt`/`LinComb`), logic (`And`/`Gt`/`Latch`/`EqConst`),
|
||||
the `Delay` z⁻¹ register, `Resample`, `RollingMin`/`RollingMax`, `Session`, the R chain
|
||||
(`Bias`/`FixedStop`/`Sizer`/`PositionManagement`), the legacy `SimBroker` pip yardstick,
|
||||
the cost-model graph (`CostNode`/`CostRunner`/`CostSum` + `ConstantCost`/`VolSlippageCost`/
|
||||
`CarryCost`), and the `Recorder`/`GatedRecorder`/`SeriesReducer` sinks. Further blocks
|
||||
land demand-driven as the walking-skeleton needs them.
|
||||
|
||||
- **Sequencing** — the runnable single-harness *substrate* comes first (walking
|
||||
skeleton: a closed harness that runs deterministically and records via a sink);
|
||||
the World/meta layer (C21) and the playground trace-explorer (C22) are the
|
||||
differentiating layers that follow — you orchestrate and visualize a thing that
|
||||
must first run once.
|
||||
+176
-2924
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,25 @@
|
||||
# C1 — Determinism and disjoint parallelism: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c01-determinism.md](c01-determinism.md).
|
||||
|
||||
**Realization note (2026-07-16, #277).** Cross-sim parallelism now also spans
|
||||
campaign cells: the executor flattens the cell matrix, groups it by instrument
|
||||
ordinal, and walks sequential chunks of K instrument groups
|
||||
(`--parallel-instruments`, default 4 — a structural bound on distinct resident
|
||||
instruments, the RAM lever for the external data-server's per-reference file
|
||||
retention); within a chunk, cells run concurrently on the process-global rayon
|
||||
pool shared with the member/window fan-out. Results are collected into
|
||||
document-order slots, so outputs stay byte-identical across worker counts and
|
||||
bounds. Two deliberate scheduling-dependent carve-outs, both outside this
|
||||
contract's per-run bit-identity (which governs successful runs): on the
|
||||
run-fatal path (non-containable faults, e.g. a dead registry store) the
|
||||
propagated fault is the lowest document-order fault among the cells that
|
||||
completed before the abort flag latched, and the set of per-cell family lines
|
||||
already written by then is scheduling-dependent — inert, because no
|
||||
campaign-run record is written on that path and store reads are name-keyed,
|
||||
never line-ordered. Duplicate campaign instruments are refused at both the
|
||||
validate tier and the executor's preflight: the per-cell family name embeds
|
||||
the raw instrument string, so uniqueness is what keeps concurrent appends from
|
||||
racing one name's run-index assignment.
|
||||
@@ -0,0 +1,69 @@
|
||||
# C1 — Determinism and disjoint parallelism
|
||||
|
||||
**Guarantee.** A backtest is a deterministic, synchronous, non-concurrent event
|
||||
loop that reaches a unique state after each input tick. Same input (incl. seed)
|
||||
→ bit-identical run. Two backtests are fully disjoint and run concurrently
|
||||
without locking. The bit-identity is *per run*: one backtest of given (inputs,
|
||||
seed) reproduces byte-for-byte. It does **not** extend to a *derived metric*
|
||||
recomputed for the same params by two *different command paths* (e.g. a swept
|
||||
member's `sqn` vs the same cell re-run under `aura generalize`), which may differ
|
||||
by floating-point reassociation (≤1 ULP) because the two paths accumulate the
|
||||
same logical reduction in a different operation order (IEEE-754 non-associativity).
|
||||
C1 governs the determinism of a single run, not the cross-command bit-identity of
|
||||
a re-derived statistic. (fieldtest 0078, ratified.)
|
||||
|
||||
**Forbids.** Concurrency *within* a single sim; any nondeterministic input that
|
||||
is not captured as an explicit input (see C11, C12).
|
||||
|
||||
**Why.** Real money rides on backtest results; reproducibility and an audit
|
||||
trail are non-negotiable. Speed comes from parallelism *across* sims, which
|
||||
disjointness makes lock-free.
|
||||
|
||||
## Current state
|
||||
|
||||
The per-run event loop is `Harness::run` (`crates/aura-engine/src/harness.rs`):
|
||||
each cycle picks the live source head with the smallest `(timestamp, source
|
||||
index)`, pops one record, increments `cycle_id`, and forwards the value —
|
||||
allocating nothing per cycle beyond a reused scratch buffer. A single run is
|
||||
fully sequential, so its output is bit-identical by construction; the
|
||||
end-of-stream `finalize` pass runs after the loop and adds no within-sim
|
||||
concurrency.
|
||||
|
||||
Cross-sim parallelism spans the sweep/member/window fan-out and campaign cells
|
||||
(#277). The campaign executor `execute` (`crates/aura-campaign/src/exec.rs`)
|
||||
flattens the (strategy, instrument, window, regime) cell matrix into
|
||||
document-order `PlannedCell`s, groups them by instrument ordinal, and walks
|
||||
sequential chunks of K instrument groups — `--parallel-instruments`
|
||||
(`DEFAULT_PARALLEL_INSTRUMENTS = 4`, `crates/aura-campaign/src/lib.rs`). K is a
|
||||
structural bound on distinct resident instruments — the RAM lever for the
|
||||
external data-server's per-reference file retention (~1.4 GB per instrument on
|
||||
the reference dataset), not a pool-worker count. Within a chunk, cells run
|
||||
concurrently via `rayon::par_iter` on the process-global pool shared with the
|
||||
member/window fan-out, and results are written into document-order `slots`, so
|
||||
outputs stay byte-identical across worker counts and bounds.
|
||||
|
||||
Two scheduling-dependent carve-outs sit outside the per-run bit-identity, which
|
||||
governs successful runs. On the run-fatal path (non-containable faults, e.g. a
|
||||
dead registry store) a shared `AtomicBool` abort flag latches, and the propagated
|
||||
fault is the lowest document-order fault among the cells that completed before
|
||||
the latch; the set of per-cell family lines already written by then is
|
||||
scheduling-dependent — inert, because no campaign-run record is written on that
|
||||
path and store reads are name-keyed, never line-ordered.
|
||||
|
||||
Duplicate campaign instruments are refused at both the validate tier
|
||||
(`crates/aura-campaign/src/lib.rs`, a `seen_instruments` scan) and the executor's
|
||||
`preflight`: the per-cell registry family name embeds the raw instrument string,
|
||||
so name uniqueness is what keeps concurrent appends from racing one name's
|
||||
run-index assignment.
|
||||
|
||||
## See also
|
||||
|
||||
- [C4](c04-cycle-granularity.md) — tie determinism preserves this contract; the
|
||||
source-index tie-break is what keeps the merge order deterministic.
|
||||
- [C11](c11-sources-record-replay.md), [C12](c12-atomic-sim-unit.md) —
|
||||
recorded/pinned inputs are how nondeterministic sources become explicit inputs;
|
||||
the harness is the atomic disjoint unit.
|
||||
- [C23](c23-graph-compilation.md) — the raw-index compilation primitive; names
|
||||
survive only as non-load-bearing debug symbols.
|
||||
|
||||
> History: [c01-determinism.history.md](c01-determinism.history.md)
|
||||
@@ -0,0 +1,36 @@
|
||||
# C2 — Causality / no look-ahead
|
||||
|
||||
**Guarantee.** A node sees only the past. Input history is a read-only window
|
||||
that ends at the current cursor; a resampler emits a bar only once it is
|
||||
complete.
|
||||
|
||||
**Forbids.** Any node access to data with `timestamp > now`; emitting a partial
|
||||
/ still-forming bar.
|
||||
|
||||
**Why.** Look-ahead is the cardinal backtester bug — a fast backtester that
|
||||
leaks the future is worse than none. Making the future *physically absent* from
|
||||
what a node receives beats merely discouraging it.
|
||||
|
||||
## Current state
|
||||
|
||||
A node's `eval` receives `Ctx::new(&inputs, ts)` (`crates/aura-engine/src/harness.rs`):
|
||||
the input windows are read-only slices whose head (`window[0]`) is the value
|
||||
pushed at or before the current `cycle_id`. The engine forwards exactly one
|
||||
merged record per cycle in ascending timestamp order, so a node structurally
|
||||
cannot reach a record with `timestamp > now` — the future is absent from what it
|
||||
receives, not merely off-limits.
|
||||
|
||||
The resampler (`crates/aura-market/src/resample.rs`) emits a completed coarse bar
|
||||
only on bucket rollover; a still-forming bar is never emitted, and an incomplete
|
||||
final bucket with no following rollover tick simply never emits. The completed
|
||||
bar carries no timestamp of its own — it takes the rollover instant, the close
|
||||
instant of the bar just emitted (C4).
|
||||
|
||||
## See also
|
||||
|
||||
- [C3](c03-single-merge.md) — the single ingestion merge is what makes a
|
||||
read-only, monotone cursor possible.
|
||||
- [C4](c04-cycle-granularity.md) — the data-driven cycle clock the cursor
|
||||
advances along.
|
||||
- [C5](c05-freshness-gating.md) — held (stale) values are real past values, not
|
||||
a look-ahead escape hatch.
|
||||
@@ -0,0 +1,36 @@
|
||||
# C3 — One merge, at ingestion only
|
||||
|
||||
**Guarantee.** Heterogeneous timestamped sources are k-way-merged by timestamp
|
||||
into one chronological cycle stream at the ingestion boundary; source-native
|
||||
time units (e.g. data-server's Unix-`time_ms`) are normalized there to the
|
||||
canonical epoch-ns `timestamp` of C7.
|
||||
|
||||
**Forbids.** Any merge / as-of join *inside* the graph.
|
||||
|
||||
**Why.** A single ordered timeline is the mechanism that makes heterogeneous-rate
|
||||
sources (news daily-bias + M5 + ticks) causally combinable without leaking the
|
||||
future. Keeping the merge at one boundary keeps the graph semantics simple.
|
||||
|
||||
## Current state
|
||||
|
||||
The k-way merge is `Harness::run`'s per-cycle pick of the live source head with
|
||||
the smallest `(timestamp, source index)` (`crates/aura-engine/src/harness.rs`).
|
||||
Each `Source` yields records in ascending timestamp order — the ingestion
|
||||
precondition the merge relies on — and there is no merge or as-of join anywhere
|
||||
inside the DAG; the engine only routes in-graph edges.
|
||||
|
||||
The ms→epoch-ns normalization is a single seam, `aura_ingest::unix_ms_to_epoch_ns`
|
||||
(`crates/aura-ingest/src/lib.rs`): the data-source ingestion edge transposes the
|
||||
data-server's Array-of-Structs `M1Parsed` records into aura's Structure-of-Arrays
|
||||
base columns (C7) and normalizes their Unix-millisecond time to the canonical
|
||||
epoch-ns `Timestamp` at this one boundary. Both the eager `load_m1_window`
|
||||
transpose and the lazy streaming `M1FieldSource` route through that same seam.
|
||||
|
||||
## See also
|
||||
|
||||
- [C4](c04-cycle-granularity.md) — the merged stream is the data-driven cycle
|
||||
clock; ties break by source declaration order.
|
||||
- [C2](c02-causality.md) — one ordered timeline is what makes heterogeneous-rate
|
||||
sources combinable without look-ahead.
|
||||
- [C7](c07-scalar-soa.md) — the canonical epoch-ns `Timestamp` and the four SoA
|
||||
base columns the merge normalizes into.
|
||||
@@ -0,0 +1,15 @@
|
||||
# C4 — Cycle granularity: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c04-cycle-granularity.md](c04-cycle-granularity.md).
|
||||
|
||||
**Realization (#275, 2026-07-15).** Ingestion sources are supplied to `run` **by
|
||||
role name**, not by list position. `SourceSpec` carries a `role: Option<String>`
|
||||
(the bound `Role`'s name, load-bearing for source binding), and `Harness::run_bound`
|
||||
resolves a keyed supply against those roles, emitting sources in `SourceSpec`
|
||||
declaration order. The C4 tie-break stays "source declaration order" — now
|
||||
independent of how the caller orders the supply — and a mis-bound feed is a named
|
||||
`SourceBindError` at start time rather than a silently wrong run. The raw-index
|
||||
`run(Vec)` primitive (positional, C23) is unchanged; every hand-built graph keeps
|
||||
`role: None`.
|
||||
@@ -0,0 +1,43 @@
|
||||
# C4 — Cycle granularity
|
||||
|
||||
**Guarantee.** The clock is data-driven: one input record = one cycle, advanced
|
||||
in global timestamp order, with a monotonic `cycle_id`. Ties (same timestamp,
|
||||
multiple sources) break by source declaration order.
|
||||
|
||||
**Forbids.** A fixed time-grid clock; nondeterministic tie ordering.
|
||||
|
||||
**Why.** The market *is* an irregular event sequence; a grid is arbitrary and
|
||||
either wastes empty cycles or clumps ticks. Backtest and live differ only in the
|
||||
origin of records, not the cycle semantics. Tie determinism preserves C1.
|
||||
|
||||
## Current state
|
||||
|
||||
`Harness::run` (`crates/aura-engine/src/harness.rs`) increments `cycle_id` once
|
||||
per popped record and each cycle picks the smallest `(timestamp, source index)`;
|
||||
the source-index tie-break is the source *declaration* order.
|
||||
|
||||
Ingestion sources are supplied to the production run path by role name, not list
|
||||
position (#275). `SourceSpec` carries `role: Option<String>` — the bound `Role`'s
|
||||
name, load-bearing for source binding only, with every other flat-graph name
|
||||
staying a non-load-bearing debug symbol (C23). `Harness::run_bound` resolves a
|
||||
keyed supply against those roles via `bind_sources`, emitting sources in
|
||||
`SourceSpec` declaration order, so the tie-break is independent of how the caller
|
||||
orders the supply. A mis-bound feed is a named `SourceBindError`
|
||||
(`MissingFeed` / `ExtraFeed` / `DuplicateFeed` / `UnnamedSource`) at start time,
|
||||
not a silently wrong run. The raw-index `run(Vec)` primitive (positional, C23) is
|
||||
unchanged: `SourceSpec::raw` carries `role: None` and every hand-built graph keeps
|
||||
`role: None`; a raw-index arity mismatch stays an engine-wiring panic, while a
|
||||
keyed role mismatch is a user data-binding `Result`.
|
||||
|
||||
## See also
|
||||
|
||||
- [C1](c01-determinism.md) — tie determinism is a precondition of the per-run
|
||||
bit-identity.
|
||||
- [C3](c03-single-merge.md) — the single ingestion merge is the clock's source
|
||||
stream.
|
||||
- [C23](c23-graph-compilation.md) — the raw-index compilation primitive; names
|
||||
non-load-bearing except the binding role.
|
||||
- [C5](c05-freshness-gating.md) — `cycle_id` advances every cycle, but recompute
|
||||
is gated.
|
||||
|
||||
> History: [c04-cycle-granularity.history.md](c04-cycle-granularity.history.md)
|
||||
@@ -0,0 +1,35 @@
|
||||
# C5 — Freshness-gated recompute and sample-and-hold
|
||||
|
||||
**Guarantee.** The `cycle_id` advances everywhere (a cheap counter), but a node
|
||||
re-evaluates only when ≥1 of its own inputs is fresh this cycle (detected by
|
||||
run-count); otherwise it holds its last output. Stale inputs contribute their
|
||||
last (held) value.
|
||||
|
||||
**Forbids.** Recomputing every node every cycle ("push all" is true for the
|
||||
*clock*, not for *recompute*); treating a held value as missing.
|
||||
|
||||
**Why.** Total recompute does not scale to many sparse high-frequency sources;
|
||||
freshness-gating is the performance discipline that keeps the synchronous model
|
||||
fast.
|
||||
|
||||
## Current state
|
||||
|
||||
Freshness is a run-count epoch. Each input slot carries `SlotState { fresh_at,
|
||||
last_ts }` (`crates/aura-engine/src/harness.rs`); a slot is fresh this cycle iff
|
||||
`fresh_at == cycle_id`. The firing predicate `fires` re-evaluates a node when any
|
||||
`Firing::Any` input is fresh this cycle, or when a barrier group completes (≥1
|
||||
member fresh this cycle and every member carrying `last_ts == ts`) — the barrier
|
||||
/ co-freshness machinery itself is C6.
|
||||
|
||||
A node that does not fire pushes nothing, so downstream consumers keep reading the
|
||||
last value via `window[0]`: sample-and-hold falls out of the push model rather
|
||||
than being a separate mechanism. A held value is a real past value, never treated
|
||||
as missing.
|
||||
|
||||
## See also
|
||||
|
||||
- [C6](c06-firing-policy.md) — the firing predicate and the barrier /
|
||||
co-freshness rule this gating builds on.
|
||||
- [C1](c01-determinism.md) — freshness is a deterministic function of `cycle_id`,
|
||||
so gating never perturbs the bit-identical run.
|
||||
- [C2](c02-causality.md) — a held value is past data, not look-ahead.
|
||||
@@ -0,0 +1,14 @@
|
||||
# C6 — Firing policy A and B, per input group: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c06-firing-policy.md](c06-firing-policy.md).
|
||||
|
||||
**Realization (cycle 0004).** Firing is tagged per input — `Firing::{Any,
|
||||
Barrier(u8)}` on `InputSpec` — and inputs sharing a `Barrier` id form a group; a
|
||||
mode-A input is its own trivial group, so "per input group" and the per-input tag
|
||||
coincide. The barrier's synchronization token is the cycle **timestamp**, not the
|
||||
`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so
|
||||
RustAst's `cycle_id`-equality barrier could never fire across them. A group fires
|
||||
when every member's last push carries the current cycle's timestamp, guarded by
|
||||
"≥1 member fresh this cycle" (so a group completed earlier does not re-fire). This
|
||||
fires both the multi-source bar and the within-source diamond rejoin (every push
|
||||
in a cycle carries that cycle's timestamp).
|
||||
@@ -0,0 +1,42 @@
|
||||
# C6 — Firing policy A and B, per input group
|
||||
|
||||
**Guarantee.** A node declares, per input group, one of two firing policies:
|
||||
**A** fire-on-any-fresh + hold (latest / as-of join — e.g. tick × held
|
||||
daily-bias); **B** all-fresh barrier (synchronizing join — e.g. O/H/L/C from
|
||||
four separate 15m sources: the candle is complete only when all four are fresh).
|
||||
A single node may mix an A input and a B group, and it fires when **any** of its
|
||||
input groups fires (OR over groups).
|
||||
|
||||
**Forbids.** A single global firing mode; forcing per-node-only granularity.
|
||||
|
||||
**Why.** Both are genuinely needed; RustAst, the retired predecessor, implemented
|
||||
only B. Per-input-group granularity is required by the OHLC-plus-bias case where
|
||||
one node needs both an A input and a B group at once.
|
||||
|
||||
## Current state
|
||||
|
||||
Firing is tagged per input — `Firing::{Any, Barrier(u8)}`
|
||||
(`crates/aura-core/src/node.rs`) carried on each input's `PortSpec.firing`, not a
|
||||
separate group object. Inputs sharing a `Barrier` id form a group; a mode-A input
|
||||
is its own trivial group, so "per input group" and the per-input tag coincide.
|
||||
|
||||
The barrier's synchronization token is the cycle **timestamp**, not the
|
||||
`cycle_id`: under C4 four same-timestamp sources are four distinct cycles, so a
|
||||
`cycle_id`-equality barrier could never fire across them — the token has to be the
|
||||
data timestamp of the push.
|
||||
|
||||
Firing is evaluated by `fires()` in `crates/aura-engine/src/harness.rs`: a mode-A
|
||||
input fires the node when it is fresh this cycle (`fresh_at == cycle_id`), a stale
|
||||
input contributing its held value; a barrier group fires when ≥1 member is fresh
|
||||
this cycle **and** every member's last push carries the current cycle's timestamp.
|
||||
The "≥1 fresh" guard is what stops a group completed in a prior cycle from
|
||||
re-firing. This one rule fires both the multi-source bar and the within-source
|
||||
diamond rejoin (C3), since every push in a cycle carries that cycle's timestamp.
|
||||
|
||||
## See also
|
||||
- [C3](c03-single-merge.md) — the one-merge boundary; the within-source diamond rejoin the barrier also completes
|
||||
- [C4](c04-cycle-granularity.md) — cycle/timestamp granularity that forces the barrier token to be a timestamp
|
||||
- [C5](c05-freshness-gating.md) — freshness-gated recompute and sample-and-hold (mode A's hold semantics)
|
||||
- [C8](c08-node-contract.md) — the node signature declares each input's firing group
|
||||
|
||||
> History: [c06-firing-policy.history.md](c06-firing-policy.history.md)
|
||||
@@ -0,0 +1,32 @@
|
||||
# C7 — Four scalar base types, streamed as SoA: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c07-scalar-soa.md](c07-scalar-soa.md).
|
||||
|
||||
**Realization (`Cell` carrier split, 2026-06).** The streamed value is now split
|
||||
into a tag-free 64-bit word and its kind. `Cell` (`crates/aura-core/src/cell.rs`)
|
||||
is a type-erased `u64`: constructed per base type (`from_i64/f64/bool/ts`) and
|
||||
read only by naming the type at the call site (`i64()/f64()/bool()/ts()` —
|
||||
branch-free bit-casts). The kind is therefore resolved once at the boundary and
|
||||
the value itself carries no tag — C7's "the type lives at the column/edge, not in
|
||||
the value" made explicit on the single-value carrier (and a single 8-byte word
|
||||
vs. the 16-byte tagged enum). `Scalar` becomes `{ kind: ScalarKind, cell: Cell }`
|
||||
— the self-describing form for the *dynamic* boundaries (builder binding, serde,
|
||||
rendering) — with `debug_assert`-guarded native accessors (caller asserts the
|
||||
kind; free in release) and a hand-written **value** `PartialEq` that preserves
|
||||
the former enum's IEEE-754 semantics (`NaN != NaN`, `+0.0 == -0.0`), pinned by
|
||||
the `scalar_eq_is_value_not_bitwise` fixture. Behaviour-preserving.
|
||||
|
||||
**Realization (`Cell` becomes the hot-path carrier, 2026-06, #74).** The carrier
|
||||
swap deferred above has landed: `Node::eval` now returns `Option<&[Cell]>` and
|
||||
every node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free
|
||||
8-byte words. The kind lives only at the schema/column: the harness forwards each
|
||||
field via the new branch-free `AnyColumn::push_cell` (infallible — the edge
|
||||
kind match is verified once at bootstrap, the surviving guard
|
||||
`bootstrap_rejects_*_kind_mismatch`). `Scalar` remains on the self-describing
|
||||
dynamic boundaries: the param plane (`build`/`bind`/`compile_with_params`/sweep
|
||||
points/`RunManifest`), `AnyColumn::get` (the type-erased read for sinks/serde),
|
||||
and source ingestion (`Source::next`, the heterogeneous C3 merge). The removed
|
||||
per-value runtime kind check on node output is the same authoring-bug class C8
|
||||
already leaves to a `debug_assert` (output width); node-output-kind correctness
|
||||
is the node's declared `FieldSpec` contract, caught by each node's own test.
|
||||
Behaviour-preserving (C1).
|
||||
@@ -0,0 +1,64 @@
|
||||
# C7 — Four scalar base types, streamed as SoA
|
||||
|
||||
**Guarantee.** Only `i64`, `f64`, `bool`, `timestamp` (newtype over i64,
|
||||
epoch-ns UTC) are streamed, as columnar Structure-of-Arrays. Composite streams
|
||||
(OHLCV) are bundles of base columns — this is the **node-output model** too: a
|
||||
node emits a record of 1..K base columns (C8), each forwarded field-wise to a
|
||||
consumer slot; the bundle is structural, never a fifth scalar type. Edges are
|
||||
type-erased to these four kinds; the type check is paid once at wiring/sim-start,
|
||||
then the topology is frozen per sim → direct dispatch, no per-event allocation.
|
||||
|
||||
**Forbids.** Streaming non-scalars (String, Records, tables, calendars) — those
|
||||
live as metadata beside the hot path; `dyn Any` payloads; per-event heap
|
||||
allocation; topology mutation mid-sim.
|
||||
|
||||
**Why.** Maximal streaming performance (SIMD/cache) needs a tiny closed scalar
|
||||
set and SoA. The open set is composites (schemas of columns), not scalar types.
|
||||
Type-erasure at the edge is also forced by the cdylib boundary (C13).
|
||||
|
||||
## Current state
|
||||
|
||||
The streamed value is split into a tag-free 64-bit word and its kind. `Cell`
|
||||
(`crates/aura-core/src/cell.rs`) is the type-erased `u64` hot-path carrier:
|
||||
constructed per base type (`from_i64/from_f64/from_bool/from_ts`) and read only by
|
||||
naming the type at the call site (`i64()/f64()/bool()/ts()` — branch-free
|
||||
bit-casts, no tag to check). The kind lives once, at the column/edge/port, never
|
||||
in the value — C7's principle made explicit on the single-value carrier (one
|
||||
8-byte word, not a 16-byte tagged enum). `Cell` deliberately has no tagged
|
||||
constructors, so a value can never re-acquire a per-value tag once it crosses onto
|
||||
the hot path.
|
||||
|
||||
`Node::eval` returns `Option<&[Cell]>` (`crates/aura-core/src/node.rs`) and every
|
||||
node out-buffer is `[Cell; N]`, so the inter-node forward carries tag-free 8-byte
|
||||
words. The harness forwards each field via `AnyColumn::push_cell`
|
||||
(`crates/aura-core/src/any.rs`) — branch-free and infallible, because the edge's
|
||||
`from_field`→slot kind match is verified once at bootstrap, not per value. There
|
||||
is deliberately no per-value runtime kind check on node output: node-output-kind
|
||||
correctness is the node's declared `FieldSpec` contract (C8), caught by each
|
||||
node's own test — the same authoring-bug class C8 already leaves to a
|
||||
`debug_assert` for output width.
|
||||
|
||||
`Scalar` (`crates/aura-core/src/scalar.rs`) is the self-describing, kind-tagged
|
||||
carrier for the *dynamic* boundaries, where a value travels detached from any
|
||||
co-present schema and so must remember its own type: the param plane (builder
|
||||
binding via `bind`, sweep points, `RunManifest`), `AnyColumn::get` (the
|
||||
type-erased read for sinks/serde), and source ingestion (`Source::next →
|
||||
Option<(Timestamp, Scalar)>`, the heterogeneous C3 merge). `Scalar` and `Cell`
|
||||
are **disjoint**, bridged by `Scalar::from_cell(kind, cell)` (decode, the kind
|
||||
supplied by the co-present schema — e.g. a validated param point carries bare
|
||||
cells, its kind living once in the co-indexed `ParamSpec`) and `Scalar::cell()`
|
||||
(encode, kind dropped once checked against the slot). Reading a `Scalar` is
|
||||
caller-asserted: `as_i64/as_f64/as_bool/as_ts` panic on a kind mismatch (a caller
|
||||
bug). `Scalar`'s value equality preserves IEEE-754 semantics (`NaN != NaN`,
|
||||
`+0.0 == -0.0`) and never equates across kinds (`i64(0) != f64(0.0)`), pinned by
|
||||
the `scalar_eq_is_value_not_bitwise` fixture; `Cell`'s own `Eq`/`Hash` are
|
||||
bit-exact (the semantics of a raw word, not of a number).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — behaviour-preservation / bit-identity across the carrier split
|
||||
- [C3](c03-single-merge.md) — heterogeneous ingestion, where `Scalar` is the ingest carrier
|
||||
- [C8](c08-node-contract.md) — the node-output record and `eval`'s Cell buffer; `FieldSpec` output-kind contract
|
||||
- [C13](c13-hot-reload-frozen-deploy.md) — the cdylib boundary that also forces edge type-erasure
|
||||
- [C23](c23-graph-compilation.md) — bootstrap compilation; the once-at-wiring edge kind check; names non-load-bearing
|
||||
|
||||
> History: [c07-scalar-soa.history.md](c07-scalar-soa.history.md)
|
||||
@@ -0,0 +1,159 @@
|
||||
# C8 — The node contract: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c08-node-contract.md](c08-node-contract.md).
|
||||
|
||||
**Realization (cycle 0005).** `NodeSchema.output` is a `Vec<FieldSpec>` (named base
|
||||
columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field`
|
||||
selects one producer column per edge; consuming a whole record is N edges (no
|
||||
"bind whole record" mechanism). The K fields of one record are **co-fresh by
|
||||
construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns
|
||||
`Option<&[Cell]>` — a borrowed row into a node-owned buffer — so the forward
|
||||
path allocates nothing per cycle (C7) (the carrier is now a tag-free `Cell` — see
|
||||
the C7 carrier note).
|
||||
|
||||
**Realization (cycle 0006).** The pure-consumer (sink) half of this contract is
|
||||
now realized at the substrate: **recording is a node role, not a type.** A
|
||||
recording node reads its typed input windows + `ctx.now()` in `eval` and pushes
|
||||
the record to a destination it holds as a field (a channel, a chart handle) — an
|
||||
**out-of-graph side effect**. There is no `Sink` type, trait, or engine flag: a
|
||||
node that only records returns `None` (pure consumer), and a node may record
|
||||
**and** return an output the engine forwards in the same `eval` (the "both"
|
||||
case). **Encoding & return contract.** A pure consumer declares `output: vec![]`
|
||||
— the empty record *is* the sink declaration; there is no separate type, trait,
|
||||
or marker. Its `eval` returns `None` or a zero-width `Some(&[])`, and the run
|
||||
loop debug-asserts the returned row's width equals the declared output width
|
||||
(`row.len() == schema.output.len()`). Field-wise wiring resolves
|
||||
`Edge::from_field` against the producer's `output` at bootstrap, so no edge can
|
||||
bind a field of a zero-output node — it fails with `BadIndex` — making a sink
|
||||
structurally unwireable as an in-graph producer; its only output is the
|
||||
out-of-graph side effect. In-graph routing stays engine-owned data (the edge table); the escape out
|
||||
of the graph is the node's own side effect — and that boundary is the
|
||||
determinism / graph-as-data boundary (C1/C7).
|
||||
|
||||
**Realization (cycle 0015 — param declaration).** The tunable-parameter half of
|
||||
this contract is now realized: a node declares its knobs in `schema()` as
|
||||
`params: Vec<ParamSpec>` (`ParamSpec { name: String, kind: ScalarKind }`), and
|
||||
`Composite::param_space()` (cycle 0024; was `Blueprint::param_space()`) aggregates
|
||||
them into one flat, path-qualified list — a
|
||||
read-only projection of the graph-as-data (C9), mirroring the inline order
|
||||
(C19/C23) so a param's slot matches the later flat-node order. Two refinements to
|
||||
the guarantee's "typed, with ranges": (1) the declaration carries `name` + `kind`
|
||||
only — the **search-range is the run's, not the node's** (which subset / grid a
|
||||
sweep covers is an experiment axis, #32/C20; the node declares the knob's existence
|
||||
and type, never its search interval). (2) Identity is **positional** (the slot,
|
||||
C23 "by index, not name"); the path-qualified `name` is a non-load-bearing debug
|
||||
symbol (like `FieldSpec.name`) — uniqueness is at the slot. **Refinement (cycle
|
||||
0032):** identity in the **flat graph** stays positional (C23, unchanged), but the
|
||||
`param_space()` **name projection** — the authoring / by-name address space (the
|
||||
surface a sweep axis or single-run binding addresses) — must be **injective** for a
|
||||
blueprint to compile (a duplicated path is unaddressable; see C9). Two layers:
|
||||
positional wiring below (the flat graph), an injective name address space above (the
|
||||
authoring boundary). So same-type siblings in one composite no longer silently
|
||||
share a name — they must be `.named(...)` apart, which is also what disambiguates a
|
||||
same-type fan-in (C9). A vector knob (`LinComb.weights`) expands to `N` flat
|
||||
`weights[i]` entries, `N` topology-fixed (C19). Permitted kinds are `i64`/`f64`/
|
||||
`bool` (a `timestamp` knob is a structural axis, C20, never a numeric sweep param).
|
||||
Binding a value to a slot landed in cycle 0016 (#31, see C19/C23); enumerating a
|
||||
sweep (#32) is the deferred next layer. The 0016 binding also moved the param
|
||||
declaration's *authoring* home into the value-empty recipe, whose `params()` is read
|
||||
pre-build. In 0016–0023 the built node's `schema().params` still reported the same
|
||||
slots, kept in lockstep by a per-node test (a duplication filed as debt, #36); **cycle
|
||||
0024 dissolved this** — the signature is declared *once* on the recipe and the built
|
||||
node no longer carries `schema()` (#36 closed). See the C8 cycle-0024 realization.
|
||||
|
||||
**Realization (cycle 0024 — the signature lives in the blueprint, #43/#36).** The
|
||||
node's whole declared interface — its `NodeSchema` (input scalar types + firing,
|
||||
output record, params) — now lives **once**, on the value-empty recipe
|
||||
`PrimitiveBuilder` (ex-`LeafFactory`), read pre-build by `param_space()`, the
|
||||
compile-time structural validation, and the render. This closes two debts: **#43**
|
||||
(a value-empty recipe used to declare *no* input/output interface pre-build — only
|
||||
params) and **#36** (params declared twice, recipe vs built node, kept in lockstep
|
||||
by a per-node test — those 8 tests are deleted, the duplication structurally gone).
|
||||
The split that makes this work: a node's signature is **fully static** per blueprint
|
||||
(input kinds/firing, output fields, params — verified across the roster; a variable-
|
||||
arity node like `LinComb` takes its arity as a *recipe argument*, not an injected
|
||||
param), so it can be declared without building; the **one** param-dependent quantity,
|
||||
an input's buffer **lookback** depth, is no longer in the signature but answered by
|
||||
`Node::lookbacks() -> Vec<usize>`, read only by bootstrap to size the windows.
|
||||
`Node::schema()` is therefore **removed** — the signature is pre-build data, not a
|
||||
built-node method. `BlueprintNode::signature()` answers uniformly for both arms (a
|
||||
primitive returns its recipe's schema; a composite *derives* it from the interior:
|
||||
role kinds in, re-exported field kinds out, aggregated params), so "every node has a
|
||||
signature in the blueprint" holds for composites too.
|
||||
|
||||
**Realization (cycle 0027 — name input ports, refs #21/#51).** `PortSpec` now
|
||||
carries a `name: String` (it drops `Copy`, like `ParamSpec`), so an input port is
|
||||
named just as `FieldSpec.name` (output) and `ParamSpec.name` (param) already are.
|
||||
The name is **non-load-bearing** (C23): wiring stays positional by slot, bootstrap
|
||||
and the run loop never read it; it exists for tracing / graph rendering (#13). Leaf
|
||||
primitives declare their slot names (`SimBroker`'s `exposure`/`price`, etc.);
|
||||
`derive_signature` carries a composite's `Role.name` into the derived input port,
|
||||
so the graph model (`model_to_json`) is homogeneously named across inputs, outputs,
|
||||
and params and across both graph levels. This does **not** close #21 (a swapped
|
||||
same-kind wiring is still only kind-checked) — it makes those slots
|
||||
self-documenting; a name-consuming validation is its own future cycle.
|
||||
|
||||
**Realization (cycle 0040 — wiring totality: every input slot connected exactly
|
||||
once, #65).** The node contract's "the engine provides a window into each input"
|
||||
presupposes each input is actually fed; until now an unwired interior input slot was
|
||||
accepted and bootstrapped to a silent empty column (`harness.rs`), and two producers
|
||||
into one slot — ill-formed, since a slot holds one column — was likewise
|
||||
uncompiled-against. Cycle 0040 makes a valid graph **total and single-valued** over
|
||||
its interior input slots: `check_ports_connected` (in `validate_wiring`, run at every
|
||||
nesting level) requires every interior node's every declared input slot to be covered
|
||||
by **exactly one** wiring act — one interior `Edge { to, slot }` or one role
|
||||
`Target { node, slot }`, edges and role targets counted uniformly. Zero coverage is
|
||||
`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`. A
|
||||
composite's **own** input roles (`source: None`) are coverage *providers* (the
|
||||
wired-by-enclosing boundary, the root case already guarded by `UnboundRootRole`) —
|
||||
never consumers, so only interior input slots are subject to the rule. There is **no
|
||||
optional-input concept**: every declared input port is required (no shipped node runs
|
||||
meaningfully without one; an unwarmed mode-A input is *wired-but-not-yet-valued*, not
|
||||
unwired). The check is **index-based and name-free** — it touches no name machinery
|
||||
and emits nothing into the flat graph, so C23 is untouched (it proves the existing
|
||||
raw-index wiring is total and single-valued). Inherited identically by the raw
|
||||
`Composite::new` path and the `GraphBuilder::build()` path (both compile via
|
||||
`compile_with_params`).
|
||||
|
||||
**Refinement (Construction-layer milestone — render labels, 2026-06-05).** A node
|
||||
additionally exposes `label() -> String`, a **single-line, non-load-bearing**
|
||||
render symbol: a default trait method the run loop never calls (wiring is by
|
||||
index, C23). Overrides carry the node's identifying params (`SMA(2)` vs `SMA(4)`)
|
||||
so a graph render (C9 graph-as-data, #13) disambiguates identical node types and
|
||||
surfaces a mis-wiring. Like `FieldSpec.name`, it is an informative debug symbol,
|
||||
not part of the C8 dataflow contract — adding it changes no run behaviour.
|
||||
|
||||
**Refinement (cycle 0070 — end-of-stream `finalize` lifecycle, 2026-06-25).** The
|
||||
node contract gains a second lifecycle phase beside `eval`: `finalize(&mut self)`,
|
||||
a **default-no-op** trait method the engine calls **once per node, in topological
|
||||
order, after the source loop drains** (the `Harness::run` epilogue). It lets a sink
|
||||
**fold online** — accumulate per `eval` into O(trades)/O(1) owned state and flush
|
||||
one compact summary at stream end — instead of pushing a record every fired cycle
|
||||
into an unbounded channel that buffers O(cycles) rows until the run ends (the
|
||||
retention BLOCKER #138 hit: a full-history Stage-1 sweep held ~2 GiB/member over
|
||||
~5.5M one-minute bars). C1/C7/C8 are preserved: `finalize` runs once *after* the
|
||||
deterministic event loop, adding no within-sim concurrency (C1); a folding sink
|
||||
holds only owned accumulator state, no interior mutability (C7); it still declares
|
||||
`output: vec![]` and its flush is the same out-of-graph side effect (C8) — one
|
||||
summary row, not a per-cycle stream. Realized by `GatedRecorder` (emits only gated
|
||||
rows + the genuine final row) and `SeriesReducer` (folds one column, emits one
|
||||
summary row), aura-std siblings of the per-cycle `Recorder`, which survives for the
|
||||
live / `--trace` / test-tap path. The recording sink's *own* per-cycle allocation
|
||||
(the `Recorder`→`Probe` rename + its accumulate-vs-stream choice, #77) stays open;
|
||||
`finalize` is the "non-channel exit from the graph" that issue's
|
||||
accumulate-then-read option named as missing.
|
||||
|
||||
**Realization (2026-07-11 — composite `doc`, #125).** `Composite` carries an
|
||||
optional authored rationale `doc: Option<String>` — the prose twin of its `name`
|
||||
and the same C23 category: a non-load-bearing debug symbol. Authored via
|
||||
`Composite::with_doc` / `GraphBuilder::doc`; persisted as a Tier-1
|
||||
additive-optional `CompositeData` field (no format-version bump; absent-field
|
||||
documents keep their exact bytes, so existing content ids are untouched);
|
||||
**blanked in the identity projection** (`strip_debug_symbols`) while staying
|
||||
canonical-byte-bearing; dissolved at inline like the name (never reaches
|
||||
`FlatGraph`). Surfaced read-only (C22): the graph model emits an optional
|
||||
trailing `"doc"` fragment per scope (`json_str` hardened for multi-line free
|
||||
text — `\n`/`\t`/`\r` named, other control chars as `\u00XX`), the viewer shows
|
||||
it in both composite view states (collapsed tooltip, expanded cluster frame) and
|
||||
the root's as a muted header line. The construction op-script vocabulary
|
||||
deliberately has no doc-carrying surface yet (scope cut recorded on #125).
|
||||
@@ -0,0 +1,146 @@
|
||||
# C8 — The node contract
|
||||
|
||||
**Guarantee.** A node is the universal composable dataflow unit — a **producer, a
|
||||
consumer, or both**. It has a **signature**, its `NodeSchema`: each input port
|
||||
(scalar kind + firing group), its output record, and its own **tunable
|
||||
parameters** — each a **typed knob** (`name` + scalar kind). The node declares a
|
||||
knob's *existence and type*, never its search interval: which subset/grid a sweep
|
||||
covers is the run's concern (an experiment axis, C20), not the node's. The knobs
|
||||
aggregate into the blueprint's flat, path-qualified param-space the optimizer
|
||||
sweeps (C12/C19/C20). The whole signature is **declared once, pre-build, on the
|
||||
value-empty recipe** (C19), so the entire interface is legible without building;
|
||||
the built node carries no `schema()`. The one param-dependent quantity — an
|
||||
input's buffer **lookback** depth (e.g. an `Sma`'s window = its `length`) — is not
|
||||
in the signature but answered by `lookbacks()`, read once at bootstrap to size the
|
||||
windows. The built node implements `eval(ctx) -> Option<&[Cell]>`: the engine
|
||||
provides read-only, zero-copy windows into each input's SoA ring buffer
|
||||
(`ctx.f64_in(i)[k]`, sized at wiring), and `eval` returns a borrowed row — one
|
||||
`Cell` per declared output column — into a node-owned buffer, so the forward path
|
||||
allocates nothing per cycle (C7). A node may *additionally* keep its own mutable
|
||||
series for derived/intermediate state. Binding is **field-wise**: one edge selects
|
||||
one producer column, so consuming a whole record is N edges — there is no
|
||||
"bind whole record" mechanism. The K columns of one record are **co-fresh by
|
||||
construction** (one `eval`, one timestamp), so C6 is untouched.
|
||||
|
||||
A **producer/transformer** exposes **one output port**, whose payload is a
|
||||
**record of 1..K base-scalar columns** (a scalar is the degenerate K=1 record); a
|
||||
**pure consumer (sink)** — chart, equity, logger — declares **no output** and
|
||||
records via an out-of-graph side effect. **Recording is a node role, not a type:**
|
||||
a node declaring `output: vec![]` *is* a sink (the empty record is the whole
|
||||
declaration — no `Sink` type, trait, or engine flag), its `eval` returns `None` or
|
||||
a zero-width `Some(&[])`, and it pushes its record to a destination it holds as a
|
||||
field (a channel, a chart handle). A node may record **and** return a forwarded
|
||||
output in the same `eval` (the "both" case). `None` = filter / not-yet-warmed-up /
|
||||
pure sink. Sources are pure producers; sinks are pure consumers. Beside `eval`,
|
||||
the contract has a second lifecycle phase, `finalize()` — a default-no-op hook the
|
||||
engine calls once per node, in topological order, after the source loop drains —
|
||||
letting a sink fold online and flush one compact summary at stream end instead of
|
||||
streaming a row every fired cycle. A node also exposes `label()`, a single-line,
|
||||
non-load-bearing render symbol (C23).
|
||||
|
||||
**Forbids.** A node sizing/growing its input lookback at runtime; more than one
|
||||
output **port** per node; a fifth scalar type or a heterogeneous output payload (a
|
||||
record is a bundle of base columns, C7); copy-on-read of input history; a `Sink`
|
||||
type/trait/engine flag or a "bind whole record" mechanism (both are structurally
|
||||
absent by design).
|
||||
|
||||
**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage lookback
|
||||
bookkeeping, and history passes through zero-copy. Fixed, pre-sized buffers suit
|
||||
deterministic, pre-dimensioned sims (no realloc in the hot loop). Declaring the
|
||||
signature once, pre-build on the recipe, keeps the whole interface legible without
|
||||
building and dissolves the drift of declaring params twice (recipe vs built node).
|
||||
The empty-output sink keeps in-graph routing engine-owned data (the edge table)
|
||||
and pushes every escape *out* of the graph onto the node's own side effect — that
|
||||
boundary is the determinism / graph-as-data boundary (C1/C7). `finalize` gives a
|
||||
folding sink O(trades)/O(1) owned accumulator state and one summary row, instead of
|
||||
an unbounded channel that buffers O(cycles) rows until the run ends; it runs once
|
||||
*after* the deterministic event loop, adding no within-sim concurrency (C1), and
|
||||
holds only owned state, no interior mutability (C7).
|
||||
|
||||
## Current state
|
||||
|
||||
The signature types and the `Node` trait live in `aura-core/src/node.rs`.
|
||||
`NodeSchema { inputs: Vec<PortSpec>, output: Vec<FieldSpec>, params: Vec<ParamSpec> }`;
|
||||
`PortSpec { kind, firing, name }`, `FieldSpec { name, kind }`, and
|
||||
`ParamSpec { name, kind }` each carry a `name` that is a **non-load-bearing** debug
|
||||
symbol (C23) — wiring is positional by slot; bootstrap and the run loop never read
|
||||
it. Permitted param kinds are `i64`/`f64`/`bool` (a `timestamp` knob is a
|
||||
structural axis, C20, never a numeric sweep param).
|
||||
|
||||
`trait Node` declares `lookbacks() -> Vec<usize>`, `eval(&mut self, ctx) ->
|
||||
Option<&[Cell]>`, `label() -> String` (default a placeholder every shipped node
|
||||
overrides; overrides carry identifying params so `SMA(2)` vs `SMA(4)` disambiguate
|
||||
in a graph render, #13), and `finalize(&mut self)` (default no-op). There is **no**
|
||||
`Node::schema()` — the signature is pre-build data, not a built-node method. The
|
||||
signature is declared once on the value-empty recipe `PrimitiveBuilder`
|
||||
(`aura-core/src/node.rs`), whose `schema()` / `params()` are read pre-build by the
|
||||
param-space aggregation, the structural validation, and the render; `named(...)`
|
||||
sets a node instance's authoring name and `bind(slot, value)` fixes a knob to a
|
||||
structural constant, removing it from the param surface.
|
||||
|
||||
Field-wise binding is `Edge::from_field` (`aura-engine/src/blueprint.rs`), resolved
|
||||
against the producer's `output` at bootstrap. A sink's empty `output` therefore
|
||||
makes it structurally unwireable as an in-graph producer — no edge can bind a field
|
||||
of a zero-output node. The run loop (`aura-engine/src/harness.rs`) debug-asserts
|
||||
`row.len()` equals the declared output width, and that `lookbacks()` arity equals
|
||||
`signature().inputs.len()`. The end-of-stream `finalize` epilogue is realized in
|
||||
`Harness::run` (`aura-engine/src/harness.rs`), which calls `finalize()` once per
|
||||
node in topological order after the source loop drains; `GatedRecorder` and
|
||||
`SeriesReducer` (`aura-std`) are the folding siblings of the per-cycle `Recorder`
|
||||
(`aura-std/src/recorder.rs`), which survives for the live / `--trace` / test-tap
|
||||
path.
|
||||
|
||||
`BlueprintNode::signature()` (`aura-engine/src/blueprint.rs`) answers uniformly for
|
||||
both arms: a primitive returns its recipe's schema, a composite *derives* it via
|
||||
`derive_signature` (input-port kinds from interior target slots, output-field kinds
|
||||
from re-exported `OutField`s, aggregated params). `Composite::param_space()`
|
||||
(same file) aggregates every node's declared params into one flat, path-qualified
|
||||
list mirroring the inline order (C19/C23) — a read-only projection of the
|
||||
graph-as-data (C9). Identity in the flat graph is positional; the `param_space()`
|
||||
**name projection** (the by-name authoring/sweep address space) must be
|
||||
**injective** for a blueprint to compile, enforced by
|
||||
`check_param_namespace_injective` → `CompileError::DuplicateParamPath`, so
|
||||
same-type siblings in one composite must be `.named(...)` apart (also what
|
||||
disambiguates a same-type fan-in, C9). A vector knob (`LinComb.weights`) expands to
|
||||
`N` flat `weights[i]` entries, `N` topology-fixed (C19).
|
||||
|
||||
Wiring is **total and single-valued** over interior input slots:
|
||||
`check_ports_connected`, run inside `validate_wiring` at every nesting level,
|
||||
requires every interior node's every declared input slot to be covered by
|
||||
**exactly one** wiring act (one interior `Edge { to, slot }` or one role
|
||||
`Target { node, slot }`, counted uniformly). Zero coverage is
|
||||
`CompileError::UnconnectedPort`, more than one is `CompileError::DoubleWiredPort`.
|
||||
A composite's **own** input roles (`source: None`) are coverage *providers* (the
|
||||
wired-by-enclosing boundary, the root case guarded by `UnboundRootRole`), never
|
||||
consumers. There is no optional-input concept — every declared input port is
|
||||
required. The check is index-based and name-free, so C23 is untouched.
|
||||
|
||||
A `Composite` carries an optional authored rationale `doc: Option<String>`
|
||||
(`aura-engine/src/builder.rs`: `GraphBuilder::doc` / `Composite::with_doc`) — the
|
||||
prose twin of its `name`, the same C23 non-load-bearing category. It is persisted
|
||||
as an additive-optional `CompositeData` field (`aura-engine/src/blueprint_serde.rs`;
|
||||
absent-field documents keep their exact bytes, so existing content ids are
|
||||
untouched), blanked in the identity projection by `strip_debug_symbols`, dissolved
|
||||
at inline like the name, and surfaced read-only in the graph model and viewer
|
||||
(C22).
|
||||
|
||||
Deferred: the recording sink's own per-cycle allocation — the `Recorder`→`Probe`
|
||||
rename and its accumulate-vs-stream choice — stays open (#77). Naming input ports
|
||||
(`PortSpec.name`) makes swap-prone same-kind slots self-documenting but does not add
|
||||
a name-consuming wiring validation; a swapped same-kind wiring is still only
|
||||
kind-checked (#21). The construction op-script vocabulary has no doc-carrying
|
||||
surface yet (scope cut on #125).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — the deterministic, non-concurrent event loop; `finalize` runs once after it; the determinism / graph-as-data boundary.
|
||||
- [C6](c06-firing-policy.md) — the co-freshness of a record's K columns.
|
||||
- [C7](c07-scalar-soa.md) — the four base scalar types, the tag-free `Cell` carrier, records as bundles of base columns, zero per-cycle allocation.
|
||||
- [C9](c09-fractal-composition.md) — graph-as-data; `param_space` as a read-only projection; same-type fan-in disambiguation by name.
|
||||
- [C12](c12-atomic-sim-unit.md) — the aggregated param-space the optimizer sweeps.
|
||||
- [C18](c18-registry.md) — `FieldSpec.name` as metadata for sinks and the playground.
|
||||
- [C19](c19-bootstrap.md) — the value-empty recipe, the pre-build signature, and inline order.
|
||||
- [C20](c20-strategy-harness.md) — the sweep axis; the search-range is the run's; a `timestamp` axis is structural.
|
||||
- [C22](c22-playground-traces.md) — read-only surfacing of the composite `doc` and the blueprint view.
|
||||
- [C23](c23-graph-compilation.md) — names as non-load-bearing debug symbols; positional (by-index) identity.
|
||||
|
||||
> History: [c08-node-contract.history.md](c08-node-contract.history.md)
|
||||
@@ -0,0 +1,85 @@
|
||||
# C9 — Fractal, acyclic composition: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c09-fractal-composition.md](c09-fractal-composition.md).
|
||||
|
||||
**Refinement (Construction-layer milestone, 2026-06-05).** "A composite is itself
|
||||
a `Node`" is an **authoring-level** identity: a composite declares the same
|
||||
interface (typed inputs + ≤1 output, C8) and is wireable wherever a node is, but it
|
||||
is **not** a runtime object. The bootstrap **compiles it away by inlining** its
|
||||
sub-graph into the one flat instance (C19/C23): the composite *boundary* dissolves
|
||||
into the raw index wiring the run loop already consumes, and names stay
|
||||
**non-load-bearing** (informative debug symbols only, as `FieldSpec.name` already
|
||||
is). So "nestable arbitrarily" and "graph-as-data" hold at the **blueprint
|
||||
(source)** level; the running graph is the flat, index-wired **`FlatGraph`**. The earlier reading — *a composite survives as a `Box<dyn Node>`
|
||||
driving a nested sub-engine* — is **explicitly rejected**: it would keep the
|
||||
interior opaque to the cross-graph optimiser (C23) and add a runtime sub-loop the
|
||||
flat model does not need. Inlining is what makes the composite boundary free.
|
||||
|
||||
**Realization (cycle 0018 — composite multi-output record, #40).** "Exposes one
|
||||
output" is one output **port** carrying a **record of 1..K re-exported fields**, not
|
||||
one field: `Composite.output` is a `Vec<OutField { node, field, name }>` (was a
|
||||
single `OutPort`). Each entry is a **named projection** of one interior
|
||||
`(node, output-field)`; a consumer selects which re-exported field it reads via the
|
||||
same `Edge::from_field` that already binds leaf record columns (C8 realization,
|
||||
cycle 0005). This is the **same arity** C8 already grants a leaf (OHLCV = one port,
|
||||
5 columns): a multi-line indicator (MACD = macd/signal/histogram) is **one** record
|
||||
of K fields, **one** port, **one** row per `eval` — C8/C7/C4 untouched, a boundary
|
||||
completion, not a contract change. A strategy composite is simply the K=1 case
|
||||
(one bias field, C10). The re-export **names** are **non-load-bearing** (C23):
|
||||
they live at the blueprint boundary and in render (cycle 0022/#46 folds each onto
|
||||
its producing node as a `name := …` binding; originally `[out:<name>]` markers, #13)
|
||||
but are dropped at lowering — `ItemLowering::Composite.output` is `Vec<(usize,
|
||||
usize)>`, raw index pairs only, so the flat graph is name-free (verified: the
|
||||
compiled-view render stayed bit-identical across this change).
|
||||
|
||||
**Realization (cycle 0019 — name the composite boundary, #41; param-overlay retired,
|
||||
cycle 0031).** The named-projection shape covers the surviving boundary edge-kinds:
|
||||
`input_roles` is a `Vec<Role { name, targets }>` (was a bare `Vec<Vec<Target>>`),
|
||||
alongside `output: Vec<OutField>`. Each is an ordered, positionally-indexed
|
||||
**named projection** of interior handles. **Param projection is no longer a
|
||||
composite overlay** (the index-addressed `ParamAlias` was retired in cycle 0031):
|
||||
a node's surface param name flows from its own **instance name** — every node
|
||||
carries a name (default = its lowercased type label, override via `.named()`), and
|
||||
`param_space()` is uniformly `<node>.<param>` at every level including the root. A
|
||||
same-type fan-in is distinguished by naming the colliding legs, the same single act
|
||||
that qualifies their param paths. Like the output and role names, **node names are
|
||||
non-load-bearing** (C23): they live at the blueprint boundary and in render but are
|
||||
dropped at lowering — the flat graph is wired by raw index. The full composite
|
||||
boundary signature (named inputs, multi-outputs) and the per-node param path are
|
||||
legible without changing the flat graph.
|
||||
|
||||
**Refinement (param-namespace injectivity, cycle 0032; supersedes the fan-in
|
||||
distinguishability check).** A blueprint compiles only if its `param_space()` name
|
||||
projection is **injective** — every path-qualified knob name is unique. A duplicated
|
||||
path is a knob no binding can select alone (it has no distinct by-name address,
|
||||
C12/C19), so it is a `CompileError::DuplicateParamPath` carrying the offending path;
|
||||
the cure is to give the colliding same-type sibling nodes distinct names with
|
||||
`.named(...)`, the same single act that qualifies their param paths. The
|
||||
param-bearing indistinguishable fan-in is *one instance* of a duplicated path (two
|
||||
default-named same-type legs share a leaf path). `signature_of`, `leaf_has_param`,
|
||||
and the fan-in-specific check (`check_fan_in_distinguishability` /
|
||||
`check_composite_fan_in`) are **retired** (cycle 0032), replaced by one structural
|
||||
`check_param_namespace_injective` over the `param_space()` names, run before name
|
||||
resolution in `compile_with_params` and both binders — so the canonical by-name
|
||||
author sees the structural cure rather than a downstream `AmbiguousKnob` symptom
|
||||
(that `BindError` arm is retired too: an injective space can never multi-match).
|
||||
Paramless interchangeable same-name sources stay legal (no path, no duplicate).
|
||||
The old signature-collision predicate had extra breadth — it also rejected an
|
||||
**asymmetric param/paramless** collision and a **role-vs-leg** collision, *neither*
|
||||
a path duplicate (each colliding configuration keeps a unique param path, or none).
|
||||
That breadth guarded **render identity**, dead since the renderer was retired in
|
||||
0026; both extra rejections are **intentionally dropped**. A future node-/wiring-name
|
||||
distinguishability check, if ever wanted (e.g. for the WASM graph view's #21
|
||||
thread), is decoupled from param-space injectivity. Construction-phase only; the
|
||||
flat graph stays name-free (C23).
|
||||
|
||||
**Refinement (2026-06-29 — graph-as-data round-trips both ways, C24).** "The built
|
||||
graph is introspectable runtime data" is now contract-level **bidirectional**: a
|
||||
blueprint is not only *emitted* as data (`model_to_json`, the render half) but is
|
||||
itself a **serializable data value with a load path** (data → blueprint →
|
||||
`FlatGraph`), so topology is a value the World generates, stores, and reproduces —
|
||||
see C24. The Rust builder API stays the primary *human / LLM* authoring surface
|
||||
(C17); the data form is the *machine* surface the World owns. (Load path **realised**
|
||||
cycle 0087 / #155, `d5602ec`: `aura-engine::blueprint_from_json`.)
|
||||
@@ -0,0 +1,88 @@
|
||||
# C9 — Fractal, acyclic composition
|
||||
|
||||
**Guarantee.** A composite is an **authoring-level** `Node`: it declares the same
|
||||
interface (typed inputs + ≤1 output, C8), is wireable wherever a node is, wires an
|
||||
interior sub-graph, and exposes one output **port** carrying a **record of 1..K
|
||||
re-exported fields** — signal, combined signal, and (with execution) strategy are
|
||||
the same abstraction, nestable arbitrarily. A composite is **not a runtime
|
||||
object**: the bootstrap compiles it away by **inlining** its sub-graph into the one
|
||||
flat instance (C19/C23), so "nestable arbitrarily" and "graph-as-data" hold at the
|
||||
**blueprint (source)** level while the running graph is the flat, index-wired
|
||||
`FlatGraph`. The dataflow graph is a DAG; the only feedback path is an explicit
|
||||
delay/state node (the RTL "register", C1). Wiring is written in Rust (builder API,
|
||||
C17); the built graph is introspectable runtime data and, per C24,
|
||||
**bidirectional** — a blueprint is not only *emitted* as data but is itself a
|
||||
serializable data value with a load path (data → blueprint → `FlatGraph`), so
|
||||
topology is a value the World generates, stores, and reproduces. The Rust builder
|
||||
stays the primary human/LLM authoring surface; the data form is the machine surface
|
||||
the World owns.
|
||||
|
||||
**Forbids.** Implicit dataflow cycles (combinational loops); special-casing
|
||||
"signal-of-signals" as separate mechanics; a composite surviving as a runtime
|
||||
sub-engine (a `Box<dyn Node>` driving a nested sub-loop); a blueprint whose
|
||||
`param_space()` name projection is non-injective — a path-qualified knob name that
|
||||
repeats, since it then has no distinct by-name address (C12/C19).
|
||||
|
||||
**Why.** Self-application of one contract gives unlimited composition with no
|
||||
adapter zoo. Acyclicity keeps the synchronous reactive model well-defined; forcing
|
||||
feedback through a visible delay node keeps per-cycle determinism intact (C1) and
|
||||
the one legitimate feedback path explicit. Graph-as-data enables visualization,
|
||||
freezing, and re-parameterization for sweeps, and — round-tripping both ways (C24)
|
||||
— lets the World own topology as content it serializes, loads, and generates rather
|
||||
than as Rust baked into the binary. Inlining, not a runtime sub-object, is what
|
||||
makes the composite boundary free: it keeps the interior transparent to the
|
||||
cross-graph optimiser (C23) and adds no runtime sub-loop the flat model does not
|
||||
need; the boundary dissolves into the raw index wiring the run loop already
|
||||
consumes, and names stay non-load-bearing (C23).
|
||||
|
||||
## Current state
|
||||
|
||||
A composite is `Composite` (`aura-engine::blueprint`), a reusable sub-graph fragment
|
||||
that is never `eval`'d. It holds interior items, interior edges (local indices),
|
||||
`input_roles: Vec<Role>`, and `output: Vec<OutField>`.
|
||||
|
||||
- **Multi-output record.** `output` is `Vec<OutField { node, field, name }>` — each
|
||||
entry a **named projection** of one interior `(node, output-field)`. A consumer
|
||||
selects which re-exported field it reads via `Edge::from_field`, the same binding
|
||||
that already selects a leaf record's columns (C8). This is the same arity C8
|
||||
grants a leaf (OHLCV = one port, 5 columns): a multi-line indicator (MACD =
|
||||
macd/signal/histogram) is one record of K fields, one port, one row per `eval`. A
|
||||
strategy composite is the K=1 case (one bias field, C10).
|
||||
- **Named boundary.** `input_roles` is `Vec<Role { name, targets }>` — role `r`
|
||||
fans its source value into the interior `targets`. Inputs and outputs are ordered,
|
||||
positionally-indexed named projections of interior handles.
|
||||
- **Per-node param path.** A node's surface param name flows from its own **instance
|
||||
name**: every node carries a name (default = its lowercased type label, override
|
||||
via `Node::named`, `aura-core::node`), and `param_space()`
|
||||
(`aura-engine::blueprint`) is uniformly `<node>.<param>` at every level including
|
||||
the root. A same-type fan-in is distinguished by naming the colliding legs — the
|
||||
same single act that qualifies their param paths.
|
||||
- **Param-namespace injectivity.** A blueprint compiles only if its `param_space()`
|
||||
name projection is injective. A duplicated path is a knob no binding can select
|
||||
alone (C12/C19), rejected as `CompileError::DuplicateParamPath` carrying the
|
||||
offending path; the cure is `.named(...)` on the colliding same-type siblings. One
|
||||
structural check, `check_param_namespace_injective` (`aura-engine::blueprint`),
|
||||
runs before name resolution in `compile_with_params` and both binders. Paramless
|
||||
interchangeable same-name sources stay legal (no path, no duplicate).
|
||||
- **Names dropped at lowering.** Output, role, and node names are non-load-bearing
|
||||
(C23): they live at the blueprint boundary and in render, and are dropped at
|
||||
inline — `ItemLowering::Composite` carries `output: Vec<(usize, usize)>` (raw index
|
||||
pairs) and `roles: Vec<Vec<Target>>`, so the flat graph is name-free and wired by
|
||||
raw index.
|
||||
- **Bidirectional data form.** The emit half is `model_to_json`
|
||||
(`aura-engine::graph_model`); the load path is realised —
|
||||
`aura-engine::blueprint_from_json` (`aura-engine::blueprint_serde`, #155) parses
|
||||
the data form back to a blueprint that compiles to a `FlatGraph` (C24).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — determinism, acyclic dataflow, the delay/state register.
|
||||
- [C8](c08-node-contract.md) — the node contract: ≤1 output, record columns, `Edge::from_field`.
|
||||
- [C10](c10-bias-r-cost.md) — the strategy as a K=1 bias composite; the cost-model graph.
|
||||
- [C12](c12-atomic-sim-unit.md) — binding by name; why a duplicated knob path is unbindable.
|
||||
- [C17](c17-authoring-surface.md) — the Rust builder as the human/LLM authoring surface.
|
||||
- [C19](c19-bootstrap.md) — bootstrap: blueprint → instance; construction-phase checks.
|
||||
- [C23](c23-graph-compilation.md) — names non-load-bearing; the cross-graph optimiser; inlining.
|
||||
- [C24](c24-blueprint-data.md) — the blueprint as a serializable, World-owned data value.
|
||||
- [C28](c28-stratification.md) — the layer ladder: construction phase vs the run loop.
|
||||
|
||||
> History: [c09-fractal-composition.history.md](c09-fractal-composition.history.md)
|
||||
@@ -0,0 +1,386 @@
|
||||
# C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c10-bias-r-cost.md](c10-bias-r-cost.md).
|
||||
|
||||
**Realization (cycle 0007).** [HISTORY — pre-reframe; `SimBroker` is now legacy per
|
||||
the 2026-06-28 reframe.] The signal-quality half was realized at the substrate as
|
||||
two `aura-std` nodes on the unchanged engine (the engine stays domain-free — it
|
||||
routes only `f64` records). The **exposure stream** was realized as `Exposure {
|
||||
scale }`: `clamp(signal / scale, -1, +1)`, one `f64` per fired cycle. The
|
||||
**sim-optimal broker** was realized as `SimBroker { pip_size }`: a two-input node
|
||||
(exposure, price) accumulating `prev_exposure · (price − prev_price) / pip_size`
|
||||
and emitting cumulative pip equity — the exposure held *into* a cycle (decided at
|
||||
t-1) earns that cycle's return (causal, C2); `pip_size` is held reference metadata
|
||||
(C7/C15). An end-to-end harness (SMA-cross → `Exposure` → `SimBroker` → recording
|
||||
sink) produced a recorded pip-equity curve, bit-identical across runs (C1).
|
||||
|
||||
**Realization (per-instrument pip channel, 2026-06, #22).** [HISTORY — pertains to
|
||||
the legacy `SimBroker` pip channel.] `SimBroker`'s `pip_size` is sourced **per
|
||||
instrument** from the recorded geometry sidecar (`instrument_geometry`, over
|
||||
data-server's `symbol_meta`), at the ingestion / source edge (never `Aura.toml`,
|
||||
never `Ctx`); the engine stays domain-free. (The original cycle-0022 form was a
|
||||
Rust-authored vetted floor `InstrumentSpec { pip_size }` + `instrument_spec(symbol)`;
|
||||
cycle 0074 removed that floor — see the C15 note — once the sidecar geometry made it
|
||||
redundant for the real path. Refuse-don't-guess on absent geometry.) The honesty
|
||||
rule is **refuse, don't guess**: a real-data run for a symbol with no vetted spec
|
||||
is a usage error (`exit 2`). Threaded through the CLI `aura run --real` path; the
|
||||
manifest broker label records the looked-up pip.
|
||||
|
||||
**Realization (position-event schema, cycle 0063, #114).** [Survives as the
|
||||
**deploy / reconciliation** schema per the 2026-06-28 reframe — no longer a
|
||||
broker-input research artifact.] A closed `PositionAction { Buy, Sell, Close }`
|
||||
enum + the `PositionEvent` row (`event_ts`, `action`, `position_id`,
|
||||
`instrument_id`, unsigned `volume`; no `open_ts`; direction *is* the action) live
|
||||
beside `RunMetrics` as a post-run value type (not a per-`eval` node — C8) — both in
|
||||
the `aura-analysis` crate since cycle 0079 (#136); originally `aura-engine`, see the
|
||||
C16 cycle-0079 note. `action` serde-encodes as a bare `i64` (Buy=0, Sell=1,
|
||||
Close=2), the C7 scalar column form, with an out-of-range code rejected on read.
|
||||
The table stays broker-independent.
|
||||
|
||||
**Reframe (2026-06-23, #117 — exposure → bias, R as the signal-quality unit).**
|
||||
[HISTORY — its R spine survives into the 2026-06-28 contract; its Stage-2 currency /
|
||||
realistic-broker / register / flat-1R-vs-compounding portions are SUPERSEDED by that
|
||||
reframe.] The contract was reframed from **exposure** (a signed fractional
|
||||
position) to an **unsized bias** plus **R** as the signal-quality unit. The
|
||||
realization notes below describe the **pre-reframe code** (`Exposure`, `SimBroker`,
|
||||
pip-equity), retained as history and as the **ancestor** of the current chain:
|
||||
`Exposure { scale }` is the ancestor of the unsized `bias` node, and `SimBroker`'s
|
||||
pip integral is the ancestor of the R-evaluator (which additionally requires a
|
||||
stop, since R is stop-defined). The `exposure → bias` rename, the RiskExecutor /
|
||||
Sizer / Veto nodes, and the R-evaluator **landed in cycle 0065**; the
|
||||
position-event schema (0063, #114) survives as the audit layer. Industry grounding
|
||||
for this reframe: LEAN / nautilus_trader / backtrader / QSTrader / vectorbt /
|
||||
zipline (see #117 decision log).
|
||||
|
||||
**Realization (cycle 0065 — Stage-1 R signal quality, #119/#126/#127/#128/#129).**
|
||||
[The R spine here is the live model; the *Stage-2 deferral* clause at its end is
|
||||
SUPERSEDED by the 2026-06-28 reframe — there is no Stage-2 currency / compounding
|
||||
layer; cost is now the cost-model graph and money lives only at the live deploy
|
||||
edge. "Stage-1" reads as a historical identifier, not a gate half.] `Exposure →
|
||||
Bias` renamed the unsized strategy output (node + output field); the persisted
|
||||
`exposure_sign_flips` metric key (serde alias), the `SimBroker` `exposure` input
|
||||
slot, and the on-disk `exposure` **tap** label retain the old name. The
|
||||
strategy-output **param namespace** (the `Bias` instance, its `bias.scale` knob,
|
||||
the `bias_scale` manifest param) was completed in #134. A **stop-rule** defines 1R:
|
||||
`FixedStop` (a triggered-constant primitive) and a `vol_stop(length, k)`
|
||||
**composition** `k·√EMA(Δ²)` (a composition of `Mul` / `Sqrt` primitives).
|
||||
**`PositionManagement`** (`aura-std`) is the stateful heart: it latches the
|
||||
entry-cycle stop distance as the immutable R-denominator, marks against the
|
||||
one-cycle-lagged fill (no look-ahead, C2), and emits a **dense 14-column per-cycle
|
||||
R-record** (one row per eval, C8; the trade ledger is the `closed_this_cycle`
|
||||
subset, the R-equity is `cum_realized_r + unrealized_r`). The **`Sizer`** (`size =
|
||||
risk_budget / stop_distance`, flat-1R) was the feed-forward sizing seam, and **R is
|
||||
size-invariant** — scaling `risk_budget` leaves every `realized_r` unchanged
|
||||
(pinned by a RED test); per the 2026-06-28 reframe the Sizer and `size` / `volume`
|
||||
are removed from research (size is a deploy concept). **`summarize_r`** is a
|
||||
post-run fold (sibling of `summarize`, **not** an in-graph node) → `RMetrics` (E[R],
|
||||
SQN, win-rate, profit-factor, max-R-drawdown, conviction terciles, net-of-cost);
|
||||
per the 2026-06-28 reframe it folds the cost-model's net-R rather than recomputing
|
||||
a scalar cost. The **RiskExecutor** ships as a public `aura-composites`
|
||||
composite-builder (`risk_executor(StopRule, risk_budget)`) with a
|
||||
`StopRule{Fixed,Vol}` **structural axis** (C20); per the 2026-06-28 reframe its
|
||||
Sizer interior and `risk_budget` arg are dropped / vestigial, the **Veto** stays a
|
||||
documented seam, not a runtime node. The layer is operable from the CLI: `aura run
|
||||
--harness <sma|macd|r-sma>` — a compile-time selector over Rust-authored
|
||||
harnesses (C9/C17) — folds `summarize_r` into `RunMetrics.r`, the `r-sma`
|
||||
harness fanning one bias into both `SimBroker` (legacy pip) and the RiskExecutor
|
||||
(R); an `r_equity` tap charts the by-trade R-equity. Composites live in the
|
||||
dedicated `aura-composites` crate, so `aura-engine`'s runtime dependency stays
|
||||
`aura-core`-only and `aura-std` is an `aura-engine` `[dev-dependencies]` (the graph
|
||||
stays acyclic).
|
||||
[HISTORY — the built-in `--harness` selector was retired with the demos →
|
||||
blueprint-data (#159, cuts 1b-4); `run` is now blueprint-driven — `aura run
|
||||
<blueprint.json>` over examples/r_*.json.]
|
||||
|
||||
**Realization (cycle 0066).** SQN is the operational single-number objective for
|
||||
ranking an r-sma sweep family by signal quality — C12 **axis-2 (argmax-metric)**
|
||||
over the C18 family store. `metric_cmp` (`aura-registry`) learns the
|
||||
higher-is-better R metrics `sqn`, `expectancy_r`, `net_expectancy_r`; a member
|
||||
with no `r` block sorts last (`NEG_INFINITY`). `aura sweep --strategy r-sma`
|
||||
produces the rankable R family, each member folding `summarize_r` into
|
||||
`RunMetrics.r`. The default grid varies **only the signal** (`fast` / `slow` SMA
|
||||
lengths), holding the stop and sizing fixed: `risk_budget` is R-invariant and
|
||||
`bias.scale` is sign-only under flat-1R, and — load-bearing — the **stop defines
|
||||
1R**, so varying it would change what R *means* per member and break cross-member
|
||||
SQN comparability. Each swept member's manifest records the fixed R-defining params
|
||||
beside the floated knobs (reproducible from its own manifest, C18).
|
||||
[HISTORY — the built-in `--strategy` sweep surface was retired with the demos →
|
||||
blueprint-data (#159, cuts 1b-4); the rankable R sweep now runs as `aura sweep
|
||||
<blueprint.json> --axis …` over r_sma_open.json (an example then; relocated to
|
||||
`crates/aura-cli/tests/fixtures/` by #248).]
|
||||
|
||||
**Realization (cycle 0067, #130 + #135).** **#130 (SQN100):** `RMetrics` gains
|
||||
`sqn_normalized = (mean_R/stdev_R)·√(min(n, 100))` — the n-normalized "SQN score"
|
||||
(`SQN_CAP = 100`), turnover-robust where raw `sqn` rewards trade count. It is an
|
||||
**opt-in** rank key (`Metric::SqnNormalized`); raw `sqn` and the default ranker
|
||||
stay byte-unchanged, and the field carries `#[serde(default)]` (C18). Below the cap
|
||||
(`n ≤ 100`) it equals raw `sqn` exactly. **#135 (r-sma `--trace`):**
|
||||
`r_sma_sweep_family` persists each member's equity / exposure / r_equity under
|
||||
`runs/traces/<n>/<member_key>/` via the same `persist_traces_r` the single run uses;
|
||||
per-member `--trace` is symmetric across all three sweep strategies.
|
||||
[HISTORY — the built-in `--strategy` sweep triple (sma / momentum / r-sma) was
|
||||
retired with the demos → blueprint-data (#159, cuts 1b-4). The per-member
|
||||
`--trace` symmetry was NOT carried to the `aura sweep <blueprint.json>` path — it
|
||||
was silently dropped at #159/#220 (`run_blueprint_sweep` never wired `persist`;
|
||||
`let _ = persist`), and #168 makes the surface honest: `sweep`/`walkforward` (like
|
||||
the pre-existing `run`/`mc`) now refuse `--trace` outright. See the CLI-`--trace`
|
||||
retirement amendment below.]
|
||||
|
||||
**Realization (cycle 0068, #115 — position-event derive).** [The derivation
|
||||
survives as the **deploy / reconciliation** layer per the 2026-06-28 reframe — its
|
||||
"realistic brokers consuming it" goal is retired; money is the live-edge concern.]
|
||||
`derive_position_events(record, instrument_id) -> Vec<PositionEvent>` (in
|
||||
`aura-analysis` since cycle 0079, #136; originally `aura-engine`, beside
|
||||
`summarize_r`) is the **first difference of the executed book**: a pure post-run
|
||||
reduction over the `PositionManagement` dense record (read positionally as
|
||||
type-erased `Scalar`s, C7 SoA — no in-graph node, so the hot path stays
|
||||
domain-free, C14), emitting a `Buy` / `Sell` at each open and a `Close` at each
|
||||
exit, a reversal (or stop-then-same-cycle reopen) emitting **Close then the
|
||||
opposite open at one `event_ts`** (close first — the C8 ">1 event per instant" case
|
||||
that forces a *derived* table, not a per-`eval` output). The close sizes the
|
||||
**actual book** (the closed position's stored volume), never an exposure delta.
|
||||
`instrument_id` is a caller-supplied scalar (`aura-analysis` depends only on
|
||||
`aura-core`). A position open at window end emits its open with **no synthetic
|
||||
`Close`** (the table records actual executed events; `summarize_r`'s force-close is
|
||||
for the R metric only). The `r_col` ⟷ PM-record lockstep is guard-pinned for
|
||||
`direction` too.
|
||||
|
||||
**Reframe (2026-06-28, #116 — realistic broker retired; cost-model graph in R; money to the live edge).**
|
||||
This is the live contract above. Ratified in an in-context design discussion
|
||||
(reference issue #116). It **preserves** the durable spine — unsized bias stream
|
||||
(sign = direction, magnitude = conviction), signal quality in R, the stop defining
|
||||
R, and the decoupling of direction from sizing from fill — and the shipped Stage-1
|
||||
realizations (the `Bias` node, `FixedStop` / `vol_stop`, `PositionManagement`,
|
||||
`summarize_r` / `RMetrics` / `sqn_normalized`, SQN as ranking objective). The
|
||||
**RiskExecutor composite survives in shape** (bias + price → stop-rule →
|
||||
position-management) but with its **Sizer interior removed** and its `risk_budget`
|
||||
argument dropped / vestigial (it sized the now-removed Sizer); the Veto remains an
|
||||
optional documented seam. The contract **supersedes** the following, which were
|
||||
**design intent, largely unbuilt** (#116 — the realistic broker and the whole
|
||||
Stage-2 currency layer were never implemented; the Stage-1 R chain was) and are
|
||||
retired:
|
||||
- the **"realistic broker"** concept — an authored-friction historical broker —
|
||||
rejected as "horseshoe-throwing" (real friction is not historically knowable),
|
||||
replaced by the **cost model**: a composable C9 graph of cost nodes (in
|
||||
`aura-std` / `aura-composites`), approximating not claiming realism,
|
||||
generalizing / subsuming `round_trip_cost` into `net_expectancy_r`;
|
||||
- the **"Currency P&L is Stage 2"** paragraph in full, the **Stage-1-vs-Stage-2
|
||||
hard sequencing gate**, and **currency / fixed-fractional / compounding** in
|
||||
research — compounding is now a post-hoc money-management transform of the
|
||||
net-R sequence at the deploy / account layer;
|
||||
- the **`z⁻¹` register on the fill edge** and the **flat-1R-vs-compounding
|
||||
structural axis** as research mechanism — there is no equity → size edge in
|
||||
research, so no register and no such axis in the loop;
|
||||
- the **Sizer in research** and currency **size / `volume`** — size is a deploy
|
||||
concept; the research executor is stop + position-management in R;
|
||||
`PositionManagement`'s `size` port is dropped or constant-driven (no dangling
|
||||
port, C8), its `size` field and the event table's `volume` are vestigial in
|
||||
research;
|
||||
- the **"a broker is an ordinary in-graph node / no special external broker
|
||||
subsystem"** forbid, **for the live edge only**: in-graph brokers are retired
|
||||
outright, and the live broker is now an explicit **I/O adapter** at the C11
|
||||
recording / C13 deploy edge — not an in-graph node and not part of the research
|
||||
graph (the no-in-graph-broker-subsystem prohibition still holds inside the
|
||||
graph);
|
||||
- the **position-event table as the realistic-broker input** and its
|
||||
first-difference-of-the-book (`deal = target − book − in_flight`) execution
|
||||
framing — the table (schema 0063 #114, derive 0068 #115) **survives** as the
|
||||
**deploy / reconciliation** artifact (real volume), not a research artifact and
|
||||
not "fed to a broker" in research.
|
||||
Money, a real broker, and cTrader Open API are a **separate, later live /
|
||||
deploy-edge** concern (C11 record-then-replay; C13 frozen-deploy invariant) — the
|
||||
only `belastbare` ground truth, measured, never modelled. The honesty principle is
|
||||
explicit: the net-R curve is a research / ranking hypothesis; the forward / live
|
||||
run is ground truth. `SimBroker` is downgraded to a legacy / optional pip
|
||||
yardstick, not to be expanded. (Terminology note: with Stage 2 gone, the
|
||||
**"Stage-1"** label below is no longer one half of a two-stage gate — it survives
|
||||
only as a historical cycle name, like the `exposure` → `bias` on-disk alias,
|
||||
denoting the shipped feed-forward R chain; the identifier family that carried it
|
||||
was renamed to the r-family — `r-sma` / `r-breakout` / `r-meanrev` — in cycle
|
||||
0100, #174.)
|
||||
|
||||
**Realization (cycle 0081 — cost-model graph, cycle 1, #148).** The first concrete
|
||||
cost node + the net-R seam shipped. `ConstantCost` (`aura-std`) is an ordinary
|
||||
downstream node (C9) emitting a 3-field cost-in-R record `{cost_in_r, cum_cost_in_r,
|
||||
open_cost_in_r}` isomorphic to `PositionManagement`'s R-triple; R-pure
|
||||
(`cost_per_trade / |entry − stop|`, notional cancels, no equity held). The scalar
|
||||
`round_trip_cost` argument of `summarize_r` is **retired**: `summarize_r` folds a
|
||||
co-temporal cost stream (positional 1:1 join — `cost[i]` is `record[i]`'s cycle) into
|
||||
`net_expectancy_r`, one home for cost / no double-count, byte-identical to the old
|
||||
cost = 0 baseline on an empty stream. The headline sink is **`net_r_equity`** =
|
||||
`LinComb(4)[cum_realized_r, unrealized_r, −cum_cost_in_r, −open_cost_in_r]` → Recorder
|
||||
(C8/C18), a sibling of `r_equity`, emitted only when a cost is authored. Wired on the
|
||||
**run path** via `--cost-per-trade` (`r_sma_graph`); sweep / walk-forward / mc pass
|
||||
`None` (cost on the reduce-mode sweep path and cost in OOS pooling deferred — the
|
||||
positional join holds only while one cost node fires in lockstep with PM).
|
||||
[Superseded 2026-07-11 (#234): the `--cost-*` run-path flags are gone (#221 removed
|
||||
that surface); cost is authored as the campaign document's `cost` block and reaches
|
||||
every family/campaign member — see the cycle-net-r realization below.] Deferred to
|
||||
later milestone cycles: the general `CostNode` trait + multi-node cost-graph
|
||||
composite-builder, data-grounded factors (realized-vol → slippage, recorded-rate →
|
||||
swap), per-cycle-held accrual (carry / funding), and the conviction-weighting
|
||||
R-aggregation axis. Decision log: #148.
|
||||
|
||||
**Realization (cycle 0082 — cost-graph composition, cycle 2, #148).** The cost graph
|
||||
**composes**: a second, *state-dependent* cost node plus an aggregator prove that two
|
||||
cost nodes sum into one net-R curve with `summarize_r` and the `net_r_equity` tap
|
||||
structurally unchanged. `VolSlippageCost` (`aura-std`) charges `slip_vol_mult · vol /
|
||||
|entry − stop|` in R, reading an **independent short-horizon realized-range** vol
|
||||
(`RollingMax − RollingMin`, window distinct from the stop's own vol — scaling slippage
|
||||
by the stop's vol would collapse cost-in-R to a constant, indistinguishable from
|
||||
`ConstantCost`). `CostSum` (`aura-std`) is the cost-graph **output node**: it sums `N`
|
||||
cost nodes' 3-field cost-in-R records **per-field** into one aggregate (`n = 1` is the
|
||||
identity), so the seam consumes a single cost stream regardless of node count — one
|
||||
home for cost, the positional join unchanged. **Co-temporality contract (load-bearing,
|
||||
generalizes to all future factors):** since `summarize_r` positionally joins `cost[i]
|
||||
↔ record[i]`, a cost node is gated **only by the PM trade-geometry**; any not-yet-warm
|
||||
state input (the vol proxy warms later than PM) contributes **0 cost** that cycle
|
||||
rather than withholding — the node still emits its row, so the cost stream stays
|
||||
co-temporal 1:1 with the PM record. This makes co-temporality structural and
|
||||
warm-up-independent, preserves the C18 golden, and is honest (no slippage estimate yet
|
||||
→ no charge). `ConstantCost` satisfies it trivially; only state-dependent nodes need
|
||||
the missing-factor → 0 rule. Wired on the **run path** via `--slip-vol-mult`,
|
||||
composable with `--cost-per-trade` (their costs sum); sweep / walk-forward / mc pass
|
||||
`None`. The 3-field cost triple is now restated by-convention across the two producers
|
||||
+ `CostSum` (a compiler-unlinked lockstep) — to be unified by the still-deferred
|
||||
general `CostNode` trait (now justified by two concrete nodes). Decision log: #148.
|
||||
[Superseded 2026-07-11 (#234): the `--slip-vol-mult`/`--cost-per-trade` flags are gone;
|
||||
authoring moved to the campaign `cost` block — see the cycle-net-r realization below.]
|
||||
|
||||
**Realization (cycle 0083 — CostNode trait + shared cost-record contract, cycle 3,
|
||||
#148).** The deferred unifier ships. A new `aura-std/src/cost.rs` owns the cost-model
|
||||
node abstraction: the 3-field cost triple is now **one source of truth**
|
||||
(`COST_FIELD_NAMES` / `COST_WIDTH`, mirroring `position_management::{FIELD_NAMES,
|
||||
WIDTH}`), read by both producers + `CostSum` + the CLI wiring — the cycle-0082
|
||||
by-convention lockstep is **structurally gone** (four restatements collapsed to one).
|
||||
The `CostNode` **factor trait** carries a cost node's only per-node difference — the
|
||||
price-unit **cost numerator** (`cost_numerator(&mut self, &Ctx) -> f64`), plus
|
||||
`extra_inputs` (default none) and `label`; everything else is the generic
|
||||
`CostRunner<F: CostNode>` **adapter**, which holds the shared `cum`/`out` state and
|
||||
implements `Node`, writing the co-temporality skeleton (geometry-only gating,
|
||||
`numerator / latched` R-normalization, the closed/open charge, the running `cum`, the
|
||||
3-field emit) **once**. `ConstantCost` and `VolSlippageCost` are now thin factors whose
|
||||
`new()` returns `CostRunner<Self>`. Honours C9 (a `CostRunner<F>` is a plain downstream
|
||||
`Node`, no runtime sub-object) and C23 (`name()` dropped as dead surface — `label()`
|
||||
remains the non-load-bearing symbol). **Behaviour-preserving**: the builders emit
|
||||
unchanged schemas, so the wiring / `net_r_equity` seam / `summarize_r` are untouched and
|
||||
the `numerator / latched` token form is byte-identical; the existing suite passes
|
||||
verbatim and two new CLI characterization goldens pin the exact flat/composed
|
||||
`net_expectancy_r` (the prior tests only asserted `net < gross`). Decision log: #148.
|
||||
|
||||
**Realization (cycle 0084 — cost-graph composite-builder, cycle 4, #148).** Decision E
|
||||
ships. A new `cost_graph(Vec<PrimitiveBuilder>) -> Composite` in `aura-composites` (the
|
||||
C16 layer that couples the engine builder + `aura-std` nodes) is the cost-model graph's
|
||||
**authoring primitive**: it fans the 4 PM-geometry inputs to `N` cost nodes, surfaces
|
||||
each node's extra inputs (discovered via `schema().inputs[GEOMETRY_WIDTH..]`,
|
||||
`GEOMETRY_WIDTH` now re-exported from `aura-std`) as `cost[k].<port>` composite roles,
|
||||
sums them through `CostSum`, and exposes the 3-field aggregate. The CLI's manual
|
||||
slot-indexed cost-wiring + the hardcoded `MAX_RUN_COST_NODES = 2` cap are deleted — the
|
||||
composite handles arbitrary arity. **Behaviour-preserving** (C11): the composite inlines
|
||||
at bootstrap to the same flat fan-in, so the cycle-0083 `net_expectancy_r` goldens are
|
||||
byte-identical (four `aura-composites` unit tests pin the exposed role-set + output
|
||||
triple, incl. arbitrary-arity per-node namespacing). Honours C9 (ordinary downstream
|
||||
nodes), C16 (wiring stays out of `aura-engine`), C23 (role/port names are
|
||||
non-load-bearing). **Carried debt (#152, for the deferred sweep-cost cycle):** the
|
||||
`cost[k].<port>` index-namespacing is restated across `CostSum` / `cost_graph` / the CLI
|
||||
(a build-validated lockstep, not the silent positional kind 0083 collapsed), and
|
||||
`cost_graph` `.leak()`s runtime port names per build — fine for one-shot run-path
|
||||
construction, but to be interned (the `COL_PORTS` production pattern, not the test-only
|
||||
`.leak()`) before cost reaches the per-member sweep path. Decision log: #148.
|
||||
[Discharged 2026-07-11 (#152/#234): `cost_port`/`intern_port` in `aura-std/src/cost.rs`
|
||||
are the process-global interned single source; both `cost_graph` `.leak()`s are gone.]
|
||||
|
||||
**Realization (cycle 0085 — per-cycle-held accrual, cycle 5, #148).** C10's
|
||||
"per-trade factors deduct at close; **per-cycle-held factors accrue over the hold**"
|
||||
clause is first realized. A cost factor now declares *when* it charges via
|
||||
`ChargeMode { AtClose, PerHeldCycle }` (a defaulted `CostNode::charge_mode()`, default
|
||||
`AtClose`), read by the **one shared `CostRunner`** — not a second runner type (a
|
||||
commission is intrinsically at-close, a carry intrinsically per-held-cycle; the timing
|
||||
belongs to the factor, preserving the cycle-0083 "skeleton written once" win). The
|
||||
`PerHeldCycle` arm accrues `per` into a per-position `acc` every held cycle, dumps the
|
||||
accrued total into `cum` at close (resetting `acc`), and marks the open position via a
|
||||
**growing `open_cost_in_r`**. The first accrual node, **`CarryCost`** (`aura-std`,
|
||||
a `ConstantCost` twin differing only in `charge_mode()`), is a labelled stress
|
||||
parameter — the flat base of the accrual family; a run-path `--carry-per-cycle` flag
|
||||
pushes it into the existing `cost_graph`/`CostSum` aggregation (no new wiring; a
|
||||
per-trade and a per-held-cycle node compose, the costs summing per-field). **Approach B
|
||||
(honest bleed), achieved without a `summarize_r` fold change:** the headline
|
||||
`net_r_equity` curve bleeds continuously over the hold because the bleed lives in
|
||||
`open_cost_in_r`, which the `net_r_equity` tap already subtracts — so `summarize_r` and
|
||||
the CLI `net_eq` wiring are **untouched**, and the cycle-0083/0084 `net_expectancy_r`
|
||||
goldens stay byte-identical (the `AtClose` arm is the pre-cycle-5 eval tokens verbatim).
|
||||
The 3-field cost record's semantics generalize cleanly: `cost_in_r` = cost realized this
|
||||
cycle (into `cum`); `open_cost_in_r` = the open position's cost marked-to-market
|
||||
as-of-now (would-be-close for `AtClose`, accrued-so-far for `PerHeldCycle`) — singly
|
||||
counted, no `cum`/`open` double-count (the two are mutually exclusive per cycle). Honours
|
||||
C9 (a `CostRunner<CarryCost>` is an ordinary downstream `Node`), C11 (byte-identity), C16
|
||||
(factor in `aura-std`, `aura-engine` domain-free), C23 (`label()` carries the rate). The
|
||||
B-vs-A discriminator (a growing intra-hold `open_cost_in_r`, invisible to the scalar
|
||||
`net_expectancy_r`) is pinned by unit B-proofs + a `net_r_equity`-bleeds-over-the-hold
|
||||
integration test. **Deferred (each its own #148 cycle):** a notional-based carry
|
||||
(`price × rate`, reads the price tap), the calendar-aware **overnight swap** proper
|
||||
(rollover-boundary timing, 3× Wednesday, long/short asymmetry — deploy-edge realism), and
|
||||
sweep-path cost. Decision log: #148.
|
||||
|
||||
**Realization (cost-flag harness scoping, #153).** Cost flags are defined only
|
||||
against an R-evaluator harness (the gross-R → net-R chain). On a non-R harness
|
||||
(`--harness sma`/`macd`, which produce no R) a cost flag is a **usage error
|
||||
(exit 2)**, not a silent no-op — refuse-don't-guess. Negative cost rates are
|
||||
likewise rejected (exit 2) with a named diagnostic that identifies the offending
|
||||
flag. CLI ergonomics only; the cost-model graph and the R math are untouched.
|
||||
[HISTORY — the built-in `--harness` selector (its `sma`/`macd` non-R examples)
|
||||
was retired with the demos → blueprint-data (#159, cuts 1b-4); the cost-flag
|
||||
scoping rule survives on the `aura <verb> <blueprint.json>` r-sma run path over
|
||||
examples/r_*.json.]
|
||||
|
||||
**Realization (2026-07-06 — the risk regime as a structural campaign axis, #210).**
|
||||
The `StopRule{Fixed,Vol}` structural axis is realized at the campaign-document
|
||||
level as `CampaignDoc.risk: [RiskRegime]` (a serializable, content-addressable
|
||||
mirror of the runtime enum — `aura-research`, variants `Vol{length,k}` and
|
||||
`VolTf{period_minutes,length,k}` (#262), the fixed-stop rule additive when
|
||||
needed). It is a **kept-separate** matrix axis, a
|
||||
peer of instruments and windows: the executor keys the nominee map by
|
||||
`(strategy, window, regime)`, so generalize aggregates across instruments
|
||||
*within* a regime, never across regimes. Regimes are therefore **compared** at
|
||||
presentation, never argmax-**selected** across — a cross-regime E[R] argmax would
|
||||
compare R-multiples in different R units (the stop defines 1R), so the legacy
|
||||
verbs' `--stop-length`/`--stop-k` joint stop grid is retired as a campaign
|
||||
methodology (finding the best regime = the same walk-forward / worst-case-R
|
||||
validation, run once per regime, and picking the most robust — a comparison, not
|
||||
a search). Each member manifest stamps its resolved stop (default included),
|
||||
closing the C18 gap; absent/empty `risk` = one implicit default regime,
|
||||
absent-serializing for content-id parity. Two default *representations*
|
||||
coexist by design (#217): a dissolved sweep binds no regime at all
|
||||
(`risk: []`), late-resolved per member by `stop_rule_for_regime` at run
|
||||
time, while walkforward/mc/generalize — whose `--stop-length`/`--stop-k`
|
||||
became optional — bind the default regime *eagerly* into the campaign
|
||||
document (`risk: [Vol{length:3,k:2.0}]`). Same R behaviour either way, but
|
||||
deliberately different document content ids: a stop-less verb invocation's
|
||||
document equals its explicit `--stop-length 3 --stop-k 2.0` spelling, not
|
||||
a stop-less sweep's document. Deferred (documented): regime-aware
|
||||
**trace** persistence — the trace re-run and cell-key dir naming still assume the
|
||||
default stop, since `CellRealization` carries no regime (`#212`); the core
|
||||
run/stamp/generalize path is unaffected.
|
||||
|
||||
**Realization (cycle net-r — cost reaches the family/campaign path, #234/#152,
|
||||
2026-07-11).** The deferred sweep-path cost ships, delivered where C24 put experiment
|
||||
intent: the campaign document gains an additive `cost: Vec<CostSpec>` block — a closed,
|
||||
externally tagged vocabulary over the three shipped nodes (`constant` / `vol_slippage`
|
||||
/ `carry`, field names = the builders' own `ParamSpec` names), mirroring the `risk`
|
||||
block (serde `default` + skip-if-empty: cost-less docs hash byte-identically, C18).
|
||||
Net is the **default**: an absent block is the explicit zero-cost model and
|
||||
`summarize_r`'s net family equals gross under it — every result is net, no second
|
||||
"gross-labelled" result kind. `cost_nodes_for` (beside `stop_rule_for_regime`) is the
|
||||
one doc→builder binding, every component fully bound (the wrapped param space stays
|
||||
cost-invariant); `wrap_r` carries an optional cost leg (the #221-deleted wiring
|
||||
rebuilt: cost_graph off the executor's four geometry outputs, the vol proxy back in
|
||||
production, a gated cost recorder in reduce mode as the `summarize_r` join input, the
|
||||
`LinComb(4)` `net_r_equity` curve in trace mode). Member manifests stamp the
|
||||
components; both re-run sides re-derive them (reproduce via `cost_specs_from_params` —
|
||||
the #233 stop pattern — and the persist re-run binds the same model, so the C1 drift
|
||||
alarm compares like with like; costed families reproduce bit-identically, pinned incl.
|
||||
`Carry`). `TapChannel::Net` routes `net_r_equity` to persisted curves; a cost-less doc
|
||||
requesting it keeps a remedy-naming skip notice. The `--cost-*` run-path flags do not
|
||||
return; the verb sugar passes no cost (docs carry it) — the same lean-flag call as the
|
||||
C26 bindings block. #152's interning ships with it (`cost_port`/`intern_port`, both
|
||||
`cost_graph` `.leak()`s gone). Decision log: #234.
|
||||
@@ -0,0 +1,361 @@
|
||||
# C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge
|
||||
|
||||
**Guarantee.** A strategy's primary, backtestable output is **not** an equity
|
||||
curve, **nor a position-event table**, **nor a position size**, but a **bias
|
||||
stream**: the DAG expresses exactly one *state* at time t (C8 — a node emits at
|
||||
most one record per `eval`), so a strategy emits one **signed, bounded bias**
|
||||
`f64 ∈ [-1, +1]` per cycle (per instrument) — the **sign is direction, the
|
||||
magnitude is conviction**, and conviction is optional (a bare ±1 / 0 is the modal
|
||||
case). Bias is **unsized**: position sizing and the protective stop do **not**
|
||||
live in the strategy. The chain is `signals (scores) → decision node → bias
|
||||
stream`.
|
||||
|
||||
**Risk-based execution is a decoupled downstream layer; in research it is Stop +
|
||||
position-management in R, no Sizer.** Turning a bias into a tracked trade is the
|
||||
job of a downstream **execution** chain, never the strategy's. The **stop-rule**
|
||||
sets a protective stop, which **defines the risk unit R** (1R = the loss taken if
|
||||
stopped). In the research loop the executor is **stop-rule → position-management**,
|
||||
operating **directly in R**, with the **Veto** an optional documented
|
||||
pre-trade-gate seam (a pass-through identity DCE'd away under C19/C23 when
|
||||
absent): there is **no Sizer**. Sizing in *currency* (`size` / `volume`) is a
|
||||
**deploy** concept. C8's wiring totality (no optional-input concept — every
|
||||
declared port is covered by exactly one wiring act) forbids a *dangling* `size`
|
||||
port, so the resolution is concrete: research position-management either **drops
|
||||
its `size` input port**, or has that port **driven by a constant unit node** (the
|
||||
flat-1R degenerate) — never an unwired "vestigial" port; the record's `size`
|
||||
field is held at unit and carries no research information, and the
|
||||
position-event table's `volume` column likewise. Consequently the
|
||||
**position-event table is demoted to a deploy / reconciliation artifact** (real
|
||||
volume lives there), no longer a research artifact and no longer "fed to a
|
||||
broker". The three-way decoupling of **direction (bias) / sizing / fill** stands
|
||||
as a *structure*; sizing and fill are simply pushed entirely to the deploy edge,
|
||||
out of the research loop.
|
||||
|
||||
**Signal quality is measured in R — gross R and net R.** A downstream
|
||||
**R-evaluator** consumes the executor run and integrates the per-trade R-outcomes
|
||||
into an **R-expectancy** / R-curve: the account- and instrument-agnostic yardstick
|
||||
for *"how much R out per 1R risked?"*. R, not pips, is the unit (pips are not
|
||||
risk-normalized). Two readings of the same unit: **gross R** (signal only) vs
|
||||
**net R** (after the cost model), with `net R = gross R − cost-in-R`. The headline
|
||||
artifact is the **net-R equity curve** — the cost-drag drawn onto the R curve,
|
||||
recorded through a named **`net_r_equity`** tap/sink (sibling of the existing
|
||||
`r_equity` tap; a sink is the only thing the registry can display, C8/C18/C22). No
|
||||
new unit is invented: it is R, gross and net, **continuous with the existing
|
||||
`net_expectancy_r`**. A bare **gross-R run with no cost model attached is valid**:
|
||||
the cost layer is optional and additively composed-on (the zero-cost baseline is
|
||||
the "default simple" floor).
|
||||
|
||||
**Cost is a COST MODEL — a composable C9 graph of cost nodes, in R, that
|
||||
approximates (never claims) realism.** The realistic broker is *retired*: real
|
||||
friction — slippage (live liquidity / order-size / volatility at fill), swaps
|
||||
(broker-set, time-varying), even recorded feed spread (often a fake constant) — is
|
||||
**not historically knowable**, so an authored-friction historical broker is
|
||||
"horseshoe-throwing". Its replacement is a **cost model**: an ordinary downstream
|
||||
**C9 graph of cost nodes** that *approximates* the cost side a broker would
|
||||
produce, explicitly as an approximation. The cost nodes live in `aura-strategy`,
|
||||
the cost-graph composite-builder (`cost_graph`) in `aura-composites` (C16), never
|
||||
in the domain-free `aura-engine` (C14/C16). They are **not** "additive on the R
|
||||
stream" in isolation: a cost node **reads the state it depends on** — the price
|
||||
stream, a realized-volatility tap, a C11-recorded interest-rate source, and the
|
||||
executor's per-cycle R-record / trade events — and emits a **cost-in-R** stream
|
||||
that is subtracted from gross R to yield net R. Cost attaches at the structurally
|
||||
correct grain: **per-trade** factors (commission, a flat cost-per-trade) deduct
|
||||
from `realized_r` at close; **per-cycle-held** factors (carry / funding / swap)
|
||||
accrue over the holding duration. The model **generalizes** the scalar
|
||||
`round_trip_cost` / `net_expectancy_r` into a possibly **state-dependent graph**:
|
||||
the scalar `round_trip_cost` is the degenerate constant-per-trade special case,
|
||||
**subsumed** by the cost graph — the post-run `summarize_r` fold no longer
|
||||
recomputes cost independently but folds the cost-model's net-R stream into
|
||||
`net_expectancy_r` (one home for cost, no double-count). Discipline: every cost
|
||||
factor is **either** a clearly-labelled **stress-parameter** (e.g. a flat
|
||||
cost-per-trade is a breakeven-threshold probe) **or** **data-grounded /
|
||||
falsifiable** (e.g. realized-volatility → slippage; recorded interest-rate data
|
||||
via C11 → funding / swap). **Default simple; complexity is earned per grounded
|
||||
factor.** Stacking unfalsifiable guesses (over-modelling) is the anti-pattern.
|
||||
|
||||
**Co-temporality contract (load-bearing, generalizes to all cost factors).**
|
||||
Because `summarize_r` positionally joins `cost[i] ↔ record[i]`, a cost node is
|
||||
gated **only by the position-management trade-geometry**; any not-yet-warm state
|
||||
input (a vol proxy warms later than the executor) contributes **0 cost** that
|
||||
cycle rather than withholding — the node still emits its row, so the cost stream
|
||||
stays co-temporal 1:1 with the executor's R-record. Co-temporality is therefore
|
||||
structural and warm-up-independent, preserves the C18 golden, and is honest (no
|
||||
slippage estimate yet → no charge). A constant-per-trade factor satisfies it
|
||||
trivially; only state-dependent factors need the missing-factor → 0 rule.
|
||||
|
||||
**The research loop is pure feed-forward — compounding is removed.** Flat-1R-style
|
||||
R accounting needs no equity, and with the Sizer gone there is **no equity → size
|
||||
edge at all** in research: the loop is **feed-forward, maximally parallel and
|
||||
deterministic (C1)**, the cost model a feed-forward subtraction on the R stream.
|
||||
**Compounding is removed from research**: it is a **post-strategy money-management
|
||||
transform** — a pure function of the per-trade **net-R sequence** and a
|
||||
bet-fraction *f*, multiplicative and path / order-dependent, the **sole** source
|
||||
of feedback — so compounding, Kelly-*f*, and drawdown-under-compounding are
|
||||
derived **post-hoc, analytically, at the deploy / account layer** from the net-R
|
||||
distribution. Consequently there is no `z⁻¹` fill-edge register and no
|
||||
flat-1R-vs-compounding structural axis in the research loop (with no in-loop
|
||||
feedback there is nothing for a register to cut; this is strictly C9 / C23-cleaner).
|
||||
|
||||
**Conviction-based risk allocation survives — as an R-aggregation axis, not a
|
||||
Sizer.** Scaling risk by bias strength is **signal-side and R-denominated**.
|
||||
Because per-trade R is **size-invariant**, conviction cannot be expressed by
|
||||
scaling position size (that is invisible to R); it is expressed by **weighting the
|
||||
per-trade R-contribution** in the R-equity: **flat** (sum of `realized_r`, sign
|
||||
only) vs **conviction-weighted** (sum of `|bias| · realized_r`, sign + magnitude).
|
||||
This is a **feed-forward, additive, order-independent research axis**, distinct
|
||||
from the removed money-Sizer; it is tested via the `conviction_at_entry` record
|
||||
field and `conviction_terciles_r` metric, and may sit in-graph or as a post-hoc
|
||||
fold.
|
||||
|
||||
**Money / real broker / cTrader Open API = a separate, later live / deploy-edge
|
||||
concern — the only reliable (`belastbare`) ground truth, measured never
|
||||
modelled.** Reliable friction statistics require **forward-trading against a real
|
||||
broker** (e.g. cTrader Open API), and are non-stationary even then. This fits C11
|
||||
(record-then-replay: real fills are recorded live, then replayable) and the
|
||||
frozen-deploy invariant (C13: deploy = frozen bot + broker connection);
|
||||
reconciliation with the real account is an **external I/O adapter** at the
|
||||
recording / deploy edge, not an in-graph node. **This is the only place account
|
||||
money appears.** Currency-denominated *reference geometry* — pip value,
|
||||
stop-distance-in-currency from the C15 `instrument_geometry` sidecar — is still
|
||||
read at the **ingestion** edge to normalize a currency / pip cost factor into R;
|
||||
notional size cancels in `cost_in_R = cost_in_currency / (size · stop_dist)`, so
|
||||
the cost model is R-pure without ever holding *equity*. It is equity / account
|
||||
money, not reference geometry, that lives only at the deploy edge. The
|
||||
broker-independent **position-event table** (`event_ts, action[buy/sell/close],
|
||||
position_id, instrument_id, volume`) is the **deploy / reconciliation** record at
|
||||
this edge (real volume), not a research artifact.
|
||||
|
||||
**Honesty principle.** The net-R curve under the cost model is a **research /
|
||||
ranking tool and a hypothesis**; the **forward / live run against a real broker is
|
||||
the ground truth**. The cost model *approximates*; it **never claims realism**.
|
||||
`SimBroker` (the legacy pip-equity, unsized-exposure node) is **redundant as a
|
||||
quality measure** with the net-R cost model in place (R displaces pips). It is retained as a **legacy / simple optional pip yardstick**
|
||||
(still wired alongside the R executor in the r-family member for an honest dual
|
||||
readout), **not part of the new model and not to be expanded**.
|
||||
|
||||
**Forbids.** Putting **sizing or the stop in the strategy** (bias is unsized; the
|
||||
stop-rule owns the stop, R is the unit); treating an **equity curve** (R or
|
||||
currency) as the strategy's direct output; **putting a Sizer / currency size /
|
||||
`volume` into the research loop** (size is a deploy concept; research is in R);
|
||||
leaving a **dangling `size` input port** on the research executor (C8 wiring
|
||||
totality forbids it — drop the port or drive it with a constant unit); **any
|
||||
equity → size / equity → anything feedback in research** (the research loop is
|
||||
pure feed-forward; compounding is a post-hoc money-management transform, not an
|
||||
in-loop edge); modelling an **authored-friction "realistic broker" over historical
|
||||
data** (real friction is not historically knowable — use the approximating cost
|
||||
model, and treat the real broker as the live-edge ground truth only); **claiming
|
||||
the cost model is realism** (it is an explicit approximation); **stacking
|
||||
unfalsifiable cost guesses** (each cost factor is a labelled stress-parameter *or*
|
||||
data-grounded — over-modelling is the anti-pattern); **computing cost in two
|
||||
homes** (the cost graph owns cost; the post-run fold subsumes the old scalar
|
||||
`round_trip_cost`, never double-counts it); expressing **conviction by scaling
|
||||
position size** (size-invisible to R; conviction is an R-aggregation weight);
|
||||
making the **position-event table the strategy's direct DAG output** (it is
|
||||
derived, not emitted per `eval` — a decision instant may need >1 event, which C8
|
||||
forbids) or a **research** measure of signal quality (it is a deploy /
|
||||
reconciliation artifact); **measuring signal quality in currency / account money**
|
||||
in the research loop at all (account money lives only at the live deploy edge;
|
||||
reference geometry at ingestion is not account money); baking a broker into the
|
||||
strategy or an **in-graph broker subsystem** (the in-graph realistic broker is
|
||||
retired; the only broker is the live-edge I/O adapter, C11 / C13); a signed-volume
|
||||
direction trick in the event table (use `action`); storing `open_ts` (derive it
|
||||
from the opening event).
|
||||
|
||||
**Why.** A strategy's edge is a *procedure*, not a currency outcome ("focus on the
|
||||
procedure, not the money"): the right primary question is *"how much R out per 1R
|
||||
risked?"*, and R — defined by the stop — is the only account- and
|
||||
instrument-agnostic, risk-normalized unit (pips are not). Separating **direction
|
||||
(bias) from sizing from fill** is the decomposition every mature system converges
|
||||
on (LEAN's Alpha → Portfolio-Construction → Execution is a near isomorphism;
|
||||
backtrader, QSTrader, zipline all emit an unsized directional signal sized
|
||||
downstream) — and aura pushes *sizing* and *fill* off the research loop for two
|
||||
**distinct** reasons: **sizing** is off because per-trade R is **size-invariant**
|
||||
(size carries no information in R — flat-1R is perfectly knowable, it just does not
|
||||
matter), and **fill / friction** is off because real friction is **not
|
||||
historically knowable** and therefore not honest over history. Keeping research
|
||||
**pure feed-forward** leaves the signal-quality layer parallel and deterministic
|
||||
(C1); the **only** real feedback (equity → bet-fraction) is **compounding**, a
|
||||
closed-form, path-dependent transform of the net-R sequence, and therefore belongs
|
||||
**after** the strategy, at the deploy / account layer, not as an in-loop register.
|
||||
The **cost model as a C9 graph** keeps cost within the one Node / graph
|
||||
abstraction (C9) and generalizes the scalar `net_expectancy_r` continuously, while
|
||||
the **gross-R / net-R** split states the cost-drag honestly without inventing a
|
||||
unit. Refusing the historical realistic broker is an **honesty** stance: the only
|
||||
reliable friction is **measured forward** against a real broker (cTrader Open API)
|
||||
— the live deploy edge, the sole place account money and ground truth appear. The
|
||||
DAG holds exactly one state at t and a node emits ≤1 record per `eval` (C8), so the
|
||||
faithful per-cycle output is the **bias** (one value); position *events* are a
|
||||
derived, deploy-side consequence. Industry grounding for the bias/R spine: LEAN /
|
||||
nautilus_trader / backtrader / QSTrader / vectorbt / zipline (#117 decision log).
|
||||
|
||||
## Current state
|
||||
|
||||
**Bias and executor.** The unsized strategy output is the `Bias` node
|
||||
(`aura-strategy/src/bias.rs`). The per-symbol **RiskExecutor** is the
|
||||
`risk_executor(StopRule, risk_budget)` composite-builder
|
||||
(`aura-composites/src/lib.rs`) with a `StopRule{Fixed, Vol, VolTf}` structural
|
||||
axis (C20); `risk_executor_vol_open` is the gridding sibling with the two vol-stop
|
||||
knobs left open as sweep axes. Stop rules: `FixedStop` (a triggered constant) and
|
||||
the `vol_stop(length, k)` composition `k·√EMA(Δ²)` built from `Mul`/`Sqrt`
|
||||
primitives; `VolTfStop` for a resampled-timeframe vol stop (`FixedStop` in
|
||||
`aura-strategy/src/stop_rule.rs`, `vol_stop` composite in `aura-composites`). The
|
||||
**Veto** is a documented seam, not a runtime node.
|
||||
|
||||
**The Sizer is a vestige, not yet removed.** The ratified design (`#116`) has *no*
|
||||
Sizer in research, but the composite still physically wires
|
||||
`stop → Sizer → PositionManagement`, with the `Sizer` bound to a constant
|
||||
`risk_budget` (`risk_budget` is a formal argument; the runner calls
|
||||
`risk_executor(stop, 1.0)` — `aura-runner/src/member.rs`). This is the
|
||||
constant-unit-driven-`size`-port resolution of C8 wiring totality (not the
|
||||
drop-the-port resolution): `risk_budget` is a constant, never equity-fed, so no
|
||||
`equity → size` edge exists and the pure-feed-forward invariant holds; under R
|
||||
size-invariance the `Sizer`'s `size = 1.0 / stop_distance` carries no research
|
||||
information. Physically deleting the `Sizer` node and its `risk_budget` argument is
|
||||
outstanding cleanup toward the ratified shape.
|
||||
|
||||
**Position management and R metrics.** `PositionManagement`
|
||||
(`aura-backtest/src/position_management.rs`) is the stateful heart: it latches the
|
||||
entry-cycle stop distance as the immutable R-denominator, marks against the
|
||||
one-cycle-lagged fill (no look-ahead, C2), and emits a dense per-cycle R-record
|
||||
(one row per eval, C8; the trade ledger is the `closed_this_cycle` subset, the
|
||||
R-equity is `cum_realized_r + unrealized_r`). `summarize_r`
|
||||
(`aura-backtest/src/metrics.rs`) is a post-run fold (sibling of `summarize`, **not**
|
||||
an in-graph node) → `RMetrics` (E[R], SQN, `sqn_normalized`, win-rate,
|
||||
profit-factor, max-R-drawdown, `conviction_terciles_r`, `net_expectancy_r`). It
|
||||
folds a co-temporal cost stream (positional 1:1 join) into `net_expectancy_r` —
|
||||
one home for cost, byte-identical to the cost = 0 baseline on an empty stream.
|
||||
`sqn_normalized = (mean_R / stdev_R)·√(min(n, 100))` (`SQN_CAP = 100`) is an opt-in
|
||||
turnover-robust rank key; below the cap it equals raw `sqn` exactly. SQN is the
|
||||
operational single-number ranking objective; a sweep's default grid varies **only
|
||||
the signal**, holding the stop fixed — the **stop defines 1R**, so varying it
|
||||
across members would change what R *means* per member and break cross-member SQN
|
||||
comparability.
|
||||
|
||||
**Cost model.** `aura-strategy/src/cost.rs` owns the abstraction: the 3-field cost
|
||||
triple `{cost_in_r, cum_cost_in_r, open_cost_in_r}` is one source of truth
|
||||
(`COST_FIELD_NAMES` / `COST_WIDTH = 3`, mirroring the position-management record),
|
||||
prefixed by the 4-wide `GEOMETRY_WIDTH` geometry inputs. The `CostNode` **factor
|
||||
trait** carries a node's only per-node difference — the price-unit cost numerator
|
||||
(`cost_numerator`), plus `extra_inputs`, `label`, and a defaulted `charge_mode()`;
|
||||
everything else is the generic `CostRunner<F: CostNode>` **adapter** (a plain
|
||||
downstream `Node`, C9; no runtime sub-object), which writes the co-temporality
|
||||
skeleton (geometry-only gating, `numerator / latched` R-normalization, the
|
||||
closed/open charge, the running `cum`, the 3-field emit) **once**. `ChargeMode
|
||||
{AtClose, PerHeldCycle}` selects timing per factor (a commission is intrinsically
|
||||
at-close, a carry per-held-cycle); the single `CostRunner` handles both arms, the
|
||||
`PerHeldCycle` arm accruing `per` into a per-position `acc` each held cycle and
|
||||
dumping it into `cum` at close. Three shipped factors, all `aura-strategy`:
|
||||
`ConstantCost` (a labelled stress-parameter, `cost_per_trade / |entry − stop|`),
|
||||
`VolSlippageCost` (`slip_vol_mult · vol / |entry − stop|`, reading an independent
|
||||
short-horizon realized-range vol proxy distinct from the stop's own vol), and
|
||||
`CarryCost` (a `ConstantCost` twin with `charge_mode() = PerHeldCycle`, the flat
|
||||
base of the accrual family). `CostSum` (`aura-strategy/src/cost_sum.rs`) is the
|
||||
cost-graph **output node**, summing `N` nodes' 3-field records per-field into one
|
||||
aggregate (`n = 1` is the identity), so the seam consumes a single cost stream
|
||||
regardless of node count. `cost_port` / `intern_port` (`aura-strategy/src/cost.rs`)
|
||||
intern runtime port names process-globally (the `COL_PORTS` production pattern,
|
||||
#152), reused across per-member rebuilds. `cost_graph(Vec<PrimitiveBuilder>)`
|
||||
(`aura-composites/src/lib.rs`) is the authoring primitive: it fans the 4 geometry
|
||||
inputs to `N` cost nodes, surfaces each node's extras
|
||||
(`schema().inputs[GEOMETRY_WIDTH..]`) as `cost[k].<port>` roles, sums through
|
||||
`CostSum`, and exposes the 3-field aggregate at arbitrary arity. The headline sink
|
||||
is **`net_r_equity`** = `LinComb(4)[cum_realized_r, unrealized_r, −cum_cost_in_r,
|
||||
−open_cost_in_r]` → Recorder (C8/C18), a sibling of `r_equity`, emitted only when a
|
||||
cost is authored; a per-held-cycle factor bleeds continuously over the hold because
|
||||
the bleed lives in `open_cost_in_r`, which this tap already subtracts (Approach B,
|
||||
no `summarize_r` fold change).
|
||||
|
||||
**Cost on the campaign / sweep path (net is the default).** Cost is authored as
|
||||
the campaign document's additive `cost: Vec<CostSpec>` block
|
||||
(`aura-research/src/lib.rs`, `CampaignDoc`) — a closed, externally-tagged
|
||||
vocabulary over the three shipped nodes (`constant` / `vol_slippage` / `carry`,
|
||||
field names = the builders' `ParamSpec` names), mirroring the `risk` block (serde
|
||||
`default` + skip-if-empty: cost-less docs hash byte-identically, C18). An absent
|
||||
block is the explicit zero-cost model and `summarize_r`'s net family equals gross
|
||||
under it — every result is net, no second gross-labelled result kind.
|
||||
`cost_nodes_for` (beside `stop_rule_for_regime`, `aura-runner/src/translate.rs`) is
|
||||
the one doc→builder binding; `wrap_r` (`aura-runner/src/member.rs`) carries the
|
||||
optional cost leg (cost_graph off the executor's four geometry outputs, the vol
|
||||
proxy in production, a gated cost recorder in reduce mode as the `summarize_r` join
|
||||
input, the `LinComb(4)` `net_r_equity` curve in trace mode). Both re-run sides
|
||||
re-derive the model (`cost_specs_from_params`), so the C1 drift alarm compares like
|
||||
with like; costed families reproduce bit-identically (incl. `Carry`).
|
||||
`TapChannel::Net` (`aura-runner/src/runner.rs`) routes `net_r_equity` to persisted
|
||||
curves; a cost-less doc requesting it keeps a remedy-naming skip notice. There are
|
||||
no `--cost-*` run-path flags (removed #221/#234); cost travels in the document.
|
||||
|
||||
**Risk regime as a structural campaign axis.** The `StopRule{Fixed, Vol}` axis is
|
||||
realized at the campaign-document level as `CampaignDoc.risk: [RiskRegime]`
|
||||
(`aura-research`, variants `Vol{length, k}` and `VolTf{period_minutes, length, k}`
|
||||
(#262), the fixed-stop rule additive when needed) — a kept-separate matrix axis,
|
||||
peer of instruments and windows. The executor keys the nominee map by `(strategy,
|
||||
window, regime)`, so `generalize` aggregates *within* a regime, never across.
|
||||
Regimes are **compared** at presentation, never argmax-**selected** across (a
|
||||
cross-regime E[R] argmax would compare R-multiples in different R units). Each
|
||||
member manifest stamps its resolved stop (default included). Two default
|
||||
representations coexist by design (#217): a dissolved sweep binds no regime
|
||||
(`risk: []`, late-resolved per member by `stop_rule_for_regime`), while
|
||||
`walkforward`/`mc`/`generalize` bind the default regime eagerly
|
||||
(`risk: [Vol{length:3, k:2.0}]`) — same R behaviour, deliberately different
|
||||
document content-ids. Deferred: regime-aware **trace** persistence — the trace
|
||||
re-run and cell-key dir naming still assume the default stop, since
|
||||
`CellRealization` carries no regime (#212).
|
||||
|
||||
**Legacy pip yardstick.** `SimBroker` (`aura-backtest/src/sim_broker.rs`, node kind
|
||||
`simbroker`) is a legacy pip-equity node, retained as an optional pip yardstick
|
||||
and still wired alongside the R executor in the r-family member for an honest dual
|
||||
pip/R readout; not part of the cost model, not to be expanded.
|
||||
|
||||
**Position-event table (deploy / reconciliation).** A closed `PositionAction {Buy,
|
||||
Sell, Close}` enum + the `PositionEvent` row (`event_ts`, `action`, `position_id`,
|
||||
`instrument_id`, unsigned `volume`; no `open_ts`; direction *is* the action) and
|
||||
`derive_position_events(record, instrument_id) -> Vec<PositionEvent>` live in
|
||||
`aura-backtest/src/metrics.rs` as post-run value types (not per-`eval` nodes, C8).
|
||||
`action` serde-encodes as a bare `i64` (Buy=0, Sell=1, Close=2), the C7 scalar
|
||||
column form, out-of-range rejected on read. `derive_position_events` is the **first
|
||||
difference of the executed book** — a pure reduction over the position-management
|
||||
dense record (read positionally as type-erased `Scalar`s, C7 SoA; no in-graph node,
|
||||
so the hot path stays domain-free, C14) — emitting a `Buy`/`Sell` at each open and
|
||||
a `Close` at each exit; a reversal (or stop-then-same-cycle reopen) emits **Close
|
||||
then the opposite open at one `event_ts`** (the C8 ">1 event per instant" case that
|
||||
forces a *derived* table). The close sizes the **actual book** (the closed
|
||||
position's stored volume), never an exposure delta. A position open at window end
|
||||
emits its open with **no synthetic `Close`** (the table records actual executed
|
||||
events; `summarize_r`'s force-close is for the R metric only). The table stays
|
||||
broker-independent.
|
||||
|
||||
**CLI.** The run/sweep surface is blueprint-driven — `aura <verb> <blueprint.json>`
|
||||
over `examples/r_*.json` / `crates/aura-cli/tests/fixtures/`; the built-in
|
||||
`--harness` / `--strategy` selectors were retired with the demos → blueprint-data
|
||||
cut (#159). The strategy identifier family is the **r-family** — `r-sma` /
|
||||
`r-breakout` / `r-meanrev` (renamed from the Stage-1 family in #174). `metric_cmp`
|
||||
(`aura-registry`) ranks the higher-is-better R metrics `sqn`, `expectancy_r`,
|
||||
`net_expectancy_r`, `sqn_normalized`; a member with no `r` block sorts last.
|
||||
|
||||
**Deferred work.** Physical removal of the vestigial `Sizer` and its `risk_budget`
|
||||
argument (ratified shape, #116). Data-grounded cost factors beyond the shipped
|
||||
stress parameters: a notional-based carry (`price × rate`, reads the price tap) and
|
||||
the calendar-aware **overnight swap** proper (rollover-boundary timing, 3× Wednesday,
|
||||
long/short asymmetry — deploy-edge realism) (#148). The **conviction-weighting**
|
||||
R-aggregation axis (flat vs `|bias|·realized_r`), tested via `conviction_at_entry` /
|
||||
`conviction_terciles_r` (#148). Regime-aware trace persistence (#212). Money, a
|
||||
real broker, and cTrader Open API remain the separate live / deploy-edge concern
|
||||
(C11 record-then-replay, C13 frozen deploy).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — determinism / bit-identity (costed families reproduce byte-for-byte)
|
||||
- [C2](c02-causality.md) — no look-ahead (one-cycle-lagged fill; the co-temporality 0-cost warm-up rule)
|
||||
- [C8](c08-node-contract.md) — ≤1 record per `eval`; sinks are the only displayable surface
|
||||
- [C9](c09-fractal-composition.md) — cost nodes are ordinary downstream nodes; the cost model is a C9 graph
|
||||
- [C11](c11-sources-record-replay.md) — record-then-replay; the live broker as recorded I/O; bit-identity of re-runs
|
||||
- [C13](c13-hot-reload-frozen-deploy.md) — the frozen-deploy invariant (deploy = frozen bot + broker connection)
|
||||
- [C14](c14-headless-two-faces.md) — the domain-free engine (no cost / broker logic in `aura-engine`)
|
||||
- [C15](c15-resampling-sessions.md) — the `instrument_geometry` sidecar (pip value / stop-distance-in-currency)
|
||||
- [C16](c16-engine-project-split.md) — the composites layer (`cost_graph`, `risk_executor`)
|
||||
- [C18](c18-registry.md) — the registry / golden (byte-identity, content-addressed docs)
|
||||
- [C20](c20-strategy-harness.md) — structural axes (`StopRule`, the risk regime)
|
||||
- [C23](c23-graph-compilation.md) — names non-load-bearing; DCE of the absent Veto
|
||||
- [C24](c24-blueprint-data.md) / [C25](c25-role-model.md) — experiment intent in the campaign document
|
||||
- [C28](c28-stratification.md) — the crate-layer ladder
|
||||
|
||||
> History: [c10-bias-r-cost.history.md](c10-bias-r-cost.history.md)
|
||||
@@ -0,0 +1,39 @@
|
||||
# C11 — Generalized sources; record-then-replay determinism boundary
|
||||
|
||||
**Guarantee.** A source is anything that produces timestamped scalar streams —
|
||||
market data (the data-server) and non-financial sources (e.g. a news-agent node
|
||||
emitting a bias) are treated identically. Anything nondeterministic, external,
|
||||
or slow (LLM/news/web) is materialized into a recorded, timestamped stream
|
||||
*before* it enters the engine; a backtest replays the recording, live computes
|
||||
fresh in real time and records it for future backtests. A bias enters as a value
|
||||
held until the next event (firing policy A, C6).
|
||||
|
||||
**Forbids.** Any live external call *inside* a backtest replay.
|
||||
|
||||
**Why.** It is the only model compatible with reproducible backtests (C1) — LLM
|
||||
calls are nondeterministic and far too slow per-cycle. External LLM (IONOS) calls
|
||||
happen only at the recording/live-source edge, with explicit per-session consent,
|
||||
never inside a sim (the IONOS consent rule lives in `~/.claude/CLAUDE.md`).
|
||||
|
||||
## Current state
|
||||
The producer seam that makes "a source is anything" concrete is the object-safe
|
||||
`Source` trait (`crates/aura-engine/src/harness.rs`), driven by the single k-way
|
||||
ingestion merge (C3); the seam itself is detailed in C12. The realized side is the
|
||||
market-data replay path: `DataServer` (the external `data-server` git-dependency
|
||||
crate, re-exported through `crates/aura-ingest`)
|
||||
parses an archived M1 window and streaming `M1FieldSource` producers
|
||||
(`crates/aura-ingest/src/lib.rs`) replay it deterministically into the merge. The
|
||||
recording *direction* — a live source materialized into a timestamped stream for
|
||||
later replay — and generalized non-price recorded sources (a news-agent bias) are
|
||||
design law that rides the same `Source` seam (the #71 source seam) but has not yet
|
||||
landed. The held-until-next-event semantics of a sparse recorded source are the
|
||||
firing-policy-A / sample-and-hold mechanism already realized under C5/C6. At the
|
||||
opposite edge the live broker is an I/O adapter at the recording/deploy boundary,
|
||||
never part of the strategy (C13; domain invariant 7).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md)
|
||||
- [C3](c03-single-merge.md)
|
||||
- [C6](c06-firing-policy.md)
|
||||
- [C12](c12-atomic-sim-unit.md)
|
||||
- [C13](c13-hot-reload-frozen-deploy.md)
|
||||
@@ -0,0 +1,36 @@
|
||||
# C12 — The atomic sim unit and the four orchestration axes: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c12-atomic-sim-unit.md](c12-atomic-sim-unit.md).
|
||||
|
||||
**Realization (cycles 0028, 0049).** Axis 1 (param-sweep) is built. `GridSpace`
|
||||
(0028) enumerates a cartesian lattice over discrete per-slot value-lists;
|
||||
`RandomSpace` (0049) draws `N` seeded points over typed continuous `ParamRange`s
|
||||
(I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`), validated against the
|
||||
param-space before any run. Both implement the `Space` trait the disjoint
|
||||
`sweep` / `run_indexed` core is generic over, so either enumeration runs through
|
||||
one execution path (C1: results in enumeration order, not completion order). The
|
||||
seeded sampler reuses the bit-stable `SplitMix64` as a **code-path-disjoint**
|
||||
instance from the data-edge seed RNG (the source-seam firewall, #52/#71: they
|
||||
share only the `u64` type, never a path).
|
||||
|
||||
**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the
|
||||
only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait
|
||||
(`peek`/`next`, object-safe) the k-way merge drives — and a streaming
|
||||
**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing
|
||||
**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and
|
||||
decoding each `Scalar` on demand: the **source ring** is resident O(one chunk),
|
||||
not O(window length) — the measured `resident_records()` predicate, a *per-source*
|
||||
bound, not whole-process RSS. (Data-server's `FileCache` retains each window's
|
||||
parsed chunks read-only for the pass — ~56 B/record — so process residency is
|
||||
O(records-touched); that is the **replay-many** sharing C12 wants — one window
|
||||
parsed once across a sweep family — not a leak. The single-pass cost is tracked as
|
||||
#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the
|
||||
gap closes by a streaming path *existing*, not by deleting the eager one). **Realized
|
||||
(2026-06-29, family cycles shipped).** cross-*sim* `Arc<[T]>` sharing — one window shared
|
||||
zero-copy across many disjoint sweep sims — is now in force: the sweep / Monte-Carlo /
|
||||
walk-forward families (axes 2–4 above) build their members over **one** shared
|
||||
`Arc<DataServer>` (one `FileCache`), so a window is parsed once and every member's
|
||||
`M1FieldSource` borrows the same cached `Arc<[M1Parsed]>` chunks
|
||||
(`crates/aura-ingest/src/lib.rs:316`). The single-pass *parse* cost stays tracked as #95.
|
||||
@@ -0,0 +1,82 @@
|
||||
# C12 — The atomic sim unit and the four orchestration axes
|
||||
|
||||
**Guarantee.** The atomic unit is `(frozen topology + param-set + data-window +
|
||||
RNG-seed) → deterministic run → metrics`. Parameters are typed, ranged, runtime
|
||||
values injected at graph build — no recompile per param-set; the optimizer sees a
|
||||
generic vector of typed ranges. A **bound blueprint param is that param's
|
||||
default**: "open" means *must be bound by an axis*, "bound" means *default,
|
||||
overridable by an axis*, so axis 1 (param-sweep) may name a bound param — the
|
||||
family boundary re-opens it on the probe and on every member reload, and the axis
|
||||
binds it per cell; `run`/`mc` still require every param resolved (a truly open
|
||||
param refuses). Identity is untouched: `content_id_of`
|
||||
(`crates/aura-research/src/lib.rs`) and `topology_hash`
|
||||
(`crates/aura-cli/src/main.rs`) read the authored document, never a re-opened
|
||||
probe, and each member manifest records its per-cell bindings (#246, ratified
|
||||
2026-07-12; the restriction this amends — axes bind only open knobs — was an
|
||||
implementation consequence of `bind()` shrinking the param surface, not a recorded
|
||||
decision). Raw data is shared read-only across sims via `Arc<[T]>` (the
|
||||
data-server is built for this). Four axes orchestrate the atomic unit:
|
||||
(1) param-sweep (grid/random), (2) optimization (argmax metric), (3) walk-forward
|
||||
(rolling in-sample optimize + out-of-sample test), (4) Monte-Carlo (N seeded
|
||||
realizations perturbing input). **MC = sweep over seeds**; each realization is
|
||||
itself deterministic given its seed.
|
||||
|
||||
**Forbids.** Baking a specific search strategy (Bayesian/genetic) into the
|
||||
primitive — those are pluggable policies atop the atomic unit; recompiling on a
|
||||
param change.
|
||||
|
||||
**Why.** A stable primitive + orchestration axes keeps "wahnsinnig schnell"
|
||||
(embarrassingly parallel across the unit) cleanly separated from search policy.
|
||||
Seed-as-input reconciles Monte-Carlo with C1. The "frozen topology" of the atomic
|
||||
unit is one harness instance, selected by the harness's **structural axes** (C20);
|
||||
the structural experiment matrix is the outer orchestration over this dimension,
|
||||
the tuning sweep the inner (C19/C20).
|
||||
|
||||
## Current state
|
||||
`Harness::run` is typed as a **producer seam** — the object-safe `Source` trait
|
||||
(`peek`/`next`, plus `bounds` and a `resident_records()` residency probe;
|
||||
`crates/aura-engine/src/harness.rs`) that the k-way merge drives (C3). Two source
|
||||
shapes back it. The eager `load_m1_window`/`close_stream` path
|
||||
(`crates/aura-ingest/src/lib.rs`), wrapped as `VecSource`, materializes a whole
|
||||
stream and is retained for bounded loads. The streaming `M1FieldSource`
|
||||
(`crates/aura-ingest/src/lib.rs`) pulls a data-server window lazily, borrowing one
|
||||
`Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and decoding each
|
||||
`Scalar` on demand, so its **source ring** is resident O(one chunk), not
|
||||
O(window length) — `resident_records()` is a *per-source* bound, not
|
||||
whole-process RSS.
|
||||
|
||||
Raw-data sharing is realized both within and across sims. The data-server's
|
||||
`FileCache` retains each window's parsed chunks read-only for the pass, so process
|
||||
residency is O(records-touched) — the **replay-many** sharing model, not a leak.
|
||||
Across sims, the sweep / Monte-Carlo / walk-forward families (axes 2–4) build their
|
||||
members over **one** shared `Arc<DataServer>` (one `FileCache`;
|
||||
`crates/aura-ingest/src/lib.rs`), so a window is parsed once and every member's
|
||||
`M1FieldSource` borrows the same cached `Arc<[M1Parsed]>` chunks. The single-pass
|
||||
*parse* cost of a window is tracked as #95.
|
||||
|
||||
Axis 1 (param-sweep) is built. `GridSpace` enumerates a cartesian lattice over
|
||||
discrete per-slot value-lists; `RandomSpace` draws `N` seeded points over typed
|
||||
continuous `ParamRange`s (I64 inclusive `[lo,hi]`, F64 half-open `[lo,hi)`),
|
||||
validated against the param-space before any run; a `ListSpace` enumerates an
|
||||
explicit point list. All implement the `Space` trait the disjoint `sweep` /
|
||||
`run_indexed` core is generic over (`crates/aura-engine/src/sweep.rs`), so every
|
||||
enumeration runs through one execution path (C1: results in enumeration order, not
|
||||
completion order). The other axes are realized as families over that core:
|
||||
`walk_forward` + `WindowRoller` (`crates/aura-engine/src/walkforward.rs`) for the
|
||||
rolling in-sample/out-of-sample split, `monte_carlo`
|
||||
(`crates/aura-backtest/src/mc.rs`) for the seed sweep; optimization is argmax over
|
||||
a sweep's metrics, a policy atop the core rather than a baked-in strategy. The
|
||||
seeded sampler reuses the bit-stable `SplitMix64`
|
||||
(`crates/aura-analysis/src/lib.rs`) as a **code-path-disjoint** instance from the
|
||||
data-edge seed RNG (the source-seam firewall, #52/#71: they share only the `u64`
|
||||
type, never a path).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md)
|
||||
- [C11](c11-sources-record-replay.md)
|
||||
- [C18](c18-registry.md)
|
||||
- [C19](c19-bootstrap.md)
|
||||
- [C20](c20-strategy-harness.md)
|
||||
- [C23](c23-graph-compilation.md)
|
||||
|
||||
> History: [c12-atomic-sim-unit.history.md](c12-atomic-sim-unit.history.md)
|
||||
@@ -0,0 +1,25 @@
|
||||
# C13 — Hot-reload is authoring-only; deploy is frozen: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract:
|
||||
> [c13-hot-reload-frozen-deploy.md](c13-hot-reload-frozen-deploy.md).
|
||||
|
||||
**Realization (cycle 0102 — the load boundary; per-invocation reload).** The
|
||||
authoring-loop half is realized: a project is an external cdylib crate loaded
|
||||
per invocation through a two-tier `#[repr(C)]` descriptor (`AURA_PROJECT`,
|
||||
`aura-core::project`) — a C-ABI stamp prefix (rustc version + aura-core
|
||||
version, baked per consuming build) validated **before** any Rust-ABI field is
|
||||
touched, then the vocabulary resolver + enumerable type-id list behind the
|
||||
stamp gate. "Hot-reload" reads, in v1, as **per-invocation load of the
|
||||
freshest build**: the author (Claude) runs `cargo build`, the next `aura`
|
||||
invocation locates the artifact via `cargo metadata` (debug default,
|
||||
`--release` opt-in) and loads it **load-and-hold** (leaked, never unloaded).
|
||||
Scope boundary: load-and-hold is trivially sound only because the CLI is a
|
||||
one-shot process — a future long-running host (the open local-server thread,
|
||||
C22) must re-solve reload (host restart or subprocess isolation), never
|
||||
in-process unload. Mismatch of either stamp refuses (exit 1) naming both
|
||||
sides; the project vocabulary is charter-checked at load (`::`-namespaced ids,
|
||||
no duplicates against std, list↔resolver cross-check) — the invariant-9
|
||||
data-plane discipline of the C24 enforcement-shift note, now enforced at the
|
||||
one seam.
|
||||
@@ -0,0 +1,63 @@
|
||||
# C13 — Hot-reload is authoring-only; deploy is frozen
|
||||
|
||||
**Guarantee.** A node/strategy is authored as a native Rust `cdylib`,
|
||||
hot-reloaded during the authoring loop (Rust-ABI; host and node built with the
|
||||
same toolchain). The live/deploy bot is a statically-linked, versioned, frozen
|
||||
artifact.
|
||||
|
||||
**Forbids.** Hot-swapping a running live bot; loading third-party / foreign-
|
||||
toolchain plugins.
|
||||
|
||||
**Why.** Hot-reload makes the research loop fast; a live artifact must be frozen
|
||||
and reproducible (audit trail: this bot = this commit). A sweep pays no
|
||||
hot-reload tax — params are runtime data ([C12](c12-atomic-sim-unit.md)), so
|
||||
the cdylib loads once.
|
||||
|
||||
## Current state
|
||||
|
||||
The authoring-loop half is realized; the frozen live/deploy artifact is not yet
|
||||
built.
|
||||
|
||||
A research project compiles to an external cdylib exporting one symbol,
|
||||
`AURA_PROJECT`, a two-tier `#[repr(C)]` `ProjectDescriptor`
|
||||
(`aura-core::project`, `crates/aura-core/src/project.rs`, emitted by the
|
||||
`aura_project!` macro). A **C tier** — `magic` (`AURAPROJ`),
|
||||
`descriptor_version`, the rustc-version and aura-core-version stamps, the
|
||||
namespace, all C-compatible field types — is validated **before** any Rust-ABI
|
||||
field is touched; a **Rust tier** — the vocabulary resolver
|
||||
`fn(&str) -> Option<PrimitiveBuilder>` and the enumerable type-id list — is read
|
||||
only once both stamps match. The stamp cannot certify the channel it rides on,
|
||||
so it does not depend on the Rust ABI it certifies; both stamps are baked at
|
||||
*aura-core* compile time (build.rs, per consuming build), so host and dylib each
|
||||
carry their own side's truth.
|
||||
|
||||
"Hot-reload" reads, in v1, as **per-invocation load of the freshest build**: the
|
||||
author runs `cargo build`, and the next `aura` invocation locates the artifact
|
||||
via `cargo metadata` (debug default, `--release` opt-in) and loads it
|
||||
**load-and-hold** — the `Library` is leaked, never unloaded, so its `'static`
|
||||
strings and fn-pointers stay valid (`aura-runner::project::load`,
|
||||
`crates/aura-runner/src/project.rs`). `validate_c_tier` checks the four stamps
|
||||
front-to-back; a mismatch of either version stamp refuses with exit 1 naming
|
||||
both sides (`ProjectError::Incompatible`). The project vocabulary is
|
||||
charter-checked at load (`check_charter`): non-empty namespace, every id
|
||||
`<ns>::`-prefixed, no duplicate, no collision against the std vocabulary,
|
||||
list↔resolver cross-check — the invariant-9 data-plane discipline (the
|
||||
[C24](c24-blueprint-data.md) enforcement-shift note) enforced at this one
|
||||
seam.
|
||||
|
||||
**Scope boundary.** Load-and-hold is trivially sound only because the CLI is a
|
||||
one-shot process. A future long-running host — the local-server thread
|
||||
([C22](c22-playground-traces.md)) — must re-solve reload by host restart or subprocess
|
||||
isolation, never in-process unload.
|
||||
|
||||
## See also
|
||||
|
||||
- [C12](c12-atomic-sim-unit.md) — params are runtime data, so the cdylib
|
||||
loads once across a sweep
|
||||
- [C16](c16-engine-project-split.md) — engine/project separation; the
|
||||
per-case dependency policy the cdylib boundary serves
|
||||
- [C22](c22-playground-traces.md) — the long-running host that must re-solve reload
|
||||
- [C24](c24-blueprint-data.md) — the enforcement-shift note whose
|
||||
data-plane discipline the charter check applies
|
||||
|
||||
> History: [c13-hot-reload-frozen-deploy.history.md](c13-hot-reload-frozen-deploy.history.md)
|
||||
@@ -0,0 +1,29 @@
|
||||
# C14 — Headless core, two faces: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract:
|
||||
> [c14-headless-two-faces.md](c14-headless-two-faces.md).
|
||||
|
||||
**Realization (cycles 0098/0099, #175 — the CLI meets GNU/clig.dev conventions
|
||||
via clap).** The programmatic/CLI face's argument surface moved from a hand-rolled
|
||||
argv parser to a `clap` derive parser (admitted under the C16 per-case dependency
|
||||
policy — research-side, a dev-loop/compile tax, never a frozen-artifact tax;
|
||||
invariant 8 untouched, the change is confined to `aura-cli`, a leaf binary the
|
||||
frozen deploy artifact cannot pick up). One declarative source now yields scoped
|
||||
`aura <sub> --help`, `--version`/`-V`, a per-flag Options section, and GNU
|
||||
`--flag=value` / `--` / long-option abbreviation. The **exit-code partition** is a
|
||||
durable part of the automation contract, so a caller can branch on the failure
|
||||
class without parsing stderr: **exit 0 = success**; **exit 2 = usage error** — a
|
||||
command-line fault (clap parse errors + aura's post-parse argument-structure
|
||||
validations, *including* the content of an argv-named blueprint file, which is
|
||||
itself an argument); **exit 1 = runtime failure** — a well-formed command whose
|
||||
needed environment / recorded state is missing, or bad piped stdin data. The four
|
||||
dual-grammar subcommands (run/sweep/walkforward/mc) keep both grammars under one
|
||||
token via an optional `[blueprint]` positional + a post-parse `is_file()`
|
||||
dispatch; the execution layer is unchanged (arg-plumbing via thin `*_from`
|
||||
adapters). Error-message casing is normalized to the clap house style —
|
||||
every hand-rolled usage line reads `Usage: aura <verb> …` (#179, cycle 0101);
|
||||
refusal diagnostics stay unprefixed (diagnostics are not usage lines). The
|
||||
machine-first help surface (JSON/manifest help, stdin op-scripts) stays on the
|
||||
#157/C21 track, not this cycle (settled as human/GNU convention compliance).
|
||||
@@ -0,0 +1,89 @@
|
||||
# C14 — Headless core, two faces
|
||||
|
||||
**Guarantee.** The engine is a UI-agnostic library. Two faces sit on it: a
|
||||
**programmatic/CLI** face (the primary surface for the LLM and automation —
|
||||
author a node, run a sim/sweep, emit structured metrics) and a **visual** face
|
||||
for human exploration. Visualization is only a downstream consumer node
|
||||
([C8](c08-node-contract.md)) on the streams.
|
||||
|
||||
**Forbids.** Any UI/pixel knowledge inside the engine.
|
||||
|
||||
**Why.** The LLM drives programmatically, the human visually; a headless core
|
||||
serves both. The visual face is the **playground** ([C22](c22-playground-traces.md)) —
|
||||
a **web frontend served from disk-persisted traces** (#101, ratified 2026-06):
|
||||
the engine writes recorded traces to disk and a browser charts them. It is
|
||||
staged after the runnable substrate but is core to aura's identity, not
|
||||
optional. Reading a serialized trace file is a stricter form of "no UI knowledge
|
||||
in the engine" than an in-process native UI reaching zero-copy into the live SoA
|
||||
columns — the pivot to the disk-trace frontend keeps this contract's core intact
|
||||
and arguably tightens it.
|
||||
|
||||
## Current state
|
||||
|
||||
**The CLI conventions.** The programmatic/CLI face is a `clap` derive parser
|
||||
(`aura-cli`, `crates/aura-cli/src/main.rs`) that meets GNU / clig.dev
|
||||
conventions: one declarative source yields scoped `aura <sub> --help`,
|
||||
`--version`/`-V`, per-flag Options sections, and GNU `--flag=value` / `--` /
|
||||
long-option abbreviation. `clap` is admitted under the
|
||||
[C16](c16-engine-project-split.md) per-case dependency policy — a
|
||||
research-side, dev-loop/compile tax confined to `aura-cli`, a leaf binary the
|
||||
frozen deploy artifact ([invariant 8](c13-hot-reload-frozen-deploy.md)) cannot
|
||||
pick up. Usage lines follow the clap house style (`Usage: aura <verb> …`);
|
||||
refusal diagnostics stay unprefixed, since a diagnostic is not a usage line.
|
||||
|
||||
**The exit-code partition** is a durable part of the automation contract: a
|
||||
caller branches on the failure class without parsing stderr.
|
||||
|
||||
- **0 — success.**
|
||||
- **2 — usage error:** a command-line fault (clap parse errors + aura's
|
||||
post-parse argument-structure validations, *including* the content of an
|
||||
argv-named blueprint file, which is itself an argument).
|
||||
- **1 — runtime failure:** a well-formed command whose needed environment /
|
||||
recorded state is missing, or bad piped stdin data.
|
||||
- **3 — campaign completed with failed cells:** a well-formed campaign that ran
|
||||
to completion but with at least one failed cell (#272; `exit_on_campaign_result`
|
||||
in `crates/aura-cli/src/main.rs`, threaded from the run registry —
|
||||
[C18](c18-registry.md)).
|
||||
|
||||
**Dual grammar.** The four dual-grammar subcommands (run/sweep/walkforward/mc)
|
||||
keep both grammars under one token via an optional `[blueprint]` positional plus
|
||||
a post-parse `is_file()` dispatch; the execution layer is unchanged (arg-plumbing
|
||||
via thin `*_from` adapters). The machine-first help surface (JSON/manifest help,
|
||||
stdin op-scripts) is deferred to the #157 / [C21](c21-world.md) track, distinct
|
||||
from this human/GNU-convention compliance.
|
||||
|
||||
**Two artifact classes, two redundancy budgets** (#249, ratified 2026-07-13).
|
||||
The data layer the programmatic face emits is a public interface read raw (by
|
||||
humans and LLMs), and its records split into two classes with opposite
|
||||
redundancy budgets. **Generated outcome records** (manifests, metrics, family
|
||||
reports) have a single writer at run time — redundancy there cannot drift and is
|
||||
deliberately spent on direct readability. **Authored intent artifacts**
|
||||
(blueprints, op-scripts, campaign documents) are author-maintained — every
|
||||
redundancy is a drift site and stays out. Consequence in the run manifest
|
||||
(`RunManifest`, `crates/aura-engine/src/report.rs`): the untouched bound
|
||||
`defaults` are stamped one-directionally beside `params` (which keeps its "what
|
||||
varied" reproduce semantics, disjoint by construction from `defaults`, "what was
|
||||
held"), so a raw reader of a fully-bound run no longer sees a misleading
|
||||
`"params": []`; the blueprint itself carries no such duplication.
|
||||
|
||||
**The programmatic face is an executor** (#295, ratified 2026-07-20). The
|
||||
member-run recipe — the definition of what a standard aura backtest is — is
|
||||
library content (`aura-runner` / `aura-backtest` / `aura-measurement`,
|
||||
[C28](c28-stratification.md) assembly position), not CLI content: the binary is
|
||||
argv → document → executor → presentation, and a downstream World program
|
||||
reaches the same recipe with no binary involved. See
|
||||
[C25](c25-role-model.md)'s control-surface amendment for the projection rule.
|
||||
|
||||
## See also
|
||||
|
||||
- [C8](c08-node-contract.md) — visualization is a downstream sink/consumer node
|
||||
- [C16](c16-engine-project-split.md) — the per-case dependency policy under
|
||||
which `clap` is admitted
|
||||
- [C18](c18-registry.md) — the run registry that yields the exit-3
|
||||
completed-with-failures class
|
||||
- [C21](c21-world.md) — the #157 machine-first help track (deferred)
|
||||
- [C22](c22-playground-traces.md) — the visual face / playground
|
||||
- [C25](c25-role-model.md) — the control-surface amendment (executor projection)
|
||||
- [C28](c28-stratification.md) — the assembly position of the member-run recipe
|
||||
|
||||
> History: [c14-headless-two-faces.history.md](c14-headless-two-faces.history.md)
|
||||
@@ -0,0 +1,38 @@
|
||||
# C15 — Resampling-as-node; sessions/calendars: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date
|
||||
> stamp and may be superseded; this file is NOT current truth and NOT a
|
||||
> grounding surface. Current contract:
|
||||
> [c15-resampling-sessions.md](c15-resampling-sessions.md).
|
||||
|
||||
**Realization (instrument specs, 2026-06, #22 → #124).** The "instrument specs are
|
||||
metadata" half of this contract is realized by the **recorded geometry sidecar**:
|
||||
`instrument_geometry(server, symbol)`, over data-server's `symbol_meta`, returns
|
||||
neutral broker-agnostic `InstrumentGeometry` (the raw provider JSON never enters this
|
||||
repo) — non-scalar reference data held beside the hot path, keyed by symbol, feeding
|
||||
the sim-optimal broker's pip divisor (C10). The real-path pip is sourced from
|
||||
`InstrumentGeometry.pip_size`; refuse-don't-guess on absent geometry. **History:**
|
||||
cycles 0022/0063 first carried a Rust-authored vetted floor (`InstrumentSpec` — a
|
||||
single `pip_size`, later a six-field deploy-grade row `instrument_id`,
|
||||
`contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`, `quote_currency`, plus
|
||||
`tick_size`/`digits`), cross-checked against the sidecar geometry. **Cycle 0074
|
||||
removed that floor**: with the sidecar geometry supplying the real-path pip, the
|
||||
authored table was redundant, so `InstrumentSpec` / `instrument_spec` / the vetted
|
||||
list were deleted. The speculative deploy-grade fields and the override/floor tiers
|
||||
of the #124 hierarchy (tier 1 authored override, tier 3 authored floor) are
|
||||
**deferred** until a consumer — the cost model's currency→R normalization (C10), or
|
||||
the live deploy edge (real broker) — needs them, read from the sidecar geometry then. The `quote_currency` runtime-type transition is likewise
|
||||
deferred to that consumer; #124 collapses to **recorded sidecar geometry → refuse**
|
||||
in the meantime.
|
||||
|
||||
**Realization (SessionNode — `bars_since_open` only, by design; #154).** The
|
||||
session-context half of this contract ships **one** scalar stream, not the three the
|
||||
Guarantee lists: `Session` (`crates/aura-std/src/session.rs`) emits only
|
||||
`bars_since_open: i64` (tz-aware, DST-correct, baked Frankfurt open). This is a deliberate
|
||||
narrowing pinned in the node's own contract — *"there is no separate in-session bool gate,
|
||||
`bars_since_open` alone is the contract"*: a downstream `EqConst(== N)` gate subsumes the
|
||||
`in_session: bool` stream (pre-open instants give `<= 0`, which never match), and
|
||||
`session_open_ts: timestamp` has no consumer. The two streams the Guarantee names remain
|
||||
the original design intent, **deferred until a consumer needs them** (default-simple;
|
||||
forward-queued as #154). The stream model is untouched — session context is still exposed
|
||||
as scalar streams fed from beside the hot path.
|
||||
@@ -0,0 +1,55 @@
|
||||
# C15 — Resampling-as-node; sessions/calendars
|
||||
|
||||
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),
|
||||
clock-sensitive, emitting a completed bar only at the boundary (C2). Calendars
|
||||
and instrument specs are **metadata** — non-scalar reference data held beside
|
||||
the hot path, never streamed. Session *context* is exposed as scalar streams
|
||||
via a `SessionNode`, so "the 3rd 15m candle after session open" is a plain node
|
||||
checking `bars_since_open == 3`, not special-cased session logic.
|
||||
|
||||
**Forbids.** Streaming the calendar; special-casing session logic outside the
|
||||
stream model.
|
||||
|
||||
**Why.** Keeps the line consistent — everything a signal needs arrives as a
|
||||
stream; reference data feeds source/session nodes from beside the hot path.
|
||||
|
||||
## Current state
|
||||
|
||||
**Session context — `bars_since_open` is the shipped contract (#154).** The
|
||||
session-context node `Session` (`crates/aura-market/src/session.rs`) emits
|
||||
**one** scalar stream, `bars_since_open: i64` (tz-aware, DST-correct, over a
|
||||
baked Frankfurt open), and takes no scalar params; the zero-arg
|
||||
`SessionFrankfurt` roster preset (#261) exposes it with a single
|
||||
`period_minutes` knob. This is a deliberate narrowing pinned in the node's own
|
||||
contract: `bars_since_open` alone is the gate — a downstream `EqConst(== N)`
|
||||
subsumes an `in_session: bool` stream (pre-open instants read `<= 0`, which
|
||||
never match), and nothing consumes a `session_open_ts: timestamp`. The two
|
||||
further session-context streams the design intent names — `in_session: bool`
|
||||
and `session_open_ts: timestamp` — remain **deferred** until a consumer needs
|
||||
them (#154); the stream model is untouched, session context is still exposed as
|
||||
scalar streams fed from beside the hot path.
|
||||
|
||||
**Instrument geometry — recorded sidecar, refuse-don't-guess.** The "instrument
|
||||
specs are metadata" half is realized by the **recorded geometry sidecar**:
|
||||
`aura_ingest::instrument_geometry(server, symbol)` returns the neutral,
|
||||
broker-agnostic `InstrumentGeometry` (re-exported from `data_server::meta`; the
|
||||
raw provider JSON never enters this repo) — non-scalar reference data keyed by
|
||||
symbol, held beside the hot path. It feeds the sim-optimal broker's pip divisor
|
||||
(C10; consumed in `aura-runner`): the real-path pip is sourced from
|
||||
`InstrumentGeometry.pip_size`, and geometry that is absent is **refused, not
|
||||
guessed**. There is no Rust-authored instrument floor — `InstrumentSpec` /
|
||||
`instrument_spec` and the vetted table were removed once the sidecar supplied
|
||||
the real-path pip. The speculative deploy-grade fields and the authored
|
||||
override/floor tiers of the #124 hierarchy (tier 1 override, tier 3 floor),
|
||||
together with the `quote_currency` runtime-type transition, are **deferred**
|
||||
until a consumer — the cost model's currency→R normalization (C10) or the live
|
||||
deploy edge (real broker) — needs them, read from the sidecar geometry then;
|
||||
#124 collapses to *recorded sidecar geometry → refuse* in the meantime.
|
||||
|
||||
## See also
|
||||
- [C2](c02-causality.md) — the resampler emits a bar only once complete
|
||||
- [C8](c08-node-contract.md) — the node contract the resampler and session node obey
|
||||
- [C10](c10-bias-r-cost.md) — the pip divisor and currency→R normalization the geometry feeds
|
||||
- [C11](c11-sources-record-replay.md) — reference data fed from beside the hot path
|
||||
|
||||
> History: [c15-resampling-sessions.history.md](c15-resampling-sessions.history.md)
|
||||
@@ -0,0 +1,60 @@
|
||||
# C16 — Engine / project separation; three-tier node reuse: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date
|
||||
> stamp and may be superseded; this file is NOT current truth and NOT a
|
||||
> grounding surface. Current contract:
|
||||
> [c16-engine-project-split.md](c16-engine-project-split.md).
|
||||
|
||||
**Realization (cycle 0079, #136 — the analysis leaf becomes its own crate).** The
|
||||
trading-domain **analysis** layer — the post-run reductions that are pure functions of a
|
||||
run's recorded data (`RunMetrics`, `RMetrics`, the R reduction `summarize_r` /
|
||||
`r_metrics_from_rs`, the position-event table `PositionEvent` / `PositionAction` /
|
||||
`derive_position_events`, and the multiple-comparison hurdle math `inv_norm_cdf` /
|
||||
`expected_max_of_normals`) — is extracted out of `aura-engine`'s `report` module into a
|
||||
dedicated **`aura-analysis`** crate (deps: `aura-core` + `serde` only; `serde_json` is a
|
||||
dev-dependency, test-only). This sharpens the engine↔domain seam: the run loop's crate no
|
||||
longer *defines* the trading metrics — it `pub use`-re-exports them for source
|
||||
compatibility this cycle, and still hosts the trace-coupled `summarize`, `RunReport`,
|
||||
`RunManifest`, and the columnar trace utils. The non-node engine-workspace crates are now
|
||||
`aura-engine` (run loop), `aura-cli` (`aura` binary), `aura-ingest` (ingestion edge),
|
||||
`aura-composites` (composite-builder convenience layer), and `aura-analysis` (post-run
|
||||
domain reductions + selection provenance). Behaviour-preserving (C1): every serde shape
|
||||
is byte-identical (C18 goldens green).
|
||||
|
||||
**Realization (cycle 0080, #136 — engine-side extraction complete).** The last
|
||||
trading-domain types still *defined* in `aura-engine`'s `report` module — the
|
||||
selection-provenance types `FamilySelection` / `SelectionMode` — also move to
|
||||
`aura-analysis` (the only non-cyclic home: `RunManifest.selection` needs the type and the
|
||||
engine already depends on `aura-analysis`; the registry/CLI reach them unchanged via the
|
||||
re-export). Behaviour-preserving (C1; suite 665/0). **Settled (2026-06-27, user-ratified
|
||||
in the #136 thread):** the `aura-registry` `Metric` / `metric_cmp` / deflation vocabulary
|
||||
**stays in the registry by design** — the registry is the trading *selection* layer, not
|
||||
the domain-free engine, and its deflation statistics branch irreducibly on metric
|
||||
identity (R-bootstrap vs. `total_pips` dispersion floor); forcing genericity would be a
|
||||
one-implementor abstraction. *(Superseded 2026-07-20: measurement's IC became the second
|
||||
implementor — #290 — and the vocabulary moved behind the `MetricVocabulary` trait
|
||||
supplied by the outer rungs; see the C28 #147 disposition.)* **Deferred to the World /
|
||||
C21 layer (explicitly *not* #136):**
|
||||
making `RunReport` generic over a metric type and turning the orchestration/registry layer
|
||||
into a reusable domain-agnostic substrate — until then `RunReport` / `RunManifest` /
|
||||
`summarize` stay trace-coupled in the engine.
|
||||
|
||||
**Realization (cycle 0107, #198 — the campaign-execution leaf).** The non-node engine
|
||||
workspace gains **`aura-campaign`**: campaign-execution *semantics* (cell enumeration,
|
||||
preflight, gate evaluation, winner selection, walk-forward rolling, realization assembly —
|
||||
see C18) as a leaf library crate whose deps deliberately exclude `aura-ingest` /
|
||||
`aura-std` / `aura-composites`; harness construction and data binding enter only through
|
||||
the one-method `MemberRunner` seam, so consumers (the `aura` CLI today; the playground and
|
||||
tests tomorrow) bind their own runners while the semantics live here once. It is
|
||||
explicitly NOT C21's World (a project-side program): it realizes one campaign document
|
||||
and owns no topology, no data sources, no UI.
|
||||
|
||||
**Realization (2026-07-12 — wiring-only tier, #241).** The smallest project
|
||||
is now data-only: `Aura.toml` + `blueprints/` + `runs/`, no crate. The load
|
||||
boundary tier-selects (a `[nodes]` pointer list in `Aura.toml` → load that
|
||||
crate; a root `Cargo.toml` → the pre-#241 native project, unchanged; neither
|
||||
→ std-vocabulary-only). `aura new` scaffolds the data-only tier;
|
||||
`aura nodes new` scaffolds a node crate **beside** the project and attaches
|
||||
it — the visible role-2 switch (C25 role model). Provenance widened
|
||||
additively: a data-only run stamps commit-only. Invariant 9's "a project is
|
||||
always a Rust crate" was deliberately amended (user decision, 2026-07-12).
|
||||
@@ -0,0 +1,100 @@
|
||||
# C16 — Engine / project separation; three-tier node reuse
|
||||
|
||||
**Guarantee.** aura is the reusable **engine**; each research project is a
|
||||
separate external repo that depends on aura via cargo (the game-engine / game
|
||||
split). **Node reuse is cargo-native, in three tiers:** `aura-std` (universal
|
||||
blocks, shipped with the engine) / **shared node crates** (cross-project-reusable,
|
||||
their own repos, pulled as cargo git deps) / **project-local node crates**
|
||||
(experimental, project-specific). A reusable node is an `rlib` dependency; the
|
||||
hot-reload unit is the project-side `cdylib` that composes it (C13).
|
||||
|
||||
**A project is a directory anchored by a static `Aura.toml`** (project context,
|
||||
paths only: data archive root, runs dir), in two tiers (#241, ratified
|
||||
2026-07-12):
|
||||
- **Data-only** — the default, smallest project: `Aura.toml` + `blueprints/` +
|
||||
`runs/`, blueprints and research documents over the std vocabulary, **no
|
||||
crate, no build step**.
|
||||
- **Native** — node logic lives in **node crates**: separate `cdylib` crates
|
||||
referenced via `Aura.toml [nodes]`, scaffolded and attached by a dedicated
|
||||
verb (the visible role-2 switch, C25).
|
||||
|
||||
The load boundary **tier-selects**: a `[nodes]` pointer list → load that crate;
|
||||
a root `Cargo.toml` → the pre-#241 native project, unchanged; neither →
|
||||
std-vocabulary-only. `aura new` scaffolds the data-only tier; `aura nodes new`
|
||||
scaffolds a node crate beside the project and attaches it. During research the
|
||||
`aura` host loads and runs the project (C13 hot-reload); for deploy the chosen
|
||||
strategy + broker freeze into a standalone binary.
|
||||
|
||||
**Dependency policy (amended 2026-06-10).** Dependencies are admitted by
|
||||
deliberate, **per-case review** — what a crate pulls in weighed against what it
|
||||
buys — with **particular scrutiny for anything that enters the frozen deploy
|
||||
artifact** (C13: this bot = this commit). Well-established standard crates
|
||||
(`serde`, `rayon`, …) pass that review and are used wherever they do the job,
|
||||
including in the bot. There is **no blanket zero-dependency commitment** and
|
||||
**no blanket admission**; hand-rolling what a vetted standard crate already does
|
||||
is the anti-pattern, not the dependency. `aura-ingest` is the data-source
|
||||
ingestion edge, **not** a dependency firewall.
|
||||
|
||||
**Forbids.** Project-specific signals in the aura repo (it keeps at most
|
||||
example/fixture nodes under `examples/` for its own tests); a multi-project
|
||||
manager inside aura; a bespoke node registry/marketplace (cargo + Gitea *is* the
|
||||
package mechanism). Dependency admission is governed by the per-case policy
|
||||
above, not a blanket ban.
|
||||
|
||||
**Why.** The engine/game split keeps the engine sharp and reusable while each
|
||||
project versions its own research with its own forward-queue. Promotion
|
||||
(local → shared → std) is the ordinary Rust reuse gradient, no new mechanism.
|
||||
The original "a project is always a Rust crate" invariant was deliberately
|
||||
amended (#241) once the role model diagnosed that the smallest project carries
|
||||
no native logic at all.
|
||||
|
||||
## Current state
|
||||
|
||||
The engine workspace carries the non-node crates beyond `aura-std`; the full
|
||||
roster and its layering are C28's ladder — this contract does not re-enumerate
|
||||
it. `aura-ingest` is the data-source ingestion edge, where the `data-server`
|
||||
external tree enters.
|
||||
|
||||
**Engine ↔ domain seam — the metric-generic engine** (decision log:
|
||||
#136 → #147/#290/#291/#292). `aura-engine` is the **domain-free, metric-generic**
|
||||
run loop: `RunReport<M>` (`crates/aura-engine/src/report.rs`) is parametrized
|
||||
over a metric type. The concrete trading instantiation `M = RunMetrics` and the
|
||||
backtest metric reductions (`RunMetrics`, `RMetrics`, `summarize_r`,
|
||||
`PositionEvent` / `PositionAction` / `derive_position_events`) live in the
|
||||
outer-rung crate **`aura-backtest`** (`crates/aura-backtest/src/metrics.rs`).
|
||||
The domain-neutral statistics substrate and selection provenance — the
|
||||
multiple-comparison hurdle math (`inv_norm_cdf`, `expected_max_of_normals`), the
|
||||
bootstrap/permutation kit (`MetricStats`, `resample_block`, `permute`), the
|
||||
`MetricVocabulary` trait, and `FamilySelection` / `SelectionMode` — live in
|
||||
**`aura-analysis`** (`crates/aura-analysis/src/lib.rs`). `aura-registry`'s
|
||||
selection vocabulary and `aura-measurement`'s IC (`IcMetrics`) are the trait's
|
||||
two production implementors. This is the realized form of the once-deferred
|
||||
"make `RunReport` generic / turn the orchestration layer into a domain-agnostic
|
||||
substrate" work.
|
||||
|
||||
**Campaign-execution leaf.** `aura-campaign` owns campaign-execution
|
||||
*semantics* — cell enumeration, preflight, gate evaluation, winner selection,
|
||||
walk-forward rolling, realization assembly (C18) — as a leaf library whose deps
|
||||
deliberately exclude `aura-ingest` / `aura-std` / `aura-composites`; harness
|
||||
construction and data binding enter only through the one-method `MemberRunner`
|
||||
seam (`&dyn MemberRunner`), so each consumer binds its own runner while the
|
||||
semantics live here once. It is explicitly **not** C21's World: it realizes one
|
||||
campaign document and owns no topology, data sources, or UI.
|
||||
|
||||
**Two-tier scaffolding.** `aura new` emits the data-only project
|
||||
(`crates/aura-cli/src/scaffold.rs`: `Aura.toml` + `blueprints/*.json` + `runs/`,
|
||||
no crate); `aura nodes new` emits a node crate beside the project and appends
|
||||
the `[nodes]` pointer (`crates/aura-cli/src/main.rs`; the load-boundary tier
|
||||
hints live in `graph_construct.rs::tier_hint_for_type_id`). Provenance widened
|
||||
additively: a data-only run stamps **commit-only**
|
||||
(`crates/aura-runner/src/project.rs::Env::provenance`).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — the behaviour-preserving invariant the crate extractions kept
|
||||
- [C13](c13-hot-reload-frozen-deploy.md) — hot-reload authoring loop / frozen deploy artifact
|
||||
- [C18](c18-registry.md) — the run registry and campaign realization assembly
|
||||
- [C21](c21-world.md) — the World the campaign leaf is deliberately not
|
||||
- [C25](c25-role-model.md) — the role-2 switch `aura nodes new` makes visible
|
||||
- [C28](c28-stratification.md) — the full crate roster and the layer ladder
|
||||
|
||||
> History: [c16-engine-project-split.history.md](c16-engine-project-split.history.md)
|
||||
@@ -0,0 +1,52 @@
|
||||
# C17 — Authoring surface
|
||||
|
||||
**Guarantee.** All *logic* — nodes, strategies, **and experiments/harnesses** —
|
||||
is authored in native Rust through **Claude Code + the skills pipeline**: the
|
||||
human describes, Claude writes the Rust, builds it, runs it via the `aura` CLI,
|
||||
and reports metrics. This binds **computation** — a node's math, a
|
||||
meta-program's control flow — to native Rust (the RustAst lesson, sharpened: a
|
||||
small LLM cannot reliably *apply* a computational DSL, and a strong one does not
|
||||
need one). It does **not** bind **topology** — which node feeds which, the
|
||||
structural axes, the param-space — to Rust *source*: topology is a
|
||||
**serializable data value** the World owns (C24), a non-Turing static DAG over
|
||||
the closed, compiled-in node vocabulary (C8/C16) that carries no computation
|
||||
(#109, resolved 2026-06-29). The no-DSL guard therefore forbids a
|
||||
*computational* experiment / strategy language, not a static topology-data
|
||||
format — the experiment-matrix *generator* may stay Rust (C20); its *output* is
|
||||
topology-data. Declarative config (`Aura.toml`) carries only **static project
|
||||
context** (paths only: data archive root, runs dir), never logic. aura ships
|
||||
**no embedded coding-LLM**; IONOS LLMs are used only as a *runtime data source*
|
||||
(news-agent bias, C11), gated by per-session consent, never in the code path.
|
||||
|
||||
**Forbids.** An in-app LLM chat that generates node code inside aura; using
|
||||
IONOS (weaker models) as the authoring brain; a *computational* experiment /
|
||||
strategy DSL the author must apply (the RustAst trap) — distinct from a static
|
||||
topology-data format, which is permitted.
|
||||
|
||||
**Why.** LLMs author Rust well in Claude Code — that is the fix to RustAst's
|
||||
failure; making weaker models the coding brain reintroduces the very problem.
|
||||
Keeps aura's scope an engine + playground, not an LLM-IDE. Topology-as-data is
|
||||
*not* the RustAst trap re-opened: that trap is a DSL the author must *apply*; a
|
||||
topology-data blueprint is machine-generated / owned and applied by no one, so
|
||||
the no-DSL guard stands unbroken — a *closed-vocabulary* data artifact is not
|
||||
the forbidden *open, logic-bearing* language (C25).
|
||||
|
||||
## Current state
|
||||
|
||||
The authoring loop is the skills pipeline over native Rust: node / strategy /
|
||||
harness logic compiles into the closed node vocabulary, and the `aura` CLI runs
|
||||
it. Topology-as-data is realized as **blueprints** — a data-only project's
|
||||
`blueprints/*.json` are serialized static DAGs over the std vocabulary (the
|
||||
`aura new` scaffold writes a starter `blueprints/signal.json`;
|
||||
`crates/aura-cli/src/scaffold.rs`); the blueprint data format itself is C24's
|
||||
contract. `Aura.toml` carries paths-only project context (C16). The IONOS
|
||||
runtime-source path (news-agent bias) is the only external-LLM use — C11's
|
||||
record-then-replay edge, consent-gated, never in the sim code path.
|
||||
|
||||
## See also
|
||||
- [C8](c08-node-contract.md) — the closed, compiled-in node vocabulary topology draws on
|
||||
- [C11](c11-sources-record-replay.md) — IONOS news bias as a recorded runtime source
|
||||
- [C16](c16-engine-project-split.md) — the engine/project split and `Aura.toml` project context
|
||||
- [C20](c20-strategy-harness.md) — the experiment-matrix generator that stays Rust
|
||||
- [C24](c24-blueprint-data.md) — the topology / blueprint data format the World owns
|
||||
- [C25](c25-role-model.md) — a closed-vocabulary artifact vs the forbidden open DSL
|
||||
@@ -0,0 +1,273 @@
|
||||
# C18 — Project management: one repo = one project, plus a run registry: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c18-registry.md](c18-registry.md).
|
||||
|
||||
**Realization (cycle 0029 — the flat run registry).** The experiments-&-results
|
||||
plane shipped as `aura-registry`: an append-only JSONL store (`runs/runs.jsonl`),
|
||||
one serde_json `RunReport` (`RunManifest{commit, params, window, seed, broker}` +
|
||||
`RunMetrics`) per line, with a typed read-path (`load`) and best-first ranking
|
||||
(`rank_by`/`optimize`). C9 holds — the registry depends on `aura-engine`, never
|
||||
the reverse.
|
||||
|
||||
**Realization (cycle 0045 — lineage as related records, #70).** The *lineage*
|
||||
depth is now realized as a **family store**: a sweep / Monte-Carlo / walk-forward
|
||||
run (the C12 axes) is persisted as a *set of related records* — each a
|
||||
`FamilyRunRecord` (a `RunReport` stamped with its `family` + `run` + `kind` +
|
||||
`ordinal`) — in a sibling JSONL (`families.jsonl`), leaving the flat `runs.jsonl`
|
||||
path and its `append`/`load`/`rank_by`/`optimize` API byte-for-byte unchanged.
|
||||
the user-facing `family_id = "{family}-{run}"` handle is **derived** from the
|
||||
stored `family` name plus a per-name `run` index (assigned as a numeric max+1 —
|
||||
not a content hash; re-running the same family mints a fresh id). `group_families` is the round-trip that re-derives a family
|
||||
from the stored links (re-listable / rankable as a unit — C21). The manifest *is*
|
||||
the re-derivation recipe (#71): no input-stream blob / path / payload enters a
|
||||
record, and a member's window is **producer-supplied** via
|
||||
`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage),
|
||||
never a materialized-`Vec` scan at the call site. CLI surface: `aura mc`,
|
||||
`aura runs families`, `aura runs family <id> [rank <metric>]`; `aura sweep` /
|
||||
`walkforward` / `mc` persist via `append_family` with an optional `--name`.
|
||||
|
||||
**Realization (cycle 0078 — cross-instrument family + instrument lineage, #146).**
|
||||
The comparison axis (C12) is realized as a `FamilyKind::CrossInstrument` family:
|
||||
`aura generalize` runs one candidate across an instrument list and persists the M
|
||||
per-instrument runs via `append_family`, each member self-identifying through a new
|
||||
first-class `RunManifest.instrument` lineage field (serde-widened with
|
||||
`skip_serializing_if`, so legacy lines and every non-cross-instrument path stay
|
||||
byte-identical — C14/C23). The cross-instrument *generalization score* (worst-case R
|
||||
floor + sign-agreement + per-instrument breakdown) is a **recomputable aggregate**
|
||||
over those members, not a persisted family-level record — distinct from #144/#145's
|
||||
per-winner selection annotation on `RunManifest.selection`.
|
||||
Deferred (Non-goals): replay-dedup (content-addressed *identity* shipped — #158,
|
||||
cycle 0094, Realization below); the "run-diff"
|
||||
depth and ranking families against each other (cross-family, vs. within-family);
|
||||
and a live producer for the flat `runs.jsonl` standalone-run path — no CLI command
|
||||
writes it (sweep/walkforward persist to the family store; `aura run` does not
|
||||
persist). **Resolved (#73, 2026-06): retired** — `aura runs list` / `rank` dropped;
|
||||
families (C21) subsume standalone over-time comparison. The `aura-registry` flat
|
||||
lib API is retained: `rank_by`/`optimize` keep live consumers (`optimize` backs
|
||||
walk-forward's in-sample step, `rank_by` backs `runs family … rank`); `append`/
|
||||
`load` (the flat-store half) remain public API with **no in-tree caller** after
|
||||
this retire — tested, available to external consumers, a latent dead-code surface
|
||||
a later sweep may revisit. **Unknown-id contract (ratified, Runway fieldtest
|
||||
2026-06).** `aura runs family <id>` treats an unknown-but-well-formed id as an
|
||||
*empty family* (prints nothing, exit 0) — the same treat-as-empty discipline as
|
||||
`Registry::load` reading a missing store as `Ok(empty)`, and deliberately
|
||||
distinct from the retired `list`/`rank` exit-2, which is argv-shape rejection
|
||||
*before* any store access, not a found-nothing lookup. Tightening to a non-zero
|
||||
`no such family <id>` exit (typo-safety) is an available future UX choice, not a
|
||||
current contract.
|
||||
|
||||
**Refinement (2026-06-29 — a generated run's topology lives in / is addressed by
|
||||
its manifest, C24).** The manifest identifies a run's topology today only via
|
||||
`commit` (this engine + a hand-coded harness) — sufficient while harnesses are a
|
||||
finite hand-coded menu. Once the World **generates or structurally searches**
|
||||
topologies (C21/C24), `commit` no longer identifies the graph, so the manifest must
|
||||
**carry or content-address the topology-data** to stay the re-derivation recipe
|
||||
(C18's "reproducible from a recorded manifest"). This pulls the previously-deferred
|
||||
**content-addressed identity** non-goal forward as the natural home for a generated
|
||||
topology's identity. The format and the carrier are C24's design; recorded here as
|
||||
the reproduction requirement it must meet.
|
||||
|
||||
**Realization (2026-07-01, cycle 0094 — content-addressed reproduction of a generated
|
||||
run, #158).** A blueprint sweep's topology is now **content-addressed**: the canonical
|
||||
`blueprint_to_json` bytes are stored once, keyed by the `topology_hash` the manifest
|
||||
already carries, in a **dumb bytes-by-key store** beside the run registry
|
||||
(`runs/blueprints/<hash>.json` — `Registry::put_blueprint`/`get_blueprint`, aura-registry;
|
||||
no `sha2`, no parse — the caller owns the hash, and reproduction's bit-identical compare is
|
||||
the integrity check). `aura reproduce <family-id>` re-derives every persisted member: load
|
||||
the member's blueprint by its `topology_hash`, reconstruct the sweep point from the
|
||||
recorded params, re-run through the **same** `run_blueprint_member` the live sweep uses
|
||||
(so bit-identity is by construction, C1), and compare metrics — refuse-don't-guess on an
|
||||
unknown id / missing stored blueprint (exit 1 — recorded state is missing, C14's
|
||||
runtime-failure class; this paragraph over-claimed exit 2 until #298 recorded the code's
|
||||
actual, C14-consistent behaviour), DIVERGED → exit 1. The id resolves first as the
|
||||
derived `{family}-{run}` handle; a bare enumeration name naming exactly one stored run
|
||||
resolves as fallback, an ambiguous name refuses listing the candidate handles (#298 —
|
||||
the list-then-reproduce seam). The manifest + the
|
||||
content-addressed store + the commit are the complete re-derivation recipe (C18). One
|
||||
blueprint is stored per family (all members share the signal `topology_hash` — C11/C12
|
||||
dedup). `aura graph introspect --content-id` exposes the same id for an op-script, via the
|
||||
one `content_id` primitive `topology_hash` also uses (acc 1); a Tier-1 optional the
|
||||
blueprint does not use leaves the id byte-stable (acc 3, composing #156/#164).
|
||||
`serde_json/float_roundtrip` is enabled so stored f64 metrics round-trip exactly through
|
||||
`families.jsonl` — the precondition for a bit-identical compare (C1). **Signal-only this
|
||||
cycle**: the id covers the signal blueprint; the fixed r-sma scaffolding stays
|
||||
commit-identified (it is not yet blueprint-data, C24). Whole-harness / structural-axis
|
||||
content-addressing remains deferred. The debug-name-in-id question was settled additively
|
||||
(cycle 0104, #171): the **identity id** — the canonical form with every non-load-bearing
|
||||
debug symbol blanked (invariant 11), hashed through the same `content_id` primitive and
|
||||
surfaced as `aura graph introspect --identity-id` beside `--content-id` (combinable) —
|
||||
makes same-topology blueprints comparable across authoring paths, while the byte-exact
|
||||
`topology_hash` keeps every role untouched (introspection-only: no manifest field, no
|
||||
store key until a dedup consumer exists). Reproduction is proven on
|
||||
synthetic (deterministic) data; recorded-dataset reproduction rides the DataServer seam
|
||||
(#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` —
|
||||
branching on the family kind, it reconstructs each member's seed-driven synthetic walk from
|
||||
the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives
|
||||
bit-identically through the same `run_blueprint_member` path. **Cycle 0097 (#173):** reproduce
|
||||
spans the third variant, `FamilyKind::WalkForward` — the same branch rebuilds each OOS member's
|
||||
windowed slice from `manifest.window` (winner params via the shared `manifest→cells` recovery),
|
||||
so an `aura walkforward` family re-derives bit-identically too. All three family kinds now
|
||||
persist *and* reproduce through the one shared `topology_hash`+`put_blueprint` hook.
|
||||
|
||||
**Realization (cycle 0106, #189 — research-artifact document stores).** The registry's
|
||||
content-addressed store family grew two siblings beside `blueprints/`: `processes/` and
|
||||
`campaigns/` hold the two research-artifact document types shipped by the #188 role-model
|
||||
pass — the **process document** (role 5: a named validation/eval methodology, a closed std
|
||||
stage vocabulary wrapping shipped primitives) and the **campaign document** (role 6b:
|
||||
persisted experiment intent — instruments × windows × strategy refs by content/identity id ×
|
||||
param axes × process ref (content-id-only) × data-level presentation). Documents are
|
||||
canonical JSON (`format_version` envelope, omit-defaults, no trailing newline) keyed by the
|
||||
**shared content-id primitive, now library-hosted** (`aura_research::content_id_of`;
|
||||
`aura-cli`'s `content_id`/`topology_hash` delegate byte-identically — the id goldens pin the
|
||||
move). Unlike `put_blueprint` (caller owns the hash), the document puts self-key from
|
||||
canonical bytes; gets are `Ok(None)` treat-as-empty. The **referential tier**
|
||||
(`validate_campaign_refs`) resolves process/strategy refs against the stores (identity refs
|
||||
by store scan in this cycle; index-first since #191, below) and checks each campaign axis — name AND declared `ScalarKind`, the axis
|
||||
carries its kind once — against the referenced blueprint's `param_space`. Campaign P1
|
||||
control constructs (axes/gates/ladders per #188) carried intent only in this cycle: no
|
||||
executor existed yet — executor need was re-tested against the intent-persistence
|
||||
diagnosis (#189 triage, decided item 6; the cycle-0106 fieldtest F7 verdict was that
|
||||
evidence), and the v1 executor shipped the next cycle (below).
|
||||
|
||||
**Realization (cycle 0107, #198/#196 — the campaign executor and its realization store).**
|
||||
`aura campaign run <file|content-id>` executes a campaign (a file is register-then-run
|
||||
sugar; the content id is canonical): zero-fault referential gate, then the process
|
||||
pipeline — v1 executable shape `std::sweep [std::gate]* [std::walk_forward]?`
|
||||
(`std::monte_carlo`/`std::generalize` refused loudly at preflight in this cycle; they
|
||||
execute since cycle 0108, below) — runs once per (strategy, instrument, window) cell in
|
||||
doc order. Execution
|
||||
*semantics* live in the **`aura-campaign` library crate** (#198 home decision: reachable
|
||||
beyond the CLI; NOT C21's project-side World): grid odometer over the campaign axes,
|
||||
members through the engine `sweep` over a **`ListSpace`** (explicit point set beside
|
||||
`GridSpace`/`RandomSpace` — a gate's survivor subset has no cartesian structure),
|
||||
per-member gates via the 14-name `member_metric` roster (an R-predicate over a missing R
|
||||
block fails conservatively), walk-forward re-rolled in the doc's epoch-ms unit
|
||||
(`WindowRoller`; IS windows search ONLY the survivor points; OOS winner reports carry
|
||||
`manifest.selection`), deflation nulls seeded from the doc's `seed` (C1: realization is a
|
||||
pure function of doc + stores + data). Harness/data binding stays consumer-side behind the
|
||||
one-method **`MemberRunner`** seam — the CLI binds the shipped loaded-blueprint reduce
|
||||
convention with unique suffix-join of raw axis names onto the wrapped `param_space`. The
|
||||
registry grew the **`campaign_runs.jsonl`** JSONL sibling (beside `runs.jsonl` /
|
||||
`families.jsonl`): one thin `CampaignRunRecord` per run — campaign/process ids, seed, and
|
||||
per-cell realized stage prefixes linking family ids, gate survivor ordinals, and sweep
|
||||
selections — over untouched family records, run-counted per campaign id. Zero survivors
|
||||
truncate a cell's realized prefix and exit 0 (a null result is a valid research result);
|
||||
`emit` is honored (`family_table`/`selection_report` lines); `persist_taps` was deferred
|
||||
LOUDLY on stderr in this cycle (the wiring shipped in cycle 0109, below). The **blueprint on-ramp** (#196) closes
|
||||
the F5 authoring gap: `aura graph register` (store put keyed by content id == topology
|
||||
hash), `aura graph introspect --params` (the RAW `param_space` namespace campaign axes
|
||||
are validated against), and a blueprint-file mode on `--content-id`. The
|
||||
`std::walk_forward` vocabulary was corrected to machinery-true fields
|
||||
(`in_sample_ms`/`out_of_sample_ms`/`step_ms`/`mode` — `WindowRoller`'s three lengths and
|
||||
both `RollMode`s; the shipped `folds` slot mapped to nothing the machinery accepts), with
|
||||
a new `ZeroWalkForwardLength` intrinsic fault; wf-bearing process docs get new content ids
|
||||
by design (the 0106 fieldtest corpus stays as the historical record). Known debt:
|
||||
metric-roster triplication (still hand-maintained, but drift from the shipped
|
||||
`aura-analysis` types is now test-caught by a cross-crate guard, #190;
|
||||
single-source removal waits on #147), deflation-constant duplication (#199).
|
||||
|
||||
#300 (2026-07-21) closed the store's read loop: `aura process|campaign show
|
||||
<content-id>` prints a registered document's canonical bytes — the
|
||||
generate → retrieve → hand-extend → re-register cycle needs no direct store
|
||||
filesystem access.
|
||||
|
||||
**Realization (cycle 0108, #200 — the annotator stages execute).** The v2 executable
|
||||
shape is `std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]?
|
||||
[std::generalize]?` — an ordered optional annotator suffix, each at most once,
|
||||
`std::generalize` strictly last; the executor preflight refuses what the intrinsic tier
|
||||
deliberately admits (`[sweep, mc, walk_forward]` is intrinsically valid — the tier
|
||||
boundary is test-pinned on both sides), plus three new static guards
|
||||
(single-instrument generalize, non-R generalize metric via the registry's
|
||||
`check_r_metric`, zero mc `resamples`/`block_len`). **Both annotators are terminal**
|
||||
(unanimous #200 triage): nothing flows out of them; filtering stays the gate's monopoly.
|
||||
`std::monte_carlo` bootstraps the stage's *incoming R-evidence* with one semantics,
|
||||
input-shaped by position — after a walk_forward, ONE `r_bootstrap` over the wf family's
|
||||
pooled per-window OOS `net_trade_rs` in roll order (`PooledOos`; #259 materialized this
|
||||
conduit as the cost-netted per-trade series `r − cost_in_r`, equal to the gross series
|
||||
bit-for-bit when no cost model is bound); after sweep/gates, one `r_bootstrap` per
|
||||
surviving member's fresh in-memory series (`PerSurvivor`, ordinals into the population
|
||||
family; a zero-trade member records the engine's defined all-zero degenerate) — seeded
|
||||
from the campaign doc's `seed` (C1; `net_trade_rs` is `#[serde(skip)]`, so annotators run
|
||||
in-executor or not at all). `std::generalize` executes at **campaign
|
||||
scope**: after all cells, per (strategy, window) the per-cell *nominees* (last wf
|
||||
window's OOS report, else the sweep winner; none on gate truncation) across instruments
|
||||
feed the shipped `generalization()` when ≥ 2 exist — divergent per-instrument winners
|
||||
are exposed via their params, never averaged away; a shortfall is recorded, not computed
|
||||
around. Realization: `StageRealization.bootstrap` (`StageBootstrap::PerSurvivor |
|
||||
PooledOos`) and `CampaignRunRecord.generalizations` (`CampaignGeneralization` keyed
|
||||
strategy × window with `winners`/`missing`), both serde-default sparse — pre-0108
|
||||
`campaign_runs.jsonl` lines parse unchanged (C14/C23). No document content id moved
|
||||
(introspection doc-strings dropped "in v1" only). Noted debt: the mc arm detects the wf
|
||||
family by the stringly `block == "std::walk_forward"` literal (a pre-existing lockstep
|
||||
pattern with the realization's block strings).
|
||||
|
||||
**Realization (cycle 0109, #201 — persist_taps wired).** Campaign presentation persists
|
||||
traces: the tap namespace is a **closed vocabulary** of the wrap convention's four sink
|
||||
names (`equity`/`exposure`/`r_equity`/`net_r_equity`; `aura_research::tap_vocabulary`,
|
||||
intrinsic `DocFault::UnknownTap` — the escalation for a new observable is a new
|
||||
vocabulary entry or an authored blueprint sink, never an open node-path namespace).
|
||||
Scope is the per-cell **nominee only**: after the pipeline settles, the CLI re-runs each
|
||||
nominee once in non-reduce mode (all four channels drained; windowed to the nominee
|
||||
manifest's own ns bounds) and **asserts metrics equality** against the recorded nominee —
|
||||
the C1 drift alarm, hard refusal on divergence (the reproduce precedent, enforced).
|
||||
Traces land in the existing TraceStore as
|
||||
`traces/{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json`
|
||||
(`ensure_name_free(Family)` once; window ordinal doc-derived), chartable by the unchanged
|
||||
viewer. The record carries ONE sparse pointer, `CampaignRunRecord.trace_name`
|
||||
(`Some("{campaign8}-{run}")` iff the doc requests taps — the claim-sentinel contract:
|
||||
`execute` claims, `append_campaign_run` composes the name via the single-sourced
|
||||
`derive_trace_name`, `execute` mirrors it onto the returned copy). Loud lines replace
|
||||
the retired deferral: per-cell no-nominee skip, per-run unproducible-tap skip
|
||||
(`net_r_equity` needs a cost leg; the campaign runner wires none), one summary.
|
||||
`aura-campaign` stays trace-agnostic (the MemberRunner seam is unchanged; the stamp is a
|
||||
pure name derivation). Noted debt: `aura chart` over the campaign family ROOT (cells
|
||||
spanning instruments) is untested/semantically undefined — only per-cell read-back is
|
||||
pinned.
|
||||
|
||||
**Realization (#272, 2026-07-14 — per-cell fault isolation).** A member
|
||||
fault (no-data, bind, run, or a caught panic) is a recorded per-cell outcome,
|
||||
never a global abort: `run_cell` returns a fault-annotated `CellRealization`
|
||||
(`fault: Option<CellFault>`, closed `CellFaultKind`) instead of `Err`, so
|
||||
`execute`'s existing accumulate-then-append-once tail persists every healthy
|
||||
cell and the one run record. Containment granularity is the cell for a sweep
|
||||
stage (a grid hole compromises selection) and the fold for walk_forward
|
||||
(surviving folds pool; failed folds recorded as `StageRealization.
|
||||
window_faults`, the summary naming the ratio). `ExecFault::Registry` and
|
||||
doc-shape preflight faults stay global. The CLI declares holes (per-cell
|
||||
notes + a completion summary) and a run with ≥1 failed cell exits **3**
|
||||
("completed with failed cells" — distinct from 0/1/2). A partially-covered
|
||||
window carries a `CellCoverage` annotation (effective bounds + interior gap
|
||||
months, from the #264 archive primitives). Generalize already treats a
|
||||
no-nominee cell as `missing`, so a failed cell surfaces there unchanged.
|
||||
Member panics are caught with `catch_unwind`(`AssertUnwindSafe`) at the three
|
||||
member-run sites and recorded as `MemberFault::Panic`; a ref-counted
|
||||
`SilencedPanic` guard (a process-global panic-hook save/no-op/restore behind a
|
||||
`static Mutex`, held only around each `catch_unwind`) suppresses the default
|
||||
crash backtrace so "recorded, campaign continues" is observably true on stderr,
|
||||
not merely true in the record. The guard's mutex serialises only the
|
||||
ref-count/hook-swap (O(1)), never the member computation, so C1 disjoint-parallel
|
||||
execution and determinism are preserved; ref-counting (save on 0→1, restore on
|
||||
1→0) keeps concurrent sweep/walk-forward threads and any caller-installed hook
|
||||
correct.
|
||||
|
||||
**Realization (#191, 2026-07-17 — identity-ref resolution is index-first).**
|
||||
`find_blueprint_by_identity` no longer re-loads the whole blueprint store per
|
||||
reference: a fourth persistent JSONL sidecar, `blueprint_identity_index.jsonl`
|
||||
(identity id → content id; fixed-name sibling of the runs store, appends under
|
||||
the #276 lock), is consulted first, and every hit is **verified** by loading
|
||||
that one blueprint under the current resolver and recomputing its identity id —
|
||||
the index is a cache, never an oracle, so resolution results stay
|
||||
scan-identical under roster drift, store surgery, or index corruption (the one
|
||||
unspecified corner is unchanged in kind: which same-identity twin answers was
|
||||
`read_dir`-order-dependent before and is index-history-dependent now). Any miss
|
||||
or failed verification runs the old scan as a **full-store repair pass**,
|
||||
collect-then-diff-append: the walk's last-wins mapping is diffed against the
|
||||
pre-walk snapshot after the walk, so a converged index — twin stores included —
|
||||
appends nothing (the twin-convergence pin). Index reads never fail a lookup
|
||||
(missing/unreadable file → empty, unparseable lines skipped), repair appends
|
||||
are best-effort, and a pre-index store backfills itself on its first miss, no
|
||||
migration; a read-only store keeps scanning as before. Write paths, the engine,
|
||||
and both callers are untouched; maintenance is lazy-only — put-time indexing
|
||||
was rejected because it would need a roster-free doc-level identity function
|
||||
whose equivalence to the loaded-composite path no green test ratifies (decision
|
||||
log: #191 comments).
|
||||
@@ -0,0 +1,287 @@
|
||||
# C18 — Project management: one repo = one project, plus a run registry
|
||||
|
||||
**Guarantee.** Management has two planes. (1) **Code & forward-queue:** git
|
||||
(commit = identity; the frozen bot *is* a commit) + Gitea (ideas/hypotheses as
|
||||
the forward-queue, a research thrust = a milestone, the
|
||||
`idea → experimental → validated → deployed` label gradient). (2) **Experiments
|
||||
& results:** an Aura-native **run registry** — one record per run = a *manifest*
|
||||
(node-commit + params + data-window + seed + broker profile + instrument +
|
||||
`topology_hash` + selection) + *metrics*, queryable, with *lineage* (composite ←
|
||||
signals; run ← inputs). Determinism (C1/C12) makes a run reproducible from its
|
||||
tiny manifest, so the registry stores manifests + metrics and re-derives full
|
||||
results on demand. Because the World may **generate or structurally search**
|
||||
topologies, `commit` alone no longer identifies the graph: the manifest carries
|
||||
a `topology_hash` and the registry keeps the canonical topology bytes in a
|
||||
content-addressed store beside it, so the **manifest + the content-addressed
|
||||
store + the commit** are the complete re-derivation recipe (C24). Depth:
|
||||
**structured** (promotion/status, lineage, run-diff).
|
||||
|
||||
**Forbids.** Storing results not reproducible from a recorded manifest;
|
||||
duplicating git/Gitea inside aura; a multi-project workspace manager.
|
||||
|
||||
**Why.** Comparing experiments over time is the heart of the research loop and
|
||||
has no home in git/Gitea; determinism makes a structured registry cheap. Every
|
||||
run emits a manifest + metrics, and the registry/index is built over manifests
|
||||
that already exist. Content-addressed identity is the natural home for a
|
||||
generated topology's identity: once a graph is no longer fully named by its
|
||||
`commit`, the re-derivation recipe must carry or content-address the
|
||||
topology-data to remain "reproducible from a recorded manifest" — the format and
|
||||
carrier are C24's design, the reproduction requirement is C18's.
|
||||
|
||||
## Current state
|
||||
|
||||
**Two planes, one crate.** The code/forward-queue plane is git + Gitea, unchanged
|
||||
by aura. The experiments-&-results plane is the run registry in `aura-registry`
|
||||
(C9: it depends on `aura-engine`, never the reverse).
|
||||
|
||||
**The store family.** All members are fixed-name siblings of the runs-store path
|
||||
(`Registry::path.with_file_name(...)`, `aura-registry`):
|
||||
|
||||
- `runs.jsonl` — the append-only flat store, one `RunReport` per line
|
||||
(`RunManifest` + `RunMetrics`), with a typed read-path (`load`) and best-first
|
||||
ranking (`rank_by`/`optimize`). No live producer writes it today — sweep /
|
||||
walk-forward / mc persist to the family store and `aura run` does not persist —
|
||||
so the flat lib API is retained but selectively live: `rank_by` backs
|
||||
`aura runs family … rank`, `optimize` backs walk-forward's in-sample step
|
||||
(`aura-campaign` and the registry's own selection helper both call it), while
|
||||
`append`/`load` remain public API with no in-tree caller (a latent surface for
|
||||
external consumers).
|
||||
- `families.jsonl` — the family store. A sweep / Monte-Carlo / walk-forward /
|
||||
cross-instrument run persists as a *set of related records*, each a
|
||||
`FamilyRunRecord` (a `RunReport` stamped with `family` + `run` + `kind` +
|
||||
`ordinal`; `FamilyKind ∈ {Sweep, MonteCarlo, WalkForward, CrossInstrument}`).
|
||||
`group_families` re-derives a family from the stored links (re-listable /
|
||||
rankable as a unit — C21). The user-facing `family_id = "{family}-{run}"` handle
|
||||
is **derived** from the stored `family` name plus a per-name `run` index
|
||||
(numeric max+1 — not a content hash, so re-running the same family mints a fresh
|
||||
id). CLI: `aura runs families`, `aura runs family <id> [rank <metric>]`;
|
||||
`aura sweep`/`walkforward`/`mc` persist via `append_family` with an optional
|
||||
`--name`.
|
||||
- `campaign_runs.jsonl` — one thin `CampaignRunRecord` per campaign run (below),
|
||||
over untouched family records.
|
||||
- `blueprints/<hash>.json`, `processes/`, `campaigns/` — the content-addressed
|
||||
document stores (below).
|
||||
- `blueprint_identity_index.jsonl` — the identity-ref resolution cache (below).
|
||||
|
||||
**The manifest is the re-derivation recipe.** No input-stream blob / path /
|
||||
payload enters a record; a member's data window is **producer-supplied** via
|
||||
`Source::bounds()`/`window_of` (eager or streamed → byte-identical lineage), never
|
||||
a materialized-`Vec` scan at the call site. New manifest fields (e.g. the
|
||||
first-class `instrument` lineage field) are serde-widened with
|
||||
`skip_serializing_if`, so legacy lines and every path that does not set them stay
|
||||
byte-identical (C14/C23). `serde_json/float_roundtrip` is enabled so stored f64
|
||||
metrics round-trip exactly through `families.jsonl` — the precondition for a
|
||||
bit-identical compare (C1).
|
||||
|
||||
**Content-addressed reproduction.** A run's topology is content-addressed: the
|
||||
canonical `blueprint_to_json` bytes are stored once, keyed by the `topology_hash`
|
||||
the manifest carries, in a **dumb bytes-by-key store**
|
||||
(`Registry::put_blueprint`/`get_blueprint` → `blueprints/<hash>.json`; no `sha2`,
|
||||
no parse — the caller owns the hash and reproduction's bit-identical compare is
|
||||
the integrity check). One blueprint is stored per family (all members share the
|
||||
signal `topology_hash` — C11/C12 dedup). `aura reproduce <id>` re-derives **every**
|
||||
persisted member: load its blueprint by `topology_hash`, reconstruct the member
|
||||
(a `Sweep` point from the recorded params; a `MonteCarlo` member's seed-driven
|
||||
synthetic walk from `manifest.seed`; a `WalkForward` OOS member's windowed slice
|
||||
from `manifest.window`, winner params via the shared manifest→cells recovery),
|
||||
re-run through the **same** `run_blueprint_member` the live path uses (bit-identity
|
||||
by construction — C1), and compare metrics. All three family kinds persist *and*
|
||||
reproduce through the one shared `topology_hash`+`put_blueprint` hook. Refuse-
|
||||
don't-guess (`aura_runner::reproduce`): an unknown id, a missing stored blueprint,
|
||||
or a DIVERGED compare each exit **1** — recorded state missing or mismatched is
|
||||
C14's runtime-failure class. Id resolution: first as the derived `{family}-{run}`
|
||||
handle; a bare enumeration name naming exactly one stored run resolves as fallback;
|
||||
an ambiguous name refuses, listing the candidate handles (the list-then-reproduce
|
||||
seam, #298). The recipe's scope is **signal-only** today: content-addressing covers
|
||||
the signal blueprint, while the fixed scaffolding stays commit-identified (not yet
|
||||
blueprint-data, C24); whole-harness / structural-axis content-addressing is
|
||||
deferred. Reproduction is proven on synthetic (deterministic) data;
|
||||
recorded-dataset reproduction rides the DataServer seam (#124).
|
||||
|
||||
**The one `content_id` primitive.** `topology_hash`, the document store keys, and
|
||||
`aura graph introspect --content-id`/`--identity-id` all route through
|
||||
`aura_research::content_id_of` (aura-cli's `content_id`/`topology_hash` delegate
|
||||
byte-identically). The **identity id** is the canonical form with every
|
||||
non-load-bearing debug symbol blanked (invariant 11 / C23), hashed through the same
|
||||
primitive: same-topology blueprints become comparable across authoring paths, while
|
||||
the byte-exact `topology_hash` keeps every debug role untouched (introspection-only
|
||||
— no manifest field, no store key, until a dedup consumer exists). A Tier-1 optional
|
||||
the blueprint does not use leaves the id byte-stable. `--content-id` and
|
||||
`--identity-id` are combinable.
|
||||
|
||||
**Cross-instrument generalization.** `FamilyKind::CrossInstrument`: `aura generalize`
|
||||
runs one candidate across an instrument list and persists the M per-instrument runs
|
||||
via `append_family`, each member self-identifying through `RunManifest.instrument`.
|
||||
The generalization score (worst-case R floor + sign-agreement + per-instrument
|
||||
breakdown) is a **recomputable aggregate** over those members, not a persisted
|
||||
family-level record.
|
||||
|
||||
**Research-artifact document stores.** `processes/` and `campaigns/` hold two
|
||||
document types (C25 roles 5 / 6b): the **process document** (a named
|
||||
validation/eval methodology — a closed std stage vocabulary wrapping shipped
|
||||
primitives) and the **campaign document** (persisted experiment intent —
|
||||
instruments × windows × strategy refs by content/identity id × param axes × process
|
||||
ref (content-id-only) × data-level presentation). Documents are canonical JSON
|
||||
(`format_version` envelope, omit-defaults, no trailing newline) keyed by the shared
|
||||
content-id primitive; unlike `put_blueprint` (caller owns the hash) the document
|
||||
puts **self-key** from their canonical bytes, and gets are `Ok(None)`
|
||||
treat-as-empty. The **referential tier** (`validate_campaign_refs`) resolves
|
||||
process/strategy refs against the stores (identity refs index-first, below) and
|
||||
checks each campaign axis — name AND declared `ScalarKind`, the axis carrying its
|
||||
kind once — against the referenced blueprint's `param_space`. `aura process|campaign
|
||||
show <content-id>` prints a registered document's canonical bytes (#300), so the
|
||||
generate → retrieve → hand-extend → re-register loop needs no direct store
|
||||
filesystem access.
|
||||
|
||||
**The campaign executor.** `aura campaign run <file|content-id>` executes a
|
||||
campaign (a file is register-then-run sugar; the content id is canonical): a
|
||||
zero-fault referential gate, then the process pipeline. The executable shape is
|
||||
`std::sweep [std::gate]* [std::walk_forward]? [std::monte_carlo]? [std::generalize]?`
|
||||
— an ordered optional annotator suffix, each at most once, `std::generalize`
|
||||
strictly last. The executor preflight is deliberately stricter than the intrinsic
|
||||
tier (`[sweep, mc, walk_forward]` is intrinsically valid — the tier boundary is
|
||||
test-pinned on both sides), plus static guards: single-instrument generalize, a
|
||||
non-R generalize metric (via the registry's `check_r_metric`), zero mc
|
||||
`resamples`/`block_len`, and `ZeroWalkForwardLength`. Execution *semantics* live in
|
||||
the **`aura-campaign`** library crate (reachable beyond the CLI; NOT C21's
|
||||
project-side World): a grid odometer over the campaign axes, members through the
|
||||
engine `sweep` over a **`ListSpace`** (an explicit point set beside
|
||||
`GridSpace`/`RandomSpace` — a gate's survivor subset has no cartesian structure),
|
||||
per-member gates via the 14-name `member_metric` roster (an R-predicate over a
|
||||
missing R block fails conservatively), walk-forward re-rolled in the doc's epoch-ms
|
||||
unit (`WindowRoller`; IS windows search only the survivor points; OOS winner reports
|
||||
carry `manifest.selection`), deflation nulls seeded from the doc's `seed` — the whole
|
||||
realization is a pure function of doc + stores + data (C1). Harness/data binding
|
||||
stays consumer-side behind the one-method **`MemberRunner`** seam (the shipped
|
||||
implementation is `aura_runner::DefaultMemberRunner`; the CLI binds the loaded-
|
||||
blueprint reduce convention with a unique suffix-join of raw axis names onto the
|
||||
wrapped `param_space`). The `campaign_runs.jsonl` sibling records one thin
|
||||
`CampaignRunRecord` per run — campaign/process ids, seed, and per-cell realized
|
||||
stage prefixes linking family ids, gate survivor ordinals, and sweep selections —
|
||||
run-counted per campaign id. Zero survivors truncate a cell's realized prefix and
|
||||
exit 0 (a null result is a valid research result); `emit` is honored
|
||||
(`family_table`/`selection_report` lines). The **blueprint on-ramp** (#196):
|
||||
`aura graph register` (store put keyed by content id == topology hash),
|
||||
`aura graph introspect --params` (the raw `param_space` namespace axes validate
|
||||
against), and a blueprint-file mode on `--content-id`. `std::walk_forward`'s
|
||||
machinery-true fields are `in_sample_ms`/`out_of_sample_ms`/`step_ms`/`mode`
|
||||
(`WindowRoller`'s three lengths + both `RollMode`s).
|
||||
|
||||
**Annotators are terminal.** Nothing flows out of an annotator; filtering stays the
|
||||
gate's monopoly. `std::monte_carlo` bootstraps the stage's *incoming* R-evidence
|
||||
with one semantics, input-shaped by position: after a walk_forward, one
|
||||
`r_bootstrap` over the wf family's pooled per-window OOS `net_trade_rs` in roll order
|
||||
(`StageBootstrap::PooledOos`; the conduit is the cost-netted per-trade series
|
||||
`r − cost_in_r`, equal to the gross series bit-for-bit when no cost model is bound,
|
||||
#259); after sweep/gates, one `r_bootstrap` per surviving member's fresh in-memory
|
||||
series (`StageBootstrap::PerSurvivor`, ordinals into the population family; a
|
||||
zero-trade member records the engine's defined all-zero degenerate) — seeded from the
|
||||
doc's `seed` (`net_trade_rs` is `#[serde(skip)]`, so annotators run in-executor or
|
||||
not at all). `std::generalize` executes at **campaign scope**: after all cells, per
|
||||
(strategy, window) the per-cell *nominees* (last wf window's OOS report, else the
|
||||
sweep winner; none on gate truncation) across instruments feed the shipped
|
||||
`generalization()` when ≥ 2 exist — divergent per-instrument winners are exposed via
|
||||
their params, never averaged away; a shortfall is recorded, not computed around.
|
||||
`StageRealization.bootstrap` and `CampaignRunRecord.generalizations`
|
||||
(`CampaignGeneralization` keyed strategy × window with `winners`/`missing`) are
|
||||
serde-default sparse (C14/C23). Known debt: the mc arm detects the wf family by the
|
||||
stringly `block == "std::walk_forward"` literal.
|
||||
|
||||
**Per-cell fault isolation (#272).** A member fault (no-data, bind, run, or a caught
|
||||
panic) is a recorded per-cell outcome, never a global abort: `run_cell` returns a
|
||||
fault-annotated `CellRealization` (`fault: Option<CellFault>`, closed
|
||||
`CellFaultKind`) instead of `Err`, so `execute`'s accumulate-then-append-once tail
|
||||
persists every healthy cell and the one run record. Containment granularity is the
|
||||
cell for a sweep stage (a grid hole compromises selection) and the fold for
|
||||
walk_forward (surviving folds pool; failed folds recorded as
|
||||
`StageRealization.window_faults`, the summary naming the ratio). `ExecFault::Registry`
|
||||
and doc-shape preflight faults stay global. The CLI declares holes (per-cell notes +
|
||||
a completion summary) and a run with ≥ 1 failed cell exits **3** ("completed with
|
||||
failed cells" — distinct from 0/1/2). A partially-covered window carries a
|
||||
`CellCoverage` annotation (effective bounds + interior gap months, #264). Generalize
|
||||
already treats a no-nominee cell as `missing`, so a failed cell surfaces there
|
||||
unchanged. Member panics are caught with `catch_unwind(AssertUnwindSafe)` at the
|
||||
three member-run sites and recorded as `MemberFault::Panic`; a ref-counted
|
||||
`SilencedPanic` guard (a process-global panic-hook save/no-op/restore behind a
|
||||
`static Mutex`, held only around each `catch_unwind`) suppresses the default crash
|
||||
backtrace so "recorded, campaign continues" is observably true on stderr. The guard's
|
||||
mutex serialises only the O(1) ref-count/hook-swap, never the member computation, so
|
||||
C1 disjoint-parallel execution and determinism hold; ref-counting (save on 0→1,
|
||||
restore on 1→0) keeps concurrent threads and any caller-installed hook correct.
|
||||
|
||||
**Persisted taps (#201).** Campaign presentation persists traces. The tap namespace
|
||||
is a **closed vocabulary** of the wrap convention's four sink names
|
||||
(`equity`/`exposure`/`r_equity`/`net_r_equity`; `aura_research::tap_vocabulary`,
|
||||
intrinsic `DocFault::UnknownTap` — the escalation for a new observable is a new
|
||||
vocabulary entry or an authored blueprint sink, never an open node-path namespace).
|
||||
Scope is the per-cell **nominee only**: after the pipeline settles the CLI re-runs
|
||||
each nominee once in non-reduce mode (all four channels drained, windowed to the
|
||||
nominee manifest's own ns bounds) and **asserts metrics equality** against the
|
||||
recorded nominee — the C1 drift alarm, a hard refusal on divergence (the reproduce
|
||||
precedent, enforced). Traces land in the existing `TraceStore` as
|
||||
`traces/{campaign8}-{run}/{strategy8}-{instrument}-w{n}/{tap}.json`, chartable by the
|
||||
unchanged viewer. The record carries one sparse pointer, `CampaignRunRecord.trace_name`
|
||||
(`Some("{campaign8}-{run}")` iff the doc requests taps — the claim-sentinel contract:
|
||||
`execute` claims, `append_campaign_run` composes the name via the single-sourced
|
||||
`derive_trace_name`, `execute` mirrors it onto the returned copy). `aura-campaign`
|
||||
stays trace-agnostic (the `MemberRunner` seam is unchanged; the stamp is a pure name
|
||||
derivation). Loud stderr lines cover a per-cell no-nominee skip and a per-run
|
||||
unproducible-tap skip (`net_r_equity` needs a cost leg the campaign runner wires none
|
||||
of). Known debt: `aura chart` over the campaign family ROOT (cells spanning
|
||||
instruments) is untested / semantically undefined — only per-cell read-back is pinned.
|
||||
|
||||
**Identity-ref resolution is index-first (#191).** `find_blueprint_by_identity`
|
||||
consults the persistent `blueprint_identity_index.jsonl` sidecar (identity id →
|
||||
content id; a fixed-name sibling of the runs store, appended under the #276 lock)
|
||||
first, and **verifies every hit** by loading that one blueprint under the current
|
||||
resolver and recomputing its identity id — the index is a cache, never an oracle, so
|
||||
resolution stays scan-identical under roster drift, store surgery, or index corruption
|
||||
(the one unspecified corner — which same-identity twin answers — is unchanged in kind:
|
||||
`read_dir`-order-dependent before, index-history-dependent now). Any miss or failed
|
||||
verification runs the old full-store scan as a **repair pass**, collect-then-diff-
|
||||
append: the walk's last-wins mapping is diffed against the pre-walk snapshot, so a
|
||||
converged index — twin stores included — appends nothing (the twin-convergence pin).
|
||||
Index reads never fail a lookup (missing/unreadable → empty, unparseable lines
|
||||
skipped); repair appends are best-effort; a pre-index store backfills on its first
|
||||
miss (no migration); a read-only store keeps scanning. Write paths, the engine, and
|
||||
both callers are untouched; maintenance is lazy-only — put-time indexing was rejected
|
||||
because it would need a roster-free doc-level identity function whose equivalence to
|
||||
the loaded-composite path no green test ratifies (decision log: #191).
|
||||
|
||||
**Retired verbs and the unknown-id contract.** Standalone `aura runs list` / `rank`
|
||||
are retired (#73): families (C21) subsume standalone over-time comparison. The
|
||||
unknown-id contract (ratified, Runway fieldtest 2026-06) is live law: `aura runs
|
||||
family <id>` treats an unknown-but-well-formed id as an **empty family** (prints
|
||||
nothing, exit 0) — the same treat-as-empty discipline as `Registry::load` reading a
|
||||
missing store as `Ok(empty)`. This is deliberately distinct from `aura reproduce
|
||||
<id>`, which refuses an unknown id with exit 1 (reproduction of a named-but-absent
|
||||
family is missing recorded state, not a found-nothing lookup). Tightening `runs
|
||||
family` to a non-zero `no such family <id>` exit (typo-safety) is an available future
|
||||
UX choice, not a current contract.
|
||||
|
||||
**Deferred.** The **run-diff** depth and **cross-family** ranking (families against
|
||||
each other, vs. within-family) remain deferred; whole-harness / structural-axis
|
||||
content-addressing remains deferred (C24). Known debt across the campaign stack:
|
||||
metric-roster triplication (test-caught by the #190 cross-crate guard; single-source
|
||||
removal waits on #147) and deflation-constant duplication (#199).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — determinism / bit-identity, the reproduction and
|
||||
drift-alarm correctness invariant
|
||||
- [C8](c08-node-contract.md) — sinks are the recording mechanism into the registry
|
||||
- [C9](c09-fractal-composition.md) — the registry depends on the engine, never the
|
||||
reverse
|
||||
- [C11](c11-sources-record-replay.md) — record-then-replay; producer-supplied
|
||||
windows; C11/C12 dedup
|
||||
- [C12](c12-atomic-sim-unit.md) — the four orchestration axes (sweep / MC /
|
||||
walk-forward / comparison) the family store persists
|
||||
- [C14](c14-headless-two-faces.md) — additive serde back-compat and the
|
||||
runtime-failure exit class
|
||||
- [C21](c21-world.md) — the World; families as the re-listable unit
|
||||
- [C23](c23-graph-compilation.md) — names non-load-bearing; the identity-id blanking
|
||||
- [C24](c24-blueprint-data.md) — the blueprint as serializable data; the topology
|
||||
data format the manifest content-addresses
|
||||
- [C25](c25-role-model.md) — the role model; process/campaign documents as
|
||||
closed-vocabulary artifacts
|
||||
|
||||
> History: [c18-registry.history.md](c18-registry.history.md)
|
||||
@@ -0,0 +1,118 @@
|
||||
# C19 — Bootstrap: blueprint → instance (recursive): history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c19-bootstrap.md](c19-bootstrap.md).
|
||||
|
||||
**Realization (cycle 0016 — param-set injection).** The bootstrap now binds an
|
||||
injected param-set, realizing C12's "params injected at graph build (the optimizer
|
||||
sees a generic vector of typed ranges)" and C19's "factory `params → sized node`"
|
||||
*literally*: a blueprint leaf is **value-empty** — `BlueprintNode::Leaf` holds a
|
||||
`LeafFactory { name, params, build }` recipe, not a built node — and the value lives
|
||||
only in the injected vector (no baked default), so the blueprint stays a pure
|
||||
param-generic recipe. (Renamed in cycle 0024: `BlueprintNode::Primitive` holds a
|
||||
`PrimitiveBuilder { name, schema, build }` — the recipe now carries the full
|
||||
signature, see the C8 0024 realization; `bootstrap_with_params`/`compile_with_params`
|
||||
moved onto `Composite` when `struct Blueprint` collapsed into it, see the C19 0024
|
||||
realization.) `bootstrap_with_params(Vec<Scalar>)` /
|
||||
`compile_with_params` **build each leaf through its own constructor** (the single
|
||||
sizing/validation gate) from its kind-checked slice **while lowering** (build-then-
|
||||
wire), consuming the vector slot-by-slot in the *same* depth-first walk
|
||||
`param_space()` projects — so the two share one traversal (subsuming the #34 dual-
|
||||
traversal hazard) and the value reaches the node at the slot the sweep enumerates.
|
||||
Arity is checked up front (`param_space().len()`); a wrong-kind or wrong-length
|
||||
vector is a typed `CompileError::{ParamKindMismatch, ParamArity}` (the typed-value
|
||||
check C8 deferred). The lowering/edge/source rewrite is structurally unchanged, so
|
||||
the **flat graph stays bit-identical** for a given point (C23, the correctness
|
||||
invariant). The value *domain* (e.g. `length ≥ 1`) stays the constructor's own
|
||||
`assert`; the search-range is still the run's (#32/C20). One value-empty leaf
|
||||
detail for C22: the blueprint view (pre-run, param-generic) labels a leaf by **bare
|
||||
type** (`PrimitiveBuilder::label`, was `LeafFactory::label`, → `[SMA]`) — the
|
||||
value-bearing `SMA(2)` of C8's render-
|
||||
label refinement now appears only in the *compiled* view (built nodes, `Node::label`).
|
||||
|
||||
**Realization (cycle 0017 — blueprint render = main graph + definitions).** The
|
||||
`aura graph` blueprint view (C9 graph-as-data, #13) renders the **authored
|
||||
structure** as a *program with subroutines*: a flat **main graph** wiring the
|
||||
harness with each composite shown as a **single opaque node** `[name]`, plus a
|
||||
`where:` section that defines each **distinct** composite type **once** (its
|
||||
interior with named input-entry nodes and outputs folded onto their producers as
|
||||
`name := …` bindings (render refined through cycles 0019–0022; originally
|
||||
`[in:k]`/`[out]` port markers); deduped by `name()`, collected
|
||||
recursively so nested composites are opaque nodes with their own definitions). This
|
||||
supersedes #13's original cluster-box model and is the durable split this view
|
||||
realizes: **blueprint = source** (composites as named subroutines, body once) vs.
|
||||
**compiled = inlined machine form** (C23, boundary dissolved). The substantive cause
|
||||
for retiring cluster boxes is a real renderer defect — `ascii-dag` 0.9.1's subgraph
|
||||
level-centering rounds sibling x-positions with `/2`, overlapping wide sibling
|
||||
labels (width/parity-sensitive, no config/padding dodge surviving unequal-width
|
||||
siblings); the **flat** layout is collision-free, and both views now build flat
|
||||
graphs only. The model also scales (blueprint size tracks top-level wiring, not
|
||||
inlined node count) and removes #13's nested-composite `unimplemented!` (the
|
||||
definitions pass recurses). The interactive *enter/focus* counterpart (a composite
|
||||
collapsed to a navigable node) is the playground's, parked as a separate concern
|
||||
(#37); the static CLI keeps the all-at-once definitions form (#38).
|
||||
|
||||
**Realization (cycle 0024 — the root is the fully-bound composite; the flat graph is
|
||||
a named type).** `struct Blueprint` is **deleted**: the root graph IS a `Composite`,
|
||||
and `compile_with_params`/`bootstrap_with_params`/`param_space` are its methods.
|
||||
What distinguished the root — its bound data sources — is now a property of its input
|
||||
roles: `Role` carries `source: Option<ScalarKind>` (`None` = an open interior port,
|
||||
wired by the enclosing graph; `Some(kind)` = a bound ingestion feed). A composite is
|
||||
**runnable iff every root role is bound** (C3: sources bind at ingestion only); an
|
||||
open root role is a compile-time `CompileError::UnboundRootRole`. So the "main graph"
|
||||
is no longer a separate kind — only the composite all of whose roles are
|
||||
source-bound, governed by the same conditions as any other node. Compilation now
|
||||
targets a **named type**: `compile` validates structurally **pre-build** (via
|
||||
`signature()`, no node constructed — an ill-typed wiring is caught before any build
|
||||
closure fires) and emits `FlatGraph { nodes, signatures, sources, edges }` — the
|
||||
C23 flat graph, now first-class — which `Harness::bootstrap` consumes (kinds/firing
|
||||
from the carried signatures, buffer depth from `lookbacks()`). The per-flat-node
|
||||
signature travels beside the node, so `bootstrap` reads it without a built-node
|
||||
`schema()` call (which no longer exists, see the C8 0024 realization).
|
||||
|
||||
**Realization (cycle 0026 — graph render redesign: model + WASM-Graphviz viewer,
|
||||
#51).** `aura graph` no longer renders ASCII. The render path is now two pieces:
|
||||
a read-only **model serializer** (`aura_engine::model_to_json`, iteration 1) that
|
||||
walks the root composite + every distinct composite type into a deterministic,
|
||||
hand-rolled JSON model (C14, golden-tested; the engine's last hand-rolled JSON
|
||||
writer after `RunReport::to_json` moved to serde in cycle 0033; the
|
||||
swapped-param mis-wire property moved here from the old compiled-view test), and a
|
||||
**self-contained HTML viewer** (`aura-cli::render::render_html`, iteration 2) that
|
||||
inlines that model, the ported prototype viewer JS, and a vendored Graphviz-WASM
|
||||
blob into one page emitted to stdout. Layout/SVG happen in the browser via
|
||||
WebAssembly — aura ships no layout engine and stays a serializer (C9: graph-as-data,
|
||||
no `eval`/build on the path). The viewer is a render asset (C10 — no node/strategy
|
||||
logic, no DSL); it labels every input pin from the model's now-real names (C23
|
||||
debug symbols, named in cycle 0027) and colours wires by the four scalar base
|
||||
types (C4). This **retires `ascii-dag`** and its adapter (`graph.rs`), the
|
||||
`--compiled`/`--macd` flag plumbing, and the invented `#Sf`/`:=`/`histogram →`
|
||||
notation — superseding the cycle-0017 flat-ascii model and its renderer-defect
|
||||
workaround. The DOT/SVG are Graphviz-version-dependent and not golden-tested; the
|
||||
deterministic JSON model is the asserted contract.
|
||||
|
||||
**Realization (cycle 0034 — structural-constant bind: a knob *removed* from
|
||||
param_space, #55).** `PrimitiveBuilder::bind(slot, value)` adds the **third** param
|
||||
category beside the topology factory-arg (C7/C19) and the tuning param (the
|
||||
cycle-0016 value-pin): a **structural constant**. The cycle-0016 binding *pins* a
|
||||
value in the injected vector while the knob **stays** in `param_space` (a tuning
|
||||
param the sweep varies); `bind` instead **removes** the slot from `param_space`
|
||||
entirely — the knob is gone, not fixed. The discriminator is the #55
|
||||
deform-vs-tune test: a value whose variation yields another valid point of the
|
||||
*same* strategy is a **tuning param** (stays in `param_space`); a value whose
|
||||
variation *deforms* the strategy into a different one (e.g. the `2` of an
|
||||
"SMA2-entry" bound to its two-candle construction) is a **structural constant**
|
||||
(bound out), so a sweep never enumerates deformed strategies as valid family
|
||||
members. Mechanically `bind` shrinks the builder's declared param surface
|
||||
(`schema.params`) and wraps its build closure to re-splice the constant at its
|
||||
original positional slot; the construction layer
|
||||
(`collect_params`/`lower_items`/`param_space`/`compile_with_params`) is
|
||||
**byte-unchanged** — both dock sites already key off `builder.params()`, so the
|
||||
shrink propagates for free, and chained binds reconstruct the correct positional
|
||||
vector because each layer computes its slot index relative to the param list it
|
||||
sees. C23 is unaffected: `bind` resolves the param **name** to a position at
|
||||
**authoring** time (the by-name authoring address space, the 0032 amendment) and
|
||||
the flat graph stays wired by raw index — the name never reaches it. The
|
||||
complementary question — exporting a **named frozen** strategy (all/most knobs
|
||||
bound) as a reusable blueprint *value* — is deferred (#60); `bind` ships only the
|
||||
per-knob overlay, no registry (C9/C10 intact).
|
||||
@@ -0,0 +1,118 @@
|
||||
# C19 — Bootstrap: blueprint → instance (recursive)
|
||||
|
||||
**Guarantee.** Construction is a distinct phase, recursive at every level. Each
|
||||
node type has a **factory** `params → sized concrete node` (e.g. `SMA(length)`
|
||||
sizes its ring buffer). A **blueprint** is the param-generic, input-role-generic
|
||||
graph-as-data produced by running a Rust builder (C9); it carries *free* numeric
|
||||
params (declared ranges) and *free* input roles. The **bootstrap** binds
|
||||
`(blueprint + param-set + data bindings + seed)` into a concrete, **frozen
|
||||
instance** — buffers sized, topology fixed. This is precisely the "wiring / graph
|
||||
build" that C7 ("sized at wiring", "topology frozen per sim") and C12 ("params
|
||||
injected at graph build") reference; the same machinery applies recursively up to
|
||||
the harness (C20). A sweep builds many instances from one blueprint; instances are
|
||||
disjoint (C1).
|
||||
|
||||
This binding is a **compilation**: the param-generic, named blueprint (the source)
|
||||
is lowered to a flat, type-erased **`FlatGraph`** (C23) **wired by raw index, not
|
||||
by name** (`Edge { from, to, slot, from_field }`) — composite boundaries dissolve
|
||||
entirely and field / role names are demoted to **non-load-bearing** debug symbols
|
||||
(exactly as `FieldSpec.name` already is, C8/C23). One narrow exception (#275): a
|
||||
`SourceSpec.role` — the lowered bound-`Role` name — is **load-bearing for source
|
||||
binding**, the key `Harness::run_bound` / `bind_sources` resolve a keyed source
|
||||
supply against; every other flat-graph name (edges, ports, composite boundaries)
|
||||
stays a non-load-bearing debug symbol, and the raw-index positional `run` path
|
||||
carries no role. "No recompile" means no **Rust / cdylib** rebuild (C12/C13: the
|
||||
cdylib loads once); re-deriving an instance per param-set is a cheap **graph
|
||||
re-compilation**, not a code recompile.
|
||||
|
||||
**Forbids.** Params that change topology (a topology change is a *different*
|
||||
blueprint — Fork A, C7 "frozen"); resizing buffers after bootstrap; running a sim
|
||||
against an un-bootstrapped blueprint.
|
||||
|
||||
**Why.** Separating the param-generic blueprint from the param-bound instance is
|
||||
what makes one strategy reusable across a whole sweep and lets the optimizer mutate
|
||||
"the 20" by *rebuilding* an instance (cheap; no recompile, C12) instead of
|
||||
rewriting code. Naming the build phase makes the implicit "wiring" of C7/C12
|
||||
explicit — and naming it a *compilation* makes its successor explicit: the flat
|
||||
graph is the target of behaviour-preserving optimisation (C23).
|
||||
|
||||
## Current state
|
||||
|
||||
**Value-empty recipe.** A blueprint leaf is **value-empty** — `BlueprintNode::Primitive`
|
||||
holds a `PrimitiveBuilder { name, schema, build }` recipe
|
||||
(`crates/aura-core/src/node.rs`), not a built node — and the tuning value lives only
|
||||
in the injected param vector (no baked default), so the blueprint stays a pure
|
||||
param-generic recipe. This realizes C12's "params injected at graph build" and C19's
|
||||
factory `params → sized node` literally. `compile_with_params(&[Scalar])` /
|
||||
`bootstrap_with_params(Vec<Scalar>)` on `Composite`
|
||||
(`crates/aura-engine/src/blueprint.rs`) **build each leaf through its own
|
||||
constructor** — the single sizing/validation gate — from its kind-checked slice
|
||||
*while lowering* (build-then-wire), consuming the vector slot-by-slot in the *same*
|
||||
depth-first walk `param_space()` projects, so the two share one traversal and the
|
||||
value reaches the node at the slot the sweep enumerates. Arity is checked up front
|
||||
(`param_space().len()`); a wrong-kind or wrong-length vector is a typed
|
||||
`CompileError::{ParamKindMismatch, ParamArity}`. The value *domain* (e.g.
|
||||
`length ≥ 1`) stays the constructor's own `assert`; the search-range is the run's
|
||||
(#32/C20, still pending). The lowering is structurally invariant under the injected
|
||||
point, so the **flat graph stays bit-identical** for a given point (C23, the
|
||||
correctness invariant).
|
||||
|
||||
**The root is the fully-bound composite.** There is no separate `Blueprint` type:
|
||||
the root graph **is** a `Composite`, and `compile` / `compile_with_params` /
|
||||
`bootstrap_with_params` / `param_space` are its methods. What once distinguished the
|
||||
root — its bound data sources — is a property of its input roles: `Role` carries
|
||||
`source: Option<ScalarKind>` (`crates/aura-engine/src/blueprint.rs`) — `None` = an
|
||||
open interior port wired by the enclosing graph, `Some(kind)` = a bound ingestion
|
||||
feed (C3: sources bind at ingestion only). A composite is **runnable iff every root
|
||||
role is bound**; an open root role is a compile-time `CompileError::UnboundRootRole`.
|
||||
`compile` validates structurally **pre-build** (via `signature()`, no node
|
||||
constructed — an ill-typed wiring is caught before any build closure fires) and
|
||||
emits the `FlatGraph`, which `Harness::bootstrap` consumes (kinds/firing from the
|
||||
carried signatures, buffer depth from `lookbacks()`); the per-flat-node signature
|
||||
travels beside the node, so `bootstrap` needs no built-node `schema()` call.
|
||||
|
||||
**`FlatGraph` shape.** `FlatGraph { nodes, signatures, sources, edges, taps }`
|
||||
(`crates/aura-engine/src/harness.rs`): `signatures[i]` is the static `NodeSchema` of
|
||||
`nodes[i]` gathered at lowering; `sources` are the lowered bound roles in
|
||||
role-declaration order; `taps` are the declared measurement points (C27), with
|
||||
interior-composite taps hoisted to the root list.
|
||||
|
||||
**Structural-constant bind (#55).** `PrimitiveBuilder::bind(slot, value)`
|
||||
(`crates/aura-core/src/node.rs`) adds the **third** param category beside the
|
||||
topology factory-arg (C7/C19) and the tuning param (the value-pin): a **structural
|
||||
constant**. Pinning a value in the injected vector leaves the knob **in**
|
||||
`param_space` (a tuning param the sweep varies); `bind` instead **removes** the slot
|
||||
from `param_space` entirely — the knob is gone, not fixed. The discriminator is the
|
||||
deform-vs-tune test: a value whose variation yields another valid point of the
|
||||
*same* strategy is a **tuning param** (stays in `param_space`); a value whose
|
||||
variation *deforms* the strategy into a different one (e.g. the `2` of an
|
||||
"SMA2-entry" bound to its two-candle construction) is a **structural constant**
|
||||
(bound out), so a sweep never enumerates deformed strategies as valid family
|
||||
members. Mechanically `bind` shrinks the builder's declared param surface
|
||||
(`schema.params`) and wraps its build closure to re-splice the constant at its
|
||||
original positional slot; both dock sites already key off `builder.params()`, so the
|
||||
construction layer is byte-unchanged and chained binds reconstruct the correct
|
||||
positional vector (each layer computes its slot index relative to the param list it
|
||||
sees). `bind` resolves the param **name** to a position at **authoring** time (the
|
||||
by-name authoring address space) and the flat graph stays wired by raw index — the
|
||||
name never reaches it (C23 unaffected). The complementary export of a **named
|
||||
frozen** strategy (all/most knobs bound) as a reusable blueprint *value* is deferred
|
||||
(#60); `bind` ships only the per-knob overlay, no registry (C9/C10 intact).
|
||||
|
||||
**Graph render.** `aura graph` renders no ASCII: the render path is a deterministic
|
||||
JSON **model serializer** (`aura_engine::model_to_json`,
|
||||
`crates/aura-engine/src/graph_model.rs`) plus a self-contained WASM-Graphviz **HTML
|
||||
viewer** (`aura-cli::render::render_html`) that inlines the model and lays it out in
|
||||
the browser — aura ships no layout engine (C9: graph-as-data). This render surface,
|
||||
and the debug-symbol labels it draws, belongs to C9/C22.
|
||||
|
||||
## See also
|
||||
- [C7](c07-scalar-soa.md), [C9](c09-fractal-composition.md), [C12](c12-atomic-sim-unit.md) — sized-at-wiring, graph-as-data, params-injected-at-build that this phase names.
|
||||
- [C8](c08-node-contract.md) — the `PrimitiveBuilder` / `NodeSchema` recipe and `FieldSpec.name` as the non-load-bearing precedent.
|
||||
- [C13](c13-hot-reload-frozen-deploy.md) — "no recompile" means no cdylib rebuild.
|
||||
- [C20](c20-strategy-harness.md) — the harness is the recursive top of the bootstrap.
|
||||
- [C22](c22-playground-traces.md) — the graph render / viewer surface.
|
||||
- [C23](c23-graph-compilation.md) — the flat graph as the optimisation target.
|
||||
- [C24](c24-blueprint-data.md) — the deferred named-frozen-blueprint value (#60).
|
||||
|
||||
> History: [c19-bootstrap.history.md](c19-bootstrap.history.md)
|
||||
@@ -0,0 +1,36 @@
|
||||
# C20 — Strategy ↔ harness; the harness is the root sim graph: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c20-strategy-harness.md](c20-strategy-harness.md).
|
||||
|
||||
**Realization (GER40 session-breakout blueprint milestone, 2026-06-17 — refs
|
||||
#94/#96/#97, spec 0051).** Made concrete on the first real-data strategy: a
|
||||
**hand-wired `FlatGraph` is not a shippable strategy** — it carries no
|
||||
`param_space()`, so the World families (sweep / walk_forward / compare, C21)
|
||||
cannot consume it without a hand re-author (the friction the GER40 deep-dive
|
||||
fieldtest surfaced). The **canonical shippable form is the `Composite`
|
||||
blueprint** (the authoring/source level, C9/C19); the `FlatGraph` is only its
|
||||
compiled substrate (C23). The breakout now ships as
|
||||
`ger40_breakout_blueprint(bar_period, …)` whose `param_space()` is exactly its
|
||||
tuning knobs (`{entry_bar.target, exit_bar.target}`); the **bar period is a
|
||||
construction argument** binding Resample + Session together — a structural
|
||||
matrix axis (C12: a different period is a *different strategy*, not a sweep
|
||||
point), never a `param_space` entry, so a sweep cannot desync the two clocks.
|
||||
This is the concrete instance of the structural-axis-vs-tuning-param split
|
||||
above (and `delay.lag`, a C8 structural constant, is bound out of the space).
|
||||
|
||||
**Refinement (2026-06-29 — experiment-matrix *members* are topology-data, C24).**
|
||||
"The experiment matrix is ordinary Rust control flow, not a config schema" holds for
|
||||
the **generator** — the loop / conditional that *enumerates* structural variants may
|
||||
stay Rust (C17). But each **member** it yields is a **topology-data value** (C24),
|
||||
not a hand-coded blueprint nor engine-baked source: a structural axis selects among
|
||||
*data* blueprints the World constructs, mutates, and serializes. This is what makes
|
||||
structural variation **first-class and searchable** rather than a hand-enumerated
|
||||
menu — `HarnessKind` and the per-strategy `*_sweep_family` functions in `aura-cli`
|
||||
are the pre-C24 scaffolding (topology-as-engine-source, a C16 tension), retired as
|
||||
C24 + the project-as-crate layer land. [C26 realization, 2026-07-10 (#231): the
|
||||
scaffolding's last data weld — `wrap_r`'s hard-wired `price`←close role and the
|
||||
`M1Field::Close`-only open sites — is retired; a strategy's input roles now bind
|
||||
archive columns by name (C26). `wrap_r`'s remaining R-scaffolding retirement
|
||||
stays #159.]
|
||||
@@ -0,0 +1,95 @@
|
||||
# C20 — Strategy ↔ harness; the harness is the root sim graph
|
||||
|
||||
**Guarantee.** A **strategy** is a reusable composite-node blueprint (C9): broker-,
|
||||
data-, and viz-independent, with inputs declared as named **roles** (symbol-agnostic
|
||||
where possible) and the **bias** stream (C10) as output. A **harness** (the
|
||||
experimental setup) is the **root sim graph** — sources bound to the strategy's
|
||||
input roles + the strategy + attached broker node(s) + sinks — and is itself
|
||||
produced by the bootstrap (C19). A harness *instance* is C1's disjoint unit (the
|
||||
root scope). The harness has **two kinds of parameterization**: **structural axes**
|
||||
(which strategy, which instrument(s), which broker(s), which window, which **risk
|
||||
regime**) whose variation selects *different* instances — the **experiment matrix**
|
||||
(the risk regime is the fourth axis, #210, carried as `CampaignDoc.risk`;
|
||||
kept-separate, compared-not-selected, see C10) — and **tuning params** (the
|
||||
strategy's numeric params) swept *within* a fixed structure (Fork A). The same
|
||||
strategy blueprint is reused across backtest, sweep, visual workspaces, and the
|
||||
frozen live bot — each a different harness. **Both strategy and harness/experiment
|
||||
are authored in Rust** via builder APIs (C17); the experiment matrix is ordinary
|
||||
Rust control flow (loops/conditionals), not a config schema. Ontologically, a node
|
||||
(incl. a strategy composite) is an **open** fragment — free input roles + ≤1 output
|
||||
(C8/C9) — that does not run alone; a **harness is the closed root graph**: a
|
||||
strategy with its input roles bound to sources and its output terminated in
|
||||
broker/sink nodes, under a clock. A harness is therefore **not a node** (no free
|
||||
inputs, no output; it does not fit `eval`) — it is the *closure that runs*, C1's
|
||||
disjoint unit / the root scope. Harnesses do not nest as nodes; the World (C21)
|
||||
orchestrates them as objects.
|
||||
|
||||
**Forbids.** Embedding data sources / brokers / sinks inside a strategy; a
|
||||
declarative experiment mini-DSL (logic is Rust — C17); modelling the harness as a
|
||||
node (it is the closed root scope, not an open composable node). The mini-DSL
|
||||
prohibition is scoped by the #188 role-model pass: it forbids an **open,
|
||||
logic-bearing** experiment language (the RustAst trap), **not** the closed-vocabulary
|
||||
**campaign document** (C25/C18) that carries persisted experiment intent — data
|
||||
windows × strategy ids × param axes × process ref, under the total, declarative P1
|
||||
construct tier (bounded axes, gates, ladders; no variables, no general recursion, no
|
||||
unbounded iteration — invariant 5's no-free-feedback move applied one level up;
|
||||
#188/#189). A *generator* that enumerates structural variants may still be
|
||||
plain Rust; what it yields, and what a campaign declares, is data. Genuinely new
|
||||
logic (metrics, analysis blocks) escalates to a new Rust block, never to a freetext
|
||||
hole in the artifact.
|
||||
|
||||
**Why.** Reusability needs the strategy to be a context-free blueprint that many
|
||||
harnesses embed. Modelling the harness as a root graph keeps it within the one
|
||||
Node/graph abstraction (C9) and makes "10 strategies in one environment" and "one
|
||||
strategy × N instruments" plain nested loops over the structural axes. Rust
|
||||
authoring (not config) preserves full programmatic power — conditional/adaptive
|
||||
matrices, generated axes, custom wiring — and avoids re-introducing the DSL trap
|
||||
C17 rejects.
|
||||
|
||||
## Current state
|
||||
|
||||
**The shippable form is the `Composite` blueprint.** A **hand-wired `FlatGraph` is
|
||||
not a shippable strategy** — it carries no `param_space()`, so the World families
|
||||
(sweep / walk_forward / compare, C21) cannot consume it without a hand re-author.
|
||||
The **canonical shippable form is the `Composite` blueprint** (the authoring source
|
||||
level, C9/C19); the `FlatGraph` is only its compiled substrate (C23). This is
|
||||
concrete in the GER40 session-breakout example (`ger40_breakout_blueprint`,
|
||||
`crates/aura-ingest/examples/shared/breakout_real.rs`), whose `param_space()` is
|
||||
exactly its tuning knobs and whose **bar period is a construction argument** binding
|
||||
Resample + Session together — a structural matrix axis (C12: a different period is a
|
||||
*different strategy*, not a sweep point), never a `param_space` entry, so a sweep
|
||||
cannot desync the two clocks. Structural constants (e.g. a `delay.lag`, C8) are
|
||||
bound out of the space.
|
||||
|
||||
**Experiment-matrix members are topology-data.** The "ordinary Rust control flow"
|
||||
clause holds for the **generator** — the loop / conditional that *enumerates*
|
||||
structural variants may stay Rust (C17). But each **member** it yields is a
|
||||
**topology-data value** (C24), not a hand-coded blueprint nor engine-baked source: a
|
||||
structural axis selects among *data* blueprints the World constructs, mutates, and
|
||||
serializes, which is what makes structural variation first-class and searchable
|
||||
rather than a hand-enumerated menu. Persisted experiment *intent* is the campaign
|
||||
document (C25/C18).
|
||||
|
||||
**Pre-C24 scaffolding retired.** The pre-C24 scaffolding — `HarnessKind` and the
|
||||
per-strategy `*_sweep_family` functions — is **gone**. The family builders are now
|
||||
generic and blueprint-driven: `blueprint_sweep_family`, `blueprint_walkforward_family`,
|
||||
`blueprint_mc_family` (and `blueprint_sweep_over`) in
|
||||
`crates/aura-runner/src/family.rs`. The R-evaluator scaffold `wrap_r`
|
||||
(defined in `crates/aura-runner/src/member.rs`; imported and called from
|
||||
`runner.rs`) survives; its last hard data weld — the
|
||||
hard-wired `price`←close role and the `M1Field::Close`-only open sites — is retired
|
||||
(a strategy's input roles now bind archive columns by name, C26), and `wrap_r`'s
|
||||
remaining R-scaffolding retirement is tracked at #159.
|
||||
|
||||
## See also
|
||||
- [C9](c09-fractal-composition.md) — the strategy as a composite blueprint.
|
||||
- [C10](c10-bias-r-cost.md) — the bias output and the risk-regime axis.
|
||||
- [C17](c17-authoring-surface.md) — Rust authoring and the DSL prohibition.
|
||||
- [C19](c19-bootstrap.md) — the bootstrap that produces the harness.
|
||||
- [C21](c21-world.md) — the World orchestrates harnesses as objects.
|
||||
- [C23](c23-graph-compilation.md) — the `FlatGraph` compiled substrate.
|
||||
- [C24](c24-blueprint-data.md) — experiment-matrix members as topology data.
|
||||
- [C25](c25-role-model.md), [C18](c18-registry.md) — the campaign document / run registry.
|
||||
- [C26](c26-input-binding.md) — roles bind archive columns by name.
|
||||
|
||||
> History: [c20-strategy-harness.history.md](c20-strategy-harness.history.md)
|
||||
@@ -0,0 +1,59 @@
|
||||
# C21 — The World: the meta-level is the product
|
||||
|
||||
**Guarantee.** Above the harness (C12's disjoint sim unit) sits the **World** — the
|
||||
project's program / "game" (a Rust crate, C16). Within it, **harnesses are
|
||||
dynamically constructible, first-class objects**: meta-programs (walk-forward,
|
||||
sweep, optimize, Monte-Carlo — C12's orchestration axes) construct harness
|
||||
instances at runtime via the bootstrap (C19), run them disjointly in parallel
|
||||
(C1), aggregate / compare their results, and discard the transient instances. A
|
||||
walk-forward rolls windows → bootstraps a harness per window → stitches
|
||||
out-of-sample equity + parameter stability into one meta-result. Orchestrating
|
||||
*families* of harnesses is **first-class**, not a headless afterthought; the run
|
||||
registry (C18) is the World's memory. A harness's **topology is a serializable
|
||||
data value the World owns** (C24), not Rust source — the precondition for the
|
||||
World's full ambition: it can **generate**, **mutate**, **structurally search**
|
||||
(genetic / NEAT-style graph operators), **compare**, and
|
||||
**serialize-for-reproduction** (C18) *families that vary structure*, not only
|
||||
numeric params. A param sweep over a fixed skeleton is the degenerate case;
|
||||
varying the skeleton itself needs topology-as-value. (#109, resolved 2026-06-29.)
|
||||
|
||||
**Forbids.** Relegating multi-harness orchestration (walk-forward / sweep /
|
||||
comparison) to second-class headless-only status; treating the single backtest as
|
||||
the product.
|
||||
|
||||
**Why.** What happens *within* one harness — backtest a strategy → equity — is
|
||||
commodity; every quant system covers it. aura exists for the meta-level: a
|
||||
programmable space where harnesses are dynamically built and families of them
|
||||
orchestrated and explored. The deterministic single-harness engine (C1–C20) is
|
||||
the **substrate**; the World is the **product**. The game-engine principle (C16)
|
||||
made literal: the engine owns the scene / blueprint graph as content, instantiates
|
||||
and runs it, the way aura owns harness topology as data (C24).
|
||||
|
||||
## Current state
|
||||
|
||||
The four orchestration axes (sweep, walk-forward, Monte-Carlo, generalize) are
|
||||
realized as families the World constructs, runs disjointly (C1), and aggregates.
|
||||
The dissolved orchestration verbs translate each invocation into a
|
||||
content-addressed process + campaign document, run through a single campaign
|
||||
executor (`aura-campaign`, driven by `aura-runner`), so every ad-hoc family run is
|
||||
durable, diffable, reproducible intent rather than a bespoke code path per verb.
|
||||
Harness topology is a World-owned serializable data value (C24): blueprints are
|
||||
JSON that the bootstrap (C19) compiles to a flat instance (C23); the run registry
|
||||
(C18) persists each run's manifest as the World's memory, and any family member is
|
||||
re-derivable from that manifest.
|
||||
|
||||
The **programmable analysis meta-level** is only partly realized. Families are
|
||||
*produced* and *compared* (the trace explorer's family view, C22) but are not yet
|
||||
themselves **composable** — sweep / MC / walk-forward as first-class meta-level
|
||||
operators one can nest and pipe — and the structural-search ambition (generate /
|
||||
mutate / genetic / NEAT-style graph operators over topology-as-value) is not built.
|
||||
That is the open search-policy direction; today the World constructs fixed family
|
||||
shapes rather than searching over the space of shapes.
|
||||
|
||||
## See also
|
||||
- [C24](c24-blueprint-data.md) — harness topology as a World-owned data value
|
||||
- [C16](c16-engine-project-split.md) — the engine / project split and the game-engine principle
|
||||
- [C12](c12-atomic-sim-unit.md) — the atomic sim unit and the four orchestration axes
|
||||
- [C19](c19-bootstrap.md) — blueprint → instance bootstrap
|
||||
- [C18](c18-registry.md) — the run registry, the World's memory
|
||||
- [C22](c22-playground-traces.md) — the trace explorer that plays these families
|
||||
@@ -0,0 +1,137 @@
|
||||
# C22 — The playground is a trace explorer; sinks are the recording mechanism: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c22-playground-traces.md](c22-playground-traces.md).
|
||||
|
||||
**Realization (cycle 0006).** Sinks-as-recording-mechanism is realized at the
|
||||
substrate level: a recorded trace is exactly what a recording node pushed out of
|
||||
the graph (no engine recording registry; the constructing World holds each
|
||||
recording node's destination). The engine's single `observe: usize` affordance is
|
||||
removed — `Harness::run` returns `()` and recording is a node-side concern, so one
|
||||
run records *many* streams (one per recording node) instead of exactly one row.
|
||||
Recorded streams are sparse and timestamped (a record per fired cycle, tagged
|
||||
`ctx.now()`), matching a trace of timestamped events (C18). No new contract; the
|
||||
`Harness` API change (observe removed, `run -> ()`) is recorded here.
|
||||
|
||||
**Realization (the web-from-disk visual face, #101).** The C14/C22 web seam shipped
|
||||
(amendment 3b56efb): a recording run persists each drained tap as a columnar (SoA,
|
||||
C7) `ColumnarTrace` to `runs/traces/<name>/` (`aura-registry::TraceStore`, beside the
|
||||
run registry's `runs.jsonl`), reachable via `aura run [--real <SYM> …]
|
||||
--trace <name>`; `aura chart <name> [--panels]` reads them back, aligns all taps on a
|
||||
synthetic-union timestamp spine via the post-run `join_on_ts` (C3 — not in-graph;
|
||||
C1 — pure, no live external call), and emits a static self-contained uPlot page
|
||||
(vendored like `render_html`'s Graphviz-WASM). The engine stays headless (C14):
|
||||
encoding lives in `aura-engine` (`ColumnarTrace`, struct→JSON only), file I/O in
|
||||
`aura-registry`, rendering in `aura-cli` (`render_chart_html` + `chart-viewer.js`).
|
||||
First cut only — overlay (per-series y-scale) / timestamp-aligned panels; the served
|
||||
page injected the chart *series* but no run-context header (the run manifest is
|
||||
persisted on disk in `index.json`, deliberately not surfaced in that first cut — a
|
||||
ratified scope call). The header (#102) and serve-time decimation (#108) landed in
|
||||
cut 2 (see the served-page-hardening amendment below); the families-comparison view
|
||||
landed too (#107). A local server and replay-clock controls remain open (see "Open
|
||||
architectural threads").
|
||||
|
||||
**Amendment (family-member traces, #104, d3cb5f8).** The disk-trace layout extends
|
||||
to family runs: `aura sweep|mc|walkforward --trace <name>` persists *each member* as
|
||||
a nested standalone run-dir `runs/traces/<name>/<member_key>/`, reusing
|
||||
`persist_traces`/`TraceStore` verbatim (engine untouched — persistence is a
|
||||
side-effect inside each per-member closure). The `member_key` is content-derived and
|
||||
deterministic — sweep: the *varying* axes rendered as a filesystem-portable
|
||||
directory component (`<axis-name>-<value>` tokens joined by `_`, charset
|
||||
`[A-Za-z0-9._-]`, case-less values, length-capped with an FNV fallback), MC
|
||||
`seed{N}`, walk-forward `oos{ns}` — never
|
||||
a runtime ordinal, because members run in parallel under the engine's `Fn + Sync`
|
||||
HOFs, where a counter would be schedule-dependent (C1); concurrent `TraceStore::write`
|
||||
targets disjoint member dirs, so it is lock-free. Opt-in: without `--trace`,
|
||||
stdout/registry are byte-unchanged. Because `TraceStore` resolves `<name>/<member_key>`
|
||||
as a subpath, `aura chart <name>/<member_key>` charts any single member with no
|
||||
view-side change — the write-side precondition for the family-comparison **view**
|
||||
(overlay / small-multiples across members) — now realized (#107; see the
|
||||
families-comparison amendment below).
|
||||
|
||||
**Amendment (CLI `--trace` retired, #168 / #224).** The CLI-verb `--trace` story above (single-run `run --trace`, amendment 3b56efb; per-member `sweep|mc|walkforward --trace`, #104) describes capabilities that no longer reach the code. `run` and `mc` refuse `--trace` (structurally parseable, refused at dispatch); the per-member family `--trace` was never wired to the blueprint sweep (`run_blueprint_sweep`'s `let _ = persist`) and was silently dropped when the welded `--strategy` triple retired (#159/#220). #168 makes the surface honest — `sweep`/`walkforward --trace` now refuse up front (exit 2, forward-pointer to #224). The single live `TraceStore::write` site is the campaign `presentation.persist_taps` → `runs/traces/<name>/` (`persist_campaign_traces`); `aura chart <name>` reads that back. Restoring CLI-side per-member trace-writing is deferred to #224. **[Delivered 2026-07-11 (#224 + fieldtest B1 fix): `sweep`/`walkforward --trace` now WRITE per-member traces on the real-data campaign path (the depth-2 fan-out `runs/traces/<name>/<cell>/<member>/`), refusing only on the synthetic path; `aura chart <family handle>` resolves both the depth-1 campaign layout and the depth-2 fan-out (members keyed `<cell>/<member>`, C1-sorted). The refusal story in this amendment is historical.]**
|
||||
|
||||
**Amendment (real-data family source, #106, 8e5d14b).** The family runs gain an
|
||||
**opt-in real-data source axis**: `aura sweep|walkforward --real <SYMBOL> [--from
|
||||
<ms>] [--to <ms>]` streams real M1 close bars from the data-server archive (the #71
|
||||
`M1FieldSource` seam — C6 record-then-replay: a pre-recorded archive, never a live
|
||||
call mid-sim) instead of the built-in synthetic stream, per family member. A
|
||||
CLI-side `DataSource` provider (synthetic | real) supplies each member's source,
|
||||
pip (resolved from the recorded geometry sidecar (`instrument_geometry`) — C10
|
||||
refuse-don't-guess; a symbol with no recorded geometry exits 2 before any data
|
||||
access), window, and — for walk-forward — its `WindowRoller` sizes
|
||||
(synthetic: bar-index 24/12/12; real: fixed calendar-time 90d/30d/30d in ns; CLI
|
||||
flags for these are deferred). **[Amended 2026-07-11 (#239): on the dissolved
|
||||
`walkforward --real`/`mc --real` sugar path the fixed 90/30/30 sizes are a
|
||||
*ceiling*, not a constant — `fit_wf_ms_sizes` (aura-cli) passes them through
|
||||
byte-identically whenever they fit the resolved campaign window, and scales them
|
||||
down preserving the 3:1 IS:OOS ratio (step = OOS, one roll at the minimum) when
|
||||
the window is shorter; authored process documents still validate their declared
|
||||
sizes as-is.]** The engine, ingest, and registry are untouched (C9 —
|
||||
the whole change is in `aura-cli`); the synthetic path is byte-unchanged.
|
||||
**MC is excluded** (#106 Fork A): its seed varies a *synthetic* price-walk
|
||||
realization (C12 axis 4), which is undefined over real data's single realization —
|
||||
`aura mc --real` refuses with exit 2; a real MC needs a bootstrap-resampling axis
|
||||
(its own cycle). The real walk-forward member key `oos{from.0}` widens from a small
|
||||
synthetic bar index to an epoch-ns integer (still portable, collision-free). The
|
||||
built-in demo strategy's **length grid is likewise data-kind-dependent**
|
||||
(`DataSource::strategy_lengths`): synthetic keeps the short lengths that fit the
|
||||
18/60-bar demo streams (trend SMA `{2,3}×{4,5}`, MACD `2/4/3`), real uses realistic
|
||||
M1 lengths (trend `{50,100}×{200,400}`, MACD `12/26/9`) so the cross is a genuine
|
||||
trend signal over tens of thousands of bars, not noise. This — like the roller
|
||||
sizes — is a *demo-strategy calibration* patch; the real answer is project-authored
|
||||
strategies (C9: a project crate owns its own grid), deferred to the project-env
|
||||
work.
|
||||
|
||||
**Amendment (families-comparison view, #107, 4c64feb).** The family-comparison
|
||||
**view** the #104 amendment left open is now realized. `aura chart <name>`
|
||||
classifies the name on disk (`TraceStore::name_kind`: a top-level `index.json` is a
|
||||
single run; else member subdirs with `index.json` are a family; else not-found) and,
|
||||
for a family, overlays **one tap** (default `equity`, `--tap` to pick) of **every
|
||||
member** in one self-contained uPlot page: `build_comparison_chart_data` emits one
|
||||
labelled series per member, all on a **single shared y-scale** — members measure one
|
||||
identical quantity, so a shared scale is what makes them comparable, unlike the
|
||||
single-run overlay of *different* taps (per-series scales). The series align on the
|
||||
same union-ts spine via `join_on_ts`, so **one mechanism yields three correct
|
||||
readings**: sweep/MC members share the data window (a true overlay), walk-forward
|
||||
members are disjoint OOS windows (null-complementary → the stitched curve). The view
|
||||
consumes a **set of runs** (`&[FamilyMember]` from `TraceStore::read_family`, sorted
|
||||
by key for determinism, C1); a family is its *first producer* — the deliberate first
|
||||
step toward the programmable analysis meta-level (C21) **without** rebuilding the
|
||||
orchestration axes (that is its own later cycle). Name resolution is made a **total
|
||||
function** by the write-guard `TraceStore::ensure_name_free`, called once per tracing
|
||||
command (`run`/`run <bp.json>`/`run --real` → Run; `sweep`/`mc`/`walkforward` → Family)
|
||||
to refuse cross-kind name reuse — so the one ambiguous on-disk state (a name used by
|
||||
both a run and a family) is unreachable. Every error path exits 2, never panics
|
||||
(C18/C10 refuse-don't-guess). Engine untouched (C9/C14): the whole change is
|
||||
`aura-registry` (the resolver + classifier + guard) + `aura-cli` (the builder +
|
||||
`emit_chart` name-kind branch + `--tap`); the single-run page is byte-unchanged when
|
||||
no `--tap` is given. **Still deferred:** multi-column tap selection
|
||||
(`build_comparison_chart_data` projects column 0 — the comparison taps in scope are
|
||||
single-column f64; #47); and the orchestration-composability rebuild (sweep/MC/WFO as
|
||||
composable meta-level tools).
|
||||
|
||||
**Amendment (served-page hardening — cut 2, #102 + #108, 476342d).** The two deferred
|
||||
follow-ons the #107 amendment named are realized, both confined to `aura-cli` (engine
|
||||
+ registry untouched, C14). **Run-context header (#102):** a `ChartMeta`
|
||||
(kind/name/commit/window/broker/seed/taps, + member count for a family, + bound params
|
||||
for a single run) is built from the `RunManifest` in
|
||||
`build_chart_data`/`build_comparison_chart_data`, injected into
|
||||
`window.AURA_TRACES.meta`, and rendered by a pure `buildHeader` in `chart-viewer.js`
|
||||
(headless `.mjs`-guarded, like `buildCharts`). The family window is the **span** across
|
||||
members `(min from, max to)` — the only reading that labels a disjoint walk-forward
|
||||
family's true OOS coverage (it collapses to the shared window for sweep/MC).
|
||||
**Decimation (#108):** a pure `decimate(ChartData, buckets) -> ChartData` min-max
|
||||
transform thins the served page to ≤ ~2·`CHART_DECIMATE_BUCKETS` spine slots
|
||||
(per-bucket min+max, nulls preserved), applied in `emit_chart` on both paths — a
|
||||
multi-year M1 family now renders a few-thousand-point page instead of 100s of MB; full
|
||||
recorded data stays on disk (view-only), and the walk-forward null-fill page collapses
|
||||
for free. Deterministic (C1): pure bucketing over the sorted-deduped spine, no-op under
|
||||
budget. **Refinement (#111, 2957561):** the per-bucket reduction is **tap-aware** — an
|
||||
*envelope* series (equity) keeps min/max, but a *bounded level* series (the C10
|
||||
bias level — pre-reframe: exposure — ∈[-1,+1]) reduces by per-bucket **mean** (a `ReduceKind` on each `Series`,
|
||||
set by `reduce_for_tap`). Without it min/max made every multi-year exposure a solid
|
||||
-1..+1 band (each bucket straddles many sign flips), reading as per-point oscillation;
|
||||
the mean shows the net/duty-cycle level. Deferred refinements: rendering the min/max
|
||||
envelope honestly as range bars / OHLC rather than a polyline (#112); a `--width`
|
||||
budget flag and true intra-bucket min/max ordering for the non-default continuous
|
||||
x-mode (#110).
|
||||
@@ -0,0 +1,144 @@
|
||||
# C22 — The playground is a trace explorer; sinks are the recording mechanism
|
||||
|
||||
**Guarantee.** The World is a *program* (C21): nothing is displayable until it
|
||||
runs, and its harnesses are transient machinery (built, run, discarded — C19). The
|
||||
only durable, displayable substance is the **recorded trace**, captured by
|
||||
**sinks** — pure consumer nodes (C8) that persist a stream (equity, a bias /
|
||||
exposure level, a node's output) into the run registry (C18). **Displayable =
|
||||
exactly what a sink recorded**; with no sink, only the input params + summary
|
||||
metrics remain. The **playground** is therefore an **execution viewer / trace
|
||||
explorer**, and it plays *any* harness (the harness *player*, not a harness, never
|
||||
bound to a default one): before a run it shows the program *structure*
|
||||
(graph-as-data, C9) + param knobs; during a run, live sink streams; after a run,
|
||||
recorded traces + metrics from the registry — including meta-views (stitched
|
||||
walk-forward, sweep surfaces, multi-strategy / instrument comparison).
|
||||
Observability is **explicit and selective** — you instrument what you want to see;
|
||||
the choice of sinks is part of the experiment. The engine ships sample blueprints
|
||||
and a scaffolded starter so a newcomer has something to run at once; a fresh
|
||||
project is empty until built, run, and instrumented. The shipped visual face is
|
||||
**web-from-disk** — disk-persisted traces rendered as self-contained HTML chart
|
||||
pages (not egui).
|
||||
|
||||
**Forbids.** A scene-editor model that assumes a persistent, populated world;
|
||||
constructing or wiring topology in the UI (topology is Rust + hot-reload, C9/C17 —
|
||||
the UI reflects the live graph-as-data and tunes runtime params via sliders, C12);
|
||||
retaining un-sinked data; binding the playground to a single / default harness. The
|
||||
"never a scene editor" forbid is precisely **no persistent scene-at-rest** (a World
|
||||
is a program) — it does *not* forbid a future blueprint *editor* over the
|
||||
World-owned topology data (C24); that data format would be the substrate such an
|
||||
editor round-trips, not a new mechanism.
|
||||
|
||||
**Why.** A program has no scene-at-rest to inspect; what persists is what it
|
||||
records. Making sinks the one recording-and-observability mechanism keeps "what can
|
||||
I see?" answerable by "what did I instrument?", and keeps the engine UI-agnostic
|
||||
(C14). Live param tuning (runtime values, no topology change — C12 / C19) gives the
|
||||
interactive feel without a wiring DSL. The visual face is **orthogonal and
|
||||
optional**: the blueprint data format (C24) is required by C21 (generation /
|
||||
structural search) and C18 (reproduction) *at the CLI level, autonomously*,
|
||||
independent of any visual surface — so the playground's form (web or egui, viewer
|
||||
or editor, or not built at all) is downstream, and topology authoring does not live
|
||||
in it. Generalized by the role model (C25): every artifact's canonical, complete
|
||||
form is **text**, every operation is executable headless, and any visual surface is
|
||||
a **stateless projection** that reads/writes that canonical form and adds no
|
||||
semantics; read-only projections (this trace explorer) rank far above write
|
||||
editors, because the Blockly-litmus discipline (C25) already keeps each vocabulary
|
||||
palette-generatable without one.
|
||||
|
||||
## Current state
|
||||
|
||||
**Sinks at the substrate.** A recorded trace is exactly what a recording (tap) node
|
||||
pushed out of the graph — there is no engine recording registry; the constructing
|
||||
World holds each recording node's destination. `Harness::run` returns `()` and
|
||||
recording is a node-side concern, so one run records *many* streams (one per tap)
|
||||
rather than exactly one row. Recorded streams are sparse and timestamped (a record
|
||||
per fired cycle, tagged `ctx.now()`), matching a trace of timestamped events (C18).
|
||||
|
||||
**Encoding / storage / rendering split (C14).** A drained tap is encoded as a
|
||||
columnar (SoA, C7) `ColumnarTrace` — struct→JSON only — in `aura-engine`
|
||||
(`report.rs`); file I/O is `aura-registry::TraceStore`, persisting each run under
|
||||
`runs/traces/<name>/` beside the run registry's `runs.jsonl`; rendering is
|
||||
`aura-cli` (`render_chart_html` + the vendored `chart-viewer.js`). The engine stays
|
||||
headless.
|
||||
|
||||
**Single run.** `aura run <blueprint.json>` persists every tap the blueprint
|
||||
declares to the trace store under the run's own name, on both shapes: a
|
||||
`bias`-output strategy (`aura-runner::member::run_signal_r`, R-wrapped) and a bare
|
||||
measurement blueprint with ≥1 declared tap but no `bias`
|
||||
(`aura-runner::measure::run_measurement`). A tap-free run writes nothing to `runs/`
|
||||
and its stdout stays byte-identical. The CLI `--trace <NAME>` flag is **retired** on
|
||||
`run` (and on `mc`): it parses but is refused at dispatch (exit 2) — naming a trace
|
||||
is the family / campaign path's job, not a per-run flag.
|
||||
|
||||
**Families.** `aura sweep|walkforward --real <SYM> … --trace <NAME>` persists *each
|
||||
member* via the campaign path: the dissolved verb is translated into a
|
||||
content-addressed process + campaign document whose `presentation.persist_taps`
|
||||
requests the tap vocabulary, run through the one campaign executor;
|
||||
`aura-runner::runner::persist_campaign_traces` writes each member under a depth-2
|
||||
fan-out `runs/traces/<name>/<cell>/<member>/`. Every written member is
|
||||
independently re-run once in non-reduce trace mode over its own recorded window,
|
||||
and its re-derived metrics are asserted equal to the recorded member's — a **C1
|
||||
drift alarm** that refuses (exit 1) rather than persist a silently-wrong trace. The
|
||||
**synthetic** sweep / walk-forward path refuses `--trace` (exit 2);
|
||||
**Monte-Carlo is excluded from trace persistence** (`mc --real` refuses
|
||||
`--name`/`--trace`, exit 2 — the real-data R-bootstrap campaign itself runs, but
|
||||
records no per-member family traces; the synthetic `--seeds` family's realization
|
||||
argument, C12, does not carry over to one real series).
|
||||
`TraceStore::ensure_name_free` makes name resolution a total function, refusing
|
||||
cross-kind reuse of one name by both a run and a family.
|
||||
|
||||
**Viewer.** `aura chart <name> [--tap <t>] [--panels]` classifies the name on disk
|
||||
(`TraceStore::name_kind`: top-level `index.json` → a single Run; member subdirs → a
|
||||
Family; else NotFound, which then tries campaign-name resolution), reads the traces
|
||||
back, and aligns all taps on a synthetic union-timestamp spine via the post-run
|
||||
`join_on_ts` (C3 — a post-run join, not an in-graph merge; C1 — pure, no live
|
||||
external call), emitting a self-contained uPlot page (vendored, like the graph
|
||||
render). A **single run** overlays all its taps, each on its own y-scale (or the
|
||||
one `--tap` selects); a **family** overlays **one tap** (default `equity`, `--tap`
|
||||
to pick) of **every member** on a **single shared** y-scale — members measure one
|
||||
identical quantity, so a shared scale is what makes them comparable. One mechanism
|
||||
yields three correct readings: sweep / MC members share the data window (a true
|
||||
overlay), walk-forward members are disjoint OOS windows (null-complementary → the
|
||||
stitched curve). A **run-context header** (`ChartMeta` → `window.AURA_TRACES.meta` →
|
||||
the pure `buildHeader`) renders chips for kind / name / members / tap / broker /
|
||||
window / seed / commit / bound-params; a family's window is the **span**
|
||||
`(min from, max to)` across members. Serve-time **min-max decimation** (`decimate`,
|
||||
`CHART_DECIMATE_BUCKETS = 2000`) thins the page to a few-thousand spine slots (full
|
||||
recorded data stays on disk, view-only), and the reduction is **tap-aware**
|
||||
(`reduce_for_tap` / `ReduceKind`): an unbounded cumulative curve (`equity`) keeps
|
||||
the per-bucket min/max **envelope** so drawdowns survive, while a bounded level
|
||||
series (`exposure` / the C10 bias level, ∈ [-1,+1]) reduces by per-bucket **mean**
|
||||
so its net / duty-cycle level shows instead of collapsing to a solid ±1 band.
|
||||
`--panels` renders stacked panels instead of the overlay.
|
||||
|
||||
**Newcomer.** `aura new <name>` scaffolds a **data-only** project — a paths-only
|
||||
`Aura.toml`, a runnable starter blueprint `blueprints/signal.json` over the std
|
||||
vocabulary (an SMA-cross → `Bias` strategy), `.gitignore`, and a project `CLAUDE.md`
|
||||
— no crate, no build step (`aura nodes new` attaches a native node crate, C16/C17).
|
||||
The engine also ships example blueprints under
|
||||
`crates/aura-cli/examples/r_{sma,breakout,channel,meanrev}.json`. Honest gap against
|
||||
the Guarantee's "populated trace immediately": these shipped blueprints are
|
||||
*strategies* (a `bias` output, no declared taps), so `aura run` prints summary
|
||||
R-metrics to stdout but writes no on-disk trace — a **chartable** trace today comes
|
||||
from a blueprint that declares taps (single run) or from a `--real … --trace` family
|
||||
campaign.
|
||||
|
||||
**Deferred.** Live sink streams *during* a run are not built — taps are
|
||||
buffer-then-drain (collected, then written after the run), and there is no local
|
||||
replay server or replay clock; the shipped face is post-run static HTML. Reworking
|
||||
tap draining into live subscribers (stream-to-disk / fold / live consumer) is #283.
|
||||
Embeddable chart / graph fragments for hosting in external pages are unbuilt (#150).
|
||||
The programmable analysis meta-level — families as composable orchestration
|
||||
operators — is C21's open direction, not this contract's.
|
||||
|
||||
> History: [c22-playground-traces.history.md](c22-playground-traces.history.md)
|
||||
|
||||
## See also
|
||||
- [C27](c27-declared-taps.md) — declared taps: the named sinks a run persists
|
||||
- [C8](c08-node-contract.md) — sinks are pure consumer nodes
|
||||
- [C21](c21-world.md) — the World is the program the playground plays
|
||||
- [C24](c24-blueprint-data.md) — the topology data a future editor would round-trip
|
||||
- [C25](c25-role-model.md) — visual surfaces as stateless projections
|
||||
- [C14](c14-headless-two-faces.md) — headless core, two faces (the encode / store / render split)
|
||||
- [C18](c18-registry.md) — the run registry the trace store sits beside
|
||||
- [C9](c09-fractal-composition.md) — topology grown in Rust; the structure the viewer reflects
|
||||
- [C10](c10-bias-r-cost.md) — the bounded bias / exposure level the viewer reduces by mean
|
||||
@@ -0,0 +1,77 @@
|
||||
# C23 — Graph compilation and behaviour-preserving optimisation
|
||||
|
||||
**Guarantee.** The bootstrap (C19) is a **compilation**: it lowers a param-generic,
|
||||
named **blueprint** (the authoring source — nodes, composites, strategy, harness;
|
||||
C8/C9/C20) into a flat, type-erased **`FlatGraph`** — the frozen runnable instance
|
||||
(C7) — **wired by raw index, not by name** (`Edge { from, to, slot, from_field }`).
|
||||
Composites are **inlined** at this step (C9): the composite *boundary* dissolves
|
||||
entirely (there is no composite in the flat graph). The blueprint's field /
|
||||
input-role *names* are **non-load-bearing** — the wiring resolves by index, and names
|
||||
survive at most as **informative debug symbols** (exactly as `FieldSpec.name` already
|
||||
is, C8), kept for tracing / rendering (C9 graph-as-data) but carrying no run
|
||||
semantics. The flat graph is then the target of **behaviour-preserving optimisation**
|
||||
— any transform that leaves every observable sink trace bit-identical (C1 is the
|
||||
correctness invariant) — on two levels:
|
||||
|
||||
- **intra-graph** (within one graph): **common-subexpression elimination** — two
|
||||
identical nodes (same type, same bound params, same input source) merge to one with
|
||||
output fan-out, so fractal composition (C9) costs no redundant compute — and
|
||||
**dead-node elimination** — a node on no path to a sink is dropped. Pure-consumer
|
||||
sinks (C8, no output) are never CSE candidates, so their out-of-graph side effects
|
||||
are preserved by construction.
|
||||
- **across the sweep family** (over the family of flat graphs one blueprint yields
|
||||
under a param sweep, C12): the sub-graph whose nodes carry **no swept param** and
|
||||
whose inputs are **all sweep-invariant** (transitively from the sources — same data
|
||||
window) is **loop-invariant** w.r.t. the sweep and is computed **once**; its output
|
||||
is materialised as a recorded stream (C11) shared read-only across the sweep
|
||||
instances (`Arc<[T]>`, C12). Only the parameter-dependent suffix re-runs per sweep
|
||||
point — loop-invariant code motion over the sweep loop.
|
||||
|
||||
The compilation also carries one structural gate that **is** load-bearing:
|
||||
**param-namespace injectivity**. `check_param_namespace_injective` runs over the
|
||||
`param_space()` name projection before name resolution, reads the **boundary** name
|
||||
projection — the by-name authoring address space — and rejects a duplicated path
|
||||
(`CompileError::DuplicateParamPath`). This makes the authoring address space
|
||||
injective **without** making names load-bearing in the flat graph: node names qualify
|
||||
the param path at construction and are dropped at lowering (the flat graph stays
|
||||
wired by raw index).
|
||||
|
||||
**Forbids.** Any "optimisation" that changes an observable trace (it breaks C1);
|
||||
optimising over a representation that is not graph-as-data — an opaque trait-object
|
||||
interior (the rejected nested-composite reading of C9) cannot be analysed or
|
||||
rewritten across its boundary, so it forecloses both levels.
|
||||
|
||||
**Why.** Determinism (C1) is not merely an audit property; it is the **licence** for
|
||||
these rewrites — a behaviour-preserving transform is only meaningful because "same
|
||||
input → bit-identical run" makes "same result" decidable, and a sweep-invariant
|
||||
sub-graph is reproducible enough to compute once and share. The flat graph
|
||||
representation (C19) is the necessary condition: only a flat, inspectable
|
||||
graph-as-data — with each node's declared param-ranges (C8) — lets the compiler
|
||||
identify identical sub-expressions, dead nodes, and the sweep-invariant frontier.
|
||||
This is precisely why a composite is an inlining (C9) and not a runtime sub-engine:
|
||||
the flat graph is what makes the whole graph optimisable.
|
||||
|
||||
## Current state
|
||||
|
||||
The flat-graph **representation** — blueprint → inline / compile → flat instance — is
|
||||
the load-bearing substrate, built **first**, and is realized: `Composite::compile`
|
||||
emits the first-class `FlatGraph` (C19). The per-node param declarations the
|
||||
sweep-level pass presupposes carry `name` + `kind` (C8); the swept *range* is still
|
||||
pending (#32/C20), as is the sweep orchestration itself (C12/C21).
|
||||
|
||||
The **optimisation passes** — intra-graph CSE/DCE and sweep-invariant hoisting — are
|
||||
**deferred, behaviour-preserving follow-on work**; no pass exists yet
|
||||
(`Composite::compile` in `crates/aura-engine/src/blueprint.rs` adds none). Each is
|
||||
gated by a "flat graph with pass ≡ flat graph without pass, bit-identical run" test
|
||||
(C1). The one compilation-time structural check that is **live** today is the
|
||||
param-namespace injectivity gate above — `check_param_namespace_injective`, defined in
|
||||
`crates/aura-engine/src/blueprint.rs` and applied at that file's own compile entry
|
||||
points (`compile`, `compile_with_params`, and the `bind` paths) as well as from
|
||||
`construction.rs`'s `finish`, raising `CompileError::DuplicateParamPath`.
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — the bit-identity correctness invariant that licences every rewrite.
|
||||
- [C8](c08-node-contract.md) — `FieldSpec.name` as the non-load-bearing precedent and the per-node param declarations.
|
||||
- [C9](c09-fractal-composition.md) — inlining vs runtime sub-engine.
|
||||
- [C11](c11-sources-record-replay.md), [C12](c12-atomic-sim-unit.md) — recorded-stream sharing and the sweep family.
|
||||
- [C19](c19-bootstrap.md) — the flat-graph representation this optimises.
|
||||
@@ -0,0 +1,160 @@
|
||||
# C24 — The blueprint is a serializable, World-owned data value (the topology data format): history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c24-blueprint-data.md](c24-blueprint-data.md).
|
||||
|
||||
**Status (2026-06-29; first cut shipped — cycle 0087 / #155, `d5602ec`;
|
||||
construction service shipped — cycle 0088 / #157).** The
|
||||
**principle** is settled (this contract, ratified in an in-context design discussion
|
||||
— the #109 resolution). The **first cut now ships**: a `Composite` blueprint
|
||||
serializes to a **canonical, versioned** data value (`format_version` envelope,
|
||||
omit-defaults JSON) and **loads back** (data → blueprint → `FlatGraph`) via
|
||||
`blueprint_to_json` / `blueprint_from_json` (`aura-engine::blueprint_serde`),
|
||||
referencing nodes by **compiled-in type identity** through an **injected resolver**
|
||||
whose concrete closed `match` over the `aura-std` vocabulary lives outside the engine
|
||||
(`aura-std::std_vocabulary`) — the engine stays domain-free, no node registry
|
||||
(invariant 9). Acceptance met: a serialized blueprint runs **bit-identical** (C1) to
|
||||
its Rust-built twin. `model_to_json` (C9) remains the render half; this closes the
|
||||
loop for the round-trippable vocabulary. **Cycle 0088 (#157) adds the
|
||||
introspectable construction service**: a declarative, replayable by-identifier
|
||||
op-script (`aura graph build` / `introspect` over a JSON op-list, the engine
|
||||
`GraphSession` / `replay`) builds a runnable blueprint through validated ops — the
|
||||
engine's construction gates split into *eager* per-op checks (name resolution, edge
|
||||
kind-match, the double-wire arm, param bind, and acyclicity — a `connect` that would close a
|
||||
cycle is rejected eagerly, #161) and *holistic* finalize checks (wiring
|
||||
totality, param-namespace injectivity, root-role boundness), **both cadences calling
|
||||
the same extracted predicates** (`edge_kind_check`, the shared resolution helpers,
|
||||
`check_root_roles_bound` — no second validator) — plus build-free introspection over
|
||||
the closed vocabulary (types, a node's ports/kinds, a partial document's unwired
|
||||
slots). It **emits** the #155 blueprint; acceptance met: a graph built purely through
|
||||
the ops compiles identical (C1) to its Rust-built twin, an invalid op is rejected at
|
||||
the op naming the cause, introspection answers without a build. The engine `Op` stays
|
||||
serde-free (the wire DTO is CLI-side); the vocabulary's enumerable companion
|
||||
(`std_vocabulary_types`) lives in `aura-std` (no registry, invariant 9). **Cycle
|
||||
0089 follow-up (#161 / #162, after the cycle-0088 fieldtest).** The eager acyclicity
|
||||
gate above (`GraphSession::closes_cycle`, a reachability check at the closing
|
||||
`connect`) closed a fieldtest-found hole where `graph build` accepted a non-DAG
|
||||
op-list (invariant 5 / C9). **Lockstep:** it is a *second* home of invariant-5
|
||||
alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two
|
||||
must co-evolve when the explicit delay/register node — invariant-5's sole legal
|
||||
feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be
|
||||
rejected at construction. The holistic *finalize* faults now also read
|
||||
**by-identifier** (#162): `finish()` translates the index-carrying `CompileError`s
|
||||
(`UnconnectedPort` / `RoleKindMismatch` / `UnboundRootRole`) into by-identifier
|
||||
`OpError` variants — still *calling* the unchanged holistic gates (the
|
||||
no-second-validator lockstep preserved), only translating their result.
|
||||
**Cycle 0090 (#156) codifies the forward-compat two-tier discipline.** Tier-1
|
||||
(additive-optional) is serde-default silent-ignore with no `format_version` bump
|
||||
(a new optional field defaults to prior behaviour, C1) — now proven by
|
||||
`unknown_optional_field_is_tolerated_byte_identically` (an unknown optional key
|
||||
loads byte-identically and runs bit-identically). Tier-2 (must-understand: a new
|
||||
node type, edge semantics, or structural-axis kind) bumps `format_version` so an
|
||||
old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`,
|
||||
already green). The per-section required-flag scheme is deferred (no current
|
||||
Tier-2 section to validate it; recorded on #156).
|
||||
Pre-ship dormancy (#61, 2026-07-10): until the first external ship there are
|
||||
no out-of-repo readers — reader and writer change atomically in one commit —
|
||||
so the Tier-2 bump discipline is dormant and structurally-semantic additions
|
||||
(the `gangs` section) land as additive-optional fields of v1; the first ship
|
||||
consciously freezes v1, gangs included, and activates the bump discipline.
|
||||
|
||||
**Milestone delivered — 2026-06-30, cycle 0090 — the serializable format + loader + construction service.** A green end-to-end milestone fieldtest (`fieldtests/milestone-topology-as-data/`) proves the full author → serialize → load → construct → introspect → reproduce story from the public surface alone, 0 behavioural bugs; the round-trippable format (#155), the construction service (#157), and the forward-compat two-tier discipline (#156) all shipped. Polish filed forward: op-script grammar docs + a stale example (#163), the canonical trailing-newline / `Composite` value ergonomics (#164), CLI discoverability of `build`/`introspect` (#159).
|
||||
|
||||
**Realization (2026-06-30, cycle 0092 — runs are built FROM blueprint-data, #165).**
|
||||
`aura run <blueprint.json>` now loads a serialized **signal** blueprint
|
||||
(`blueprint_from_json` → the closed `std_vocabulary`; an unknown type fails clean as
|
||||
`UnknownNodeType`, the data-plane face of invariant 9), wraps it in the r-sma run
|
||||
scaffolding (sinks / broker / data supplied **at run**, not serialized — C24's deferred
|
||||
set), runs it, and emits a `RunReport` **bit-identical** (C1) to its Rust-built twin —
|
||||
proven by `loaded_signal_runs_bit_identical_to_rust_built`. The `RunManifest` now carries
|
||||
a **`topology_hash`**: SHA256 of the canonical (#164) `blueprint_to_json`, the #158
|
||||
reproducibility anchor, a Tier-1 optional field (#156). The hash + helper live
|
||||
research-side (`aura-cli` + `sha2`), off the frozen engine (invariant 8). This is the
|
||||
**keystone of the World/C21 milestone**: topology-as-data is now *runnable*, not only
|
||||
serializable. The harness wrap was made shareable — `r_sma_graph()` =
|
||||
`wrap_r(sma_signal(...))`, so the Rust path and the data path are the same seam
|
||||
over the same signal (a behaviour-preserving C19/C23 restructure). Scope note: the hash
|
||||
covers the **signal** only (the fixed scaffolding is identified by `commit`); a content-id
|
||||
distinguishing harness-structural variants is a #158/#166 concern once scaffolding varies.
|
||||
|
||||
**Realization (2026-07-01, cycle 0093 — the World orchestrates FAMILIES from blueprint-data,
|
||||
#166).** `aura sweep <blueprint.json> --axis <name>=<csv>` loads an **open** signal blueprint,
|
||||
grids the by-name axes against its `param_space()`, builds each member through cycle-1's
|
||||
`wrap_r` seam, and aggregates to a `FamilyKind::Sweep` family — byte-identical in shape
|
||||
to the hard-wired sweep (`append_family` / `sweep_member_reports` reused verbatim). This is the
|
||||
C21 step beyond a single run: the World now constructs and orchestrates **families** of
|
||||
harnesses from topology-data, not just one. Each member manifest carries the **shared**
|
||||
`topology_hash` (one signal topology, only params vary; `member_key` distinguishes members) —
|
||||
reproducible per C18. Two design facts: a sweep needs an **open** blueprint (a fully-bound one
|
||||
has an empty `param_space`, distinct from cycle-1's bound run fixture), and the signal is
|
||||
**re-loaded per member** from its serialized doc (the `Composite` is `!Clone` — its
|
||||
`PrimitiveBuilder`s hold `Box<dyn Fn>` build closures, #164 — and `bootstrap_with_cells`
|
||||
consumes the graph). **Monte-Carlo over a loaded blueprint shipped (cycle 0095, #170).** `aura mc
|
||||
<blueprint.json> --seeds N` runs a **closed** blueprint across N seeds — each seed a distinct
|
||||
synthetic price walk (`synthetic_walk_sources`, the `mc_family` `SyntheticSpec` pattern), the
|
||||
draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and aggregates to a
|
||||
`FamilyKind::MonteCarlo` family with one stored blueprint per family (the C18 hook), so it
|
||||
`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint
|
||||
(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered
|
||||
exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the
|
||||
pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward
|
||||
<blueprint.json> --axis <name>=<csv> [--select argmax|plateau:mean|plateau:worst]` re-optimizes
|
||||
the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each
|
||||
24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric +
|
||||
`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a
|
||||
`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical
|
||||
(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One
|
||||
substitution deep from the hard-wired r-sma WF arm (the loaded blueprint via
|
||||
`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure
|
||||
exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form
|
||||
honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the
|
||||
meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP
|
||||
is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the
|
||||
`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint
|
||||
warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle
|
||||
0096, #169):** `aura sweep <blueprint.json> --list-axes` lists a loaded blueprint's open
|
||||
sweepable knobs (one `<name>:<kind>` per line, `param_space()` order), then exits — the names it
|
||||
prints are exactly what `--axis` binds. Every listed name is **mandatory** on a `sweep` /
|
||||
`walkforward`: the blueprint must be fully bound before it runs, so a subset grid is refused with
|
||||
the missing knob named (`BindError::MissingKnob`) and there is no default — pin a knob you do not
|
||||
want to vary with a single-value axis (`--axis <name>=<one-value>`). A single `blueprint_axis_probe` helper now single-sources
|
||||
the wrapped probe (`wrap_r(loaded_signal).param_space()`) for the sweep terminal, the MC
|
||||
closed-check, AND the listing (three former inline copies → one), so **listed == swept by
|
||||
construction** (and stays so across #159's harness retirement — the listing tracks whatever the
|
||||
sweep actually resolves, never a second source of truth). The names are prefixed by the current
|
||||
r-sma wrapping (`sma_signal.fast.length`, the nested-composite prefix), not the raw
|
||||
`param_space` — which is why the discovery lives on the sweep verb (it owns the wrapping), not
|
||||
`graph introspect`. CLI `--trace` is refused on all verbs (#168 for sweep/walkforward; run/mc already refused); the live trace-writer is the campaign `presentation.persist_taps` (`persist_campaign_traces`), and restoring per-member CLI traces is tracked by #224. **[Delivered 2026-07-11 (#224): `sweep`/`walkforward --trace` write per-member traces on the real-data campaign path (depth-2 fan-out, chartable by the printed family handle); only the synthetic path still refuses.]**
|
||||
|
||||
**Content-addressed reproduction shipped (cycle 0094, #158, C18)**: `topology_hash`
|
||||
landed cycle 0092; re-deriving a member's FlatGraph from a stored manifest + the
|
||||
content-addressed blueprint store (`aura reproduce`) shipped cycle 0094 (see C18
|
||||
Realization). What **remains** is a content-id that covers **structural-axis / whole-harness
|
||||
variants** (the scaffolding is not yet blueprint-data); the debug-name-in-id question is
|
||||
settled by cycle 0104's **identity id** (#171: `blueprint_identity_json` + `graph
|
||||
introspect --identity-id`, an additive sibling — the byte-exact content id keeps the
|
||||
store/reproduce roles; introspection-only until a dedup consumer exists). Still deferred: retiring the pre-C24
|
||||
hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `*_sweep_family`) once
|
||||
the project-as-crate layer lands (#159, paired with #157's data-authoring surface). **Out of the first cut's round-trippable set**
|
||||
(deliberate; fails clean as `UnknownNodeType`, never a silent wrong graph): recording
|
||||
sinks (capture an `mpsc::Sender` — runtime identity, not param-generic data, C19) and
|
||||
construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / `Session` —
|
||||
structural-axis args, a C20 concern), additively addable later (#156). The
|
||||
**project-as-crate load boundary landed in cycle 0102** (`Aura.toml` discovery +
|
||||
`cdylib` loading + merged project ∪ std vocabulary — see the C13 realization
|
||||
note; the `aura new` scaffolder followed in cycle 0103 — one command emits a
|
||||
buildable project whose blueprint runs through the merged vocabulary); of the
|
||||
two layers this paragraph used to name as sequencing-coupled, what remains
|
||||
open is the **composable-orchestration** thread (#109): topology-as-data is
|
||||
the substrate it stands on. **Canonical project shape (#181, resolved
|
||||
2026-07-02):** the `aura new` templates (`scaffold.rs`) are the canonical
|
||||
authoring shape and evolve with the engine; the cycle-0102 `demo-project`
|
||||
fixture is an intentionally frozen known-good twin for the load-boundary
|
||||
tests. The two are deliberately **not** lockstep-guarded: no consumer requires
|
||||
them to match, each is e2e-guarded on its fitness for purpose (build →
|
||||
descriptor load → charter check → deterministic run — blueprint *wiring*
|
||||
content is pinned in neither, stated honestly), and an equality guard would
|
||||
convert every deliberate template improvement into forced churn of the frozen
|
||||
fixture — the same cross-purpose coupling that rules out regenerating the
|
||||
fixture from the scaffolder.
|
||||
|
||||
**[C26, 2026-07-10 (#231): the single-price data weld inside the surviving `wrap_r` scaffolding is retired — input roles bind archive columns by name; the wrapper's remaining R-scaffolding retirement stays #159.]** (From the C24 Forbids clause as of that date; the single-price weld was retired at #231/C26, but `wrap_r` itself survives — its full R-scaffolding retirement stays deferred, #159.)
|
||||
@@ -0,0 +1,300 @@
|
||||
# C24 — The blueprint is a serializable, World-owned data value (the topology data format)
|
||||
|
||||
**Guarantee.** A **blueprint** — the param-generic, named graph-as-data a Rust
|
||||
builder produces (C9/C19) — is a **first-class serializable data value** with a
|
||||
stable, versioned format, and the engine has a **load path** (data → blueprint →
|
||||
`FlatGraph`, C23), not only the Rust-builder construction path. Topology is
|
||||
therefore a **value the World owns** (C21): constructed, **generated**, mutated,
|
||||
**structurally searched**, serialized for **reproduction** (C18), and (optionally)
|
||||
rendered or edited by a visual face (C22). `model_to_json` (C9) is the **existing
|
||||
reverse half** (blueprint → data, for render); this contract closes the loop. The
|
||||
format carries **topology + structural axes + the param-space**, referencing nodes
|
||||
by their **compiled-in type identity** — the closed, typed node vocabulary of the
|
||||
std set plus the project `cdylib` (C8/C16) — and carries **no node logic**
|
||||
(computation is compiled Rust, C17). It is **non-Turing-complete by construction**
|
||||
(a static DAG, no control flow), so it structurally cannot become a second RustAst.
|
||||
|
||||
**Forbids.** Putting **node logic / computation** in the format (the RustAst trap —
|
||||
logic is Rust, C17); a **Turing-complete or control-flow-bearing** blueprint format
|
||||
(it is static data; the *generator* that emits it may be Rust, C20, but the artifact
|
||||
is not a language); a **by-name node marketplace / registry** (the vocabulary is the
|
||||
compiled-in closed set referenced by type identity — the format is not a distribution
|
||||
mechanism, C9/C16); treating the **visual face (C22) as the motivation or the
|
||||
authoring brain** (the format is required by C21/C18 at the CLI level, independent of
|
||||
any UI); a **generated / searched run whose topology is not recoverable from its
|
||||
manifest** (C18 reproduction); baking topology as **engine source** compiled into the
|
||||
binary. The hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `r_sma_graph`,
|
||||
`sample_blueprint_with_sinks`, the built-in `*_sweep_family`) were pre-C24 scaffolding,
|
||||
a C16 tension, and are now **retired** (#159/#220): topology reaches the engine only as
|
||||
data, resolved by the injected vocabulary. The `wrap_r` R-scaffolding wrap
|
||||
(`aura-runner`) **survives** — it is the r-sma wrap every `run` / `sweep` / `mc` /
|
||||
`walkforward` path applies to the loaded signal at run (supplied at run, not serialized);
|
||||
its full R-scaffolding retirement stays deferred (#159). Harness input roles bind archive
|
||||
columns **by name**, not by a single baked-in price weld (C26).
|
||||
|
||||
**Why.** The World (C21) is the product, and it cannot orchestrate, **structurally
|
||||
search**, or **reproduce** *families that vary topology* while topology stays opaque
|
||||
Rust source baked into the engine. The **game-engine principle** (C16's engine / game
|
||||
split) makes it concrete: the engine is compiled native systems; the "game" is
|
||||
**content** — and topology *is* content, a data value the engine owns, serializes,
|
||||
instantiates, and mutates, exactly as a game engine owns its scene / prefab /
|
||||
blueprint graphs. The #109 fork was never "*who types the wiring*" — a small LLM
|
||||
cannot reliably *apply* a DSL and a strong one does not need one, so a
|
||||
manifest-as-authoring-surface dies on both horns (C17). It is "**is topology a
|
||||
compile-time *source* artifact or a runtime, serializable, World-owned *value*?**",
|
||||
and determinism (C1) + reproduction (C18) + structural search (the open search-policy
|
||||
thread) force the **value**. The engine already treats topology as a runtime value
|
||||
(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the
|
||||
flat graph as the optimisation target); C24 only adds that the value **serializes out
|
||||
of Rust and loads back in**.
|
||||
|
||||
## Current state
|
||||
|
||||
The **principle** is settled — the #109 resolution, ratified in an in-context design
|
||||
discussion. The format, its loader, the construction service, and the family
|
||||
orchestration all ship. The acceptance criterion is green: a serialized blueprint
|
||||
runs **bit-identical** (C1) to its Rust-built twin.
|
||||
|
||||
### The format and load path
|
||||
|
||||
A `Composite` serializes to a **canonical, versioned** data value — a
|
||||
`format_version` envelope (`BlueprintDoc`), omit-defaults JSON — and **loads back**
|
||||
(data → blueprint → `FlatGraph`) via `blueprint_to_json` / `blueprint_from_json`
|
||||
(`aura-engine::blueprint_serde`). Nodes are referenced by **compiled-in type
|
||||
identity** through an **injected resolver** whose concrete closed `match` over the
|
||||
std vocabulary lives outside the engine (`aura-vocabulary::std_vocabulary`), so the
|
||||
engine stays domain-free, with no node registry (invariant 9). An unknown type fails
|
||||
clean as `LoadError::UnknownNodeType` — the data-plane face of invariant 9, never a
|
||||
silent wrong graph. `model_to_json` (C9) remains the render half; this closes the
|
||||
loop for the round-trippable vocabulary.
|
||||
|
||||
**Out of the round-trippable set** (deliberate; fails clean as `UnknownNodeType`):
|
||||
recording sinks (they capture an `mpsc::Sender` — runtime identity, not param-generic
|
||||
data, C19) and construction-arg builders (`LinComb` / `CostSum` / `SimBroker` /
|
||||
`Session` — structural-axis args, a C20 concern), additively addable later (#156).
|
||||
|
||||
### Runs and families are built FROM blueprint-data
|
||||
|
||||
`aura run <blueprint.json>` loads a serialized **signal** blueprint and emits a
|
||||
`RunReport` **bit-identical** (C1) to its Rust-built twin; the run scaffolding (sinks
|
||||
/ broker / data) is supplied **at run**, not serialized. Beyond a single run, the
|
||||
World constructs and orchestrates **families** of harnesses from topology-data:
|
||||
|
||||
- `aura sweep <blueprint.json> --axis <name>=<csv>` → `FamilyKind::Sweep`. A sweep
|
||||
needs an **open** blueprint (a fully-bound one has an empty `param_space`).
|
||||
- `aura mc <blueprint.json> --seeds N` → `FamilyKind::MonteCarlo`, each seed a
|
||||
distinct synthetic price walk drawn disjoint-parallel through the engine
|
||||
`monte_carlo` seam (invariant 1). MC binds no axis, so it needs a **closed**
|
||||
blueprint (the sweep's distinction inverted); an open one returns a named `Err`,
|
||||
rendered exit-2 at the builder boundary — no hidden exit in the pure builder.
|
||||
- `aura walkforward <blueprint.json> --axis <name>=<csv> [--select
|
||||
argmax|plateau:mean|plateau:worst]` → `FamilyKind::WalkForward`: re-optimizes the
|
||||
loaded blueprint's params over the `--axis` grid on each 24/12/12 IS window,
|
||||
selects the winner by `sqn_normalized`, runs it out-of-sample. Reduce-mode members
|
||||
are R-measured (`oos_r` the meaningful summary; stitched pip-equity empty, C10).
|
||||
|
||||
The family builders live in `aura-runner::family` (`blueprint_sweep_family` /
|
||||
`blueprint_mc_family` / `blueprint_walkforward_family`). Every member manifest carries
|
||||
the **shared** `topology_hash` (one signal topology, only params vary; `member_key`
|
||||
distinguishes members), and each family stores its blueprint(s) content-addressed so
|
||||
`aura reproduce` re-derives every member bit-identically (`aura-runner::reproduce`,
|
||||
C18). The synthetic-walk DGP is the MC machinery, not trader-grade statistics; a
|
||||
real-data block-bootstrap — and retiring the `synthetic_walk_sources` `len:60`↔warm-up
|
||||
coupling — rides #172.
|
||||
|
||||
### Reproduction identity
|
||||
|
||||
The `RunManifest` carries a **`topology_hash`**: SHA256 of the canonical
|
||||
`blueprint_to_json` (see *Canonical form*), the #158 reproducibility anchor, a Tier-1
|
||||
optional field (#156). The hash and its helper live research-side (`aura-cli` +
|
||||
`sha2`), off the frozen engine (invariant 8). The hash covers the **signal** only —
|
||||
the fixed scaffolding is identified by `commit`; a content-id distinguishing
|
||||
structural-axis / whole-harness variants is a #158/#166 concern once scaffolding is
|
||||
itself blueprint-data. The **identity id** (#171) is an additive sibling:
|
||||
`blueprint_identity_json` (`aura-engine::blueprint_serde`) + `graph introspect
|
||||
--identity-id`. The byte-exact content id keeps the store/reproduce roles; the
|
||||
identity id is introspection-only until a dedup consumer exists.
|
||||
|
||||
### Axis discovery
|
||||
|
||||
`aura sweep <blueprint.json> --list-axes` lists a loaded blueprint's open sweepable
|
||||
knobs (one `<name>:<kind>` per line, `param_space()` order) and exits; the printed
|
||||
names are exactly what `--axis` binds. Every listed name is **mandatory** on `sweep` /
|
||||
`walkforward` — the blueprint must be fully bound before it runs — so a subset grid is
|
||||
refused with the missing knob named (`BindError::MissingKnob`, `aura-engine`) and
|
||||
there is no default; pin a knob you do not want to vary with a single-value axis. A
|
||||
single `blueprint_axis_probe` (`aura-runner`) single-sources the wrapped probe for the
|
||||
sweep terminal, the MC closed-check, and the listing, so **listed == swept by
|
||||
construction** (and stays so across the harness retirement — the listing tracks
|
||||
whatever the sweep actually resolves). The names are prefixed by the current wrapping
|
||||
(`sma_signal.fast.length`, the nested-composite prefix), which is why discovery lives
|
||||
on the sweep verb (it owns the wrapping), not `graph introspect`. `--trace` on
|
||||
`sweep` / `walkforward` writes per-member traces on the real-data campaign path
|
||||
(depth-2 fan-out, chartable by the printed family handle, #224); the synthetic path
|
||||
still refuses (`run` / `mc` refuse `--trace` outright). The live trace-writer is the
|
||||
campaign `presentation.persist_taps` (`persist_campaign_traces`, `aura-runner::runner`).
|
||||
|
||||
### The construction service (op-script)
|
||||
|
||||
`aura graph build` / `introspect` over a JSON op-list (#157) is the introspectable,
|
||||
replayable, by-identifier construction surface; it **emits** the serializable
|
||||
blueprint. The engine `GraphSession` / `replay` (`aura-engine::construction`) build a
|
||||
runnable blueprint through validated ops. Construction gates split into **eager**
|
||||
per-op checks — name resolution, edge kind-match (`edge_kind_check`), the double-wire
|
||||
arm, param bind, and acyclicity (a `connect` that would close a cycle is rejected
|
||||
eagerly at the closing op, `GraphSession::closes_cycle`, #161) — and **holistic**
|
||||
finalize checks — wiring totality (`validate_wiring`), param-namespace injectivity
|
||||
(`check_param_namespace_injective`), root-role boundness (`check_root_roles_bound`).
|
||||
Both cadences call the **same extracted predicates** — no second validator. The
|
||||
holistic faults read **by-identifier** (#162): `finish()` translates the
|
||||
index-carrying `CompileError`s (`UnconnectedPort` / `RoleKindMismatch` /
|
||||
`UnboundRootRole`) into by-identifier `OpError` variants while still *calling* the
|
||||
unchanged holistic gates. Build-free introspection answers over the closed vocabulary:
|
||||
`graph introspect --vocabulary | --node <T> | --unwired`. The engine `Op` stays
|
||||
serde-free; the wire DTO (`OpDoc`) is CLI-side (`aura-cli::graph_construct`).
|
||||
Acceptance: a graph built purely through the ops compiles identical (C1) to its
|
||||
Rust-built twin, an invalid op is rejected at the op naming the cause, introspection
|
||||
answers without a build.
|
||||
|
||||
**Invariant-5 lockstep.** The eager acyclicity gate is a *second* home of invariant 5
|
||||
alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two
|
||||
must co-evolve when the explicit delay/register node — invariant 5's sole legal
|
||||
feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be
|
||||
rejected at construction.
|
||||
|
||||
### Op-script grammar
|
||||
|
||||
The op-script is a JSON **array of ops**, each object internally tagged by `"op"`,
|
||||
replayed in order; nodes are referenced **by identifier**, ports as dotted
|
||||
`<identifier>.<port>`. The eight verbs:
|
||||
|
||||
- `source` — `{"op":"source","role":<str>,"kind":<ScalarKind>}` — declare a root
|
||||
source role producing a base column of `kind`.
|
||||
- `input` — `{"op":"input","role":<str>}` — declare a root input role (kind inferred
|
||||
from the slots it feeds).
|
||||
- `add` — `{"op":"add","type":<TypeId>,"name":<str>?,"bind":{<param>:<Scalar>}?}` —
|
||||
add a node of compiled-in type identity `type`; **`name` is its identifier**
|
||||
(mirrors the builder's `.named(...)`; defaults to the lowercased type label, so
|
||||
two unnamed nodes of one type collide); `bind` sets params.
|
||||
- `feed` — `{"op":"feed","role":<str>,"into":[<port>,…]}` — fan a root role into
|
||||
interior input slots.
|
||||
- `connect` — `{"op":"connect","from":<port>,"to":<port>}` — wire an interior output
|
||||
field to an input slot; a `connect` closing a cycle is rejected eagerly
|
||||
(invariant 5).
|
||||
- `expose` — `{"op":"expose","from":<port>,"as":<str>}` — promote an interior output
|
||||
field to a boundary output, aliased by `as` (a real alias, not a naming; contrast
|
||||
`add`'s `name`) — one of the two verbs that keep `as` (with `tap`).
|
||||
- `tap` — `{"op":"tap","from":<port>,"as":<str>}` — declare a measurement **tap** on
|
||||
an interior output field under `as` (#284) — the output-side twin of `expose`: a
|
||||
recorded observation point (a `Composite.taps` entry, C27), not a boundary output.
|
||||
Name-addressed like every other op (no raw index); tap names are their own
|
||||
namespace (a duplicate refuses). The `finish` gate threads op-declared taps into
|
||||
the built `Composite` (`.with_taps`); a single `aura run` records each, a sweep
|
||||
leaves them inert.
|
||||
- `gang` — `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}`
|
||||
— fuse two or more sibling params into ONE public knob: the member addresses
|
||||
leave the sweepable param space and `as` replaces them; the bound or swept
|
||||
value fans out to every member at bootstrap. Members must share one scalar
|
||||
kind and stay open (un-bound).
|
||||
|
||||
Value forms are the serialized representations: a `Scalar` bind value is the typed tag
|
||||
form `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` / `{"Timestamp":<i64>}`, and `kind`
|
||||
is the capitalized `ScalarKind` (`"F64"`). Worked example:
|
||||
`fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json` (an
|
||||
SMA-crossover bias).
|
||||
|
||||
### Canonical form (#164)
|
||||
|
||||
The canonical blueprint artifact is exactly the bytes `blueprint_to_json` returns — a
|
||||
JSON value with **no trailing newline** (`serde_json::to_string`, not framed with a
|
||||
display `println!`), so the CLI and library emit paths are **byte-identical** — a
|
||||
prerequisite for content-addressed topology identity (#158, a hash over the canonical
|
||||
form). The canonical JSON is also the blueprint's **equality / identity surface**:
|
||||
`Composite` (`aura-engine::blueprint`) and `PrimitiveBuilder` (`aura-core::node`)
|
||||
deliberately carry **no in-memory `PartialEq` / `Debug`** — the recipe holds a
|
||||
`build: Box<dyn Fn>` closure (non-comparable, non-printable), so equality could only be
|
||||
defined over the serialized *data*, of which `blueprint_to_json` is already the single
|
||||
source; a second in-memory notion would be a drift hazard against the very form #158
|
||||
content-addresses. The loader stays lenient (a trailing newline on input still parses,
|
||||
Tier-1 robustness).
|
||||
|
||||
### Forward-compat: the two-tier discipline (#156)
|
||||
|
||||
Tier-1 (additive-optional) is serde-default silent-ignore with **no** `format_version`
|
||||
bump — a new optional field defaults to prior behaviour (C1). Tier-2 (must-understand:
|
||||
a new node type, edge semantics, or structural-axis kind) **bumps** `format_version` so
|
||||
an old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`).
|
||||
**Pre-ship dormancy (#61):** until the first external ship there are no out-of-repo
|
||||
readers — reader and writer change atomically in one commit — so the Tier-2 bump
|
||||
discipline is dormant and structurally-semantic additions (the `gangs` section) land as
|
||||
additive-optional fields of v1; the first ship consciously freezes v1, gangs included,
|
||||
and activates the bump discipline. The per-section required-flag scheme is deferred (no
|
||||
current Tier-2 section to validate it; recorded on #156).
|
||||
|
||||
### Enforcement shift — invariant 9 on the data plane
|
||||
|
||||
Lifting topology onto the data plane relocates *where* the engine/project boundary
|
||||
(invariant 9, no bespoke node registry/marketplace) is enforced, without changing what
|
||||
it forbids. **Pre-C24** a blueprint referencing another project's node was a **compile
|
||||
error** — topology was in-process Rust, so cross-boundary node resolution was
|
||||
*structurally impossible*, compiler-guaranteed for free. **Post-C24** that same
|
||||
reference is a **type-id string in a serialized blueprint**, refused only by the
|
||||
injected resolver's closed set (`UnknownNodeType`). The anti-NIH rationale survives
|
||||
intact (node *logic* stays cargo+Gitea-only; the format distributes
|
||||
**topology-as-content**, never logic — C17), and `std_vocabulary` stays a compiled-in
|
||||
dispatch table, not a registry — but its enforcement **degrades from
|
||||
compiler-guaranteed to resolver-seam discipline**. Two consequences:
|
||||
|
||||
1. **#160** (the guard that the closed vocabulary stays honest) is no longer droppable
|
||||
hygiene — it guards the data-plane face of the boundary (its own drift direction
|
||||
still fails *safe*, `UnknownNodeType`). The `std_vocabulary_roster!` macro
|
||||
(`aura-vocabulary`) expands the resolver `match` and the enumerable
|
||||
`std_vocabulary_types()` list from **one roster**, closing the resolver-vs-list
|
||||
drift mode by construction; the residual (a new zero-arg node never rostered at all)
|
||||
stays fail-safe, guarded by the one-line maintainer surface plus the count pins (the
|
||||
in-crate shape test and aura-cli's cross-boundary `--vocabulary` count e2e). The
|
||||
per-project **merged resolver** (project ∪ std, project first) lives in
|
||||
`aura-runner::project`.
|
||||
2. The **injected per-project resolver** (C16) is the genuinely new, *deferred*
|
||||
invariant-9 surface: nothing structurally stops a project resolver from accepting
|
||||
arbitrary type-ids, or a project from layering a blueprint *marketplace* (store +
|
||||
type-discovery API) **on top of** the format — one layer *above* the engine
|
||||
contract. Invariant 9 holds **at the engine**; the project/ecosystem boundary (what
|
||||
an injected resolver may resolve, where closed-set discipline draws the line) is a
|
||||
**World/C21-layer charter point**, not an engine fix.
|
||||
|
||||
Invariant 8 (frozen) is likewise reframed by C24/C21: from "topology baked at build
|
||||
time" to "the World may *generate* topology at runtime (research plane), the chosen
|
||||
blueprint is frozen into the deploy artifact, and any run is deterministic once
|
||||
instantiated" (C1).
|
||||
|
||||
### Adjacent / deferred
|
||||
|
||||
The **project-as-crate load boundary** has landed (`Aura.toml` discovery + `cdylib`
|
||||
loading + merged project ∪ std vocabulary; the `aura new` scaffolder emits a buildable
|
||||
project whose blueprint runs through the merged vocabulary) — see C13/C16. **Canonical
|
||||
project shape (#181):** the `aura new` templates (`scaffold.rs`) are the canonical
|
||||
authoring shape and evolve with the engine; the `demo-project` fixture is an
|
||||
intentionally frozen known-good twin. The two are deliberately **not** lockstep-guarded
|
||||
— no consumer requires them to match, each is e2e-guarded on its fitness for purpose
|
||||
(build → descriptor load → charter check → deterministic run; blueprint *wiring* content
|
||||
is pinned in neither, stated honestly), and an equality guard would convert every
|
||||
deliberate template improvement into forced churn of the frozen fixture (the same
|
||||
cross-purpose coupling that rules out regenerating the fixture from the scaffolder).
|
||||
What remains open is the **composable-orchestration** thread (#109): topology-as-data is
|
||||
the substrate it stands on.
|
||||
|
||||
## See also
|
||||
|
||||
- [C9](c09-fractal-composition.md) — graph-as-data; `model_to_json` render half
|
||||
- [C16](c16-engine-project-split.md) — engine / project split; the injected resolver
|
||||
- [C17](c17-authoring-surface.md) — logic is Rust; the closed-vocabulary-data-artifact clarification
|
||||
- [C18](c18-registry.md) — reproduction, the run registry, content-addressed store
|
||||
- [C19](c19-bootstrap.md) — bootstrap: blueprint → frozen instance
|
||||
- [C21](c21-world.md) — the World owns and orchestrates topology
|
||||
- [C23](c23-graph-compilation.md) — bootstrap-as-compilation to `FlatGraph`; the optimisation target
|
||||
- [C26](c26-input-binding.md) — role names bind archive columns
|
||||
- [C27](c27-declared-taps.md) — declared taps as named measurement points
|
||||
- [C1](c01-determinism.md) — determinism / bit-identity; [C10](c10-bias-r-cost.md), [C13](c13-hot-reload-frozen-deploy.md), [C20](c20-strategy-harness.md), [C22](c22-playground-traces.md), [C25](c25-role-model.md), [C28](c28-stratification.md)
|
||||
|
||||
> History: [c24-blueprint-data.history.md](c24-blueprint-data.history.md)
|
||||
@@ -0,0 +1,90 @@
|
||||
# C25 — The role model: nine authoring roles, cut by artifact + surface + iteration cost
|
||||
|
||||
**Guarantee.** aura's user-facing design is organized by **roles**, cut by the
|
||||
artifact a role owns, the surface/language it works in, and the iteration cost it
|
||||
tolerates — never by org chart. One trader — or an LLM actor, which is the whole
|
||||
point of the authoring design — must be able to fill *every* role; the separation
|
||||
separates *surfaces with different iteration costs and failure modes*, not people,
|
||||
and switching roles is a visible act. The nine roles (ratified 2026-07-03, #188):
|
||||
|
||||
1. **system programmer** — the engine; Rust, this repo; all invariants.
|
||||
2. **extension dev** — native nodes + analysis blocks; Rust in project/shared
|
||||
crates; [C7](c07-scalar-soa.md)/[C8](c08-node-contract.md), determinism in
|
||||
`eval`.
|
||||
3. **toolchain dev** — playground/viewer/skills pipeline; vendor-side; authoring
|
||||
ergonomics.
|
||||
4. **data curator** — recorded streams, sidecars, derived/news recordings; CLI +
|
||||
recording edge; [C2](c02-causality.md)/[C3](c03-single-merge.md)/[C6](c06-firing-policy.md).
|
||||
5. **methodology designer** — validation/eval **process documents** ("the game
|
||||
rules"); closed data vocabulary; anti-false-discovery.
|
||||
6. **6a strategy designer** — blueprint topologies; data (op-script/blueprint
|
||||
JSON, the graph idiom); [C5](c05-freshness-gating.md)/[C7](c07-scalar-soa.md).
|
||||
7. **6b campaign designer** — **campaign documents** (persisted experiment
|
||||
intent); data, the pipeline/block idiom; [C1](c01-determinism.md).
|
||||
8. **player** — owns nothing; explores traces, tunes runtime params; the
|
||||
playground, [C22](c22-playground-traces.md)/[C12](c12-atomic-sim-unit.md);
|
||||
read-only by design.
|
||||
9. **operator/auditor** — frozen bots, reconciliation, money management;
|
||||
deploy-edge CLI, manifests, reproduce; [C13](c13-hot-reload-frozen-deploy.md).
|
||||
|
||||
The 6a/6b split follows invariant 12's tier boundary (node/harness vs World); the
|
||||
interface between them is the id machinery (content id = byte-exact, identity id =
|
||||
topological). The param gradient runs 6a defines topology + open params, 6b spans
|
||||
spaces over them, 7 moves inside those spaces live. A role is **recognizable** when
|
||||
four things exist: a named artifact type it owns, a surface addressed to it, an
|
||||
entry path, and its invariants enforced at its boundary.
|
||||
|
||||
**Text-first / headless-first is an invariant, not a taste** (owner-ratified
|
||||
2026-07-03: editors must never be the only way; LLM operability demands it): every artifact class has a canonical,
|
||||
complete, text-serialized form; every operation is executable headless via
|
||||
CLI/API; visual surfaces are *stateless projections* that read/write the same
|
||||
canonical form and add no semantics — read-only projections (viewers) rank far
|
||||
above write editors. The **Blockly litmus test** is the acceptance criterion for
|
||||
every artifact vocabulary, editor or no editor: a block palette must be
|
||||
*generatable* from the vocabulary — every block with typed slots, every valid
|
||||
composition snappable, every invalid one not.
|
||||
|
||||
**Control surfaces are projections** (#295, ratified 2026-07-20). The text-first
|
||||
invariant fixes the control-surface layering: the text artifact vocabulary
|
||||
(blueprints, process/campaign documents, registry records) is the canonical layer,
|
||||
and every control surface — the one-shot CLI's executor verbs, any future
|
||||
long-running host, any MCP face, a World program — is a projection/executor over
|
||||
those artifacts, never a second home for intent.
|
||||
|
||||
**Forbids.** An artifact class whose only authoring path is a visual editor or a
|
||||
Rust compile cycle when the role's iteration cost demands data (the role-6b
|
||||
lesson: a tiny campaign change must never cost a compile); stringly-typed fields,
|
||||
implicit inter-block coupling, or "arbitrary JSON here" holes in a role's
|
||||
vocabulary (they fail the litmus test); collapsing the 6a/6b idioms into one
|
||||
surface (dataflow boxes-and-wires vs sequential pipeline blocks are different
|
||||
languages).
|
||||
|
||||
**Why.** The game-engine analogy that seeded aura ([C16](c16-engine-project-split.md))
|
||||
extends to its people: engines separate system programmers, toolchain devs, asset
|
||||
creators, level designers, and players because their artifacts, tools, and
|
||||
iteration loops differ — a design that forces one role's technique on another
|
||||
(Rust for campaign tweaks, an editor as the only entry) mis-prices iteration
|
||||
exactly where research throughput lives.
|
||||
|
||||
## Current state
|
||||
|
||||
Roles 5 and 6b have their artifacts — process and campaign documents
|
||||
([C18](c18-registry.md); #189). Roles 4, 7, and 8 remain technically
|
||||
present but faceless: no addressed verb families yet. Role homes in the project
|
||||
layout and docs-by-role are open (#192).
|
||||
|
||||
**Document-first completion** is the resolved direction on the control-surface
|
||||
amendment (owner-minuted 2026-07-21 on #295), delivered by #300. The executor verb
|
||||
set is settled: `run`, the four thin per-verb generators, and the document verbs
|
||||
validate/introspect/register/show/run/runs — `show` being #300's read-back
|
||||
addition. The verbs' per-verb identity is re-ratified (#300 F8 — reduction, not
|
||||
grammar collapse). A typed-protocol host or MCP face remains demand-driven and
|
||||
unbuilt.
|
||||
|
||||
## See also
|
||||
- [C16](c16-engine-project-split.md) — the game-engine analogy this extends to people
|
||||
- [C17](c17-authoring-surface.md) — all logic is Rust; closed-vocabulary data artifacts are not the forbidden DSL
|
||||
- [C18](c18-registry.md) — the process/campaign document artifacts and the run registry
|
||||
- [C20](c20-strategy-harness.md), [C21](c21-world.md) — the node/harness/World tiers behind the 6a/6b split
|
||||
- [C24](c24-blueprint-data.md) — topology-as-data, the strategy designer's artifact
|
||||
- [C26](c26-input-binding.md) — a role's input contract carried in blueprint data
|
||||
@@ -0,0 +1,73 @@
|
||||
# C26 — Harness input binding: role names bind archive columns
|
||||
|
||||
**Guarantee.** A strategy blueprint declares WHICH data it consumes as the NAMES
|
||||
of its root input roles: a role whose name is in the **closed column vocabulary** —
|
||||
`open`, `high`, `low`, `close`, `spread`, `volume`, plus `price` as the
|
||||
backward-compatible alias of `close` — binds by default to that column of the
|
||||
campaign cell's (or run invocation's) instrument. A campaign document may
|
||||
**override** the name default per role via the additive `data.bindings` block
|
||||
(`role name → column name`; serde `default` + skip-if-empty, so binding-less
|
||||
documents keep their content ids) — the 6b rebind seam: the blueprint's content
|
||||
identity never changes.
|
||||
|
||||
Resolution (`aura-runner`'s `binding` module, `resolve_binding`) produces ONE
|
||||
ordered plan consumed by BOTH halves of the old weld: the columns are opened
|
||||
(`aura_ingest::open_columns`, the generalization of `open_ohlc`/#92) and the
|
||||
wrapped root roles declared (`aura-runner::member`'s `wrap_r`) in the **same
|
||||
canonical order** — `M1Field` declaration order (open, high, low, close, spread,
|
||||
volume), filtered to the consumed set — which is the [C4](c04-cycle-granularity.md)
|
||||
merge tie-break order, so role *i* receives source *i* by construction. A close
|
||||
column is always part of the plan (shared when the strategy consumes
|
||||
`close`/`price`, else opened for the broker/executor pair alone).
|
||||
|
||||
The vocabulary is **Blockly-litmus-clean** ([C25](c25-role-model.md)): role-name
|
||||
slots draw from a closed enum, the bindings block is typed `role → column` with no
|
||||
free-form holes, and validation is two-tier — values against the column vocabulary
|
||||
in the intrinsic tier (`validate_campaign`, `aura-research`), keys against the
|
||||
strategies' `input_roles()` in the resolver tier (`validate_campaign_refs`,
|
||||
`aura-registry`).
|
||||
|
||||
**Forbids.** Guessing a column for an unknown role name (refuse with the vocabulary
|
||||
+ the override remedy, never default); opening columns in any order other than the
|
||||
canonical declaration order (the [C4](c04-cycle-granularity.md) tie-break contract);
|
||||
free-form or logic-bearing binding values (the [C17](c17-authoring-surface.md)/[C25](c25-role-model.md)
|
||||
line — a binding value is a typed column reference, nothing else); fabricating
|
||||
synthetic OHLC when a multi-column strategy meets synthetic data (the walk
|
||||
generates a close series only — refuse with the `--real` remedy).
|
||||
|
||||
**Why.** The single-price weld — `wrap_r`'s hard-wired `price`←close and the six
|
||||
`M1Field::Close`-only open sites — made every strategy monocular regardless of what
|
||||
its blueprint declared. Binding by role name keeps the blueprint the single source
|
||||
of its own data needs ([C24](c24-blueprint-data.md): topology-as-data carries its
|
||||
input contract), lets a campaign re-aim a strategy without touching its content id
|
||||
([C18](c18-registry.md) identity), and pins opening and declaring to one
|
||||
shared, canonically-ordered plan so the two halves cannot drift
|
||||
([C1](c01-determinism.md)/[C4](c04-cycle-granularity.md) determinism).
|
||||
|
||||
## Current state
|
||||
|
||||
`resolve_binding` lives in `aura-runner`'s `binding` module
|
||||
(`crates/aura-runner/src/binding.rs`), moved out of the shell with #295; it takes
|
||||
the strategy name, its `input_roles()`, and the campaign override map and returns
|
||||
one `ResolvedBinding` plan. Column opening is `aura_ingest::open_columns`
|
||||
(`crates/aura-ingest`; the `M1Field` enum fixes the declaration order
|
||||
open/high/low/close/spread/volume); role declaration is `wrap_r` in
|
||||
`aura-runner::member`. Both validation tiers are realized: `validate_campaign`
|
||||
(`crates/aura-research`) checks binding values against the column vocabulary;
|
||||
`validate_campaign_refs` (`crates/aura-registry`) checks binding keys against the
|
||||
strategy's declared roles.
|
||||
|
||||
**Extension point (deferred).** When recorded non-price sources land (the #71
|
||||
Source seam), the binding VALUE-space grows additively from archive columns to
|
||||
recorded-stream references — a root role like `sentiment` becomes bindable to a
|
||||
recorded stream then, and until then refuses with the vocabulary-naming message.
|
||||
The role-name key-space and the campaign `bindings` carrier are unchanged by that
|
||||
growth.
|
||||
|
||||
## See also
|
||||
- [C4](c04-cycle-granularity.md) — the merge tie-break order that is the canonical open/declare order
|
||||
- [C24](c24-blueprint-data.md) — topology-as-data carrying its own input contract
|
||||
- [C18](c18-registry.md) — the content-id identity a rebind must not touch
|
||||
- [C25](c25-role-model.md) — the Blockly litmus the vocabulary satisfies; the 6b rebind seam
|
||||
- [C1](c01-determinism.md) — the determinism the shared plan protects
|
||||
- [C27](c27-declared-taps.md) — the output-side twin (`taps` to `input_roles`)
|
||||
@@ -0,0 +1,69 @@
|
||||
# C27 — Declared taps: named measurement points bind sinks run-mode-aware
|
||||
|
||||
**Guarantee.** A blueprint may declare **taps** — named, pure output-side
|
||||
declarations `{ name, from: {node, field} }`, the output-side twin of `input_roles`
|
||||
([C26](c26-input-binding.md)). A tap names an interior producer's output field
|
||||
without naming a sink, exactly as a `Role` names an abstract input without naming a
|
||||
source. At compile the tap resolves — and, for an interior composite, **hoists** to
|
||||
the root — through the same lowering remap edges and `OutField` re-exports use
|
||||
(`resolve_tap_wire`, a `flat_taps` accumulator threaded through the lowering
|
||||
recursion), landing in `FlatGraph.taps` as a `FlatTap { name, node, field }` whose
|
||||
name survives compile and is load-bearing for by-name binding (like
|
||||
`SourceSpec.role`, #275).
|
||||
|
||||
Binding is **run-mode-aware**: the run-mode-owning layer constructs a sink (a
|
||||
`Recorder`) at a bound tap via `FlatGraph::bind_tap`, which takes a **caller-built**
|
||||
`Box<dyn Node>` sink (so the engine keeps its `aura-core`-only production
|
||||
dependency — it never constructs a domain sink type) and appends it plus an edge
|
||||
before bootstrap. The single-run path (`run_signal_r`) binds and records each
|
||||
declared tap, persisting the series through the existing trace store
|
||||
(`env.trace_store()`), so the tap columns surface through the same tooling the
|
||||
campaign path feeds; a sweep/reduce run leaves taps unbound.
|
||||
|
||||
**Forbids.** A tap carrying a channel endpoint or effect in the serialized artefact
|
||||
— recording policy is run-mode authority, not fragment-embedded (a fragment must
|
||||
not drag its measurement decisions into every harness that embeds it; the tier
|
||||
ontology, [C20](c20-strategy-harness.md)/[C21](c21-world.md)). The engine
|
||||
constructing a domain sink type (the `aura-core`-only wall — the sink is
|
||||
caller-built). Order statistics (median, etc.) inside the graph — they stay
|
||||
sink/analysis-side ([C18](c18-registry.md)); multi-instrument study
|
||||
inputs stay harness/World tier.
|
||||
|
||||
**Non-error.** An **unbound** tap is inert, not a fault — unlike an unbound root
|
||||
input role, which `check_root_roles_bound` rejects ([C26](c26-input-binding.md)):
|
||||
observation is optional, a fed input is mandatory. A declared-but-unbound tap
|
||||
compiles and runs, its producer evaluating and its output discarded (a no-out-edge
|
||||
producer is a valid runnable sink — the Kahn sort emits it,
|
||||
`check_ports_connected` gates only inputs).
|
||||
|
||||
**Why.** Observability must be expressible in a hand-authored blueprint — the
|
||||
measurement-shaped study computes in the graph and surfaces via taps, no throwaway
|
||||
Rust harness — while recording stays a run-mode decision, not a fragment-embedded
|
||||
effect. Taps are designed **DCE-compatible** (a bound tap is a natural DCE root, an
|
||||
unbound tap a dead declaration) but this contract does **not** depend on DCE: an
|
||||
unbound tap's sink is simply never constructed (build-time elision, which the
|
||||
engine already tolerates). (#282, 2026-07-18.)
|
||||
|
||||
## Current state
|
||||
|
||||
The tap types are realized in `aura-engine`: the authoring-level `Tap`
|
||||
(`crates/aura-engine/src/blueprint.rs`, the output-side twin of `Role`), the
|
||||
compiled `FlatTap { name, node, field }` and the `FlatGraph.taps` field
|
||||
(`crates/aura-engine/src/harness.rs`), and `FlatGraph::bind_tap`, which appends a
|
||||
caller-built `Box<dyn Node>` sink plus an edge and raises a typed `UndeclaredTap`
|
||||
on a tap the graph does not declare (duplicate detection across binds is the
|
||||
caller's — the method keeps no cross-call state). Lowering resolves and hoists taps
|
||||
via `resolve_tap_wire` and the `flat_taps` accumulator (`blueprint.rs`). The
|
||||
single-run bind/record path is `run_signal_r` in `aura-runner::member`; the
|
||||
sweep/reduce path never calls `bind_tap`.
|
||||
|
||||
The chain-pruning benefit — a sweep paying zero for the study wires behind an
|
||||
unbound tap — is **deferred to the future DCE cycle** ([C23](c23-graph-compilation.md));
|
||||
the mechanism ships now, verified sound.
|
||||
|
||||
## See also
|
||||
- [C26](c26-input-binding.md) — the input-side twin (`input_roles`); `check_root_roles_bound`, the mandatory-input counterpart
|
||||
- [C23](c23-graph-compilation.md) — compilation/lowering and the deferred DCE cycle the tap design anticipates
|
||||
- [C18](c18-registry.md) — the trace store and registry the tap series feed; order statistics live sink-side
|
||||
- [C20](c20-strategy-harness.md), [C21](c21-world.md) — the tier ontology behind run-mode recording authority
|
||||
- [C8](c08-node-contract.md) — the node/sink contract (a no-out-edge producer as a valid runnable sink)
|
||||
@@ -0,0 +1,85 @@
|
||||
# C28 — Internal stratification: the trading ladder, the process column, the shell: history
|
||||
|
||||
> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp
|
||||
> and may be superseded; this file is NOT current truth and NOT a grounding
|
||||
> surface. Current contract: [c28-stratification.md](c28-stratification.md).
|
||||
|
||||
**Status (2026-07-19, milestone "Stratification — ladder, process column,
|
||||
shell", #286).** This contract states the *target*. At HEAD the crates cut by
|
||||
mechanical role (vocabulary / reductions / documents / orchestration / CLI), not
|
||||
cleanly by layer, so the layering is realized only partially:
|
||||
- The dependency *direction* now obeys the rule across the engine stack. The
|
||||
`aura-engine → aura-backtest` production violation is cut (phase 2, #292):
|
||||
`RunReport` is generic over its metric payload `M` (the engine names no
|
||||
concrete metric type), the pip/R reductions and the MC assembly (`summarize`,
|
||||
`McAggregate`/`RBootstrap`/`monte_carlo`) moved to `aura-backtest`, and the
|
||||
statistics kernel (`MetricStats`/`quantile`/`resample_block`/`SplitMix64`)
|
||||
moved to the `aura-analysis` foundation. The engine's `[dependencies]` are now
|
||||
`aura-core` + `aura-analysis` only; the backtest layer instantiates
|
||||
`M = RunMetrics` (`aura-backtest → aura-engine`, an outer→inner edge). The
|
||||
ladder direction is enforced for the engine/backtest/analysis rungs by the
|
||||
structural test (`c28_layering`, seven-row table).
|
||||
- **#147 retired (user-ratified direction 2026-07-20; shipped the same day).**
|
||||
Item 1 — the metric genericity `RunReport<M>` — shipped as phase 2 above
|
||||
(#292). Item 2 — the registry deflation vocabulary — shipped as the ratified
|
||||
"A1" cut, its deferral trigger having fired when measurement supplied its
|
||||
first deflatable metric (the IC, #290): the deliberately narrow
|
||||
`MetricVocabulary` trait (resolve/roster/direction/value/one null draw)
|
||||
lives in `aura-analysis`, re-exported through `aura-engine`; the R
|
||||
vocabulary (`RunMetricKey`, `r_based`, the rosters, the centred
|
||||
moving-block `null_draw`) is supplied by `aura-backtest`; the registry's
|
||||
rank/optimize/deflate machinery is generic over `M: MetricVocabulary` with
|
||||
refusal prose derived from the carried roster; the IC (`aura-cli`) is the
|
||||
second production implementor, bringing its within-run permutation null.
|
||||
The C10 wall stays monomorphic (`check_r_metric`/`generalization` accept
|
||||
only the R vocabulary; `r_based` is the enforcement point). Explicitly
|
||||
still deferred ("A2"): measurement runs as sweep-family citizens (report
|
||||
unification, campaign engine generic-over-M) — until a concrete
|
||||
family/campaign demand exists. The #136 one-implementor rationale is
|
||||
superseded by the second implementor; the registry's process-column
|
||||
trading-awareness holds unchanged. See #147 for the full disposition.
|
||||
- The `aura-std` four-layer roster is now cut by layer (phase 4, #288): `aura-std`
|
||||
holds the engine nodes only (arithmetic/logic/rolling + the generic sinks);
|
||||
`aura-market` (`session`, `resample`), `aura-strategy` (`bias`/`stop_rule`/
|
||||
`sizer` + the cost nodes) and `aura-backtest` (`sim_broker`,
|
||||
`position_management`) carry the outer rungs, each depending only on `aura-core`;
|
||||
the closed node roster moved to `aura-vocabulary`. The `aura-analysis`
|
||||
interweave is resolved (phase 5, #291): the backtest reductions live in
|
||||
`aura-backtest::metrics` beside their `position_management` producer, and
|
||||
`aura-analysis` is reduced to domain-free statistics + selection provenance
|
||||
(`[dependencies]` = serde only). The *measurement* rung is seeded (#295):
|
||||
`aura-measurement` carries the IC vocabulary + reduction (its run verb remains
|
||||
the additive shape dispatch, #286). No crate exists yet for *execution*
|
||||
(unbuilt — the C10/C13 edge only). Under this model the
|
||||
`aura-registry → aura-research` edge is column-internal and legal.
|
||||
- Phased realization (each independently shippable; behaviour byte-identical
|
||||
except the purely additive shape dispatch): (1) this contract; (2) cut the
|
||||
engine's backtest-metrics edge via `RunReport<M>` — **done** (#292:
|
||||
metric-generic run record, the pip/R reductions + MC assembly relocated to
|
||||
`aura-backtest`, the statistics kernel to `aura-analysis`, dependency
|
||||
inversion, ladder enforced by the structural test; realizes item 1 of #147);
|
||||
(3) the #286
|
||||
shape dispatch plus the per-run scaffold as a library — **done** (#295: the
|
||||
pure per-run scaffold pieces live in `aura-backtest::scaffold`; the composed
|
||||
recipe — harness assembly, binding, translators, the default `MemberRunner`
|
||||
— is the assembly crate `aura-runner`; the measurement rung is seeded as
|
||||
`aura-measurement` with the IC; the shell is reduced to
|
||||
argv/translation/presentation, enforced structurally. Residual: ~20 refusal
|
||||
sites inside `aura-runner`'s single-run verb paths still terminate the
|
||||
process; their conversion to `RunnerError` propagation is tracked as #297 —
|
||||
the campaign path already refuses via `MemberFault`, never a process exit);
|
||||
(4) split the `aura-std` roster along engine / market / strategy /
|
||||
backtest — **done** (#288: four `aura-core`-only node crates, the closed roster
|
||||
moved to `aura-vocabulary`, the ladder direction enforced by a structural test);
|
||||
(5) split `aura-analysis` into generic statistics and backtest metrics —
|
||||
**done** (#291: metrics moved to `aura-backtest::metrics`, `aura-analysis`
|
||||
reduced to the domain-free half, engine re-export surface name-unchanged); (6)
|
||||
generify the column's metric interface (demand-driven — this is #147). Crate
|
||||
names now realized (`aura-market`/`aura-strategy`/`aura-backtest`/
|
||||
`aura-vocabulary`); full evidence on #288, #286 and the milestone.
|
||||
|
||||
Provenance note (#295): the shell-boundary cut closes a ratified structural
|
||||
debt (this contract's own phase plan) and un-blocks the future World program;
|
||||
it was not a demonstrated downstream blocker — the first downstream project's
|
||||
observed friction was document-vocabulary gaps, never Rust-level recipe
|
||||
reachability.
|
||||
@@ -0,0 +1,179 @@
|
||||
# C28 — Internal stratification: the trading ladder, the process column, the shell
|
||||
|
||||
**Guarantee.** The aura workspace's crates realize a layered responsibility
|
||||
model along two axes. A **ladder**, ordered by domain specificity from inner to
|
||||
outer:
|
||||
|
||||
1. **engine** — the domain-free deterministic streaming runtime and vocabulary
|
||||
(cells/kinds/freshness/firing, graph build, compile, run, taps/trace — C1–C9,
|
||||
C19, C23, C24), the domain-free node roster (arithmetic/logic/rolling nodes
|
||||
plus the generic `Recorder`/`GatedRecorder`/`SeriesReducer` sinks), and
|
||||
generic statistics (the Monte-Carlo and moving-block-bootstrap kernels). The
|
||||
layer name is wider than the crate `aura-engine`: it spans `aura-core`,
|
||||
`aura-engine`, the domain-free node roster (`aura-std`), and the
|
||||
generic-statistics surface (`aura-analysis`).
|
||||
2. **market** — what a market *is*, carrying no analysis or strategy intent:
|
||||
instruments and pip/point geometry (C15), session anchoring, bar resampling,
|
||||
market-data ingestion (`aura-ingest`).
|
||||
3. **measurement** — descriptive statistics over market streams (session
|
||||
statistics, conditional rates, distributions). Sibling of **strategy**: both
|
||||
consume *market*, neither imports the other.
|
||||
4. **strategy** — the *definition* of trading signals: bias, stop rules, sizing,
|
||||
cost-model nodes.
|
||||
5. **backtest** — the *evaluation* of strategies: simulated execution without
|
||||
money (the `SimBroker`, position-management), the R-metric reductions and the
|
||||
position-event table, and the per-run scaffold. Backtest mirrors *execution*
|
||||
on the research side — execution semantics in R units, money exiled.
|
||||
6. **execution** — money, account-term sizing, the live broker; already exiled to
|
||||
the deploy edge by C10/C13, existing today only as that ratified boundary.
|
||||
|
||||
Beside the ladder stands the **process column** — the research-process machinery
|
||||
(the run registry C18, the process/campaign document types, campaign execution).
|
||||
It consumes run *artifacts*, not market streams, and orchestrates runs of any
|
||||
ladder layer; it stands beside the ladder, not on a rung. Between ladder/column
|
||||
and the shell stands the **assembly** position (`aura-runner`): it composes the
|
||||
rungs into the canonical member-run recipe — harness assembly, input binding
|
||||
(C26), the C1-load-bearing param↔config translators, and the shipped default
|
||||
`MemberRunner` (`DefaultMemberRunner`) — importing the ladder rungs,
|
||||
`aura-composites`, `aura-ingest`, and the process column, and imported only by
|
||||
the shell and by downstream World programs. It also carries a direct production
|
||||
dependency on the external `data-server` tree (inherited from the shell with the
|
||||
recipe: the runner constructs archive servers itself). The external tree
|
||||
therefore enters the workspace at exactly three points — the ingestion edge
|
||||
(`aura-ingest`), the assembly position (`aura-runner`), and the shell
|
||||
(`aura-cli`, which imports everything by definition) — and at no other ladder or
|
||||
column crate; the `c28_layering` guard pins the exact set. The **shell** (the
|
||||
`aura` CLI, `aura-cli`) imports everything, is imported by nothing, exports
|
||||
nothing, and holds no domain logic: argv/dispatch, argv→document translation
|
||||
(including the op-script construction front-end, `graph_construct`),
|
||||
presentation, and the `aura new` project scaffolder (authoring-tooling template
|
||||
emission — shell-resident, like rendering, until a second consumer wants it).
|
||||
Rendering stays in the shell until the C22 web face provides its second consumer.
|
||||
|
||||
**Import rule.** An outer ladder layer may import inner layers, never the
|
||||
reverse; *measurement* and *strategy* are siblings (no import either way);
|
||||
*backtest* may import *strategy*. The process column imports the engine layer
|
||||
plus the run-artifact and metric interfaces — a production dependency on
|
||||
`aura-backtest` for the trading instantiation `M = RunMetrics` (#291/#292) — and
|
||||
is imported by no ladder crate; column-internal edges are free (the
|
||||
`aura-registry → aura-research` edge is legal). The assembly position imports
|
||||
ladder and column alike and is imported by neither; the shell imports all;
|
||||
nothing imports the shell. **`dev-dependencies` are exempt** — tests may cross
|
||||
layers (the existing `engine`/`ingest`/`registry` → `std` edges are dev-only and
|
||||
stay). The rule binds **crate structure, not graph composition**: a blueprint may
|
||||
feed a measurement statistic into a bias input — data flow inside a graph is free
|
||||
(C24); layers govern imports, graphs compose freely.
|
||||
|
||||
**Forbids.** An inner layer importing an outer one — most sharply the engine
|
||||
layer importing backtest reductions, or market importing strategy. A ladder crate
|
||||
importing the process column. Money (currency P&L, account sizing) inside the
|
||||
strategy or backtest layers (C10/C13 — it lives only at the execution/deploy
|
||||
edge). Once a second consumer warrants the split, keeping multiple layers welded
|
||||
in one crate's roster.
|
||||
|
||||
**Why.** The stack is a framework specialized for traders: the inner engine is a
|
||||
domain-free analysis substrate (the same machinery would measure robot telemetry
|
||||
or any timestamped process), the outer rungs the trading specialization, and the
|
||||
two together are the "game engine for traders" (C16's game/engine split applied
|
||||
*inside* the workspace, from the engine|project boundary to the intra-stack
|
||||
layering). Separable responsibilities behind narrow, contract-defined seams let a
|
||||
contributor focus on one rung without absorbing the rest, and make each boundary
|
||||
auditable. Enforcement is the crate graph itself: once crates are cut along these
|
||||
cells, a layer violation is a compile error and every new edge a visible
|
||||
`Cargo.toml` diff — no linter, no discipline appeals. Five of the six seams are
|
||||
pre-existing ratified contracts and this model only *names* them as layer
|
||||
boundaries: the node contract (C8, engine ↔ any vocabulary), blueprint-as-data
|
||||
(C24, topology ↔ engine — its shape selecting the run scaffold is the additive
|
||||
#286 refinement), declared taps (C27, run ↔ analysis — the only way values leave
|
||||
a run), the run registry (C18, run ↔ process column), and the deploy edge
|
||||
(C10/C13, everything ↔ money). The sixth — the process column ↔ metric interface
|
||||
— is the one this contract introduced: a named metric vocabulary *supplied* by
|
||||
measurement and backtest instead of baked in R-only (#147, now realized).
|
||||
|
||||
## Current state
|
||||
|
||||
The workspace realizes the full ladder today, cut by layer, with only the
|
||||
*execution* rung unbuilt. The rung→crate map:
|
||||
|
||||
- **engine** — `aura-core` + `aura-engine` + `aura-std` (the engine node roster:
|
||||
arithmetic/logic/rolling + the generic sinks) + `aura-analysis` (domain-free
|
||||
statistics + selection provenance).
|
||||
- **market** — `aura-market` (`session`, `resample`) + `aura-ingest`.
|
||||
- **measurement** — `aura-measurement` (the IC vocabulary + reduction).
|
||||
- **strategy** — `aura-strategy` (`bias`/`stop_rule`/`sizer` + the cost nodes).
|
||||
- **backtest** — `aura-backtest` (`sim_broker`, `position_management`, the
|
||||
R-metric reductions, and the per-run scaffold `aura-backtest::scaffold`).
|
||||
- **execution** — no crate exists; it is the C10/C13 deploy edge only.
|
||||
|
||||
The closed node roster lives in `aura-vocabulary`. The process column is
|
||||
`aura-registry` (C18) + `aura-research` + `aura-campaign`; the assembly position
|
||||
is `aura-runner`; the shell is `aura-cli`.
|
||||
|
||||
The dependency direction obeys the rule across the whole stack. The engine names
|
||||
no concrete metric type: `aura_engine::RunReport<M>` is generic over its metric
|
||||
payload `M` (`crates/aura-engine/src/report.rs`), and `aura-backtest` supplies the
|
||||
trading instantiation `M = RunMetrics` (`pub type RunReport =
|
||||
aura_engine::RunReport<RunMetrics>`, `crates/aura-backtest/src/lib.rs`) — the
|
||||
`aura-backtest → aura-engine` edge is outer→inner. The engine's production
|
||||
`[dependencies]` are `aura-core` + `aura-analysis` only; `aura-analysis` is
|
||||
reduced to the domain-free half (`[dependencies]` = serde only), holding the
|
||||
statistics kernel (`MetricStats`/`quantile`/`resample_block`/`SplitMix64`) and the
|
||||
selection-provenance types, while the pip/R reductions and the Monte-Carlo
|
||||
assembly live in `aura-backtest`. The former monolithic `aura-std` node roster is
|
||||
split by layer: `aura-std` (engine nodes), `aura-market`, and `aura-strategy` are
|
||||
each `aura-core`-only node crates, while `aura-backtest` carries both the backtest
|
||||
node modules (`sim_broker`, `position_management`) and the outer-rung metric
|
||||
layer, so it depends on `aura-engine` as well as `aura-core`. The additive #286
|
||||
shape dispatch (blueprint shape selecting the run scaffold) is realized; the pure
|
||||
per-run scaffold pieces live in `aura-backtest::scaffold`, and `aura-runner`
|
||||
composes them with harness assembly, input binding, the param↔config translators,
|
||||
and the shipped `DefaultMemberRunner`. The shell is reduced to argv/dispatch,
|
||||
argv→document translation, and presentation.
|
||||
|
||||
The direction is enforced structurally, not by discipline. The Rust compiler only
|
||||
rejects import *cycles*; an acyclic-but-outward edge (e.g. `aura-std → aura-market`)
|
||||
would compile silently, so `crates/aura-vocabulary/tests/c28_layering.rs` reads
|
||||
every crate's production `[dependencies]` and asserts each intra-workspace edge
|
||||
stays within the inner set C28 permits for that layer. The `allowed` table is the
|
||||
**full workspace** — all 17 crates under `crates/` — and the test asserts the
|
||||
table enumerates exactly the crates found on disk (`table_names == disk_names`),
|
||||
so a newly added crate cannot escape the guard silently: even a foundation crate
|
||||
with no `aura-*` deps must be listed. `[dev-dependencies]` are not inspected
|
||||
(C28-exempt). Two further checks ride the same file: the shell-content check
|
||||
(`aura-cli` is a pure binary with no `lib` target and a fixed module set —
|
||||
argv/translation/presentation only), and the external-`data-server` pin (the tree
|
||||
may be a production dependency of exactly `aura-ingest`, `aura-runner`, and
|
||||
`aura-cli`).
|
||||
|
||||
**#147 disposition (retired; user-ratified direction 2026-07-20, shipped the same
|
||||
day).** The process-column metric interface is generic. The deliberately narrow
|
||||
`MetricVocabulary` trait (resolve/roster/direction/value/one null draw) lives in
|
||||
`aura-analysis` (`crates/aura-analysis/src/lib.rs`) and is re-exported through
|
||||
`aura-engine`; the R vocabulary (`RunMetricKey`, `r_based`, the rosters, the
|
||||
centred moving-block `null_draw`) is supplied by `aura-backtest`; the registry's
|
||||
rank/optimize/deflate machinery is generic over `M: MetricVocabulary`, with
|
||||
refusal prose derived from the carried roster. The IC (`aura-measurement`) is the
|
||||
second production implementor, bringing its within-run permutation null. The C10
|
||||
money wall stays monomorphic — `check_r_metric`/`generalization` accept only the R
|
||||
vocabulary (`r_based` is the enforcement point). Still deferred ("A2",
|
||||
demand-gated, no tracking issue): measurement runs as sweep-family citizens
|
||||
(report unification, campaign engine generic-over-`M`), until a concrete
|
||||
family/campaign demand exists.
|
||||
|
||||
**Deferred.** ~20 refusal sites inside `aura-runner`'s single-run verb paths still
|
||||
terminate the process (`std::process::exit`); their conversion to `RunnerError`
|
||||
propagation is tracked as **#297** (the campaign path already refuses via
|
||||
`MemberFault`, never a process exit).
|
||||
|
||||
## See also
|
||||
- [C1](c01-determinism.md) — determinism / bit-identity, the correctness invariant the layer cuts preserve
|
||||
- [C8](c08-node-contract.md) — the node contract (engine ↔ vocabulary seam)
|
||||
- [C10](c10-bias-r-cost.md) / [C13](c13-hot-reload-frozen-deploy.md) — the deploy edge (everything ↔ money)
|
||||
- [C16](c16-engine-project-split.md) — game/engine split, applied here inside the workspace
|
||||
- [C18](c18-registry.md) — the run registry (run ↔ process column seam)
|
||||
- [C23](c23-graph-compilation.md) — names are non-load-bearing
|
||||
- [C24](c24-blueprint-data.md) — blueprint-as-data (topology ↔ engine seam)
|
||||
- [C26](c26-input-binding.md) — input binding
|
||||
- [C27](c27-declared-taps.md) — declared taps (run ↔ analysis seam)
|
||||
|
||||
> History: [c28-stratification.history.md](c28-stratification.history.md)
|
||||
Reference in New Issue
Block a user