Aura/docs/design/contracts/c24-blueprint-data.history.md

# C24 — The blueprint is a serializable, World-owned data value (the topology data format): history

> FROZEN HISTORICAL RECORD. Each block below was true as of its cycle/date stamp and may be superseded; this file is NOT current truth and NOT a grounding surface. Current contract: [c24-blueprint-data.md](c24-blueprint-data.md).

**Status (2026-06-29; first cut shipped — cycle 0087 / #155, `d5602ec`;
construction service shipped — cycle 0088 / #157).** The
**principle** is settled (this contract, ratified in an in-context design discussion
— the #109 resolution). The **first cut now ships**: a `Composite` blueprint
serializes to a **canonical, versioned** data value (`format_version` envelope,
omit-defaults JSON) and **loads back** (data → blueprint → `FlatGraph`) via
`blueprint_to_json` / `blueprint_from_json` (`aura-engine::blueprint_serde`),
referencing nodes by **compiled-in type identity** through an **injected resolver**
whose concrete closed `match` over the `aura-std` vocabulary lives outside the engine
(`aura-std::std_vocabulary`) — the engine stays domain-free, no node registry
(invariant 9). Acceptance met: a serialized blueprint runs **bit-identical** (C1) to
its Rust-built twin. `model_to_json` (C9) remains the render half; this closes the
loop for the round-trippable vocabulary. **Cycle 0088 (#157) adds the
introspectable construction service**: a declarative, replayable by-identifier
op-script (`aura graph build` / `introspect` over a JSON op-list, the engine
`GraphSession` / `replay`) builds a runnable blueprint through validated ops — the
engine's construction gates split into *eager* per-op checks (name resolution, edge
kind-match, the double-wire arm, param bind, and acyclicity — a `connect` that would close a
cycle is rejected eagerly, #161) and *holistic* finalize checks (wiring
totality, param-namespace injectivity, root-role boundness), **both cadences calling
the same extracted predicates** (`edge_kind_check`, the shared resolution helpers,
`check_root_roles_bound` — no second validator) — plus build-free introspection over
the closed vocabulary (types, a node's ports/kinds, a partial document's unwired
slots). It **emits** the #155 blueprint; acceptance met: a graph built purely through
the ops compiles identical (C1) to its Rust-built twin, an invalid op is rejected at
the op naming the cause, introspection answers without a build. The engine `Op` stays
serde-free (the wire DTO is CLI-side); the vocabulary's enumerable companion
(`std_vocabulary_types`) lives in `aura-std` (no registry, invariant 9). **Cycle
0089 follow-up (#161 / #162, after the cycle-0088 fieldtest).** The eager acyclicity
gate above (`GraphSession::closes_cycle`, a reachability check at the closing
`connect`) closed a fieldtest-found hole where `graph build` accepted a non-DAG
op-list (invariant 5 / C9). **Lockstep:** it is a *second* home of invariant-5
alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two
must co-evolve when the explicit delay/register node — invariant-5's sole legal
feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be
rejected at construction. The holistic *finalize* faults now also read
**by-identifier** (#162): `finish()` translates the index-carrying `CompileError`s
(`UnconnectedPort` / `RoleKindMismatch` / `UnboundRootRole`) into by-identifier
`OpError` variants — still *calling* the unchanged holistic gates (the
no-second-validator lockstep preserved), only translating their result.
**Cycle 0090 (#156) codifies the forward-compat two-tier discipline.** Tier-1
(additive-optional) is serde-default silent-ignore with no `format_version` bump
(a new optional field defaults to prior behaviour, C1) — now proven by
`unknown_optional_field_is_tolerated_byte_identically` (an unknown optional key
loads byte-identically and runs bit-identically). Tier-2 (must-understand: a new
node type, edge semantics, or structural-axis kind) bumps `format_version` so an
old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`,
already green). The per-section required-flag scheme is deferred (no current
Tier-2 section to validate it; recorded on #156).
Pre-ship dormancy (#61, 2026-07-10): until the first external ship there are
no out-of-repo readers — reader and writer change atomically in one commit —
so the Tier-2 bump discipline is dormant and structurally-semantic additions
(the `gangs` section) land as additive-optional fields of v1; the first ship
consciously freezes v1, gangs included, and activates the bump discipline.

**Milestone delivered — 2026-06-30, cycle 0090 — the serializable format + loader + construction service.** A green end-to-end milestone fieldtest (`fieldtests/milestone-topology-as-data/`) proves the full author → serialize → load → construct → introspect → reproduce story from the public surface alone, 0 behavioural bugs; the round-trippable format (#155), the construction service (#157), and the forward-compat two-tier discipline (#156) all shipped. Polish filed forward: op-script grammar docs + a stale example (#163), the canonical trailing-newline / `Composite` value ergonomics (#164), CLI discoverability of `build`/`introspect` (#159).

**Realization (2026-06-30, cycle 0092 — runs are built FROM blueprint-data, #165).**
`aura run <blueprint.json>` now loads a serialized **signal** blueprint
(`blueprint_from_json` → the closed `std_vocabulary`; an unknown type fails clean as
`UnknownNodeType`, the data-plane face of invariant 9), wraps it in the r-sma run
scaffolding (sinks / broker / data supplied **at run**, not serialized — C24's deferred
set), runs it, and emits a `RunReport` **bit-identical** (C1) to its Rust-built twin —
proven by `loaded_signal_runs_bit_identical_to_rust_built`. The `RunManifest` now carries
a **`topology_hash`**: SHA256 of the canonical (#164) `blueprint_to_json`, the #158
reproducibility anchor, a Tier-1 optional field (#156). The hash + helper live
research-side (`aura-cli` + `sha2`), off the frozen engine (invariant 8). This is the
**keystone of the World/C21 milestone**: topology-as-data is now *runnable*, not only
serializable. The harness wrap was made shareable — `r_sma_graph()` =
`wrap_r(sma_signal(...))`, so the Rust path and the data path are the same seam
over the same signal (a behaviour-preserving C19/C23 restructure). Scope note: the hash
covers the **signal** only (the fixed scaffolding is identified by `commit`); a content-id
distinguishing harness-structural variants is a #158/#166 concern once scaffolding varies.

**Realization (2026-07-01, cycle 0093 — the World orchestrates FAMILIES from blueprint-data,
#166).** `aura sweep <blueprint.json> --axis <name>=<csv>` loads an **open** signal blueprint,
grids the by-name axes against its `param_space()`, builds each member through cycle-1's
`wrap_r` seam, and aggregates to a `FamilyKind::Sweep` family — byte-identical in shape
to the hard-wired sweep (`append_family` / `sweep_member_reports` reused verbatim). This is the
C21 step beyond a single run: the World now constructs and orchestrates **families** of
harnesses from topology-data, not just one. Each member manifest carries the **shared**
`topology_hash` (one signal topology, only params vary; `member_key` distinguishes members) —
reproducible per C18. Two design facts: a sweep needs an **open** blueprint (a fully-bound one
has an empty `param_space`, distinct from cycle-1's bound run fixture), and the signal is
**re-loaded per member** from its serialized doc (the `Composite` is `!Clone` — its
`PrimitiveBuilder`s hold `Box<dyn Fn>` build closures, #164 — and `bootstrap_with_cells`
consumes the graph). **Monte-Carlo over a loaded blueprint shipped (cycle 0095, #170).** `aura mc
<blueprint.json> --seeds N` runs a **closed** blueprint across N seeds — each seed a distinct
synthetic price walk (`synthetic_walk_sources`, the `mc_family` `SyntheticSpec` pattern), the
draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and aggregates to a
`FamilyKind::MonteCarlo` family with one stored blueprint per family (the C18 hook), so it
`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint
(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered
exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the
pure builder). **IS-refit walk-forward shipped (cycle 0097, #173).** `aura walkforward
<blueprint.json> --axis <name>=<csv> [--select argmax|plateau:mean|plateau:worst]` re-optimizes
the loaded blueprint's params over the user `--axis` grid (the #169 prefixed names) on each
24/12/12 IS window, selects the winner by `sqn_normalized` (the hard-wired arm's metric +
`select_winner` reused verbatim), runs it out-of-sample, and aggregates to a
`FamilyKind::WalkForward` family — persisted + content-addressed + `aura reproduce` bit-identical
(the read-side WalkForward branch rebuilds each OOS window from `manifest.window`). One
substitution deep from the hard-wired r-sma WF arm (the loaded blueprint via
`blueprint_axis_probe` replaces the built-in strategy); a bad `--axis` is a clean in-closure
exit 2, never a panic. This is Arm A (the settled direction): the loaded IS-optimizing form
honestly carries the `walkforward` name. Reduce-mode members are R-measured (`oos_r` the
meaningful summary; stitched pip-equity empty, C10). The synthetic-walk DGP
is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the
`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint
warms poorly → silent-vacuous draws today) — ride #172. **Axis-name discovery shipped (cycle
0096, #169):** `aura sweep <blueprint.json> --list-axes` lists a loaded blueprint's open
sweepable knobs (one `<name>:<kind>` per line, `param_space()` order), then exits — the names it
prints are exactly what `--axis` binds. Every listed name is **mandatory** on a `sweep` /
`walkforward`: the blueprint must be fully bound before it runs, so a subset grid is refused with
the missing knob named (`BindError::MissingKnob`) and there is no default — pin a knob you do not
want to vary with a single-value axis (`--axis <name>=<one-value>`). A single `blueprint_axis_probe` helper now single-sources
the wrapped probe (`wrap_r(loaded_signal).param_space()`) for the sweep terminal, the MC
closed-check, AND the listing (three former inline copies → one), so **listed == swept by
construction** (and stays so across #159's harness retirement — the listing tracks whatever the
sweep actually resolves, never a second source of truth). The names are prefixed by the current
r-sma wrapping (`sma_signal.fast.length`, the nested-composite prefix), not the raw
`param_space` — which is why the discovery lives on the sweep verb (it owns the wrapping), not
`graph introspect`. CLI `--trace` is refused on all verbs (#168 for sweep/walkforward; run/mc already refused); the live trace-writer is the campaign `presentation.persist_taps` (`persist_campaign_traces`), and restoring per-member CLI traces is tracked by #224. **[Delivered 2026-07-11 (#224): `sweep`/`walkforward --trace` write per-member traces on the real-data campaign path (depth-2 fan-out, chartable by the printed family handle); only the synthetic path still refuses.]**

**Content-addressed reproduction shipped (cycle 0094, #158, C18)**: `topology_hash`
landed cycle 0092; re-deriving a member's FlatGraph from a stored manifest + the
content-addressed blueprint store (`aura reproduce`) shipped cycle 0094 (see C18
Realization). What **remains** is a content-id that covers **structural-axis / whole-harness
variants** (the scaffolding is not yet blueprint-data); the debug-name-in-id question is
settled by cycle 0104's **identity id** (#171: `blueprint_identity_json` + `graph
introspect --identity-id`, an additive sibling — the byte-exact content id keeps the
store/reproduce roles; introspection-only until a dedup consumer exists). Still deferred: retiring the pre-C24
hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `*_sweep_family`) once
the project-as-crate layer lands (#159, paired with #157's data-authoring surface). **Out of the first cut's round-trippable set**
(deliberate; fails clean as `UnknownNodeType`, never a silent wrong graph): recording
sinks (capture an `mpsc::Sender` — runtime identity, not param-generic data, C19) and
construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / `Session` —
structural-axis args, a C20 concern), additively addable later (#156). The
**project-as-crate load boundary landed in cycle 0102** (`Aura.toml` discovery +
`cdylib` loading + merged project ∪ std vocabulary — see the C13 realization
note; the `aura new` scaffolder followed in cycle 0103 — one command emits a
buildable project whose blueprint runs through the merged vocabulary); of the
two layers this paragraph used to name as sequencing-coupled, what remains
open is the **composable-orchestration** thread (#109): topology-as-data is
the substrate it stands on. **Canonical project shape (#181, resolved
2026-07-02):** the `aura new` templates (`scaffold.rs`) are the canonical
authoring shape and evolve with the engine; the cycle-0102 `demo-project`
fixture is an intentionally frozen known-good twin for the load-boundary
tests. The two are deliberately **not** lockstep-guarded: no consumer requires
them to match, each is e2e-guarded on its fitness for purpose (build →
descriptor load → charter check → deterministic run — blueprint *wiring*
content is pinned in neither, stated honestly), and an equality guard would
convert every deliberate template improvement into forced churn of the frozen
fixture — the same cross-purpose coupling that rules out regenerating the
fixture from the scaffolder.

**[C26, 2026-07-10 (#231): the single-price data weld inside the surviving `wrap_r` scaffolding is retired — input roles bind archive columns by name; the wrapper's remaining R-scaffolding retirement stays #159.]** (From the C24 Forbids clause as of that date; the single-price weld was retired at #231/C26, but `wrap_r` itself survives — its full R-scaffolding retirement stays deferred, #159.)