Aura/docs/design/contracts/c24-blueprint-data.md

# C24 — The blueprint is a serializable, World-owned data value (the topology data format)

**Guarantee.** A **blueprint** — the param-generic, named graph-as-data a Rust
builder produces (C9/C19) — is a **first-class serializable data value** with a
stable, versioned format, and the engine has a **load path** (data → blueprint →
`FlatGraph`, C23), not only the Rust-builder construction path. Topology is
therefore a **value the World owns** (C21): constructed, **generated**, mutated,
**structurally searched**, serialized for **reproduction** (C18), and (optionally)
rendered or edited by a visual face (C22). `model_to_json` (C9) is the **existing
reverse half** (blueprint → data, for render); this contract closes the loop. The
format carries **topology + structural axes + the param-space**, referencing nodes
by their **compiled-in type identity** — the closed, typed node vocabulary of the
std set plus the project `cdylib` (C8/C16) — and carries **no node logic**
(computation is compiled Rust, C17). It is **non-Turing-complete by construction**
(a static DAG, no control flow), so it structurally cannot become a second RustAst.

**Forbids.** Putting **node logic / computation** in the format (the RustAst trap —
logic is Rust, C17); a **Turing-complete or control-flow-bearing** blueprint format
(it is static data; the *generator* that emits it may be Rust, C20, but the artifact
is not a language); a **by-name node marketplace / registry** (the vocabulary is the
compiled-in closed set referenced by type identity — the format is not a distribution
mechanism, C9/C16); treating the **visual face (C22) as the motivation or the
authoring brain** (the format is required by C21/C18 at the CLI level, independent of
any UI); a **generated / searched run whose topology is not recoverable from its
manifest** (C18 reproduction); baking topology as **engine source** compiled into the
binary. The hard-wired `aura-cli` harnesses (`HarnessKind`, `run_r_sma`, `r_sma_graph`,
`sample_blueprint_with_sinks`, the built-in `*_sweep_family`) were pre-C24 scaffolding,
a C16 tension, and are now **retired** (#159/#220): topology reaches the engine only as
data, resolved by the injected vocabulary. The `wrap_r` R-scaffolding wrap
(`aura-runner`) **survives** — it is the r-sma wrap every `run` / `sweep` / `mc` /
`walkforward` path applies to the loaded signal at run (supplied at run, not serialized);
its full R-scaffolding retirement stays deferred (#159). Harness input roles bind archive
columns **by name**, not by a single baked-in price weld (C26).

**Why.** The World (C21) is the product, and it cannot orchestrate, **structurally
search**, or **reproduce** *families that vary topology* while topology stays opaque
Rust source baked into the engine. The **game-engine principle** (C16's engine / game
split) makes it concrete: the engine is compiled native systems; the "game" is
**content** — and topology *is* content, a data value the engine owns, serializes,
instantiates, and mutates, exactly as a game engine owns its scene / prefab /
blueprint graphs. The #109 fork was never "*who types the wiring*" — a small LLM
cannot reliably *apply* a DSL and a strong one does not need one, so a
manifest-as-authoring-surface dies on both horns (C17). It is "**is topology a
compile-time *source* artifact or a runtime, serializable, World-owned *value*?**",
and determinism (C1) + reproduction (C18) + structural search (the open search-policy
thread) force the **value**. The engine already treats topology as a runtime value
(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the
flat graph as the optimisation target); C24 only adds that the value **serializes out
of Rust and loads back in**.

## Current state

The **principle** is settled — the #109 resolution, ratified in an in-context design
discussion. The format, its loader, the construction service, and the family
orchestration all ship. The acceptance criterion is green: a serialized blueprint
runs **bit-identical** (C1) to its Rust-built twin.

### The format and load path

A `Composite` serializes to a **canonical, versioned** data value — a
`format_version` envelope (`BlueprintDoc`), omit-defaults JSON — and **loads back**
(data → blueprint → `FlatGraph`) via `blueprint_to_json` / `blueprint_from_json`
(`aura-engine::blueprint_serde`). Nodes are referenced by **compiled-in type
identity** through an **injected resolver** whose concrete closed `match` over the
std vocabulary lives outside the engine (`aura-vocabulary::std_vocabulary`), so the
engine stays domain-free, with no node registry (invariant 9). An unknown type fails
clean as `LoadError::UnknownNodeType` — the data-plane face of invariant 9, never a
silent wrong graph. `model_to_json` (C9) remains the render half; this closes the
loop for the round-trippable vocabulary.

**Out of the round-trippable set** (deliberate; fails clean as `UnknownNodeType`):
recording sinks (they capture an `mpsc::Sender` — runtime identity, not param-generic
data, C19) and `SimBroker` (a scalar-typed baked value, a narrower gap than the args
channel). `LinComb` / `CostSum` / `Session` — formerly listed here — entered the
round-trippable set with #271's typed construction args (see the add-op `args`
clause and the data-driven `format_version` below).

### Runs and families are built FROM blueprint-data

`aura run <blueprint.json>` loads a serialized **signal** blueprint and emits a
`RunReport` **bit-identical** (C1) to its Rust-built twin; the run scaffolding (sinks
/ broker / data) is supplied **at run**, not serialized. Beyond a single run, the
World constructs and orchestrates **families** of harnesses from topology-data:

- `aura sweep <blueprint.json> --axis <name>=<csv>` → `FamilyKind::Sweep`. A sweep
  needs an **open** blueprint (a fully-bound one has an empty `param_space`).
- `aura mc <blueprint.json> --seeds N` → `FamilyKind::MonteCarlo`, each seed a
  distinct synthetic price walk drawn disjoint-parallel through the engine
  `monte_carlo` seam (invariant 1). MC binds no axis, so it needs a **closed**
  blueprint (the sweep's distinction inverted); an open one returns a named `Err`,
  rendered exit-2 at the builder boundary — no hidden exit in the pure builder.
- `aura walkforward <blueprint.json> --axis <name>=<csv> [--select
  argmax|plateau:mean|plateau:worst]` → `FamilyKind::WalkForward`: re-optimizes the
  loaded blueprint's params over the `--axis` grid on each 24/12/12 IS window,
  selects the winner by `sqn_normalized`, runs it out-of-sample. Reduce-mode members
  are R-measured (`oos_r` the meaningful summary; stitched pip-equity empty, C10).

The family builders live in `aura-runner::family` (`blueprint_sweep_family` /
`blueprint_mc_family` / `blueprint_walkforward_family`). Every member manifest carries
the **shared** `topology_hash` (one signal topology, only params vary; `member_key`
distinguishes members), and each family stores its blueprint(s) content-addressed so
`aura reproduce` re-derives every member bit-identically (`aura-runner::reproduce`,
C18). The synthetic-walk DGP is the MC machinery, not trader-grade statistics; a
real-data block-bootstrap — and retiring the `synthetic_walk_sources` `len:60`↔warm-up
coupling — rides #172.

### Reproduction identity

The `RunManifest` carries a **`topology_hash`**: SHA256 of the canonical
`blueprint_to_json` (see *Canonical form*), the #158 reproducibility anchor, a Tier-1
optional field (#156). The hash and its helper live research-side (`aura-cli` +
`sha2`), off the frozen engine (invariant 8). The hash covers the **signal** only —
the fixed scaffolding is identified by `commit`; a content-id distinguishing
structural-axis / whole-harness variants is a #158/#166 concern once scaffolding is
itself blueprint-data. The **identity id** (#171) is an additive sibling:
`blueprint_identity_json` (`aura-engine::blueprint_serde`) + `graph introspect
--identity-id`. The byte-exact content id keeps the store/reproduce roles; the
identity id is introspection-only until a dedup consumer exists.

### Axis discovery

`aura sweep <blueprint.json> --list-axes` lists a loaded blueprint's open sweepable
knobs (one `<name>:<kind>` per line, `param_space()` order) and exits; the printed
names are exactly what `--axis` binds. Every listed name is **mandatory** on `sweep` /
`walkforward` — the blueprint must be fully bound before it runs — so a subset grid is
refused with the missing knob named (`BindError::MissingKnob`, `aura-engine`) and
there is no default; pin a knob you do not want to vary with a single-value axis. A
single `blueprint_axis_probe` (`aura-runner`) single-sources the wrapped probe for the
sweep terminal, the MC closed-check, and the listing, so **listed == swept by
construction** (and stays so across the harness retirement — the listing tracks
whatever the sweep actually resolves). One raw namespace, `<node>.<param>` (a splice
path keeps its interior path, e.g. `anchor.sess.period_minutes`), is the only
user-facing axis name (#328): `graph introspect --params` and `--list-axes` are
line-identical (open params bare, bound params with `default=`) and both print
exactly what `--axis` binds, on both sweep routes. The wrapped
`<blueprint>.<node>.<param>` form the probe still resolves against internally is
retired from the surface, with a translation refusal naming the raw candidate on
both intake seams (`--axis` and `campaign validate`). `--trace` on
`sweep` / `walkforward` writes per-member traces on the real-data campaign path
(depth-2 fan-out, chartable by the printed family handle, #224); the synthetic path
still refuses (`run` / `mc` refuse `--trace` outright). The live trace-writer is the
campaign `presentation.persist_taps` (`persist_campaign_traces`, `aura-runner::runner`).

### The construction service (op-script)

`aura graph build` / `introspect` over a JSON op-list (#157) is the introspectable,
replayable, by-identifier construction surface; it **emits** the serializable
blueprint. The engine `GraphSession` / `replay` (`aura-engine::construction`) build a
runnable blueprint through validated ops. Construction gates split into **eager**
per-op checks — name resolution, edge kind-match (`edge_kind_check`), the double-wire
arm, param bind, and acyclicity (a `connect` that would close a cycle is rejected
eagerly at the closing op, `GraphSession::closes_cycle`, #161) — and **holistic**
finalize checks — wiring totality (`validate_wiring`), param-namespace injectivity
(`check_param_namespace_injective`). Both cadences call the **same extracted
predicates** — no second validator. The holistic faults read **by-identifier**
(#162): `finish()` translates the index-carrying `CompileError`s (`UnconnectedPort` /
`RoleKindMismatch`) into by-identifier `OpError` variants while still *calling* the
unchanged holistic gates. **Root-role boundness moved off this list (#317, open
patterns):** `check_root_roles_bound` no longer runs in `finish()` — a `finish()`ed
op-script may still carry an open (unbound) root role, declaring a reusable
pattern's formal parameter; only `compile`/bootstrap (the runnability gate, not a
construction gate) still call it, unchanged. Build-free introspection answers over
the closed vocabulary: `graph introspect --vocabulary | --node <T> | --unwired`. The
engine `Op` stays serde-free; the wire DTO (`OpDoc`) is CLI-side
(`aura-cli::graph_construct`). Acceptance: a graph built purely through the ops
compiles identical (C1) to its Rust-built twin, an invalid op is rejected at the op
naming the cause, introspection answers without a build.

**Registered-blueprint splice (#317).** The `use` op resolves a reference — a store
content id, a unique content-id prefix (#302 semantics), or a registry **name
label** (`graph register --name`, latest-wins, `aura-registry`'s
`blueprint_names.jsonl` sidecar) — to a previously registered blueprint and
splices it into the building graph as a nested `BlueprintNode::Composite` under an
instance identifier. The **engine never resolves**: `GraphSession`/`replay` take a
second injected closure, `subgraph: &dyn Fn(&str) -> Option<Composite>`, mirroring
`vocab`'s closed-lookup posture exactly — this is the same enforcement-shift
permission §"Enforcement shift" above already grants the node vocabulary, extended
to registered-blueprint references. All store I/O — label/prefix resolution, the
`get_blueprint` fetch, the C29 doc-gate re-walk over the fetched composite (before
it ever reaches the session, so a doc-less fetched entry cannot enter a NEW
composition — a **backstop**: the register verb already gates both input forms, so
this fires only for store content written before C29 or through the raw in-crate
path), and the resolution echo — happens CLI-side, at DTO conversion, before
replay; build-free introspection paths pass `&|_| None`. The echo
(`aura: note: use "<instance>": <label-or-prefix> -> <full id>`) is the existing
`aura: note:` benign-diagnostic marker (C14) — a new instance of the existing
class, not a new one, so the exit-code/marker taxonomy is unchanged.

**Invariant-5 lockstep.** The eager acyclicity gate is a *second* home of invariant 5
alongside the bootstrap Kahn sort (`harness.rs`, `BootstrapError::Cycle`); the two
must co-evolve when the explicit delay/register node — invariant 5's sole legal
feedback — lands, or a valid delay-feedback graph the bootstrap accepts would be
rejected at construction.

### Op-script grammar

The op-script is a JSON **array of ops**, each object internally tagged by `"op"`,
replayed in order; nodes are referenced **by identifier**, ports as dotted
`<identifier>.<port>`. The eleven verbs:

- `name` — `{"op":"name","name":<str>}` — set the composite's render name
  (#331), script-level and at-most-once (a second refuses); omitted, the
  built composite keeps the CLI's own seed default (`"graph"`). Gated by a
  shared shape check (non-empty, single path segment, no `/`, `\`, `.` or
  `..`) applied at this op intake **plus every CLI intake that reads an
  authored blueprint envelope from a file** — `register`, `introspect
  --content-id <FILE>`, the bare graph-file viewer, `run`, `introspect
  --params <FILE>`, and each of `sweep`/`walkforward`/`mc`/`generalize`'s
  file-reading entry points (synthetic and `--real` alike) — one class of
  intake, one gate. Store read-back (`reproduce`, `use` resolution,
  `introspect`/`--params` by content id) stays deliberately ungated — C29: a
  registered artifact is never retroactively invalidated.
- `source` — `{"op":"source","role":<str>,"kind":<ScalarKind>}` — declare a root
  source role producing a base column of `kind`.
- `input` — `{"op":"input","role":<str>}` — declare a root input role (kind inferred
  from the slots it feeds) — a reusable **pattern**'s formal parameter, left open
  until an enclosing graph wires it (#317's open patterns): `finish()` accepts an
  op-script that never binds it, only `compile`/bootstrap require it bound.
- `add` — `{"op":"add","type":<TypeId>,"name":<str>?,"args":{<arg>:<str>}?,"bind":{<param>:<Scalar>}?}` —
  add a node of compiled-in type identity `type`; **`name` is its identifier**
  (mirrors the builder's `.named(...)`; defaults to the lowercased type label, so
  two unnamed nodes of one type collide); `args` (#271) configures an **arg-bearing**
  type's structural, non-scalar construction (`Session`'s IANA timezone/open,
  `LinComb`/`CostSum`'s arity) through the closed, load-time-validated `ArgKind`
  table (`Tz`/`TimeOfDay`/`Count`) — applied BEFORE `bind`, so a bound param
  `args` unlocks (e.g. `LinComb`'s `weights[i]`) is only bindable once `arity` is
  consumed; an arg-bearing type given no `args` refuses (`BadArg(MissingArg)`), a
  `Plain` type given `args` it does not declare also refuses
  (`BadArg(NotArgBearing)`); `bind` sets the (now real) params.
- `feed` — `{"op":"feed","role":<str>,"into":[<port>,…]}` — fan a root role into
  interior input slots.
- `connect` — `{"op":"connect","from":<port>,"to":<port>}` — wire an interior output
  field to an input slot; a `connect` closing a cycle is rejected eagerly
  (invariant 5).
- `expose` — `{"op":"expose","from":<port>,"as":<str>}` — promote an interior output
  field to a boundary output, aliased by `as` (a real alias, not a naming; contrast
  `add`'s `name`) — one of the two verbs that keep `as` (with `tap`).
- `tap` — `{"op":"tap","from":<port>,"as":<str>}` — declare a measurement **tap** on
  an interior output field under `as` (#284) — the output-side twin of `expose`: a
  recorded observation point (a `Composite.taps` entry, C27), not a boundary output.
  Name-addressed like every other op (no raw index); tap names are their own
  namespace (a duplicate refuses). The `finish` gate threads op-declared taps into
  the built `Composite` (`.with_taps`); a single `aura run` records each, a sweep
  leaves them inert.
- `gang` — `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}`
  — fuse two or more sibling params into ONE public knob: the member addresses
  leave the sweepable param space and `as` replaces them; the bound or swept
  value fans out to every member at bootstrap. Members must share one scalar
  kind and stay open (un-bound).
- `doc` — `{"op":"doc","text":<str>}` — declare the composite's one-line meaning
  (C29, #316) — the op-script twin of the builder's `.doc(...)`; at most one per
  script, a second is a fault (a declarative script carries no last-wins
  ambiguity).
- `use` — `{"op":"use","ref":{"content_id":<str>}|{"name":<str>},"name":<str>?,"bind":{<path>:<Scalar>}?}`
  — splice a **registered** blueprint into the building graph as a nested
  composite under an instance identifier (`name`, defaulting to the fetched
  composite's own name); `bind` path-qualifies the spliced instance's params
  (e.g. `"sma.length"`), applied after the splice (#317). An instance's open
  input roles become its `<instance>.<role>` in-ports, its output boundary
  aliases its `<instance>.<field>` out-fields — the same `Composite` arm every
  other add/wiring path walks, not a new mechanism. See §"The construction
  service" below for the resolution split.

Value forms are the serialized representations: a `Scalar` bind value is the typed tag
form `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` / `{"Timestamp":<i64>}`, and `kind`
is the capitalized `ScalarKind` (`"F64"`). Worked example:
`fieldtests/cycle-0088-construction-op-script/c0088_1_sma_crossover.json` (an
SMA-crossover bias).

### Canonical form (#164)

The canonical blueprint artifact is exactly the bytes `blueprint_to_json` returns — a
JSON value with **no trailing newline** (`serde_json::to_string`, not framed with a
display `println!`), so the CLI and library emit paths are **byte-identical** — a
prerequisite for content-addressed topology identity (#158, a hash over the canonical
form). The canonical JSON is also the blueprint's **equality / identity surface**:
`Composite` (`aura-engine::blueprint`) and `PrimitiveBuilder` (`aura-core::node`)
deliberately carry **no in-memory `PartialEq` / `Debug`** — the recipe holds a
`build: Box<dyn Fn>` closure (non-comparable, non-printable), so equality could only be
defined over the serialized *data*, of which `blueprint_to_json` is already the single
source; a second in-memory notion would be a drift hazard against the very form #158
content-addresses. The loader stays lenient (a trailing newline on input still parses,
Tier-1 robustness).

### Forward-compat: the two-tier discipline (#156)

Tier-1 (additive-optional) is serde-default silent-ignore with **no** `format_version`
bump — a new optional field defaults to prior behaviour (C1). Tier-2 (must-understand:
a new node type, edge semantics, or structural-axis kind) **bumps** `format_version` so
an old reader refuses cleanly (`LoadError::UnsupportedVersion` / `UnknownNodeType`).
**Pre-ship dormancy (#61, ended by #271):** until the first external ship there were
no out-of-repo readers — reader and writer changed atomically in one commit — so the
Tier-2 bump discipline lay dormant and structurally-semantic additions (the `gangs`
section) landed as additive-optional fields of v1. #271's construction args are the
first exercised Tier-2 bump (data-driven, next paragraph); v1 remains the version of
every args-free document, gangs included. The per-section required-flag scheme stays
deferred (no per-section case yet; recorded on #156).

**First data-driven bump (#271):** construction args are the first Tier-2 case whose
version is decided PER DOCUMENT, not a single global bump. The writer emits
`format_version: 1` for an args-free document (byte-identical to every pre-#271
document — content ids stable) and `2` the moment any primitive, at any composite
nesting depth, carries `args` — the must-understand signal an old (pre-#271) reader
needs, since it cannot construct a pending arg-bearing type at all. The loader accepts
the closed range `1..=2`.

### Enforcement shift — invariant 9 on the data plane

Lifting topology onto the data plane relocates *where* the engine/project boundary
(invariant 9, no bespoke node registry/marketplace) is enforced, without changing what
it forbids. **Pre-C24** a blueprint referencing another project's node was a **compile
error** — topology was in-process Rust, so cross-boundary node resolution was
*structurally impossible*, compiler-guaranteed for free. **Post-C24** that same
reference is a **type-id string in a serialized blueprint**, refused only by the
injected resolver's closed set (`UnknownNodeType`). The anti-NIH rationale survives
intact (node *logic* stays cargo+Gitea-only; the format distributes
**topology-as-content**, never logic — C17), and `std_vocabulary` stays a compiled-in
dispatch table, not a registry — but its enforcement **degrades from
compiler-guaranteed to resolver-seam discipline**. Two consequences:

1. **#160** (the guard that the closed vocabulary stays honest) is no longer droppable
   hygiene — it guards the data-plane face of the boundary (its own drift direction
   still fails *safe*, `UnknownNodeType`). The `std_vocabulary_roster!` macro
   (`aura-vocabulary`) expands the resolver `match` and the enumerable
   `std_vocabulary_types()` list from **one roster**, closing the resolver-vs-list
   drift mode by construction; the residual (a new zero-arg node never rostered at all)
   stays fail-safe, guarded by the one-line maintainer surface plus the count pins (the
   in-crate shape test and aura-cli's cross-boundary `--vocabulary` count e2e). The
   per-project **merged resolver** (project ∪ std, project first) lives in
   `aura-runner::project`.
2. The **injected per-project resolver** (C16) is the genuinely new, *deferred*
   invariant-9 surface: nothing structurally stops a project resolver from accepting
   arbitrary type-ids, or a project from layering a blueprint *marketplace* (store +
   type-discovery API) **on top of** the format — one layer *above* the engine
   contract. Invariant 9 holds **at the engine**; the project/ecosystem boundary (what
   an injected resolver may resolve, where closed-set discipline draws the line) is a
   **World/C21-layer charter point**, not an engine fix.

Invariant 8 (frozen) is likewise reframed by C24/C21: from "topology baked at build
time" to "the World may *generate* topology at runtime (research plane), the chosen
blueprint is frozen into the deploy artifact, and any run is deterministic once
instantiated" (C1).

### Adjacent / deferred

The **project-as-crate load boundary** has landed (`Aura.toml` discovery + `cdylib`
loading + merged project ∪ std vocabulary; the `aura new` scaffolder emits a buildable
project whose blueprint runs through the merged vocabulary) — see C13/C16. **Canonical
project shape (#181):** the `aura new` templates (`scaffold.rs`) are the canonical
authoring shape and evolve with the engine; the `demo-project` fixture is an
intentionally frozen known-good twin. The two are deliberately **not** lockstep-guarded
— no consumer requires them to match, each is e2e-guarded on its fitness for purpose
(build → descriptor load → charter check → deterministic run; blueprint *wiring* content
is pinned in neither, stated honestly), and an equality guard would convert every
deliberate template improvement into forced churn of the frozen fixture (the same
cross-purpose coupling that rules out regenerating the fixture from the scaffolder).
What remains open is the **composable-orchestration** thread (#109): topology-as-data is
the substrate it stands on.

## See also

- [C9](c09-fractal-composition.md) — graph-as-data; `model_to_json` render half
- [C16](c16-engine-project-split.md) — engine / project split; the injected resolver
- [C17](c17-authoring-surface.md) — logic is Rust; the closed-vocabulary-data-artifact clarification
- [C18](c18-registry.md) — reproduction, the run registry, content-addressed store
- [C19](c19-bootstrap.md) — bootstrap: blueprint → frozen instance
- [C21](c21-world.md) — the World owns and orchestrates topology
- [C23](c23-graph-compilation.md) — bootstrap-as-compilation to `FlatGraph`; the optimisation target
- [C26](c26-input-binding.md) — role names bind archive columns
- [C27](c27-declared-taps.md) — declared taps as named measurement points
- [C1](c01-determinism.md) — determinism / bit-identity; [C10](c10-bias-r-cost.md), [C13](c13-hot-reload-frozen-deploy.md), [C20](c20-strategy-harness.md), [C22](c22-playground-traces.md), [C25](c25-role-model.md), [C28](c28-stratification.md)

> History: [c24-blueprint-data.history.md](c24-blueprint-data.history.md)