audit(0087): cycle close — drift-clean after C24/C9 realization note; rm ephemeral spec/plan
Architect drift review of cycle 0087 (#155, the C24 blueprint-serialization-and- loader first cut). What holds (confirmed against the diff): C17 / no-RustAst (the format carries only type identity + name + bound params + topology; the build closure is dropped and re-derived, no node logic in the data); invariant 9 / domain-free engine (injected resolver, the closed match in aura-std, engine lib deps aura-core + aura-analysis only — a compiled-in closed set, not a dynamic registry); C9 round-trip / C23 bootstrap untouched (loader's terminal step is the unchanged Composite::new -> bootstrap_with_params; validation deferred to the existing gate, not bypassed). Resolution: - fix (ledger): C24 Status refreshed from "unbuilt" to "first cut shipped" with a cycle-0087 realization note; C9's round-trips-both-ways refinement marked realised. The ledger no longer contradicts the code. - backlog (debt-med): the hand-written std_vocabulary match has no guard test that fails when a new zero-arg aura-std node lacks an arm — fails safe today (clean UnknownNodeType, never silent-wrong). Filed as #160. - carry-on (scaling): construction-arg builders + sinks excluded, so no real harness round-trips yet; harness retirement blocks on the construction + project-as-crate layers — already tracked #156/#157/#159. No architect-sweep / regression script is declared in the project facts, so the suite is the gate: cargo test --workspace 748 passed (737 -> 748 = the 11 new tests); clippy --all-targets -D warnings clean. Ephemeral cycle artifacts removed (lifecycle convention): docs/specs/0087-*, docs/plans/0087-*.
This commit is contained in:
+25
-10
@@ -486,7 +486,8 @@ blueprint is not only *emitted* as data (`model_to_json`, the render half) but i
|
||||
itself a **serializable data value with a load path** (data → blueprint →
|
||||
`FlatGraph`), so topology is a value the World generates, stores, and reproduces —
|
||||
see C24. The Rust builder API stays the primary *human / LLM* authoring surface
|
||||
(C17); the data form is the *machine* surface the World owns.
|
||||
(C17); the data form is the *machine* surface the World owns. (Load path **realised**
|
||||
cycle 0087 / #155, `d5602ec`: `aura-engine::blueprint_from_json`.)
|
||||
|
||||
### C10 — Strategy output is a bias stream; signal quality is measured in R; cost is a composable downstream graph (gross R → net R); money is decoupled to the live deploy edge
|
||||
**Guarantee.** A strategy's primary, backtestable output is **not** an equity
|
||||
@@ -1779,16 +1780,30 @@ thread) force the **value**. The engine already treats topology as a runtime val
|
||||
(C9 graph-as-data, C19 "cheap graph re-compilation, not a code recompile", C23 the
|
||||
flat graph as the optimisation target); C24 only adds that the value **serializes out
|
||||
of Rust and loads back in**.
|
||||
**Status (2026-06-29).** The **principle** is settled (this contract, ratified in an
|
||||
in-context design discussion — the #109 resolution). The **format itself** — the
|
||||
concrete data structure, serialization, the data → blueprint loader, and
|
||||
content-addressing for identity (C18) — is **unbuilt**, the next brainstorm /
|
||||
milestone. `model_to_json` (C9) is the one-directional half that exists.
|
||||
Sequencing-coupled to the **project-as-crate authoring layer** (`aura new` /
|
||||
**Status (2026-06-29; first cut shipped — cycle 0087 / #155, `d5602ec`).** The
|
||||
**principle** is settled (this contract, ratified in an in-context design discussion
|
||||
— the #109 resolution). The **first cut now ships**: a `Composite` blueprint
|
||||
serializes to a **canonical, versioned** data value (`format_version` envelope,
|
||||
omit-defaults JSON) and **loads back** (data → blueprint → `FlatGraph`) via
|
||||
`blueprint_to_json` / `blueprint_from_json` (`aura-engine::blueprint_serde`),
|
||||
referencing nodes by **compiled-in type identity** through an **injected resolver**
|
||||
whose concrete closed `match` over the `aura-std` vocabulary lives outside the engine
|
||||
(`aura-std::std_vocabulary`) — the engine stays domain-free, no node registry
|
||||
(invariant 9). Acceptance met: a serialized blueprint runs **bit-identical** (C1) to
|
||||
its Rust-built twin. `model_to_json` (C9) remains the render half; this closes the
|
||||
loop for the round-trippable vocabulary. **Remaining** (each its own milestone
|
||||
issue): the forward-compat *must-understand* gate (#156); the introspectable
|
||||
construction service that *emits* blueprints (#157); content-addressed topology
|
||||
identity in the manifest (#158, C18); retiring the pre-C24 hard-wired `aura-cli`
|
||||
harnesses (`HarnessKind`, `run_stage1_r`, `*_sweep_family`) once the construction +
|
||||
project-as-crate layers land (#159). **Out of the first cut's round-trippable set**
|
||||
(deliberate; fails clean as `UnknownNodeType`, never a silent wrong graph): recording
|
||||
sinks (capture an `mpsc::Sender` — runtime identity, not param-generic data, C19) and
|
||||
construction-arg builders (`LinComb` / `CostSum` / `SimBroker` / `Session` —
|
||||
structural-axis args, a C20 concern), additively addable later (#156). Still
|
||||
sequencing-coupled to the **project-as-crate authoring layer** (`aura new` /
|
||||
`Aura.toml` / `cdylib` loading, C16/C17) and the **composable-orchestration** thread
|
||||
(#109): topology-as-data is the substrate both stand on. The pre-C24 hard-wired
|
||||
harnesses (`HarnessKind`, `run_stage1_r`, the per-strategy `*_sweep_family` in
|
||||
`aura-cli`) are scaffolding retired as it lands.
|
||||
(#109): topology-as-data is the substrate both stand on.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,667 +0,0 @@
|
||||
# Blueprint serialization & loader (cycle 0087) — Implementation Plan
|
||||
|
||||
> **Parent spec:** `docs/specs/0087-blueprint-serialization-loader.md`
|
||||
>
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
||||
|
||||
**Goal:** Close the C24 graph-as-data loop (#155): a canonical, versioned
|
||||
serialization of a param-generic `Composite` blueprint plus a `data → blueprint`
|
||||
loader, so a serialized blueprint runs bit-identical (C1) to its Rust-built twin.
|
||||
|
||||
**Architecture:** A serde DTO layer in a new `aura-engine` module projects a
|
||||
`&Composite` to canonical JSON (the build closures dropped) and reconstructs it
|
||||
through an **injected** node resolver; the concrete closed `match` over the
|
||||
`aura-std` vocabulary lives in `aura-std` (the engine never depends on the node
|
||||
vocabulary crate — its only library deps are `aura-core` + `aura-analysis` +
|
||||
serde; `aura-std` is a test-only dep). Sinks and structural-construction-arg
|
||||
nodes are out of the round-trippable set (see spec; recorded on #155).
|
||||
|
||||
**Tech Stack:** `serde` + `serde_json` (both already deps of `aura-engine`;
|
||||
`serde` derive already in `aura-core`), the existing `Composite` /
|
||||
`PrimitiveBuilder` / `compile_with_params` machinery.
|
||||
|
||||
---
|
||||
|
||||
**Files this plan creates or modifies:**
|
||||
|
||||
- Modify: `crates/aura-core/src/scalar.rs:14` — add serde derive to `ScalarKind`
|
||||
- Modify: `crates/aura-core/src/node.rs:91` — add serde derive to `BoundParam`
|
||||
- Modify: `crates/aura-engine/src/harness.rs:29,38` — add serde derive to `Edge`, `Target`
|
||||
- Modify: `crates/aura-engine/src/blueprint.rs:29,126` — add serde derive to `OutField`, `Role` (`source` skip-when-None)
|
||||
- Create: `crates/aura-std/src/vocabulary.rs` — `std_vocabulary(type_id)` closed match (22 zero-arg builders)
|
||||
- Modify: `crates/aura-std/src/lib.rs` — `mod vocabulary;` + `pub use vocabulary::std_vocabulary;`
|
||||
- Create: `crates/aura-engine/src/blueprint_serde.rs` — DTOs, `BLUEPRINT_FORMAT_VERSION`, `blueprint_to_json`, `blueprint_from_json`, `SerializeError`, `LoadError`
|
||||
- Modify: `crates/aura-engine/src/lib.rs:45-79` — `mod blueprint_serde;` + re-exports
|
||||
- Modify: `crates/aura-engine/src/test_fixtures.rs` — add `sink_free_sma_cross_signal()` fixture
|
||||
- Test: `crates/aura-core/src/scalar.rs` (tests) — `ScalarKind` serde = variant-name
|
||||
- Test: `crates/aura-std/src/vocabulary.rs` (tests) — resolver hits/misses
|
||||
- Test: `crates/aura-engine/src/blueprint_serde.rs` (tests) — canonical golden, named failures, bit-identical run, idempotence, recursion
|
||||
|
||||
Note on the spec's type-list: `Firing` (spec Components table) is **not** in the
|
||||
serialized closure — the DTO carries only `type_id`/`name`/`bound`; `Firing`
|
||||
lives in `NodeSchema`/`PortSpec`, which are never serialized (schemas are
|
||||
re-derived from the resolver). It gets no serde derive. The spec also mis-locates
|
||||
`ScalarKind` in `node.rs`; it is in `scalar.rs:15`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: serde derives on the plain serialized types
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-core/src/scalar.rs:14` (`ScalarKind`), `crates/aura-core/src/node.rs:91` (`BoundParam`)
|
||||
- Modify: `crates/aura-engine/src/harness.rs:29` (`Edge`), `:38` (`Target`)
|
||||
- Modify: `crates/aura-engine/src/blueprint.rs:29` (`OutField`), `:126` (`Role`)
|
||||
- Test: `crates/aura-core/src/scalar.rs` (tests module)
|
||||
|
||||
- [ ] **Step 1: Add the serde encoding test (RED)**
|
||||
|
||||
In `crates/aura-core/src/scalar.rs`, inside the existing `#[cfg(test)] mod tests`
|
||||
block (the one holding `scalar_serde_round_trips`), add:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn scalar_kind_serde_is_variant_name() {
|
||||
// canonical kind encoding: a bare variant-name string, both directions.
|
||||
assert_eq!(serde_json::to_string(&ScalarKind::I64).unwrap(), "\"I64\"");
|
||||
assert_eq!(serde_json::to_string(&ScalarKind::F64).unwrap(), "\"F64\"");
|
||||
let back: ScalarKind = serde_json::from_str("\"I64\"").unwrap();
|
||||
assert_eq!(back, ScalarKind::I64);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails to compile**
|
||||
|
||||
Run: `cargo test -p aura-core scalar_kind_serde_is_variant_name`
|
||||
Expected: FAIL — compile error, `ScalarKind` does not implement `Serialize`.
|
||||
|
||||
- [ ] **Step 3: Add the derives**
|
||||
|
||||
`crates/aura-core/src/scalar.rs:14` — change
|
||||
`#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]` (on `ScalarKind`) to:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
|
||||
```
|
||||
|
||||
`crates/aura-core/src/node.rs:91` — change `#[derive(Clone, Debug, PartialEq)]`
|
||||
(on `BoundParam`) to:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
|
||||
```
|
||||
|
||||
`crates/aura-engine/src/harness.rs:29` (`Edge`) and `:38` (`Target`) — change each
|
||||
`#[derive(Clone, Copy, Debug, PartialEq, Eq)]` to:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
```
|
||||
|
||||
`crates/aura-engine/src/blueprint.rs:29` (`OutField`) — change
|
||||
`#[derive(Clone, Debug, PartialEq, Eq)]` to:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
```
|
||||
|
||||
`crates/aura-engine/src/blueprint.rs:126` (`Role`) — change
|
||||
`#[derive(Clone, Debug, PartialEq, Eq)]` to the line below, and add the field
|
||||
attribute on `source`:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct Role {
|
||||
pub name: String,
|
||||
pub targets: Vec<Target>,
|
||||
/// `None` = an open interior port (wired by the enclosing graph's edges);
|
||||
/// `Some(kind)` = a bound ingestion feed of `kind` (only meaningful at the root,
|
||||
/// where it lowers to a `FlatGraph` source). C3: sources bind at ingestion only.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub source: Option<ScalarKind>,
|
||||
}
|
||||
```
|
||||
|
||||
(Deriving serde directly on `Role` — rather than the spec's separate `RoleData`
|
||||
mirror — yields the identical wire bytes with one fewer type; the format is
|
||||
unchanged. `Edge`/`Target`/`OutField`/`BoundParam` likewise serialize 1:1, so the
|
||||
DTO embeds the real types.)
|
||||
|
||||
- [ ] **Step 4: Run the test to verify it passes**
|
||||
|
||||
Run: `cargo test -p aura-core scalar_kind_serde_is_variant_name`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Build the engine to confirm the engine-side derives compile**
|
||||
|
||||
Run: `cargo build -p aura-engine`
|
||||
Expected: clean build (0 errors).
|
||||
|
||||
---
|
||||
|
||||
### Task 2: the closed aura-std node-vocabulary resolver
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/aura-std/src/vocabulary.rs`
|
||||
- Modify: `crates/aura-std/src/lib.rs` (`mod` list + `pub use` block)
|
||||
|
||||
- [ ] **Step 1: Create `crates/aura-std/src/vocabulary.rs`**
|
||||
|
||||
```rust
|
||||
//! The closed, compiled-in dispatch from a serialized node **type identity** to
|
||||
//! its `aura-std` builder factory — the load-side counterpart to the type label
|
||||
//! a blueprint serializes (`PrimitiveBuilder::label`). This is **not** a dynamic
|
||||
//! registry (domain invariant 9): it is a `match` the crate that *owns* the
|
||||
//! nodes writes, over its own closed, compiled-in vocabulary (C24 — nodes are
|
||||
//! referenced "by compiled-in type identity", never a by-name marketplace). The
|
||||
//! engine cannot hold this table (it does not depend on `aura-std`); a loader
|
||||
//! takes it as an injected resolver, and a project `cdylib` later supplies its
|
||||
//! own (C16) through the same seam.
|
||||
//!
|
||||
//! Scope (#155): only the **zero-argument** `Type::builder()` factories are
|
||||
//! resolvable. Builders that take structural construction arguments
|
||||
//! (`LinComb(arity)`, `CostSum(n_costs)`, `SimBroker(pip_size)`, `Session(..)`)
|
||||
//! and the recording sinks (`Recorder`/`GatedRecorder`/`SeriesReducer`, which
|
||||
//! capture an `mpsc::Sender`) are deliberately absent — an absent `type_id`
|
||||
//! yields `None`, so the loader fails cleanly (`LoadError::UnknownNodeType`)
|
||||
//! rather than guessing. Serialising structural-axis construction args is a
|
||||
//! later, additive extension (#156/C20).
|
||||
|
||||
use crate::{
|
||||
Add, And, Bias, CarryCost, ConstantCost, Delay, Ema, EqConst, FixedStop, Gt, Latch, LongOnly,
|
||||
Mul, PositionManagement, Resample, RollingMax, RollingMin, Sizer, Sma, Sqrt, Sub,
|
||||
VolSlippageCost,
|
||||
};
|
||||
use aura_core::PrimitiveBuilder;
|
||||
|
||||
/// Resolve a serialized node type identity (the `PrimitiveBuilder::label`, e.g.
|
||||
/// `"SMA"`) to a fresh, unbound builder from the node's own factory. Returns
|
||||
/// `None` for any identity outside the zero-arg `aura-std` vocabulary.
|
||||
pub fn std_vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
|
||||
Some(match type_id {
|
||||
"Add" => Add::builder(),
|
||||
"And" => And::builder(),
|
||||
"Bias" => Bias::builder(),
|
||||
"CarryCost" => CarryCost::builder(),
|
||||
"ConstantCost" => ConstantCost::builder(),
|
||||
"Delay" => Delay::builder(),
|
||||
"EMA" => Ema::builder(),
|
||||
"EqConst" => EqConst::builder(),
|
||||
"FixedStop" => FixedStop::builder(),
|
||||
"Gt" => Gt::builder(),
|
||||
"Latch" => Latch::builder(),
|
||||
"LongOnly" => LongOnly::builder(),
|
||||
"Mul" => Mul::builder(),
|
||||
"PositionManagement" => PositionManagement::builder(),
|
||||
"Resample" => Resample::builder(),
|
||||
"RollingMax" => RollingMax::builder(),
|
||||
"RollingMin" => RollingMin::builder(),
|
||||
"Sizer" => Sizer::builder(),
|
||||
"SMA" => Sma::builder(),
|
||||
"Sqrt" => Sqrt::builder(),
|
||||
"Sub" => Sub::builder(),
|
||||
"VolSlippageCost" => VolSlippageCost::builder(),
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::std_vocabulary;
|
||||
|
||||
#[test]
|
||||
fn std_vocabulary_resolves_known_and_rejects_unknown() {
|
||||
// a known zero-arg node resolves, and the builder carries the type label back
|
||||
let sma = std_vocabulary("SMA").expect("SMA is in the vocabulary");
|
||||
assert_eq!(sma.label(), "SMA");
|
||||
// an unknown id, a construction-arg node, and a sink are all absent
|
||||
assert!(std_vocabulary("nope").is_none());
|
||||
assert!(std_vocabulary("LinComb").is_none());
|
||||
assert!(std_vocabulary("Recorder").is_none());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Wire the module into `crates/aura-std/src/lib.rs`**
|
||||
|
||||
Add `mod vocabulary;` to the private `mod` list (after `mod vol_slippage_cost;`):
|
||||
|
||||
```rust
|
||||
mod vocabulary;
|
||||
```
|
||||
|
||||
Add the re-export at the end of the `pub use` block (after
|
||||
`pub use vol_slippage_cost::VolSlippageCost;`):
|
||||
|
||||
```rust
|
||||
pub use vocabulary::std_vocabulary;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the resolver test**
|
||||
|
||||
Run: `cargo test -p aura-std std_vocabulary_resolves_known_and_rejects_unknown`
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: DTO layer + canonical serializer + envelope
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/aura-engine/src/blueprint_serde.rs`
|
||||
- Modify: `crates/aura-engine/src/lib.rs` (`mod` + re-export)
|
||||
- Modify: `crates/aura-engine/src/test_fixtures.rs` (add the sink-free signal fixture)
|
||||
|
||||
- [ ] **Step 1: Add the sink-free signal fixture to `test_fixtures.rs`**
|
||||
|
||||
Append to `crates/aura-engine/src/test_fixtures.rs` (its imports already cover
|
||||
`BlueprintNode`/`Composite`/`Edge`/`OutField`/`Role`/`Target`, `Scalar`, and
|
||||
`aura_std::{Bias, Sma, Sub}`):
|
||||
|
||||
```rust
|
||||
/// The SMA-cross **signal** graph as a sink-free, param-generic composite: fast
|
||||
/// SMA (length bound to 2) and slow SMA over one `price` role, their spread, a
|
||||
/// `Bias`. Output is the `bias` field. No recorder/broker — observation is
|
||||
/// attached by the caller. This is the canonical round-trip fixture (cycle 0087).
|
||||
pub(crate) fn sink_free_sma_cross_signal() -> Composite {
|
||||
Composite::new(
|
||||
"sma_cross",
|
||||
vec![
|
||||
Sma::builder().named("fast").bind("length", Scalar::i64(2)).into(),
|
||||
Sma::builder().named("slow").into(),
|
||||
Sub::builder().into(),
|
||||
Bias::builder().into(),
|
||||
],
|
||||
vec![
|
||||
Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast -> Sub slot 0
|
||||
Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow -> Sub slot 1
|
||||
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // spread -> Bias
|
||||
],
|
||||
vec![Role {
|
||||
name: "price".into(),
|
||||
targets: vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }],
|
||||
source: None,
|
||||
}],
|
||||
vec![OutField { node: 3, field: 0, name: "bias".into() }],
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Create `crates/aura-engine/src/blueprint_serde.rs` (DTO + serializer)**
|
||||
|
||||
```rust
|
||||
//! Blueprint serialization (C24): a `Composite` blueprint is projected to a
|
||||
//! canonical, versioned data value and reconstructed from it (`blueprint_from_json`
|
||||
//! in this module's loader half). `Composite`/`PrimitiveBuilder` cannot derive
|
||||
//! serde — they hold a `Box<dyn Fn>` build closure — so the format is a faithful
|
||||
//! serde **projection**: it carries each primitive's compiled-in **type identity**
|
||||
//! (`PrimitiveBuilder::label`), its instance name, and its bound params; the build
|
||||
//! closure and the declared schema are re-derived on load from the injected
|
||||
//! resolver. This is the inverse of the lossy render half (`model_to_json`); it
|
||||
//! carries no node logic (C17) and references a closed vocabulary (C24).
|
||||
|
||||
use crate::blueprint::{BlueprintNode, Composite, OutField, Role};
|
||||
use crate::harness::Edge;
|
||||
use aura_core::{BoundParam, PrimitiveBuilder};
|
||||
|
||||
/// The format version the loader understands. Bumped only by a load-bearing
|
||||
/// (Tier-2) change; additive optional fields do not bump it (#156).
|
||||
pub const BLUEPRINT_FORMAT_VERSION: u32 = 1;
|
||||
|
||||
/// Top-level envelope: the version is read before the payload is interpreted.
|
||||
#[derive(serde::Serialize, serde::Deserialize)]
|
||||
pub struct BlueprintDoc {
|
||||
pub format_version: u32,
|
||||
pub blueprint: CompositeData,
|
||||
}
|
||||
|
||||
/// Serde mirror of a `Composite` (the in-memory type holds build closures and
|
||||
/// cannot derive serde). Field order here is the canonical JSON field order.
|
||||
#[derive(serde::Serialize, serde::Deserialize)]
|
||||
pub struct CompositeData {
|
||||
pub name: String,
|
||||
pub nodes: Vec<NodeData>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub edges: Vec<Edge>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub input_roles: Vec<Role>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub output: Vec<OutField>,
|
||||
}
|
||||
|
||||
/// A blueprint item: a primitive (referenced by type identity) or a nested
|
||||
/// composite (recursion). Externally tagged: `{"primitive": ..}` / `{"composite": ..}`.
|
||||
#[derive(serde::Serialize, serde::Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum NodeData {
|
||||
Primitive(PrimitiveData),
|
||||
Composite(CompositeData),
|
||||
}
|
||||
|
||||
/// A primitive node as data: its compiled-in type identity, optional instance
|
||||
/// name, and bound params. The schema + build closure are re-derived on load.
|
||||
#[derive(serde::Serialize, serde::Deserialize)]
|
||||
pub struct PrimitiveData {
|
||||
#[serde(rename = "type")]
|
||||
pub type_id: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
pub bound: Vec<BoundParam>,
|
||||
}
|
||||
|
||||
/// Serializer failure (typed, named).
|
||||
#[derive(Debug)]
|
||||
pub enum SerializeError {
|
||||
Json(serde_json::Error),
|
||||
}
|
||||
|
||||
fn project(c: &Composite) -> CompositeData {
|
||||
CompositeData {
|
||||
name: c.name().to_string(),
|
||||
nodes: c.nodes().iter().map(project_node).collect(),
|
||||
edges: c.edges().to_vec(),
|
||||
input_roles: c.input_roles().to_vec(),
|
||||
output: c.output().to_vec(),
|
||||
}
|
||||
}
|
||||
|
||||
fn project_node(n: &BlueprintNode) -> NodeData {
|
||||
match n {
|
||||
BlueprintNode::Primitive(b) => NodeData::Primitive(PrimitiveData {
|
||||
type_id: b.label(),
|
||||
name: b.instance_name().map(str::to_string),
|
||||
bound: b.bound_params().to_vec(),
|
||||
}),
|
||||
BlueprintNode::Composite(c) => NodeData::Composite(project(c)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Serialize a blueprint to canonical, versioned JSON: compact, struct-declaration
|
||||
/// field order, defaults omitted (`skip_serializing_if`). An absent optional is
|
||||
/// byte-identical to the pre-extension form.
|
||||
pub fn blueprint_to_json(c: &Composite) -> Result<String, SerializeError> {
|
||||
let doc = BlueprintDoc { format_version: BLUEPRINT_FORMAT_VERSION, blueprint: project(c) };
|
||||
serde_json::to_string(&doc).map_err(SerializeError::Json)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn signal_serializes_to_canonical_golden() {
|
||||
let signal = crate::test_fixtures::sink_free_sma_cross_signal();
|
||||
let json = blueprint_to_json(&signal).expect("serializes");
|
||||
let golden = r#"{"format_version":1,"blueprint":{"name":"sma_cross","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}]}],"output":[{"node":3,"field":0,"name":"bias"}]}}"#;
|
||||
// pins: canonical field order, format_version envelope, AND omit-defaults
|
||||
// (the unnamed/unbound `Sub` + `Bias` carry no `name`/`bound` keys; the
|
||||
// `price` role carries no `source` key).
|
||||
assert_eq!(json, golden);
|
||||
assert!(json.contains(r#"{"primitive":{"type":"Sub"}}"#), "omit-defaults");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Wire the module into `crates/aura-engine/src/lib.rs`**
|
||||
|
||||
Add to the private `mod` list (around `:45-52`, next to `mod blueprint;`):
|
||||
|
||||
```rust
|
||||
mod blueprint_serde;
|
||||
```
|
||||
|
||||
Add to the re-export block (around `:54-79`):
|
||||
|
||||
```rust
|
||||
pub use blueprint_serde::{blueprint_to_json, BlueprintDoc, CompositeData, NodeData, PrimitiveData, SerializeError, BLUEPRINT_FORMAT_VERSION};
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the golden test**
|
||||
|
||||
Run: `cargo test -p aura-engine signal_serializes_to_canonical_golden`
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: the loader + typed load errors
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-engine/src/blueprint_serde.rs` (add loader + `LoadError`)
|
||||
- Modify: `crates/aura-engine/src/lib.rs` (extend the re-export)
|
||||
|
||||
- [ ] **Step 1: Add the loader + `LoadError` to `blueprint_serde.rs`**
|
||||
|
||||
Append after `blueprint_to_json` (before the `#[cfg(test)]` module):
|
||||
|
||||
```rust
|
||||
/// Loader failure (typed, named — never a panic, never a silent wrong graph).
|
||||
#[derive(Debug)]
|
||||
pub enum LoadError {
|
||||
/// Malformed or structurally-invalid JSON.
|
||||
Json(serde_json::Error),
|
||||
/// `format_version` the loader does not understand. The full must-understand
|
||||
/// horizon (multiple versions, unknown-section policy) is #156; #155 supports
|
||||
/// exactly one version and refuses any other, cleanly.
|
||||
UnsupportedVersion { found: u32, supported: u32 },
|
||||
/// `resolve` returned `None` for a serialized `type_id` (outside the injected
|
||||
/// vocabulary — unknown node, or a construction-arg/sink node not in #155's set).
|
||||
UnknownNodeType(String),
|
||||
}
|
||||
|
||||
fn reconstruct(
|
||||
d: &CompositeData,
|
||||
resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>,
|
||||
) -> Result<Composite, LoadError> {
|
||||
let mut nodes = Vec::with_capacity(d.nodes.len());
|
||||
for nd in &d.nodes {
|
||||
nodes.push(match nd {
|
||||
NodeData::Primitive(p) => {
|
||||
let mut b = resolve(&p.type_id)
|
||||
.ok_or_else(|| LoadError::UnknownNodeType(p.type_id.clone()))?;
|
||||
if let Some(n) = &p.name {
|
||||
b = b.named(n);
|
||||
}
|
||||
// Re-apply bound params BY NAME. The effective param vector is
|
||||
// order-independent (each `bind` computes its slot against the
|
||||
// shrunk schema); ascending original `pos` is the canonical order.
|
||||
let mut bound: Vec<&BoundParam> = p.bound.iter().collect();
|
||||
bound.sort_by_key(|bp| bp.pos);
|
||||
for bp in bound {
|
||||
b = b.bind(&bp.name, bp.value);
|
||||
}
|
||||
BlueprintNode::Primitive(b)
|
||||
}
|
||||
NodeData::Composite(c) => BlueprintNode::Composite(reconstruct(c, resolve)?),
|
||||
});
|
||||
}
|
||||
Ok(Composite::new(
|
||||
d.name.clone(),
|
||||
nodes,
|
||||
d.edges.clone(),
|
||||
d.input_roles.clone(),
|
||||
d.output.clone(),
|
||||
))
|
||||
}
|
||||
|
||||
/// Load a blueprint from canonical JSON, resolving each primitive's type identity
|
||||
/// through the injected `resolve` (e.g. `aura_std::std_vocabulary`). The version
|
||||
/// envelope is checked before the payload is reconstructed.
|
||||
pub fn blueprint_from_json(
|
||||
data: &str,
|
||||
resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>,
|
||||
) -> Result<Composite, LoadError> {
|
||||
let doc: BlueprintDoc = serde_json::from_str(data).map_err(LoadError::Json)?;
|
||||
if doc.format_version != BLUEPRINT_FORMAT_VERSION {
|
||||
return Err(LoadError::UnsupportedVersion {
|
||||
found: doc.format_version,
|
||||
supported: BLUEPRINT_FORMAT_VERSION,
|
||||
});
|
||||
}
|
||||
reconstruct(&doc.blueprint, resolve)
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Extend the `lib.rs` re-export**
|
||||
|
||||
Change the Task-3 re-export line to add `blueprint_from_json` and `LoadError`:
|
||||
|
||||
```rust
|
||||
pub use blueprint_serde::{blueprint_from_json, blueprint_to_json, BlueprintDoc, CompositeData, LoadError, NodeData, PrimitiveData, SerializeError, BLUEPRINT_FORMAT_VERSION};
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add the named-failure tests**
|
||||
|
||||
In `blueprint_serde.rs`'s `#[cfg(test)] mod tests`, add (the resolver
|
||||
`aura_std::std_vocabulary` is available — `aura-std` is a test-only dep of the
|
||||
engine):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn unknown_node_type_fails_named() {
|
||||
// a valid envelope naming a type outside the vocabulary -> clean, named error
|
||||
let json = r#"{"format_version":1,"blueprint":{"name":"x","nodes":[{"primitive":{"type":"NoSuchNode"}}]}}"#;
|
||||
let err = blueprint_from_json(json, &|t| aura_std::std_vocabulary(t)).unwrap_err();
|
||||
assert!(matches!(err, LoadError::UnknownNodeType(t) if t == "NoSuchNode"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unsupported_version_fails_named() {
|
||||
let json = r#"{"format_version":2,"blueprint":{"name":"x","nodes":[]}}"#;
|
||||
let err = blueprint_from_json(json, &|t| aura_std::std_vocabulary(t)).unwrap_err();
|
||||
assert!(matches!(err, LoadError::UnsupportedVersion { found: 2, supported: 1 }));
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the loader failure tests**
|
||||
|
||||
Run: `cargo test -p aura-engine unknown_node_type_fails_named`
|
||||
Expected: PASS.
|
||||
|
||||
Run: `cargo test -p aura-engine unsupported_version_fails_named`
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: round-trip integration — bit-identical run, idempotence, recursion
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/aura-engine/src/blueprint_serde.rs` (`#[cfg(test)] mod tests`)
|
||||
|
||||
- [ ] **Step 1: Add a recorder-attaching run helper + the bit-identical test**
|
||||
|
||||
In `blueprint_serde.rs`'s `#[cfg(test)] mod tests`, add the imports and helper.
|
||||
The helper nests a signal composite under a root with one `Recorder` on its
|
||||
`bias` output (the same `Composite::new` nesting idiom as
|
||||
`composite_sma_cross_harness`), bootstraps with the open point, and returns the
|
||||
recorded trace:
|
||||
|
||||
```rust
|
||||
use crate::blueprint::{Composite, Role};
|
||||
use crate::harness::{Edge, Target};
|
||||
use crate::VecSource; // crate-root re-export (blueprint.rs tests import it the same way, e.g. :923)
|
||||
use aura_core::{Scalar, ScalarKind, Timestamp};
|
||||
use aura_std::Recorder;
|
||||
use std::sync::mpsc;
|
||||
|
||||
// Open param point for the sink-free signal (fast.length is bound): [slow.length, bias.scale].
|
||||
fn signal_point() -> Vec<Scalar> {
|
||||
vec![Scalar::i64(4), Scalar::f64(0.5)]
|
||||
}
|
||||
|
||||
// Nest `signal` under a root that records its single `bias` output, run the
|
||||
// 7-tick synthetic price fixture, and collect the recorded trace.
|
||||
fn run_recording(signal: Composite) -> Vec<(Timestamp, Vec<Scalar>)> {
|
||||
let (tx, rx) = mpsc::channel();
|
||||
let root = Composite::new(
|
||||
"h",
|
||||
vec![
|
||||
crate::blueprint::BlueprintNode::Composite(signal),
|
||||
Recorder::builder(vec![ScalarKind::F64], aura_core::Firing::Any, tx).into(),
|
||||
],
|
||||
vec![Edge { from: 0, to: 1, slot: 0, from_field: 0 }], // bias -> recorder
|
||||
vec![Role {
|
||||
name: "src".into(),
|
||||
targets: vec![Target { node: 0, slot: 0 }], // price -> nested signal's price port
|
||||
source: Some(ScalarKind::F64),
|
||||
}],
|
||||
vec![],
|
||||
);
|
||||
let prices = crate::test_fixtures::synthetic_prices();
|
||||
let mut h = root.bootstrap_with_params(signal_point()).expect("bootstraps");
|
||||
h.run(vec![Box::new(VecSource::new(prices))]);
|
||||
rx.try_iter().collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serialized_blueprint_runs_bit_identical_to_rust_built() {
|
||||
let rust_built = crate::test_fixtures::sink_free_sma_cross_signal();
|
||||
let json = blueprint_to_json(&rust_built).expect("serializes");
|
||||
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
|
||||
|
||||
let trace_rust = run_recording(rust_built);
|
||||
let trace_loaded = run_recording(loaded);
|
||||
|
||||
assert_eq!(trace_rust, trace_loaded, "serialized-then-loaded run diverged");
|
||||
assert!(!trace_loaded.is_empty(), "trace must be populated (non-degenerate)");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the bit-identical test**
|
||||
|
||||
Run: `cargo test -p aura-engine serialized_blueprint_runs_bit_identical_to_rust_built`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 3: Add the canonical-idempotence and recursion tests**
|
||||
|
||||
In the same test module:
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn serializer_and_loader_are_inverse() {
|
||||
let original = crate::test_fixtures::sink_free_sma_cross_signal();
|
||||
let json = blueprint_to_json(&original).expect("serializes");
|
||||
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
|
||||
// serialize -> load -> serialize is byte-stable (canonical round-trip identity)
|
||||
assert_eq!(json, blueprint_to_json(&loaded).expect("re-serializes"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nested_composite_round_trips() {
|
||||
// recursion + a nested bound param + a role: wrap the signal composite in
|
||||
// an outer composite that re-exports its `bias` output. Proves the loop is
|
||||
// not flat-SMA-specific.
|
||||
let inner = crate::test_fixtures::sink_free_sma_cross_signal();
|
||||
let outer = Composite::new(
|
||||
"outer",
|
||||
vec![crate::blueprint::BlueprintNode::Composite(inner)],
|
||||
vec![],
|
||||
vec![Role {
|
||||
name: "price".into(),
|
||||
targets: vec![Target { node: 0, slot: 0 }],
|
||||
source: None,
|
||||
}],
|
||||
vec![crate::blueprint::OutField { node: 0, field: 0, name: "bias".into() }],
|
||||
);
|
||||
let json = blueprint_to_json(&outer).expect("serializes");
|
||||
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
|
||||
assert_eq!(json, blueprint_to_json(&loaded).expect("re-serializes"));
|
||||
// the nested composite survived as a composite, not flattened
|
||||
assert!(json.contains(r#"{"composite":{"name":"sma_cross""#), "nested composite preserved");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the idempotence + recursion tests**
|
||||
|
||||
Run: `cargo test -p aura-engine serializer_and_loader_are_inverse`
|
||||
Expected: PASS.
|
||||
|
||||
Run: `cargo test -p aura-engine nested_composite_round_trips`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Full workspace gate**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: PASS (all pre-existing tests + the new cycle-0087 tests; ~737 + new).
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: clean (0 warnings).
|
||||
@@ -1,333 +0,0 @@
|
||||
# Blueprint serialization & loader (close the graph-as-data loop) — Design Spec
|
||||
|
||||
**Date:** 2026-06-29
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
**Cycle:** 0087
|
||||
**Issue:** #155 (milestone "Topology-as-data: blueprint serialization & loader (C24)")
|
||||
|
||||
## Goal
|
||||
|
||||
Close the graph-as-data loop mandated by **C24** (ledger, committed `f80571a`):
|
||||
make a **blueprint** — the param-generic, named graph-as-data a Rust builder
|
||||
produces (C9/C19) — a **first-class serializable data value** with a stable,
|
||||
versioned, canonical format, *and* give the engine the missing **load path**
|
||||
(`data → blueprint → FlatGraph`). Today only the forward render half exists
|
||||
(`model_to_json`, `crates/aura-engine/src/graph_model.rs`); there is no faithful
|
||||
serializer and no loader.
|
||||
|
||||
Scope is **format + serializer + loader only** (#155). Explicitly out of scope,
|
||||
each its own milestone issue:
|
||||
|
||||
- the forward-compat *must-understand* refusal gate (Tier-2 unknown node type /
|
||||
too-new version) — #156;
|
||||
- the introspectable construction service that *emits* blueprints — #157;
|
||||
- content-addressed topology identity in the run manifest — #158.
|
||||
|
||||
### Acceptance criterion (applied prospectively)
|
||||
|
||||
aura's feature-acceptance default: the feature solves a real design-mandated
|
||||
problem and contradicts no stated design commitment. This cycle:
|
||||
|
||||
- **Solves a mandated problem.** C24/C21/C18 cannot orchestrate, structurally
|
||||
search, or *reproduce* topology-varying families while topology stays opaque
|
||||
Rust source. The load path is the substrate they stand on.
|
||||
- **Reintroduces no forbidden failure class.** The format carries **no node
|
||||
logic** (C17 — computation stays compiled Rust; no RustAst regression), is a
|
||||
**static DAG** (non-Turing by construction), and references nodes by
|
||||
**compiled-in type identity over a closed vocabulary** — *not* a by-name
|
||||
marketplace/registry (C24 forbid; domain invariant 9).
|
||||
- **Empirical evidence** (this is an infrastructure cycle; the evidence is the
|
||||
delivered code + a must-pass test, per the no-surface-still-show-code rule):
|
||||
the round-trip test below, whose correct behaviour is a **bit-identical run**
|
||||
(C1) of a serialized-then-loaded blueprint against its Rust-built twin, plus
|
||||
the canonical JSON a real example blueprint serializes to (shown under
|
||||
*Concrete code shapes*).
|
||||
|
||||
## Architecture
|
||||
|
||||
Four pieces, placed to honour the engine-stays-domain-free boundary
|
||||
(`aura-engine → aura-core` only; `aura-std` is a *test-only* dep of the engine —
|
||||
see `crates/aura-engine/Cargo.toml`):
|
||||
|
||||
1. **A serde DTO layer** (new module `crates/aura-engine/src/blueprint_serde.rs`).
|
||||
`Composite` and `PrimitiveBuilder` cannot derive serde — they transitively
|
||||
hold `build: Box<dyn Fn(&[Cell]) -> Box<dyn Node>>` (`node.rs:115`). So the
|
||||
format is a **separate, faithful projection** to serde-derivable DTO mirrors,
|
||||
*not* an extension of the lossy render model `model_to_json` (which carries no
|
||||
constructor link and no param values — confirmed lossy). This mirrors the
|
||||
established read-widening idiom in `crates/aura-registry/src/compat.rs`.
|
||||
|
||||
2. **The serializer** `blueprint_to_json(&Composite) -> Result<String, _>`:
|
||||
projects a `&Composite` to the DTO (dropping the build closures, which the
|
||||
loader re-derives) and emits **canonical** JSON — compact, struct-declaration
|
||||
field order, defaults omitted via `skip_serializing_if`. Canonicality is the
|
||||
precondition for stable content-addressing (#158) and for aura's existing
|
||||
byte-identical-golden discipline.
|
||||
|
||||
3. **The loader** `blueprint_from_json(data, resolve) -> Result<Composite, _>`:
|
||||
reads the `format_version` envelope first, deserializes the DTO, and
|
||||
reconstructs the `Composite`. The build closures are recovered **by type
|
||||
identity** through an **injected node resolver** (next point); bound params
|
||||
and instance names are re-applied from the DTO. The terminal step downstream
|
||||
is the unchanged `Composite::compile_with_params` (`blueprint.rs:261`).
|
||||
|
||||
4. **The node resolver seam.** The loader is **generic over an injected
|
||||
resolver** `Fn(&str) -> Option<PrimitiveBuilder>`, defined engine-side
|
||||
(`aura-core`/`aura-engine`, so the engine stays domain-free). The **concrete
|
||||
closed dispatch over the `aura-std` vocabulary** lives in the crate that owns
|
||||
those nodes — a new `std_vocabulary(type_id)` in `aura-std`, a plain
|
||||
compiled-in `match` over its ~29 `builder()` factories. This is **not** the
|
||||
forbidden registry (no dynamic registration, no marketplace, no engine-side
|
||||
node table): it is exactly C24's "compiled-in closed-set referenced by type
|
||||
identity". The injected seam is also where a project `cdylib` later supplies
|
||||
its own vocabulary (C16/#157) — but loading a cdylib is out of #155's scope;
|
||||
this cut wires the seam and one `aura-std` implementation.
|
||||
|
||||
### The load-bearing fork and its resolution (recorded on #155)
|
||||
|
||||
**Fork:** the loader fundamentally needs a `type-id → constructor` resolution,
|
||||
and domain invariant 9 forbids "a node registry inside aura"; the recon flagged
|
||||
this as a possible ledger-level decision.
|
||||
|
||||
**Decision (derived, not a user preference):** the *closed, compiled-in,
|
||||
injected* resolver is **not** the forbidden registry — it realises the mechanism
|
||||
C24 *already* mandates ("compiled-in type identity — the closed, typed node
|
||||
vocabulary of `aura-std` + the project `cdylib`"; forbids only "a by-name node
|
||||
marketplace/registry … the format is not a distribution mechanism"). Invariant
|
||||
9's "no node registry" governs *multi-project management / distribution*, not a
|
||||
crate's compile-time `match` over its own known nodes. The resolution is further
|
||||
**forced** by the crate graph: `aura-engine → aura-core` only, so the engine
|
||||
*cannot* hold an `aura-std` node table — the resolver *must* be injected and the
|
||||
vocabulary table *must* live outside the engine. This is a realisation of C24,
|
||||
not a contract change, so it is a `specify` decision, not a `brainstorm` bounce.
|
||||
(Basis: C24 body `docs/design/INDEX.md:1751-1766`, `f80571a`; invariant 9 in
|
||||
`crates/aura/CLAUDE.md`; `crates/aura-engine/Cargo.toml` engine-domain-free
|
||||
comment.)
|
||||
|
||||
### What is and is not in the serialized blueprint
|
||||
|
||||
- **In:** topology (nodes, edges, input roles, exposed outputs), each primitive
|
||||
node's **type identity**, its instance name, and its **bound params** (the
|
||||
knobs the author fixed). Nested composites recurse structurally.
|
||||
- **Out (param point):** the *free* param-space is **not** baked to a single
|
||||
point — a blueprint is param-generic (C19). The free point is supplied at
|
||||
`compile_with_params` exactly as today; the run manifest records the point
|
||||
separately (C18, `report.rs`). The round-trip test binds the **same** point to
|
||||
both twins.
|
||||
- **Out (sinks / observation):** a recording sink (`Recorder::builder(.., tx)`,
|
||||
`test_fixtures.rs:67`) captures an `mpsc::Sender` in its build closure —
|
||||
runtime identity, neither a param nor topology, so it structurally cannot live
|
||||
in a param-generic, value-empty blueprint (C19). Sink-destination addressing
|
||||
is the C18-registry / trace-persistence concern (#101), out of #155. The
|
||||
round-trip test attaches **identical sinks post-load** to both twins and
|
||||
compares the recorded traces. (How a serialized harness later re-acquires its
|
||||
sinks — observation is part of the harness in C12 — is #101/#158's question,
|
||||
not #155's; this cut serialises the closure-free topology and stops there.)
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### Delivered API + the canonical artifact (the north-star slice)
|
||||
|
||||
What a World / CLI eventually does — serialize an authored signal graph, store
|
||||
the bytes, reload and run them (the capability this cycle delivers the substrate
|
||||
for):
|
||||
|
||||
```rust
|
||||
let bp: Composite = sma_cross_signal(); // authored via the Rust builder (C17)
|
||||
let json = blueprint_to_json(&bp)?; // canonical, versioned bytes
|
||||
std::fs::write("sma_cross.bp.json", &json)?; // topology is now a stored value (C24)
|
||||
|
||||
// later / elsewhere — no Rust rebuild, just data:
|
||||
let data = std::fs::read_to_string("sma_cross.bp.json")?;
|
||||
let bp2 = blueprint_from_json(&data, &|t| aura_std::std_vocabulary(t))?;
|
||||
let flat = bp2.compile_with_params(&[Scalar::i64(2), Scalar::i64(4)])?; // runs (C1)
|
||||
```
|
||||
|
||||
The canonical bytes for a minimal SMA-cross **signal** graph (`sma_fast`,
|
||||
`sma_slow`, a `Sub`, a `Bias`; fast length bound to 2, slow open):
|
||||
|
||||
```json
|
||||
{"format_version":1,"blueprint":{"name":"sma_cross","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}]}],"output":[{"node":3,"field":0,"name":"bias"}]}}
|
||||
```
|
||||
|
||||
Note: `slow` carries no `name`-less-than-needed clutter — `bound` and `name` are
|
||||
**omitted** when empty/None (`skip_serializing_if`), so an absent optional is
|
||||
byte-identical to the pre-extension form (the additive-extension precondition
|
||||
#156 builds on).
|
||||
|
||||
### DTO layer (new — `blueprint_serde.rs`), before → after
|
||||
|
||||
There is no "before"; this is new. Shape:
|
||||
|
||||
```rust
|
||||
pub const BLUEPRINT_FORMAT_VERSION: u32 = 1;
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct BlueprintDoc { pub format_version: u32, pub blueprint: CompositeData }
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct CompositeData {
|
||||
pub name: String,
|
||||
pub nodes: Vec<NodeData>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")] pub edges: Vec<Edge>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")] pub input_roles: Vec<RoleData>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")] pub output: Vec<OutField>,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum NodeData { Primitive(PrimitiveData), Composite(CompositeData) }
|
||||
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct PrimitiveData {
|
||||
#[serde(rename = "type")] pub type_id: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")] pub name: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")] pub bound: Vec<BoundParam>,
|
||||
}
|
||||
|
||||
pub fn blueprint_to_json(c: &Composite) -> Result<String, SerializeError>;
|
||||
pub fn blueprint_from_json(data: &str, resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>)
|
||||
-> Result<Composite, LoadError>;
|
||||
```
|
||||
|
||||
`RoleData` mirrors `Role { name, targets: Vec<Target>, source: Option<ScalarKind> }`
|
||||
(`source` skipped when `None`). The plain data types `Edge`, `Target`, `OutField`
|
||||
(`harness.rs`, `blueprint.rs:30`) and `BoundParam`, `ScalarKind`, `Firing`
|
||||
(`node.rs`) gain `#[derive(Serialize, Deserialize)]` — they hold no closures and
|
||||
already derive `Clone/Debug/PartialEq`. `Scalar` already derives serde
|
||||
(`scalar.rs:40`). **No `HashMap` enters any DTO** — every field is a `Vec` or
|
||||
scalar, so struct-declaration field order is the only ordering and JSON output is
|
||||
deterministic (canonical).
|
||||
|
||||
### Reconstruction (the loader's crux), shape
|
||||
|
||||
```rust
|
||||
fn reconstruct(d: &CompositeData, resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>)
|
||||
-> Result<Composite, LoadError>
|
||||
{
|
||||
let mut nodes = Vec::new();
|
||||
for nd in &d.nodes {
|
||||
nodes.push(match nd {
|
||||
NodeData::Primitive(p) => {
|
||||
let mut b = resolve(&p.type_id)
|
||||
.ok_or_else(|| LoadError::UnknownNodeType(p.type_id.clone()))?;
|
||||
if let Some(n) = &p.name { b = b.named(n); }
|
||||
// bound params re-applied by NAME; order-independent in the
|
||||
// effective param vector (each bind computes pos against the
|
||||
// shrunk schema) — ascending original pos is the canonical order.
|
||||
let mut bound: Vec<&BoundParam> = p.bound.iter().collect();
|
||||
bound.sort_by_key(|b| b.pos);
|
||||
for bp in bound { b = b.bind(&bp.name, bp.value); }
|
||||
BlueprintNode::Primitive(b)
|
||||
}
|
||||
NodeData::Composite(c) => BlueprintNode::Composite(reconstruct(c, resolve)?),
|
||||
});
|
||||
}
|
||||
Ok(Composite::new(d.name.clone(), nodes, d.edges.clone(),
|
||||
d.input_roles.iter().map(Into::into).collect(),
|
||||
d.output.clone()))
|
||||
}
|
||||
```
|
||||
|
||||
### Resolver (new — `aura-std/src/vocabulary.rs`), shape
|
||||
|
||||
```rust
|
||||
/// Closed, compiled-in dispatch over aura-std's OWN node vocabulary. Not a
|
||||
/// dynamic registry (invariant 9): a `match` the crate that owns the nodes
|
||||
/// writes, referenced by compiled-in type identity (C24).
|
||||
pub fn std_vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
|
||||
Some(match type_id {
|
||||
"SMA" => Sma::builder(),
|
||||
"Sub" => Sub::builder(),
|
||||
"Bias" => Bias::builder(),
|
||||
"VolStop" => VolStop::builder(),
|
||||
// … the remaining aura-std builder() factories …
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
The `type_id` is the `PrimitiveBuilder.name` (`&'static str`, e.g. `"SMA"`,
|
||||
`node.rs:108`) — the type label, *not* the lowercased instance default.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Crate / file | Responsibility |
|
||||
|---|---|---|
|
||||
| DTO mirrors + envelope | `aura-engine/src/blueprint_serde.rs` (new) | serde-derivable projection of `Composite`; `format_version` |
|
||||
| `blueprint_to_json` | same | `&Composite` → canonical JSON |
|
||||
| `blueprint_from_json` + `reconstruct` | same | JSON + resolver → `Composite` |
|
||||
| `SerializeError` / `LoadError` | same | typed, named failures |
|
||||
| serde derives on plain types | `aura-core/src/node.rs`, `aura-engine/src/harness.rs`, `blueprint.rs` | `Edge`, `Target`, `OutField`, `BoundParam`, `ScalarKind`, `Firing`, `RoleData` source |
|
||||
| `std_vocabulary` | `aura-std/src/vocabulary.rs` (new) | closed `type_id → builder()` dispatch |
|
||||
| round-trip test | `aura-engine` tests (test-only `aura-std` dep, mirroring `blueprint.rs:1957`) | bit-identity + canonical-idempotence |
|
||||
|
||||
## Data flow
|
||||
|
||||
**Serialize:** `&Composite` → walk nodes/edges/roles/output → `BlueprintDoc`
|
||||
(drop build closures, keep `name`/`type_id`/`bound`) → `serde_json::to_string`
|
||||
(compact, defaults omitted) → canonical bytes.
|
||||
|
||||
**Load:** bytes → `serde_json::from_str::<BlueprintDoc>` → check `format_version
|
||||
== 1` (else `LoadError::UnsupportedVersion`) → `reconstruct`: per primitive,
|
||||
`resolve(type_id)` → `PrimitiveBuilder`, re-apply `named` + `bind`; per nested
|
||||
composite, recurse → `Composite::new(...)` → caller runs
|
||||
`compile_with_params(point)` → `FlatGraph` → run.
|
||||
|
||||
## Error handling
|
||||
|
||||
`LoadError` (typed, each names its cause — never a panic, never a silent wrong
|
||||
graph):
|
||||
|
||||
- `Json(serde_json::Error)` — malformed/structurally-invalid JSON.
|
||||
- `UnsupportedVersion { found, supported }` — `format_version` mismatch. This is
|
||||
the *minimal* version-awareness #155 needs; the full must-understand horizon
|
||||
(multiple versions, unknown-section policy) is #156.
|
||||
- `UnknownNodeType(String)` — `resolve` returned `None` for a `type_id`. (The
|
||||
*richer* Tier-2 refusal semantics are #156; #155 fails cleanly and named.)
|
||||
|
||||
`bind`'s existing preconditions (unknown/ambiguous param name, kind mismatch)
|
||||
panic by design at authoring (`node.rs:205`); a serialized blueprint that names a
|
||||
param the resolved builder does not declare is a corrupt/incompatible artifact —
|
||||
#156 owns turning that into a clean Tier-2 refusal; #155 inherits the existing
|
||||
`bind` panic as the honest first-cut failure and does not widen scope to catch
|
||||
it.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
1. **Bit-identical run (the headline acceptance, C1).** Mirror
|
||||
`composite_sma_cross_runs_bit_identical_to_hand_wired` (`blueprint.rs:1957`):
|
||||
author a **sink-free** SMA-cross *signal* composite; `blueprint_to_json` it;
|
||||
`blueprint_from_json` with `std_vocabulary`; attach **identical** recording
|
||||
sinks post-load to both the Rust twin and the loaded blueprint; bootstrap both
|
||||
with the **same** param point; assert the recorded traces are byte-for-byte
|
||||
equal and non-empty.
|
||||
2. **Canonical round-trip idempotence.** `blueprint_to_json(loaded) ==
|
||||
blueprint_to_json(original)` — serialize→load→serialize is byte-stable.
|
||||
3. **Omit-defaults canonicality.** A node with no instance name and no bound
|
||||
params serializes with neither `name` nor `bound` key present (golden
|
||||
substring), proving an absent optional is byte-identical to the pre-extension
|
||||
form (#156's additive precondition).
|
||||
4. **Named failures.** An unknown `type_id` → `LoadError::UnknownNodeType`; a
|
||||
`format_version` of `2` → `LoadError::UnsupportedVersion` — both asserted by
|
||||
variant, neither panicking.
|
||||
5. **A richer composite round-trips** (e.g. a nested `risk_executor`-style
|
||||
composite from `aura-composites`, sinks excluded) to exercise recursion +
|
||||
bound params + roles, proving the loop is not SMA-specific.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A serialized blueprint loads and runs to a **bit-identical** recorded trace
|
||||
(C1) against its Rust-built twin — same wiring, same sink output (test 1).
|
||||
- [ ] Serializer and loader are **inverse** on the in-repo example blueprints —
|
||||
canonical round-trip identity (tests 2, 5).
|
||||
- [ ] The format carries a top-level **`format_version`** and a **canonical
|
||||
omit-defaults** encoding (test 3).
|
||||
- [ ] Unknown node type and unsupported version fail **cleanly and named**, never
|
||||
panicking, never silently building a different graph (test 4).
|
||||
- [ ] The engine stays **domain-free** (`aura-engine → aura-core` only); the
|
||||
vocabulary table lives in `aura-std`; the resolver is injected (no
|
||||
engine-side node table — invariant 9, C24).
|
||||
- [ ] `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D
|
||||
warnings` are green.
|
||||
Reference in New Issue
Block a user