feat(0087): blueprint serialization & loader — close the graph-as-data loop

C24 first cut (#155): a blueprint — the param-generic, named graph-as-data a
Rust builder produces — is now a first-class serializable data value with a
canonical, versioned format, and the engine has the missing load path
(data -> blueprint -> FlatGraph), not only the Rust-builder construction path.

What ships:
- `blueprint_to_json` / `blueprint_from_json` in a new `aura-engine`
  blueprint_serde module: a faithful serde DTO projection (the build closures
  dropped; re-derived on load), top-level `format_version` envelope, canonical
  compact JSON (struct field order, defaults omitted via skip_serializing_if).
- A resolver seam: the loader is generic over an injected
  `Fn(&str) -> Option<PrimitiveBuilder>`; the concrete closed `match` over the
  aura-std vocabulary lives in aura-std (`std_vocabulary`), never in the engine
  (the engine stays domain-free: lib deps aura-core + aura-analysis only, never
  the node-vocabulary crate). This is C24's compiled-in closed-set referenced by
  type identity, NOT a dynamic/marketplace node registry (domain invariant 9).
- serde derives on the serialized plain types (Edge, Target, OutField, Role,
  BoundParam, ScalarKind); BoundParam additionally re-exported from aura-core's
  root (an additive fix the plan's file list missed).

Load-bearing scope decisions (derived, logged on #155):
- Sinks are excluded from the serialized blueprint — a recording sink captures an
  mpsc::Sender (runtime identity, not param/topology, C19 value-empty); sink
  addressing is the C18-registry / #101 concern. The round-trip test attaches
  identical sinks post-load to both twins.
- Construction-arg builders (LinComb(arity), CostSum(n), SimBroker(pip),
  Session(..)) are out of the round-trippable set: their structural args are a
  C20 structural-axis concern the param-generic format does not yet encode; an
  absent type_id fails the load cleanly (UnknownNodeType), never a silent wrong
  graph. #155's vocabulary is the 22 zero-arg aura-std builders. Additively
  extensible later (#156).

Orchestrator hardening on review: the serializer now emits bound params in
ascending original-slot order (mirroring the loader), so the canonical form is
bind-order-independent — a precondition for content-addressing (#158). Identity
today (every #155 node has <=1 param); holds by construction for future
multi-param nodes.

Acceptance (all green): a serialized blueprint runs bit-identical (C1) to its
Rust-built twin; serializer/loader are inverse (canonical idempotence, incl. a
recursive nested composite); unknown-type and unsupported-version fail named,
never panicking. cargo test --workspace 748 passed; clippy --all-targets
-D warnings clean.

closes #155
This commit is contained in:
2026-06-29 14:54:45 +02:00
parent b2278538ab
commit d5602ec5ad
11 changed files with 609 additions and 7 deletions
+3 -2
View File
@@ -26,7 +26,7 @@ use crate::{GridSpace, ParamRange, RandomSpace, RunReport, SweepFamily};
/// `(node, output-field)` surfaced at the boundary under `name`. `name` is a
/// non-load-bearing render/debug symbol (C23) — like `FieldSpec.name` and
/// `Composite.name`, it does not reach the flat graph.
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct OutField {
pub node: usize,
pub field: usize,
@@ -123,13 +123,14 @@ fn interior_slot_kind(nodes: &[BlueprintNode], _edges: &[Edge], t: &Target) -> S
/// One named input role: role `r` (by position) fans the source value into
/// `targets`. The `name` is a non-load-bearing render symbol (C23); identity is
/// the role index, which survives lowering — the name does not.
#[derive(Clone, Debug, PartialEq, Eq)]
#[derive(Clone, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct Role {
pub name: String,
pub targets: Vec<Target>,
/// `None` = an open interior port (wired by the enclosing graph's edges);
/// `Some(kind)` = a bound ingestion feed of `kind` (only meaningful at the root,
/// where it lowers to a `FlatGraph` source). C3: sources bind at ingestion only.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source: Option<ScalarKind>,
}
+283
View File
@@ -0,0 +1,283 @@
//! Blueprint serialization (C24): a `Composite` blueprint is projected to a
//! canonical, versioned data value and reconstructed from it (`blueprint_from_json`
//! in this module's loader half). `Composite`/`PrimitiveBuilder` cannot derive
//! serde — they hold a `Box<dyn Fn>` build closure — so the format is a faithful
//! serde **projection**: it carries each primitive's compiled-in **type identity**
//! (`PrimitiveBuilder::label`), its instance name, and its bound params; the build
//! closure and the declared schema are re-derived on load from the injected
//! resolver. This is the inverse of the lossy render half (`model_to_json`); it
//! carries no node logic (C17) and references a closed vocabulary (C24).
use crate::blueprint::{BlueprintNode, Composite, OutField, Role};
use crate::harness::Edge;
use aura_core::{BoundParam, PrimitiveBuilder};
/// The format version the loader understands. Bumped only by a load-bearing
/// (Tier-2) change; additive optional fields do not bump it (#156).
pub const BLUEPRINT_FORMAT_VERSION: u32 = 1;
/// Top-level envelope: the version is read before the payload is interpreted.
#[derive(serde::Serialize, serde::Deserialize)]
pub struct BlueprintDoc {
pub format_version: u32,
pub blueprint: CompositeData,
}
/// Serde mirror of a `Composite` (the in-memory type holds build closures and
/// cannot derive serde). Field order here is the canonical JSON field order.
#[derive(serde::Serialize, serde::Deserialize)]
pub struct CompositeData {
pub name: String,
pub nodes: Vec<NodeData>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub edges: Vec<Edge>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub input_roles: Vec<Role>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub output: Vec<OutField>,
}
/// A blueprint item: a primitive (referenced by type identity) or a nested
/// composite (recursion). Externally tagged: `{"primitive": ..}` / `{"composite": ..}`.
#[derive(serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum NodeData {
Primitive(PrimitiveData),
Composite(CompositeData),
}
/// A primitive node as data: its compiled-in type identity, optional instance
/// name, and bound params. The schema + build closure are re-derived on load.
#[derive(serde::Serialize, serde::Deserialize)]
pub struct PrimitiveData {
#[serde(rename = "type")]
pub type_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub bound: Vec<BoundParam>,
}
/// Serializer failure (typed, named).
#[derive(Debug)]
pub enum SerializeError {
Json(serde_json::Error),
}
fn project(c: &Composite) -> CompositeData {
CompositeData {
name: c.name().to_string(),
nodes: c.nodes().iter().map(project_node).collect(),
edges: c.edges().to_vec(),
input_roles: c.input_roles().to_vec(),
output: c.output().to_vec(),
}
}
fn project_node(n: &BlueprintNode) -> NodeData {
match n {
BlueprintNode::Primitive(b) => {
// Canonical: bound params in ascending original-slot order, mirroring
// the loader's re-bind canonicalization, so serialization is
// bind-order-independent (a precondition for content-addressing, #158).
// Identity today — every #155 node has <=1 param — but it holds by
// construction once a multi-param node enters the vocabulary.
let mut bound = b.bound_params().to_vec();
bound.sort_by_key(|bp| bp.pos);
NodeData::Primitive(PrimitiveData {
type_id: b.label(),
name: b.instance_name().map(str::to_string),
bound,
})
}
BlueprintNode::Composite(c) => NodeData::Composite(project(c)),
}
}
/// Serialize a blueprint to canonical, versioned JSON: compact, struct-declaration
/// field order, defaults omitted (`skip_serializing_if`). An absent optional is
/// byte-identical to the pre-extension form.
pub fn blueprint_to_json(c: &Composite) -> Result<String, SerializeError> {
let doc = BlueprintDoc { format_version: BLUEPRINT_FORMAT_VERSION, blueprint: project(c) };
serde_json::to_string(&doc).map_err(SerializeError::Json)
}
/// Loader failure (typed, named — never a panic, never a silent wrong graph).
#[derive(Debug)]
pub enum LoadError {
/// Malformed or structurally-invalid JSON.
Json(serde_json::Error),
/// `format_version` the loader does not understand. The full must-understand
/// horizon (multiple versions, unknown-section policy) is #156; #155 supports
/// exactly one version and refuses any other, cleanly.
UnsupportedVersion { found: u32, supported: u32 },
/// `resolve` returned `None` for a serialized `type_id` (outside the injected
/// vocabulary — unknown node, or a construction-arg/sink node not in #155's set).
UnknownNodeType(String),
}
fn reconstruct(
d: &CompositeData,
resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>,
) -> Result<Composite, LoadError> {
let mut nodes = Vec::with_capacity(d.nodes.len());
for nd in &d.nodes {
nodes.push(match nd {
NodeData::Primitive(p) => {
let mut b = resolve(&p.type_id)
.ok_or_else(|| LoadError::UnknownNodeType(p.type_id.clone()))?;
if let Some(n) = &p.name {
b = b.named(n);
}
// Re-apply bound params BY NAME. The effective param vector is
// order-independent (each `bind` computes its slot against the
// shrunk schema); ascending original `pos` is the canonical order.
let mut bound: Vec<&BoundParam> = p.bound.iter().collect();
bound.sort_by_key(|bp| bp.pos);
for bp in bound {
b = b.bind(&bp.name, bp.value);
}
BlueprintNode::Primitive(b)
}
NodeData::Composite(c) => BlueprintNode::Composite(reconstruct(c, resolve)?),
});
}
Ok(Composite::new(
d.name.clone(),
nodes,
d.edges.clone(),
d.input_roles.clone(),
d.output.clone(),
))
}
/// Load a blueprint from canonical JSON, resolving each primitive's type identity
/// through the injected `resolve` (e.g. `aura_std::std_vocabulary`). The version
/// envelope is checked before the payload is reconstructed.
pub fn blueprint_from_json(
data: &str,
resolve: &dyn Fn(&str) -> Option<PrimitiveBuilder>,
) -> Result<Composite, LoadError> {
let doc: BlueprintDoc = serde_json::from_str(data).map_err(LoadError::Json)?;
if doc.format_version != BLUEPRINT_FORMAT_VERSION {
return Err(LoadError::UnsupportedVersion {
found: doc.format_version,
supported: BLUEPRINT_FORMAT_VERSION,
});
}
reconstruct(&doc.blueprint, resolve)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::blueprint::{Composite, Role};
use crate::harness::{Edge, Target};
use crate::VecSource; // crate-root re-export (blueprint.rs tests import it the same way, e.g. :923)
use aura_core::{Scalar, ScalarKind, Timestamp};
use aura_std::Recorder;
use std::sync::mpsc;
// Open param point for the sink-free signal (fast.length is bound): [slow.length, bias.scale].
fn signal_point() -> Vec<Scalar> {
vec![Scalar::i64(4), Scalar::f64(0.5)]
}
// Nest `signal` under a root that records its single `bias` output, run the
// 7-tick synthetic price fixture, and collect the recorded trace.
fn run_recording(signal: Composite) -> Vec<(Timestamp, Vec<Scalar>)> {
let (tx, rx) = mpsc::channel();
let root = Composite::new(
"h",
vec![
crate::blueprint::BlueprintNode::Composite(signal),
Recorder::builder(vec![ScalarKind::F64], aura_core::Firing::Any, tx).into(),
],
vec![Edge { from: 0, to: 1, slot: 0, from_field: 0 }], // bias -> recorder
vec![Role {
name: "src".into(),
targets: vec![Target { node: 0, slot: 0 }], // price -> nested signal's price port
source: Some(ScalarKind::F64),
}],
vec![],
);
let prices = crate::test_fixtures::synthetic_prices();
let mut h = root.bootstrap_with_params(signal_point()).expect("bootstraps");
h.run(vec![Box::new(VecSource::new(prices))]);
rx.try_iter().collect()
}
#[test]
fn serialized_blueprint_runs_bit_identical_to_rust_built() {
let rust_built = crate::test_fixtures::sink_free_sma_cross_signal();
let json = blueprint_to_json(&rust_built).expect("serializes");
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
let trace_rust = run_recording(rust_built);
let trace_loaded = run_recording(loaded);
assert_eq!(trace_rust, trace_loaded, "serialized-then-loaded run diverged");
assert!(!trace_loaded.is_empty(), "trace must be populated (non-degenerate)");
}
#[test]
fn serializer_and_loader_are_inverse() {
let original = crate::test_fixtures::sink_free_sma_cross_signal();
let json = blueprint_to_json(&original).expect("serializes");
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
// serialize -> load -> serialize is byte-stable (canonical round-trip identity)
assert_eq!(json, blueprint_to_json(&loaded).expect("re-serializes"));
}
#[test]
fn nested_composite_round_trips() {
// recursion + a nested bound param + a role: wrap the signal composite in
// an outer composite that re-exports its `bias` output. Proves the loop is
// not flat-SMA-specific.
let inner = crate::test_fixtures::sink_free_sma_cross_signal();
let outer = Composite::new(
"outer",
vec![crate::blueprint::BlueprintNode::Composite(inner)],
vec![],
vec![Role {
name: "price".into(),
targets: vec![Target { node: 0, slot: 0 }],
source: None,
}],
vec![crate::blueprint::OutField { node: 0, field: 0, name: "bias".into() }],
);
let json = blueprint_to_json(&outer).expect("serializes");
let loaded = blueprint_from_json(&json, &|t| aura_std::std_vocabulary(t)).expect("loads");
assert_eq!(json, blueprint_to_json(&loaded).expect("re-serializes"));
// the nested composite survived as a composite, not flattened
assert!(json.contains(r#"{"composite":{"name":"sma_cross""#), "nested composite preserved");
}
#[test]
fn signal_serializes_to_canonical_golden() {
let signal = crate::test_fixtures::sink_free_sma_cross_signal();
let json = blueprint_to_json(&signal).expect("serializes");
let golden = r#"{"format_version":1,"blueprint":{"name":"sma_cross","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}]}],"output":[{"node":3,"field":0,"name":"bias"}]}}"#;
// pins: canonical field order, format_version envelope, AND omit-defaults
// (the unnamed/unbound `Sub` + `Bias` carry no `name`/`bound` keys; the
// `price` role carries no `source` key).
assert_eq!(json, golden);
}
#[test]
fn unknown_node_type_fails_named() {
// a valid envelope naming a type outside the vocabulary -> clean, named error
let json = r#"{"format_version":1,"blueprint":{"name":"x","nodes":[{"primitive":{"type":"NoSuchNode"}}]}}"#;
// `.err().unwrap()` (not `.unwrap_err()`): the Ok type `Composite` holds a
// build closure and is not `Debug`, which `unwrap_err`'s bound would require.
let err = blueprint_from_json(json, &|t| aura_std::std_vocabulary(t)).err().unwrap();
assert!(matches!(err, LoadError::UnknownNodeType(t) if t == "NoSuchNode"));
}
#[test]
fn unsupported_version_fails_named() {
let json = r#"{"format_version":2,"blueprint":{"name":"x","nodes":[]}}"#;
let err = blueprint_from_json(json, &|t| aura_std::std_vocabulary(t)).err().unwrap();
assert!(matches!(err, LoadError::UnsupportedVersion { found: 2, supported: 1 }));
}
}
+2 -2
View File
@@ -26,7 +26,7 @@ use aura_core::{AnyColumn, Cell, Ctx, Firing, Node, NodeSchema, Scalar, ScalarKi
/// Forwards one field (`from_field`) of a producer's output record into a
/// consumer's input slot. Consuming a whole record is N such edges, one per field
/// (there is no "bind whole record" mechanism).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct Edge {
pub from: usize,
pub to: usize,
@@ -35,7 +35,7 @@ pub struct Edge {
}
/// An input slot the source value is forwarded into each cycle.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct Target {
pub node: usize,
pub slot: usize,
+5
View File
@@ -43,6 +43,7 @@
//! Visualization is never here: it is a downstream consumer node on the streams.
mod blueprint;
mod blueprint_serde;
mod builder;
mod graph_model;
mod harness;
@@ -55,6 +56,10 @@ pub use blueprint::{
BindError, Binder, BlueprintNode, CompileError, Composite, OutField, RandomBinder,
Role, SweepBinder,
};
pub use blueprint_serde::{
blueprint_from_json, blueprint_to_json, BlueprintDoc, CompositeData, LoadError, NodeData,
PrimitiveData, SerializeError, BLUEPRINT_FORMAT_VERSION,
};
pub use builder::{BuildError, GraphBuilder, InPort, NodeHandle, OutPort, RoleHandle};
pub use graph_model::model_to_json;
pub use harness::{
+27
View File
@@ -85,3 +85,30 @@ pub(crate) fn composite_sma_cross_harness() -> (
);
(bp, rx_eq, rx_ex)
}
/// The SMA-cross **signal** graph as a sink-free, param-generic composite: fast
/// SMA (length bound to 2) and slow SMA over one `price` role, their spread, a
/// `Bias`. Output is the `bias` field. No recorder/broker — observation is
/// attached by the caller. This is the canonical round-trip fixture (cycle 0087).
pub(crate) fn sink_free_sma_cross_signal() -> Composite {
Composite::new(
"sma_cross",
vec![
Sma::builder().named("fast").bind("length", Scalar::i64(2)).into(),
Sma::builder().named("slow").into(),
Sub::builder().into(),
Bias::builder().into(),
],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // fast -> Sub slot 0
Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // slow -> Sub slot 1
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // spread -> Bias
],
vec![Role {
name: "price".into(),
targets: vec![Target { node: 0, slot: 0 }, Target { node: 1, slot: 0 }],
source: None,
}],
vec![OutField { node: 3, field: 0, name: "bias".into() }],
)
}