From 78d68fa8ad090ab384363e46e3e28874f72bdaba Mon Sep 17 00:00:00 2001 From: Brummel Date: Tue, 9 Jun 2026 11:09:06 +0200 Subject: [PATCH] spec: 0024 node signature in blueprint Consolidate the node data structure so every node's signature (NodeSchema: inputs/output/params) exists in the blueprint pre-build, declared once. - Signature vs sizing split: NodeSchema becomes the static signature (InputSpec -> PortSpec, lookback removed); the one param-dependent quantity (input buffer lookback) moves to a build-time Node::lookbacks() query. Node::schema() is removed. - PrimitiveBuilder (ex-LeafFactory) carries the schema; the built node no longer re-declares it -> closes the params-declared-twice drift. - Blueprint collapses into Composite: Role gains source: Option; the root is the fully-bound composite (no "main graph"); new error UnboundRootRole. - compile -> FlatGraph -> bootstrap: FlatGraph carries node + signature in parallel; bootstrap sizes from lookbacks(), kind-checks from the signatures. - Renames: BlueprintNode::Leaf -> Primitive, LeafFactory -> PrimitiveBuilder. Behaviour-preserving (C1). Render is out of scope beyond compiling (next cycle). refs #43 #36 --- .../specs/0024-node-signature-in-blueprint.md | 297 ++++++++++++++++++ 1 file changed, 297 insertions(+) create mode 100644 docs/specs/0024-node-signature-in-blueprint.md diff --git a/docs/specs/0024-node-signature-in-blueprint.md b/docs/specs/0024-node-signature-in-blueprint.md new file mode 100644 index 0000000..86103a3 --- /dev/null +++ b/docs/specs/0024-node-signature-in-blueprint.md @@ -0,0 +1,297 @@ +# Node Signature Lives in the Blueprint — Design Spec + +**Date:** 2026-06-09 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +Make every node's **signature** — its declared inputs, output, and params — +exist in the blueprint, **pre-build**, as a single source of truth. Today a +primitive node's I/O interface exists only on the *built* node's `schema()` +(post-build), and its params are declared **twice** (on the value-empty factory +*and* on the built node's schema), kept in agreement only by a per-node lockstep +test. This cycle consolidates the node data structure so that: + +- `NodeSchema` becomes the static **signature** and is declared **once**, on the + pre-build recipe — closing #43 (a value-empty recipe declared no I/O interface) + and #36 (params declared twice). +- The root graph stops being a special type. `Blueprint` collapses into + `Composite`: the root is just the `Composite` whose input roles are all bound + to data feeds. There is no "main graph" — only the **Main-Composite-Node**, the + entry node, governed by the same conditions as every other node. + +This is a **behaviour-preserving** structural refactor (C1): every existing +backtest, composite, and harness test stays green. Graph rendering +(`aura-cli/src/graph.rs`) is **out of scope** beyond *making it compile* — render +tuning is the next cycle, and is explicitly deferred here. + +## Architecture + +One structural axis (a node is `Primitive` or `Composite`), one signature +question for both arms, one runnable form (a fully-bound composite). + +The signature is **fully static** per blueprint — verified across the current +node roster: input kinds + firing, output fields, and params are all +param-independent. `LinComb`'s variable arity is a **factory argument** +(`LinComb::factory(arity)`, fixed per blueprint, C19), not an injected param. The +**only** param-dependent quantity in today's `NodeSchema` is an input's buffer +**lookback** depth (`Sma`'s lookback *is* its injected `length`). That one value +is not a signature concern but a **sizing** concern; it moves out of the +signature to a build-time query consumed only by `bootstrap`. + +The construction pipeline becomes a clean three-noun chain: + +``` +Blueprint ──compile──▶ FlatGraph ──bootstrap──▶ Harness +(nested, named, (flat, index-wired, (runnable: buffers + param-generic) carries node+signature) sized, topo-sorted) +``` + +`compile` does all structural validation **pre-build** through `signature()` +(kind-checks, output-range, fan-in distinguishability — no node built to check +it), then builds each primitive and emits the `FlatGraph`. `bootstrap` reads +kinds/output from the carried signatures and calls `node.lookbacks()` for buffer +depth. + +## Concrete code shapes + +### User-facing: how a node author declares a node (before → after) + +The authoring surface is the node author writing a `Node` impl + its recipe. The +redundancy this cycle removes is visible right here — the params declared twice +collapse to once, and the I/O interface that was trapped in the build closure is +now declared on the recipe. + +```rust +// BEFORE — crates/aura-std/src/sma.rs: params declared twice; I/O only post-build +impl Sma { + pub fn factory() -> LeafFactory { + LeafFactory::new( + "SMA", + vec![ParamSpec { name: "length".into(), kind: ScalarKind::I64 }], // (1) here + |p| Box::new(Sma::new(p[0].as_i64().expect("length slot is I64") as usize)), + ) + } +} +impl Node for Sma { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![InputSpec { kind: ScalarKind::F64, lookback: self.length, firing: Firing::Any }], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + params: vec![ParamSpec { name: "length".into(), kind: ScalarKind::I64 }], // (1) AGAIN — #36 drift + } + } + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { /* unchanged */ } + fn label(&self) -> String { format!("SMA({})", self.length) } +} +``` + +```rust +// AFTER — the signature is declared ONCE on the builder; the built node answers +// only the value-dependent sizing question. +impl Sma { + pub fn builder() -> PrimitiveBuilder { + PrimitiveBuilder::new( + "SMA", + NodeSchema { + inputs: vec![PortSpec { kind: ScalarKind::F64, firing: Firing::Any }], // no lookback + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + params: vec![ParamSpec { name: "length".into(), kind: ScalarKind::I64 }], // the ONE declaration + }, + |p| Box::new(Sma::new(p[0].as_i64().expect("length slot is I64") as usize)), + ) + } +} +impl Node for Sma { + fn lookbacks(&self) -> Vec { vec![self.length] } // the only param-dependent quantity + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { /* unchanged */ } + fn label(&self) -> String { format!("SMA({})", self.length) } + // no schema() — the signature lives on the builder +} +``` + +For a paramless node (`Sub`, `Add`) `lookbacks()` is `vec![1; n_inputs]`; for a +sink (`Recorder`, `output: vec![]`) the signature's `output` stays empty (C8 +unchanged) and `lookbacks()` is one entry per recorded input. + +### Engine type shapes (before → after) — secondary + +```rust +// crates/aura-core/src/node.rs — the signature loses lookback +// BEFORE +struct InputSpec { kind: ScalarKind, lookback: usize, firing: Firing } +struct NodeSchema { inputs: Vec, output: Vec, params: Vec } +trait Node { fn schema(&self) -> NodeSchema; fn eval(..) -> ..; fn label(&self) -> String; } + +// AFTER +struct PortSpec { kind: ScalarKind, firing: Firing } // lookback removed +struct NodeSchema { inputs: Vec, output: Vec, params: Vec } +trait Node { fn lookbacks(&self) -> Vec; fn eval(..) -> ..; fn label(&self) -> String; } +``` + +```rust +// crates/aura-core/src/node.rs — the recipe carries the signature +// BEFORE +struct LeafFactory { name: &'static str, params: Vec, build: Box Box> } +// AFTER +struct PrimitiveBuilder { name: &'static str, schema: NodeSchema, build: Box Box> } +// accessors: name(), schema() -> &NodeSchema, build(slice) -> Box +// params() is now schema().params (no second list) +``` + +```rust +// crates/aura-engine/src/blueprint.rs — Blueprint collapses into Composite; renames; FlatGraph +// BEFORE +enum BlueprintNode { Leaf(LeafFactory), Composite(Composite) } +struct Blueprint { nodes: Vec, sources: Vec, edges: Vec } +struct Composite { name, nodes, edges, input_roles: Vec, params, output } +struct Role { name: String, targets: Vec } + +// AFTER — `Blueprint` deleted; root is a Composite whose roles are all bound +enum BlueprintNode { Primitive(PrimitiveBuilder), Composite(Composite) } +struct Composite { name, nodes, edges, input_roles: Vec, params, output } +struct Role { name: String, targets: Vec, source: Option } +// source == None → open interior port +// source == Some(kind) → bound ingestion feed of `kind`; runnable iff every root role is Some + +impl BlueprintNode { + // ONE signature question for both arms (the heart of "every node has a signature in the blueprint") + fn signature(&self) -> NodeSchema { + match self { + BlueprintNode::Primitive(b) => b.schema().clone(), // declared + BlueprintNode::Composite(c) => derive_signature(c), // inputs = role kinds, output = OutField kinds, + // params = aggregated — pre-build, no build + } + } +} +``` + +```rust +// crates/aura-engine/src/{blueprint.rs,harness.rs} — compile emits FlatGraph; bootstrap consumes it +// BEFORE +fn compile_with_params(self, params: &[Scalar]) -> Result<(Vec>, Vec, Vec), CompileError>; +fn bootstrap(nodes: Vec>, sources: Vec, edges: Vec) -> Result; + +// AFTER +struct FlatGraph { + nodes: Vec>, + signatures: Vec, // parallel to nodes — every flat node is a primitive with a known signature + sources: Vec, // lowered bound roles, in role-declaration order (deterministic, C1) + edges: Vec, +} +fn compile_with_params(self, params: &[Scalar]) -> Result; +fn bootstrap(flat: FlatGraph) -> Result; +// SourceSpec the TYPE survives as the flat ingestion descriptor { kind, targets }; +// what dissolves is the standalone Blueprint.sources field — sources now come from bound roles. +``` + +## Components + +- **`crates/aura-core/src/node.rs`** — `InputSpec` → `PortSpec` (drop `lookback`); + `NodeSchema.inputs: Vec`; `Node::schema()` removed, `Node::lookbacks() + -> Vec` added; `LeafFactory` → `PrimitiveBuilder` carrying `schema: + NodeSchema` (its `params()` is now `schema().params`). +- **`crates/aura-std/src/*.rs`** — every node in the current roster (`sma`, `ema`, + `lincomb`, `exposure`, `add`, `sub`, `sim_broker`, `recorder`): `factory()` → + `builder()` declaring the + full `schema`; `impl Node` drops `schema()`, adds `lookbacks()`. Per-node + `factory_params_match_built_node_schema` lockstep tests are **deleted** (their + subject is now structurally impossible). +- **`crates/aura-engine/src/blueprint.rs`** — delete `struct Blueprint`; promote + its `compile*`/`bootstrap*`/`param_space` methods onto `Composite`; rename + `BlueprintNode::Leaf` → `Primitive`; add `Role.source`; `derive_signature` for + the composite arm; structural validation reads `signature()` **pre-build**; + `compile` emits `FlatGraph` (gathering each primitive's signature parallel to + the built node); add `CompileError::UnboundRootRole { role }`. +- **`crates/aura-engine/src/harness.rs`** — `Harness::bootstrap` takes a + `FlatGraph`; sizing reads `node.lookbacks()`, kind/output checks read the carried + signatures (no `schema()` call on built nodes). +- **`crates/aura-cli/src/graph.rs`** + sample constructors + tests — migrate all + `Blueprint`/`LeafFactory`/`.factory()` call sites to the new types/names. The + renderer is brought to **compile only**; its output may regress this cycle and + is re-tuned next cycle. + +## Data flow + +1. **Authoring.** A blueprint is a root `Composite`: interior `nodes` (each a + `Primitive(PrimitiveBuilder)` or nested `Composite`), `edges`, and `input_roles` + where the root's roles each carry `source: Some(kind)`. +2. **`param_space()`** (pre-build, on `Composite`) walks the tree and concatenates + each node's declared params (from `signature().params`) in lowering order — the + same deterministic order `compile` uses, unchanged. +3. **`compile_with_params`** runs structural validation through `signature()` + **without building** (fan-in distinguishability, role kind-checks, output-range); + then lowers: builds each primitive from its kind-checked param slice, pushing the + built node **and** its `PrimitiveBuilder::schema` into the parallel + `FlatGraph.{nodes,signatures}`; inlines composites (boundary dissolves, C23); + lowers each bound root role into a `FlatGraph.sources` entry `{ kind, flat + targets }` in role order. Returns the `FlatGraph`. +4. **`bootstrap`** sizes each node's input columns from `signatures[i].inputs[k].kind` + and `node.lookbacks()[k]`, lifts firing from the signature, kind-checks sources + and edges against the carried signatures, topo-sorts (Kahn), and returns the + `Harness`. Run-loop and `eval` are untouched. + +## Error handling + +`CompileError` keeps every existing variant (`BadInteriorIndex`, +`RoleKindMismatch`, `OutputPortOutOfRange`, `ParamKindMismatch`, `ParamArity`, +`IndistinguishableFanIn`, `Bootstrap`). The kind/output/fan-in checks now fire +**pre-build** (off `signature()`) — same errors, no node built to reach them. + +One variant is added: + +- **`UnboundRootRole { role }`** — compiling/bootstrapping a root `Composite` that + has an `input_role` with `source: None` (an open port at the root, which has no + parent to bind it). This is the structural form of "only a fully-bound composite + is runnable." + +`BootstrapError` is unchanged in spirit; its kind-mismatch checks now read the +`FlatGraph.signatures` rather than a built node's `schema()`. + +## Testing strategy + +- **Behaviour preservation (C1) — the clamp.** Every existing engine/std/cli test + for backtest output, the MACD composite, fan-in, output bindings, and harness + semantics must stay green unchanged. Same input → same run. +- **Delete the #36 lockstep tests.** `factory_params_match_built_node_schema` (per + node) is removed: with the signature declared once, the drift it guarded is + structurally impossible. Its removal *is* the proof #36 is closed. +- **New: signature is pre-build and uniform.** + - `BlueprintNode::Primitive(b).signature() == b.schema()` (declared), and a + `Composite`'s `signature()` equals the interface derived from its interior + (role kinds in, OutField kinds out, aggregated params) — asserted against the + MACD composite (`fast:i64, slow:i64, signal:i64` in; `macd, signal, histogram` + f64 out). + - `compile` rejects a kind mismatch **without building** — a blueprint wiring an + f64 producer into an i64 slot errors `RoleKindMismatch`/`KindMismatch` with no + node constructed (assert via a builder whose `build` closure panics if called). +- **New: the root is just a bound composite.** A root `Composite` with every role + `source: Some(kind)` bootstraps to a `Harness` that produces byte-identical + output to the equivalent pre-refactor `Blueprint` (pin one existing sample). +- **New: `UnboundRootRole`.** A root `Composite` with one `source: None` role + errors `UnboundRootRole { role }` at compile. +- **New: sizing decoupled.** `node.lookbacks().len() == signature().inputs.len()` + for every std node (a cheap invariant test); `Sma::new(k).lookbacks() == vec![k]`. + +## Acceptance criteria + +aura's criterion: the change measurably **removes redundancy** and reintroduces no +failure class the core constraints exist to eliminate. This cycle: + +- [ ] `NodeSchema` is declared once per node (on `PrimitiveBuilder`); `Node::schema()` + no longer exists; #36's lockstep tests are deleted and the suite is still green. +- [ ] A value-empty node recipe exposes its full I/O signature pre-build + (`PrimitiveBuilder::schema()`), closing #43; `BlueprintNode::signature()` + answers uniformly for both arms without building. +- [ ] `struct Blueprint` is gone; the root is a `Composite` whose bound roles carry + `source: Some(kind)`; a fully-bound root bootstraps with byte-identical output + to the pre-refactor sample (C1); an open root role errors `UnboundRootRole`. +- [ ] `compile` emits a `FlatGraph` and validates structure pre-build; `bootstrap` + consumes the `FlatGraph`, sizing from `lookbacks()` and kind-checking from the + carried signatures. +- [ ] Renames landed (`Leaf`→`Primitive`, `LeafFactory`→`PrimitiveBuilder`); the + workspace builds and `cargo clippy --workspace --all-targets -- -D warnings` + is clean. `aura-cli/src/graph.rs` compiles (render tuning deferred to the next + cycle; visible render regression is acceptable and tracked). +- [ ] Closes #43 and #36.