diff --git a/docs/plans/0005-node-output-record.md b/docs/plans/0005-node-output-record.md new file mode 100644 index 0000000..37594e2 --- /dev/null +++ b/docs/plans/0005-node-output-record.md @@ -0,0 +1,1140 @@ +# Node output is a record — Implementation Plan + +> **Parent spec:** `docs/specs/0005-node-output-record.md` +> +> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run +> this plan. Steps use `- [ ]` checkboxes for tracking. + +**Goal:** Generalize a node's output from one scalar to a record of K ≥ 1 +base-scalar columns (scalar = degenerate K = 1), bound field-wise via +`Edge::from_field`, proven with a neutral OHLCV bundler. + +**Architecture:** `aura-core` changes the contract (`NodeSchema.output: Vec`, +`Node::eval -> Option<&[Scalar]>`); `aura-std` migrates Sma/Sub to the 1-field +degenerate case; `aura-engine` adds `Edge::from_field`, a per-field bootstrap +kind/range check, and a reused-scratch field-indexed forward in the run loop +(`run -> Vec>>`). The design ledger records the C8 revision and +C7 sharpening. + +**Tech Stack:** Rust workspace (`aura-core`, `aura-std`, `aura-engine`); the +`Node`/`NodeSchema`/`Edge`/`Harness` types; `cargo build/test/clippy`. + +--- + +**Files this plan creates or modifies:** + +- Modify: `crates/aura-core/src/node.rs:1-47` — add `FieldSpec`; `NodeSchema.output` → `Vec`; `Node::eval` → `Option<&[Scalar]>`; module/struct/trait docs. +- Modify: `crates/aura-core/src/lib.rs:21,42` — re-export `FieldSpec`; doc wording. +- Modify: `crates/aura-std/src/sma.rs:6-83` — `out` buffer; 1-field schema; slice return; in-crate tests. +- Modify: `crates/aura-std/src/sub.rs:6-62` — `out` buffer; 1-field schema; slice return; manual `Default`; in-crate test. +- Modify: `crates/aura-engine/src/harness.rs:21-329` — `Edge.from_field`; `NodeBox.out_len`; per-field bootstrap check; run loop scratch + return type; module/Edge docs. +- Modify: `crates/aura-engine/src/harness.rs:331-669` (Test) — migrate 3 fixtures to slice return; add `Ohlcv`/`TwoField`; field-binding + must-fail tests; adapt all Edge literals + expected vectors. +- Modify: `docs/design/INDEX.md:168-197` — C8 revision + C7 sharpening. + +--- + +## Task 1: aura-core contract + aura-std degenerate migration + +**Files:** +- Modify: `crates/aura-core/src/node.rs` +- Modify: `crates/aura-core/src/lib.rs` +- Modify: `crates/aura-std/src/sma.rs` +- Modify: `crates/aura-std/src/sub.rs` + +> **Compile-gate note:** this task's gate is **scoped to `aura-core` + `aura-std`**. +> Changing `Node::eval`/`NodeSchema.output` knowingly breaks `aura-engine` (its +> production reads `from.output` and its fixtures impl the old `eval` signature); +> `aura-engine` is repaired in Tasks 2–3. A workspace-wide gate here is +> unsatisfiable by design — do **not** run `--workspace` in this task. + +- [ ] **Step 1: Add `FieldSpec` and change `NodeSchema`/`Node` in `node.rs`** + +In `crates/aura-core/src/node.rs`, replace the `NodeSchema` struct (currently +lines 32-38) and the `Node` trait (currently lines 40-47) with the block below. +This adds `FieldSpec` immediately before `NodeSchema`, changes the `output` field +type, the `eval` return type, and the surrounding doc comments: + +```rust +/// One declared output column of a node's record: its name (metadata for sinks / +/// the playground, C18) and its scalar kind. The position of a `FieldSpec` in +/// `NodeSchema.output` is what an `Edge` binds (`Edge::from_field`); the name is +/// not load-bearing for wiring. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct FieldSpec { + pub name: &'static str, + pub kind: ScalarKind, +} + +/// A node's declared interface: its inputs (in order) and its output record — an +/// ordered list of named base columns; length 1 is a scalar (the degenerate +/// case). Built once at wiring, never on the hot path — the `Vec`s are fine here. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct NodeSchema { + pub inputs: Vec, + pub output: Vec, +} + +/// The universal composable dataflow unit (C8): one output port carrying a record +/// of 1..K base columns, a producer or transformer. `schema` declares the +/// interface; `eval` computes one cycle's row, returning a borrowed slice into a +/// buffer the node owns (`None` = filter / not-yet-warmed-up). `&mut self` because +/// a node may keep its own derived state (and its output buffer). +pub trait Node { + fn schema(&self) -> NodeSchema; + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]>; +} +``` + +- [ ] **Step 2: Update the `node.rs` module doc** + +In `crates/aura-core/src/node.rs`, replace the module-doc line (currently line 3): + +```rust +//! and its output kind via `schema`, and computes one cycle's output via `eval`. +``` + +with: + +```rust +//! and its output record (1..K base columns, C7) via `schema`, and computes one +//! cycle's row via `eval`. +``` + +- [ ] **Step 3: Re-export `FieldSpec` and fix the lib doc in `aura-core/src/lib.rs`** + +In `crates/aura-core/src/lib.rs`, replace the `node` re-export (currently line 42): + +```rust +pub use node::{Firing, InputSpec, Node, NodeSchema}; +``` + +with: + +```rust +pub use node::{FieldSpec, Firing, InputSpec, Node, NodeSchema}; +``` + +Then replace the doc line (currently line 21) that reads: + +```rust +//! [`NodeSchema`] / [`InputSpec`] declaring inputs (kind + lookback) and the +//! single output kind; +``` + +with: + +```rust +//! [`NodeSchema`] / [`InputSpec`] declaring inputs (kind + lookback) and the +//! output record ([`FieldSpec`] columns; length 1 = scalar); +``` + +- [ ] **Step 4: Migrate `Sma` (`aura-std/src/sma.rs`)** + +In `crates/aura-std/src/sma.rs`, replace the import (line 6): + +```rust +use aura_core::{Ctx, Firing, InputSpec, Node, NodeSchema, Scalar, ScalarKind}; +``` + +with: + +```rust +use aura_core::{Ctx, FieldSpec, Firing, InputSpec, Node, NodeSchema, Scalar, ScalarKind}; +``` + +Replace the struct (lines 9-11) and ctor (lines 13-19): + +```rust +/// Simple moving average over the last `length` values of one f64 input. +pub struct Sma { + length: usize, + out: [Scalar; 1], +} + +impl Sma { + /// Build an SMA of window `length` (must be >= 1). + pub fn new(length: usize) -> Self { + assert!(length >= 1, "SMA length must be >= 1"); + Self { length, out: [Scalar::F64(0.0)] } + } +} +``` + +Replace the `impl Node for Sma` body (lines 21-44) with: + +```rust +impl Node for Sma { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![InputSpec { + kind: ScalarKind::F64, + lookback: self.length, + firing: Firing::Any, + }], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + } + } + + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + let w = ctx.f64_in(0); + if w.len() < self.length { + return None; // not yet warmed up + } + let mut sum = 0.0; + for k in 0..self.length { + sum += w[k]; // index 0 = newest (financial indexing) + } + self.out[0] = Scalar::F64(sum / self.length as f64); + Some(&self.out) + } +} +``` + +- [ ] **Step 5: Adapt the `Sma` in-crate tests** + +In `crates/aura-std/src/sma.rs`, replace the body of `sma_warms_up_then_tracks_the_window_mean` +(the loop currently at lines 66-69) with: + +```rust + for (v, want) in feed.iter().zip(expect) { + inputs[0].push(Scalar::F64(*v)).unwrap(); + let got = sma.eval(Ctx::new(&inputs)); + match want { + None => assert_eq!(got, None), + Some(m) => assert_eq!(got, Some([Scalar::F64(m)].as_slice())), + } + } +``` + +Replace the two assertions in `sma_length_one_is_identity` (currently lines 77-81): + +```rust + inputs[0].push(Scalar::F64(7.0)).unwrap(); + assert_eq!(sma.eval(Ctx::new(&inputs)), Some([Scalar::F64(7.0)].as_slice())); + + inputs[0].push(Scalar::F64(9.0)).unwrap(); + assert_eq!(sma.eval(Ctx::new(&inputs)), Some([Scalar::F64(9.0)].as_slice())); +``` + +- [ ] **Step 6: Migrate `Sub` (`aura-std/src/sub.rs`)** + +In `crates/aura-std/src/sub.rs`, replace the import (line 6): + +```rust +use aura_core::{Ctx, Firing, InputSpec, Node, NodeSchema, Scalar, ScalarKind}; +``` + +with: + +```rust +use aura_core::{Ctx, FieldSpec, Firing, InputSpec, Node, NodeSchema, Scalar, ScalarKind}; +``` + +Replace the struct + ctor (lines 10-18) with (the `#[derive(Default)]` is dropped +because `Scalar` has no `Default`; a manual `Default` keeps clippy's +`new_without_default` quiet): + +```rust +/// Two-input f64 difference: input 0 minus input 1. Emits `None` until both +/// inputs have a value. +pub struct Sub { + out: [Scalar; 1], +} + +impl Sub { + /// Build a `Sub` node. + pub fn new() -> Self { + Self { out: [Scalar::F64(0.0)] } + } +} + +impl Default for Sub { + fn default() -> Self { + Self::new() + } +} +``` + +Replace the `impl Node for Sub` body (lines 20-39) with: + +```rust +impl Node for Sub { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![ + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }, + ], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + } + } + + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + let a = ctx.f64_in(0); + let b = ctx.f64_in(1); + if a.is_empty() || b.is_empty() { + return None; + } + self.out[0] = Scalar::F64(a[0] - b[0]); + Some(&self.out) + } +} +``` + +- [ ] **Step 7: Adapt the `Sub` in-crate test** + +In `crates/aura-std/src/sub.rs`, replace the two assertions in +`sub_is_difference_once_both_inputs_present` (currently lines 55-60): + +```rust + // only input 0 present -> None + inputs[0].push(Scalar::F64(10.0)).unwrap(); + assert_eq!(sub.eval(Ctx::new(&inputs)), None); + + // both present -> a - b + inputs[1].push(Scalar::F64(4.0)).unwrap(); + assert_eq!(sub.eval(Ctx::new(&inputs)), Some([Scalar::F64(6.0)].as_slice())); +``` + +- [ ] **Step 8: Scoped gate — aura-core + aura-std build/test/clippy** + +Run: `cargo build -p aura-core -p aura-std --all-targets` +Expected: `Finished` — 0 errors. (`aura-engine` is NOT built here; it is knowingly broken until Task 3.) + +Run: `cargo test -p aura-core -p aura-std` +Expected: PASS — `aura-core` 19 passed, `aura-std` 3 passed (`sma` 2 + `sub` 1), 0 failed. + +Run: `cargo clippy -p aura-core -p aura-std --all-targets -- -D warnings` +Expected: `Finished` — 0 warnings. + +--- + +## Task 2: aura-engine production — Edge.from_field, per-field bootstrap, field-indexed forward + +**Files:** +- Modify: `crates/aura-engine/src/harness.rs` (production only; the `#[cfg(test)]` module is Task 3) + +> **Compile-gate note:** this task changes the `bootstrap`/`run` signatures and the +> `Edge` struct, which knowingly breaks the in-module `#[cfg(test)]` fixtures and +> tests. Its gate is therefore a **partial `--lib`** build that excludes test code; +> the tests are rewritten and the full gate runs in Task 3. Do **not** run +> `cargo test` or `--all-targets` in this task. + +- [ ] **Step 1: Add `from_field` to `Edge` and update its doc** + +In `crates/aura-engine/src/harness.rs`, replace the `Edge` doc + struct (lines 23-29): + +```rust +/// Forwards one field (`from_field`) of a producer's output record into a +/// consumer's input slot. Consuming a whole record is N such edges, one per field +/// (there is no "bind whole record" mechanism). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Edge { + pub from: usize, + pub to: usize, + pub slot: usize, + pub from_field: usize, +} +``` + +- [ ] **Step 2: Add `out_len` to `NodeBox`** + +In `crates/aura-engine/src/harness.rs`, replace the `NodeBox` struct (lines 72-77): + +```rust +struct NodeBox { + node: Box, + inputs: Vec, + firing: Vec, + slots: Vec, + out_len: usize, +} +``` + +- [ ] **Step 3: Set `out_len` at bootstrap** + +In `crates/aura-engine/src/harness.rs`, replace the per-node box construction loop +(lines 123-137) with (it lifts the output width from the schema alongside firing +and slots): + +```rust + let mut boxes: Vec = Vec::with_capacity(n); + for (nd, schema) in nodes.into_iter().zip(schemas.iter()) { + let inputs: Vec = schema + .inputs + .iter() + .map(|spec| AnyColumn::with_capacity(spec.kind, spec.lookback)) + .collect(); + let firing: Vec = schema.inputs.iter().map(|spec| spec.firing).collect(); + let slots: Vec = schema + .inputs + .iter() + .map(|_| SlotState { fresh_at: 0, last_ts: Timestamp(i64::MIN) }) + .collect(); + let out_len = schema.output.len(); + boxes.push(NodeBox { node: nd, inputs, firing, slots, out_len }); + } +``` + +- [ ] **Step 4: Per-field edge kind/range check at bootstrap** + +In `crates/aura-engine/src/harness.rs`, replace the edge-check loop (lines 153-166). +Also update the leading comment so it reads "field kind" rather than "output kind": + +```rust + // edges: indices in range, producer output field kind == consumer slot kind + let mut out_edges: Vec> = vec![Vec::new(); n]; + for &e in &edges { + let from = schemas.get(e.from).ok_or(BootstrapError::BadIndex)?; + let to = schemas.get(e.to).ok_or(BootstrapError::BadIndex)?; + let field = from.output.get(e.from_field).ok_or(BootstrapError::BadIndex)?; + let slot = to.inputs.get(e.slot).ok_or(BootstrapError::BadIndex)?; + if field.kind != slot.kind { + return Err(BootstrapError::KindMismatch { + producer: field.kind, + consumer: slot.kind, + }); + } + out_edges[e.from].push(e); + } +``` + +- [ ] **Step 5: Change the `run` signature and declare the scratch row** + +In `crates/aura-engine/src/harness.rs`, replace the `run` signature (line 205): + +```rust + pub fn run(&mut self, streams: Vec>) -> Vec>> { +``` + +Then, immediately after the `let mut out = Vec::new();` line (line 221), add one +reused scratch buffer (allocated once; its capacity stabilizes after the first +fires, so the inter-node forward path allocates nothing per cycle): + +```rust + let mut scratch: Vec = Vec::new(); +``` + +- [ ] **Step 6: Field-indexed forward in the run loop** + +In `crates/aura-engine/src/harness.rs`, replace the per-node evaluate/forward block +(lines 254-278) — from `let mut observed = None;` through `out.push(observed);` — +with the block below. `result` is `Option<&[Scalar]>` (which is `Copy`, since +`&[Scalar]` is a shared reference); the observed row is cloned out, the producer +row is copied into `scratch` (ending the borrow), and each out-edge forwards its +single field: + +```rust + let mut observed: Option> = None; + for &nidx in topo.iter() { + let out_len = nodes[nidx].out_len; + let fired = { + let nb = &nodes[nidx]; + fires(&nb.firing, &nb.slots, cycle_id, ts) + }; + if !fired { + continue; // hold: no eval, no push + } + let result: Option<&[Scalar]> = { + let nb = &mut nodes[nidx]; + nb.node.eval(Ctx::new(&nb.inputs)) + }; + if nidx == observe { + observed = result.map(|row| row.to_vec()); + } + if let Some(row) = result { + debug_assert_eq!(row.len(), out_len, "node returned a row of the wrong width"); + scratch.clear(); + scratch.extend_from_slice(row); + for e in out_edges[nidx].iter() { + let nb = &mut nodes[e.to]; + nb.inputs[e.slot] + .push(scratch[e.from_field]) + .expect("edge kind checked at wiring"); + nb.slots[e.slot] = SlotState { fresh_at: cycle_id, last_ts: ts }; + } + } + } + out.push(observed); +``` + +- [ ] **Step 7: Update the harness module doc for the record-forward framing** + +In `crates/aura-engine/src/harness.rs`, replace the module-doc sentence about the +node owning its input columns (line 8, currently +`//! the cycle-0002 shape), so \`Ctx\` is unchanged.`) with the two lines below +(the `Ctx`-unchanged claim stays true; the addition states the output-record +forward): + +```rust +//! the cycle-0002 shape), so `Ctx` is unchanged. A node's `eval` returns a +//! borrowed record (`Option<&[Scalar]>`); each out-edge forwards one field of it +//! (`Edge::from_field`) into a consumer slot, so the K fields are co-fresh (C6). +``` + +- [ ] **Step 8: Partial gate — aura-engine lib build/clippy (tests excluded)** + +Run: `cargo build -p aura-engine --lib` +Expected: `Finished` — 0 errors. (The `#[cfg(test)]` module is NOT compiled by +`--lib`; it is rewritten in Task 3.) + +Run: `cargo clippy -p aura-engine --lib -- -D warnings` +Expected: `Finished` — 0 warnings. + +--- + +## Task 3: aura-engine tests — fixtures migration, OHLCV proof, must-fail cases + +**Files:** +- Modify: `crates/aura-engine/src/harness.rs` (the `#[cfg(test)] mod tests` block, lines 331-669) + +- [ ] **Step 1: Add `FieldSpec` to the test-module imports** + +In `crates/aura-engine/src/harness.rs`, replace the test-module import (line 337): + +```rust + use aura_core::{FieldSpec, InputSpec, NodeSchema}; +``` + +- [ ] **Step 2: Migrate `AsOfSum` to the slice return** + +In `crates/aura-engine/src/harness.rs`, replace the `AsOfSum` fixture (lines 350-369) +with (it gains a 1-field output buffer; behaviour identical): + +```rust + struct AsOfSum { + out: [Scalar; 1], + } + impl Node for AsOfSum { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![ + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }, + ], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + } + } + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + let a = ctx.f64_in(0); + let b = ctx.f64_in(1); + if a.is_empty() || b.is_empty() { + return None; + } + self.out[0] = Scalar::F64(a[0] + b[0]); + Some(&self.out) + } + } +``` + +- [ ] **Step 3: Migrate `BarrierSum` to the slice return** + +In `crates/aura-engine/src/harness.rs`, replace the `BarrierSum` fixture (lines 373-387): + +```rust + struct BarrierSum { + out: [Scalar; 1], + } + impl Node for BarrierSum { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![ + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + ], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + } + } + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + self.out[0] = Scalar::F64(ctx.f64_in(0)[0] + ctx.f64_in(1)[0]); + Some(&self.out) + } + } +``` + +- [ ] **Step 4: Migrate `MixedSum` to the slice return** + +In `crates/aura-engine/src/harness.rs`, replace the `MixedSum` fixture (lines 392-413): + +```rust + struct MixedSum { + out: [Scalar; 1], + } + impl Node for MixedSum { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![ + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }, + ], + output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }], + } + } + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + let a = ctx.f64_in(0); + let b = ctx.f64_in(1); + let c = ctx.f64_in(2); + if a.is_empty() || b.is_empty() || c.is_empty() { + return None; + } + self.out[0] = Scalar::F64(a[0] + b[0] + c[0]); + Some(&self.out) + } + } +``` + +- [ ] **Step 5: Add the `Ohlcv` and `TwoField` fixtures** + +In `crates/aura-engine/src/harness.rs`, immediately after the `MixedSum` fixture +(after the block replaced in Step 4), insert these two new fixtures: + +```rust + /// A neutral multi-field producer: five f64 inputs bundled into one 5-field + /// record. No trading-domain logic — it proves the K > 1 output mechanism in + /// isolation. The five inputs are a Barrier(0) group, so the node emits one + /// complete bar only when all five share the cycle's timestamp. + struct Ohlcv { + out: [Scalar; 5], + } + impl Node for Ohlcv { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![ + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) }, + ], + output: vec![ + FieldSpec { name: "open", kind: ScalarKind::F64 }, + FieldSpec { name: "high", kind: ScalarKind::F64 }, + FieldSpec { name: "low", kind: ScalarKind::F64 }, + FieldSpec { name: "close", kind: ScalarKind::F64 }, + FieldSpec { name: "volume", kind: ScalarKind::F64 }, + ], + } + } + fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> { + for i in 0..5 { + let w = ctx.f64_in(i); + if w.is_empty() { + return None; // not yet warmed + } + self.out[i] = Scalar::F64(w[0]); + } + Some(&self.out) // one 5-field record, all fields co-fresh + } + } + + /// A producer whose output record mixes kinds: field 0 is f64, field 1 is i64. + /// Used only to prove the bootstrap kind check is per-field (field 0 would bind + /// into an f64 slot; field 1 would not). Its `eval` never runs in these tests — + /// bootstrap rejects the wiring first. + struct TwoField { + out: [Scalar; 2], + } + impl Node for TwoField { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Any }], + output: vec![ + FieldSpec { name: "f", kind: ScalarKind::F64 }, + FieldSpec { name: "i", kind: ScalarKind::I64 }, + ], + } + } + fn eval(&mut self, _ctx: Ctx<'_>) -> Option<&[Scalar]> { + self.out[0] = Scalar::F64(0.0); + self.out[1] = Scalar::I64(0); + Some(&self.out) + } + } +``` + +- [ ] **Step 6: Adapt `chain_source_sma_runs`** + +In `crates/aura-engine/src/harness.rs`, replace the expected vector of +`chain_source_sma_runs` (the `assert_eq!` currently at lines 426-435) with the +record-wrapped form: + +```rust + assert_eq!( + out, + vec![ + None, + None, + Some(vec![Scalar::F64(2.0)]), + Some(vec![Scalar::F64(3.0)]), + Some(vec![Scalar::F64(4.0)]), + ] + ); +``` + +- [ ] **Step 7: Adapt `fan_out_join_dag_runs_deterministically`** + +In `crates/aura-engine/src/harness.rs`, replace the two `Edge` literals (line 449) +with `from_field`-bearing forms: + +```rust + vec![Edge { from: 0, to: 2, slot: 0, from_field: 0 }, Edge { from: 1, to: 2, slot: 1, from_field: 0 }], +``` + +Replace the expected vector (currently lines 459-469) with: + +```rust + assert_eq!( + out, + vec![ + None, + None, + None, + Some(vec![Scalar::F64(2.0)]), + Some(vec![Scalar::F64(2.0)]), + Some(vec![Scalar::F64(2.0)]), + ] + ); +``` + +- [ ] **Step 8: Adapt the three fixture-construction sites and the mode-A/mode-B/mixed expected vectors** + +In `crates/aura-engine/src/harness.rs`: + +Replace `vec![Box::new(AsOfSum)]` (line 482) with: + +```rust + vec![Box::new(AsOfSum { out: [Scalar::F64(0.0)] })], +``` + +Replace the `mode_a_as_of_fires_on_any_fresh_and_holds` expected vector (lines 498-507): + +```rust + assert_eq!( + out, + vec![ + None, + None, + Some(vec![Scalar::F64(120.0)]), + Some(vec![Scalar::F64(130.0)]), + Some(vec![Scalar::F64(140.0)]), + Some(vec![Scalar::F64(240.0)]), + ] + ); +``` + +Replace `vec![Box::new(BarrierSum)]` in `mode_b_barrier_fires_only_on_timestamp_coincidence` +(line 519) with: + +```rust + vec![Box::new(BarrierSum { out: [Scalar::F64(0.0)] })], +``` + +Replace the `mode_b_barrier_fires_only_on_timestamp_coincidence` expected vector (lines 535-544): + +```rust + assert_eq!( + out, + vec![ + None, + None, + Some(vec![Scalar::F64(120.0)]), + None, + None, + Some(vec![Scalar::F64(240.0)]), + ] + ); +``` + +Replace `vec![Box::new(MixedSum)]` (line 597) with: + +```rust + vec![Box::new(MixedSum { out: [Scalar::F64(0.0)] })], +``` + +Replace the `mixed_a_and_b_or_combine_on_one_node` expected vector (lines 616-625): + +```rust + assert_eq!( + out, + vec![ + None, + None, + Some(vec![Scalar::F64(221.0)]), + Some(vec![Scalar::F64(223.0)]), + None, + ] + ); +``` + +- [ ] **Step 9: Adapt `within_source_diamond_rejoin_barrier_fires`** + +In `crates/aura-engine/src/harness.rs`, replace the `BarrierSum` construction in the +node list (line 560, `vec![Box::new(Sma::new(2)), Box::new(Sma::new(4)), Box::new(BarrierSum)]`): + +```rust + vec![Box::new(Sma::new(2)), Box::new(Sma::new(4)), Box::new(BarrierSum { out: [Scalar::F64(0.0)] })], +``` + +Replace the two `Edge` literals (line 565): + +```rust + vec![Edge { from: 0, to: 2, slot: 0, from_field: 0 }, Edge { from: 1, to: 2, slot: 1, from_field: 0 }], +``` + +Replace the expected vector (lines 577-586): + +```rust + assert_eq!( + out, + vec![ + None, + None, + None, + Some(vec![Scalar::F64(28.0)]), + Some(vec![Scalar::F64(32.0)]), + Some(vec![Scalar::F64(36.0)]), + ] + ); +``` + +- [ ] **Step 10: Adapt `bootstrap_rejects_a_cycle`** + +In `crates/aura-engine/src/harness.rs`, replace the two `Edge` literals in +`bootstrap_rejects_a_cycle` (line 634): + +```rust + vec![Edge { from: 0, to: 1, slot: 0, from_field: 0 }, Edge { from: 1, to: 0, slot: 0, from_field: 0 }], +``` + +(`bootstrap_rejects_a_kind_mismatch` and `bootstrap_rejects_a_bad_index` use a +source kind mismatch and a bad observe index respectively — no `Edge` literal, no +`run` output — so they need no change.) + +- [ ] **Step 11: Add the K > 1 output test** + +In `crates/aura-engine/src/harness.rs`, at the end of the `mod tests` block (after +`bootstrap_rejects_a_bad_index`, before the closing `}` of the module), add: + +```rust + /// Build five timestamp-aligned f64 sources feeding Ohlcv's five barrier slots. + fn ohlcv_streams() -> Vec> { + vec![ + f64_stream(&[(1, 10.0), (2, 20.0)]), // open + f64_stream(&[(1, 15.0), (2, 25.0)]), // high + f64_stream(&[(1, 8.0), (2, 19.0)]), // low + f64_stream(&[(1, 12.0), (2, 22.0)]), // close + f64_stream(&[(1, 100.0), (2, 200.0)]), // volume + ] + } + + fn ohlcv_sources() -> Vec { + (0..5) + .map(|slot| SourceSpec { + kind: ScalarKind::F64, + targets: vec![Target { node: 0, slot }], + }) + .collect() + } + + #[test] + fn ohlcv_bundles_five_field_record() { + // node 0 = Ohlcv; five sources feed O/H/L/C/V; observe node 0. The barrier + // fires once all five share the timestamp, so each bar appears on the fifth + // cycle of its timestamp (the four partial cycles hold -> None). + let mut h = Harness::bootstrap( + vec![Box::new(Ohlcv { out: [Scalar::F64(0.0); 5] })], + ohlcv_sources(), + vec![], + 0, + ) + .expect("valid"); + let out = h.run(ohlcv_streams()); + assert_eq!( + out, + vec![ + None, + None, + None, + None, + Some(vec![ + Scalar::F64(10.0), + Scalar::F64(15.0), + Scalar::F64(8.0), + Scalar::F64(12.0), + Scalar::F64(100.0), + ]), + None, + None, + None, + None, + Some(vec![ + Scalar::F64(20.0), + Scalar::F64(25.0), + Scalar::F64(19.0), + Scalar::F64(22.0), + Scalar::F64(200.0), + ]), + ] + ); + } +``` + +- [ ] **Step 12: Add the field-wise binding proof (high − low) + determinism** + +In `crates/aura-engine/src/harness.rs`, after the test added in Step 11, add: + +```rust + #[test] + fn edge_binds_single_field_high_minus_low() { + // nodes [Ohlcv (0), Sub (1)]; Sub binds field 1 (high) and field 2 (low) + // of the Ohlcv record -> high - low == the bar range. Observing Sub proves + // from_field routes the right columns (not field 0), and that the two bound + // fields are co-fresh (Sub's Any inputs both fire in the bar's cycle). + let build = || { + Harness::bootstrap( + vec![ + Box::new(Ohlcv { out: [Scalar::F64(0.0); 5] }), + Box::new(Sub::new()), + ], + ohlcv_sources(), + vec![ + Edge { from: 0, to: 1, slot: 0, from_field: 1 }, // high + Edge { from: 0, to: 1, slot: 1, from_field: 2 }, // low + ], + 1, + ) + .expect("valid DAG") + }; + let mut h = build(); + let out = h.run(ohlcv_streams()); + // bar1: 15 - 8 = 7; bar2: 25 - 19 = 6 (each on the bar's fifth cycle). + assert_eq!( + out, + vec![ + None, + None, + None, + None, + Some(vec![Scalar::F64(7.0)]), + None, + None, + None, + None, + Some(vec![Scalar::F64(6.0)]), + ] + ); + + // determinism (C1): a second identical run is bit-identical + let mut h2 = build(); + assert_eq!(h2.run(ohlcv_streams()), out); + } +``` + +- [ ] **Step 13: Add the distinct-fields proof (close − open)** + +In `crates/aura-engine/src/harness.rs`, after the test added in Step 12, add: + +```rust + #[test] + fn distinct_edges_read_distinct_fields() { + // Same Ohlcv, a different consumer: Sub binds field 3 (close) and field 0 + // (open) -> close - open. Proves two different edges on one record read two + // different fields (3 and 0, neither of them the high/low pair above). + let mut h = Harness::bootstrap( + vec![ + Box::new(Ohlcv { out: [Scalar::F64(0.0); 5] }), + Box::new(Sub::new()), + ], + ohlcv_sources(), + vec![ + Edge { from: 0, to: 1, slot: 0, from_field: 3 }, // close + Edge { from: 0, to: 1, slot: 1, from_field: 0 }, // open + ], + 1, + ) + .expect("valid DAG"); + let out = h.run(ohlcv_streams()); + // bar1: 12 - 10 = 2; bar2: 22 - 20 = 2. + assert_eq!( + out, + vec![ + None, + None, + None, + None, + Some(vec![Scalar::F64(2.0)]), + None, + None, + None, + None, + Some(vec![Scalar::F64(2.0)]), + ] + ); + } +``` + +- [ ] **Step 14: Add the must-fail tests (from_field OOB; per-field kind mismatch)** + +In `crates/aura-engine/src/harness.rs`, after the test added in Step 13, add: + +```rust + #[test] + fn bootstrap_rejects_from_field_out_of_range() { + // Sma(0) has a 1-field output (index 0 only); an edge reading field 9 is + // out of range -> BadIndex (caught before any kind check). + let err = Harness::bootstrap( + vec![Box::new(Sma::new(1)), Box::new(Sma::new(1))], + vec![SourceSpec { kind: ScalarKind::F64, targets: vec![Target { node: 0, slot: 0 }] }], + vec![Edge { from: 0, to: 1, slot: 0, from_field: 9 }], + 0, + ) + .unwrap_err(); + assert_eq!(err, BootstrapError::BadIndex); + } + + #[test] + fn bootstrap_rejects_per_field_kind_mismatch() { + // TwoField(0) output: field 0 f64, field 1 i64. Binding field 1 (i64) into + // Sma(1)'s f64 input slot is a per-field kind mismatch -> KindMismatch. (The + // mismatch is field-specific: from_field 0 would have matched.) + let err = Harness::bootstrap( + vec![Box::new(TwoField { out: [Scalar::F64(0.0), Scalar::I64(0)] }), Box::new(Sma::new(1))], + vec![SourceSpec { kind: ScalarKind::F64, targets: vec![Target { node: 0, slot: 0 }] }], + vec![Edge { from: 0, to: 1, slot: 0, from_field: 1 }], + 1, + ) + .unwrap_err(); + assert_eq!( + err, + BootstrapError::KindMismatch { producer: ScalarKind::I64, consumer: ScalarKind::F64 } + ); + } +``` + +- [ ] **Step 15: Full workspace gate + purity grep** + +Run: `cargo build --workspace --all-targets` +Expected: `Finished` — 0 errors. + +Run: `cargo test --workspace` +Expected: PASS — 0 failed. Per-crate: `aura-core` 19 passed; `aura-std` 3 passed; +`aura-engine` 14 passed (the 9 prior tests + `ohlcv_bundles_five_field_record`, +`edge_binds_single_field_high_minus_low`, `distinct_edges_read_distinct_fields`, +`bootstrap_rejects_from_field_out_of_range`, `bootstrap_rejects_per_field_kind_mismatch`). + +Run: `cargo clippy --workspace --all-targets -- -D warnings` +Expected: `Finished` — 0 warnings. + +Run: `grep -rnE 'RefCell|Rc<|dyn Any' crates/*/src` +Expected: no output (exit code 1) — the record is a bundle of base columns, no +fifth type, no heterogeneous payload. + +--- + +## Task 4: Design ledger — C8 revision + C7 sharpening + +**Files:** +- Modify: `docs/design/INDEX.md` (C7 at lines 168-179; C8 at lines 181-197) + +> This is a docs-only task (no compile gate). The glossary `composite`/`node` +> record-reality pass is explicitly an **audit-time** follow-up, NOT this plan. + +- [ ] **Step 1: Sharpen C7 — composite-bundle is the node-output model** + +In `docs/design/INDEX.md`, in the C7 guarantee, replace the sentence (line 170-171): + +``` +sim, as columnar Structure-of-Arrays. Composite streams +(OHLCV) are bundles of base columns. Edges are type-erased to these four kinds; +``` + +with: + +``` +sim, as columnar Structure-of-Arrays. Composite streams +(OHLCV) are bundles of base columns — this is the **node-output model** too: a +node emits a record of 1..K base columns (C8), each forwarded field-wise to a +consumer slot; the bundle is structural, never a fifth scalar type. Edges are +type-erased to these four kinds; +``` + +- [ ] **Step 2: Revise C8 — one output port carrying a record** + +In `docs/design/INDEX.md`, in the C8 guarantee, replace `eval(ctx) -> Option` +(line 185) with `eval(ctx) -> Option<&[Scalar]>`. Concretely, replace (lines 184-185): + +``` +param-space the optimizer sweeps, C12/C19/C20) + `eval(ctx) -> Option`. The +engine provides read-only, zero-copy windows into each input's SoA ring buffer +``` + +with: + +``` +param-space the optimizer sweeps, C12/C19/C20) + `eval(ctx) -> Option<&[Scalar]>`. The +engine provides read-only, zero-copy windows into each input's SoA ring buffer +``` + +- [ ] **Step 3: Revise the C8 output-arity sentence** + +In `docs/design/INDEX.md`, replace (lines 189-192): + +``` +not-yet-warmed-up. A node is a **producer, a consumer, or both**: a +producer/transformer exposes **at most one** output (one series per node); a +**pure consumer (sink)** — chart, equity, logger — has **no** output. Sources +are pure producers; sinks are pure consumers. +``` + +with: + +``` +not-yet-warmed-up. A node is a **producer, a consumer, or both**: a +producer/transformer exposes **one output port**, whose payload is a **record of +1..K base-scalar columns** (a scalar is the degenerate K=1 record; an `eval` +returns a borrowed row, one value per column); a **pure consumer (sink)** — chart, +equity, logger — has **no** output. Sources are pure producers; sinks are pure +consumers. +``` + +- [ ] **Step 4: Revise the C8 Forbids + add the cycle-0005 realization** + +In `docs/design/INDEX.md`, replace the C8 Forbids/Why block (lines 193-197): + +``` +**Forbids.** A node sizing/growing its input lookback at runtime; more than one +output per node (model as multiple nodes); copy-on-read of input history. +**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage +lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized +buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop). +``` + +with: + +``` +**Forbids.** A node sizing/growing its input lookback at runtime; more than one +output **port** per node; a fifth scalar type or a heterogeneous output payload +(a record is a bundle of base columns, C7); copy-on-read of input history. +**Why.** Engine-provided windows mean LLM-authored code cannot mis-manage +lookback bookkeeping, and history passes through zero-copy. Fixed, pre-sized +buffers suit deterministic, pre-dimensioned sims (no realloc in the hot loop). +**Realization (cycle 0005).** `NodeSchema.output` is a `Vec` (named base +columns; length 1 = scalar). Binding is **field-wise only**: `Edge::from_field` +selects one producer column per edge; consuming a whole record is N edges (no +"bind whole record" mechanism). The K fields of one record are **co-fresh by +construction** (one `eval`, one timestamp), so C6 is untouched. `eval` returns +`Option<&[Scalar]>` — a borrowed row into a node-owned buffer — so the forward +path allocates nothing per cycle (C7). +``` + +- [ ] **Step 5: Sanity-check the ledger edits** + +Run: `grep -n 'Option<&\[Scalar\]>' docs/design/INDEX.md` +Expected: at least one hit (the C8 guarantee line). + +Run: `grep -n 'one series per node' docs/design/INDEX.md` +Expected: no output (exit 1) — the old single-output framing is gone from C8. + +Run: `grep -n 'Realization (cycle 0005)' docs/design/INDEX.md` +Expected: one hit — the cycle-0005 realization note is present.