From 4f72701e3bd3acf021d46bb31452a3775e046d5d Mon Sep 17 00:00:00 2001 From: Brummel Date: Wed, 3 Jun 2026 12:05:38 +0200 Subject: [PATCH] spec: cycle 0002 node contract and ctx MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second walking-skeleton cycle, on top of the 0001 streaming substrate: the Node contract (C8) and its evaluation context. Scope (one iteration): - `Node` trait — `schema() -> NodeSchema` (inputs: kind + lookback; output kind) + `eval(&mut self, Ctx) -> Option` (None = filter / not warmed). - `Ctx` — a Copy borrow-wrapper handing eval zero-copy, financial-indexed Windows per input (`ctx.f64_in(i)[k]`, index 0 = newest). - Read-side `AnyColumn::as_f64/as_i64/as_bool/as_ts` — mirrors the existing write-side `as_*_mut` and closes the cycle-0001 audit gap. - `Sma` in aura-std — the worked producer node (the skeleton's first block), driven by a hand-written test that mimics the future sim loop. Deliberate deferrals (recorded as decisions, not gaps): the sim loop (C4), freshness gating (C5), firing policies (C6, so InputSpec carries no firing field yet), schema-level tunable params (C12/C19), and sinks / no-output nodes (C8 consumer side). The hand-driven test stands in for the loop. Grounding-check PASS: all load-bearing assumptions about the 0001 substrate (Column/Window/AnyColumn/Scalar, financial indexing) ratified by green tests. refs walking-skeleton --- docs/specs/0002-node-contract-and-ctx.md | 294 +++++++++++++++++++++++ 1 file changed, 294 insertions(+) create mode 100644 docs/specs/0002-node-contract-and-ctx.md diff --git a/docs/specs/0002-node-contract-and-ctx.md b/docs/specs/0002-node-contract-and-ctx.md new file mode 100644 index 0000000..e383a9e --- /dev/null +++ b/docs/specs/0002-node-contract-and-ctx.md @@ -0,0 +1,294 @@ +# The Node Contract and Ctx — Design Spec + +**Date:** 2026-06-03 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +Deliver the **node contract** (C8) and its **evaluation context** on top of the +cycle-0001 streaming substrate: the `Node` trait (`schema` + `eval`), a `Ctx` +that hands a node read-only, zero-copy windows into its inputs, and one worked +producer node (a simple moving average) that proves a node is authorable and +evaluable. The node is exercised by a hand-driven test that mimics, by hand, +exactly what the sim loop will later do — push a fresh input value, call `eval`, +collect the output — so the contract is fully proven this cycle without an engine. + +This is the first slice of the walking-skeleton milestone *after* the substrate. +It deliberately stops short of the sim loop (C4), freshness gating (C5), firing +policies (C6), sources (C11), and ingestion (C3): those build on a proven node +contract in later cycles. This cycle nails the single highest-leverage interface +— the one every future node forever implements — and closes the read-side gap the +cycle-0001 audit noted (`AnyColumn` had write-side `as_*_mut` accessors but no +symmetric read-side window). + +## Architecture + +Three additions to `aura-core`, plus one worked node in `aura-std`: + +1. **`Node` trait** (`aura-core/src/node.rs`). `schema(&self) -> NodeSchema` + declares the node's inputs (scalar kind + required lookback depth) and its + single output kind; `eval(&mut self, ctx: Ctx<'_>) -> Option` computes + this cycle's output. `&mut self` because a node may keep its own derived + state (C8); `Option` because `None` = filter / not-yet-warmed-up. + +2. **`Ctx`** (`aura-core/src/ctx.rs`). A thin, `Copy` borrow-wrapper over the + node's input columns, in schema-declared order. It exposes one typed accessor + per scalar kind — `f64_in(i)`, `i64_in(i)`, `bool_in(i)`, `ts_in(i)` — each + returning a zero-copy `Window` (cycle 0001) into input `i`, with financial + indexing (index 0 = newest). The kind was checked at wiring; a kind mismatch + here is an engine bug and panics with a clear message. + +3. **Read-side accessors on `AnyColumn`** (`aura-core/src/any.rs`). `as_f64`, + `as_i64`, `as_bool`, `as_ts` return `Option<&Column>` — the symmetric + read-side of the existing `as_*_mut`, and the mechanism `Ctx` uses to obtain a + typed window from a type-erased edge. This closes the cycle-0001 audit gap. + +4. **`Sma`** (`aura-std/src/sma.rs`). The worked example: a producer node with one + `f64` input and one `f64` output, computing the arithmetic mean of the last + `length` values, emitting `None` until warmed up. It lives in `aura-std` (not + in `aura-core`'s tests) on purpose — it is the walking skeleton's first real + block (C16 names SMA an `aura-std` block), and authoring it in a *downstream* + crate proves `aura-core`'s `Node`/`Ctx` are usable across the crate boundary + exactly as a real project-side node would use them. + +## Concrete code shapes + +### The worked node — what a node author writes (the headline) + +This is the empirical evidence for the feature-acceptance criterion: the actual +Rust a node author produces against `aura-core`. If this is natural to write and +fully testable, the contract is right. + +```rust +// aura-std/src/sma.rs +use aura_core::{Ctx, InputSpec, Node, NodeSchema, Scalar, ScalarKind}; + +/// Simple moving average over the last `length` values of one f64 input. +pub struct Sma { + length: usize, +} + +impl Sma { + pub fn new(length: usize) -> Self { + assert!(length >= 1, "SMA length must be >= 1"); + Self { length } + } +} + +impl Node for Sma { + fn schema(&self) -> NodeSchema { + NodeSchema { + inputs: vec![InputSpec { kind: ScalarKind::F64, lookback: self.length }], + output: ScalarKind::F64, + } + } + + fn eval(&mut self, ctx: Ctx<'_>) -> Option { + let w = ctx.f64_in(0); + if w.len() < self.length { + return None; // not yet warmed up + } + let mut sum = 0.0; + for k in 0..self.length { + sum += w[k]; // index 0 = newest (financial indexing) + } + Some(Scalar::F64(sum / self.length as f64)) + } +} +``` + +### The hand-driven test — proving it runs without an engine (the proof) + +The test builds the input column the way the engine eventually will (sized from +`schema`), then drives the node by hand — push, `eval`, assert — which is exactly +the loop the sim engine will generalize in a later cycle. + +```rust +// aura-std/src/sma.rs (tests) +#[test] +fn sma_warms_up_then_tracks_the_window_mean() { + let mut sma = Sma::new(3); + let schema = sma.schema(); + + // size the input column from the schema, as the engine will at wiring + let mut inputs = + vec![AnyColumn::with_capacity(schema.inputs[0].kind, schema.inputs[0].lookback)]; + + let feed = [1.0_f64, 2.0, 3.0, 4.0, 5.0]; + let expect = [None, None, Some(2.0), Some(3.0), Some(4.0)]; // means of [1,2,3],[2,3,4],[3,4,5] + + for (v, want) in feed.iter().zip(expect) { + inputs[0].push(Scalar::F64(*v)).unwrap(); + let got = sma.eval(Ctx::new(&inputs)); + assert_eq!(got, want.map(Scalar::F64)); + } +} +``` + +### Implementation shapes (supporting, secondary) + +**New — `aura-core/src/node.rs`:** + +```rust +use crate::{Ctx, Scalar, ScalarKind}; + +/// One declared input of a node: its scalar kind and the lookback depth the +/// engine must pre-size for it (>= 1). Firing policy (C6) and tunable params +/// (C12/C19) are deliberately not declared yet — see "Out of scope". +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct InputSpec { + pub kind: ScalarKind, + pub lookback: usize, +} + +/// A node's declared interface: its inputs (in order) and its single output kind. +/// Built once at wiring; never on the hot path, so the `Vec` is fine here. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct NodeSchema { + pub inputs: Vec, + pub output: ScalarKind, +} + +/// The universal composable dataflow unit (C8): at most one output, a producer +/// or transformer. `schema` declares the interface; `eval` computes one cycle's +/// output (`None` = filter / not-yet-warmed-up). +pub trait Node { + fn schema(&self) -> NodeSchema; + fn eval(&mut self, ctx: Ctx<'_>) -> Option; +} +``` + +**New — `aura-core/src/ctx.rs`:** + +```rust +use crate::{AnyColumn, Timestamp, Window}; + +/// Read-only, zero-copy view of a node's inputs for one `eval`, in schema order. +/// `Copy` because it is just a borrow of the input slice. +#[derive(Clone, Copy)] +pub struct Ctx<'a> { + inputs: &'a [AnyColumn], +} + +impl<'a> Ctx<'a> { + pub fn new(inputs: &'a [AnyColumn]) -> Self { + Self { inputs } + } + + pub fn f64_in(&self, i: usize) -> Window<'a, f64> { + self.inputs[i] + .as_f64() + .expect("input kind mismatch (checked at wiring) — engine bug") + .window() + } + // i64_in / bool_in / ts_in are identical against as_i64 / as_bool / as_ts. +} +``` + +**Modify — `aura-core/src/any.rs`** (add the read-side, mirroring `as_*_mut`): + +```rust +impl AnyColumn { + pub fn as_f64(&self) -> Option<&Column> { + match self { + AnyColumn::F64(c) => Some(c), + _ => None, + } + } + // as_i64 / as_bool / as_ts mirror this against their arms. +} +``` + +**Modify — `aura-core/src/lib.rs`**: `mod ctx; mod node;` + re-export +`Ctx`, `Node`, `NodeSchema`, `InputSpec`; update the roadmap doc comment. + +**Modify — `aura-std/src/lib.rs`**: `mod sma; pub use sma::Sma;` and drop the +placeholder doc-only body. + +## Components + +| Component | Crate | Responsibility | +|-----------|-------|----------------| +| `Node` trait | aura-core | the `schema` + `eval` contract every node implements | +| `NodeSchema`, `InputSpec` | aura-core | a node's declared inputs (kind + lookback) and output kind | +| `Ctx` | aura-core | per-`eval` typed, zero-copy window access into inputs | +| `AnyColumn::as_*` | aura-core | read-side type-erased→typed column access (closes 0001 gap) | +| `Sma` | aura-std | the worked producer node; the walking skeleton's first block | + +## Data flow + +For one `eval` (mimicking one future sim cycle): + +1. Caller (the test now; the engine later) holds one `AnyColumn` per input, sized + from the node's `schema` at wiring. +2. A fresh value is `push`ed onto an input column (newest at index 0). +3. The caller wraps the input slice in a `Ctx` and calls `node.eval(ctx)`. +4. `eval` reads its inputs through `ctx.f64_in(i)` → a zero-copy `Window`, indexes + newest-first, and returns `Some(Scalar)` or `None`. + +No allocation occurs inside `eval` (the `Window` borrows; the SMA sum is a stack +scalar). The `Vec` in `NodeSchema` is wiring-time only, never on the hot path. + +## Error handling + +- **Kind mismatch in `Ctx` accessors** — `ctx.f64_in(i)` on a non-f64 input + panics with an explicit "engine bug" message. This is not a user-facing error: + the engine sizes and types inputs from `schema` at wiring, so a mismatch can + only mean the wiring layer is broken. Panicking (vs. returning `Option`) keeps + node-author code clean (`w[k]`, not `w?[k]`) and surfaces wiring bugs loudly. +- **Out-of-range input index `i`** — slice indexing panics; same rationale (the + engine addresses only declared inputs). +- **Not warmed up** — `eval` returns `None` (a normal value, not an error): the + window is shorter than the required lookback. The SMA shows the idiom. +- **`AnyColumn::as_*` on a wrong kind** — returns `None` (the low-level, + non-panicking primitive; `Ctx` is the layer that turns the wiring-guaranteed + case into a panic). + +## Testing strategy + +**aura-core (substrate-level, in the new modules):** + +- `any.rs` — read-side accessors: `as_f64`/`as_i64`/`as_bool`/`as_ts` each return + `Some(&Column)` for the matching kind and `None` for every other kind. +- `ctx.rs` — `f64_in` returns a window with financial indexing (newest at 0) over + a multi-input slice (addressing input 1, not just 0); a typed accessor on a + mismatched input panics (`#[should_panic]`). + +**aura-std (node-level, the worked example):** + +- SMA warm-up: `None` while the window is shorter than `length`. +- SMA value: tracks the moving mean exactly over `[1..5]` with `length = 3`. +- SMA degenerate `length = 1`: output equals the newest input each cycle. + +All four workspace gates stay green: `cargo build/test/clippy --workspace` and the +surface-purity grep (no `dyn Any` / `Rc<` / `RefCell` / per-event heap alloc). + +## Acceptance criteria + +1. `cargo build --workspace`, `cargo test --workspace`, + `cargo clippy --workspace --all-targets -- -D warnings` all green. +2. The worked `Sma` node compiles and its hand-driven test passes: `None` through + warm-up, then the exact window mean — proving a node is authorable in a + downstream crate and evaluable with no engine present. +3. `Ctx` hands `eval` zero-copy, financial-indexed windows per input; no + allocation inside `eval`. +4. `AnyColumn` read-side accessors return the typed column for the matching kind + and `None` otherwise (cycle-0001 audit gap closed). +5. Surface purity preserved: the new code adds no `dyn Any`, `Rc`, `RefCell`, or + per-event heap allocation on the eval path. + +## Out of scope (deliberate deferrals, recorded so they are decisions not gaps) + +- **Sim loop / cycle clock (C4)** and **freshness-gated recompute (C5)** — the + hand-driven test stands in for the loop this cycle; the engine generalizes it + next. +- **Firing policies (C6)** — `InputSpec` declares kind + lookback only; the + `firing` group is added when the sim loop that consumes it lands (C6 is + meaningless without the loop, so declaring it now would be a field nothing + reads). +- **Tunable params in `schema` (C12/C19)** — the param-space is a bootstrap-cycle + concern; `Sma`'s `length` is a plain constructor arg for now. +- **Sinks / no-output nodes (C8 consumer side)** — `NodeSchema.output` is a single + `ScalarKind`; the no-output sink refinement arrives with the broker/sink cycle. +- **Sources / ingestion (C3/C11)** and **composites (C9)** — later cycles.