Aura/docs/specs/0005-node-output-record.md

Node output is a record (composite stream) — Design Spec

Date: 2026-06-03 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Parent issue: Gitea #1 (BLOCKER, milestone Walking skeleton). Revises contract C8, sharpens C7.

Goal

Generalize a node's output from a single scalar to a record of base-scalar columns (K ≥ 1), with a scalar being the degenerate K = 1 case. A node still has exactly one output port; that port's payload is an ordered list of named, typed base columns (a composite stream, C7). This unblocks every multi-column producer the engine needs next — OHLCV bars and, later, the position-event table (C10) — none of which can exist while a node emits at most one column.

This cycle delivers the substrate only: the contract change, the field-wise binding mechanism, the Sma/Sub migration, and a neutral OHLCV proof. The position table (C10) and broker nodes are explicitly out of scope — they are later cycles that consume this capability.

Architecture

Three coordinated changes, in compile-gate order:

1. aura-core — the contract. NodeSchema.output changes from one ScalarKind to Vec<FieldSpec> (an ordered list of { name, kind } base columns). Node::eval changes from -> Option<Scalar> to -> Option<&[Scalar]>: a node fills a buffer it owns (sized once, at construction) and returns a borrowed row of K scalars; None still means filter / not-yet-warmed. Scalar output is the degenerate 1-field record — there is no separate scalar path.

2. aura-std — the degenerate migration. Sma and Sub declare a 1-field output (name: "value"), hold a [Scalar; 1] buffer, and return Some(&out). Behaviour is unchanged: a 1-field record is a scalar.

3. aura-engine — field-wise binding. Edge gains from_field: usize — which field of the producer's record this edge forwards. Bootstrap kind-checks per field (producer.output[from_field].kind == consumer.inputs[slot].kind) and range-checks from_field. The run loop copies a producer's returned row into one reused scratch buffer (resolving the borrow, zero per-cycle heap allocation once its capacity stabilizes), then scatters scratch[from_field] into each out-edge's input slot. Freshness stamping is per slot, unchanged, so the K fields of one record are co-fresh by construction (one eval, one timestamp).

The record is structural bundling over the four base types (C7 intact), not a fifth scalar type: no dyn Any, no heterogeneous buffer. Every field lands in a base column at the edge exactly as today's single scalar does; the streamed / stored form stays SoA.

Concrete code shapes

What the cycle delivers — the neutral OHLCV proof (test-local fixtures)

The cycle has no end-user surface; its deliverable is the executable proof that a node can emit K > 1 fields and a consumer can bind them individually. The north-star slice: OHLCV as a 5-field record (the position-event table will be the same shape), with a downstream node binding single fields by index.

// A neutral multi-field producer: five f64 inputs bundled into one 5-field
// record. No trading-domain logic — it proves the output mechanism in isolation.
struct Ohlcv {
    out: [Scalar; 5],
}
impl Node for Ohlcv {
    fn schema(&self) -> NodeSchema {
        NodeSchema {
            inputs: vec![
                InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) },
                InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) },
                InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) },
                InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) },
                InputSpec { kind: ScalarKind::F64, lookback: 1, firing: Firing::Barrier(0) },
            ],
            output: vec![
                FieldSpec { name: "open", kind: ScalarKind::F64 },
                FieldSpec { name: "high", kind: ScalarKind::F64 },
                FieldSpec { name: "low", kind: ScalarKind::F64 },
                FieldSpec { name: "close", kind: ScalarKind::F64 },
                FieldSpec { name: "volume", kind: ScalarKind::F64 },
            ],
        }
    }
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> {
        for i in 0..5 {
            let w = ctx.f64_in(i);
            if w.is_empty() {
                return None; // not yet warmed
            }
            self.out[i] = Scalar::F64(w[0]);
        }
        Some(&self.out) // one 5-field record, all fields co-fresh
    }
}

// The proof wiring: five timestamp-aligned sources feed O/H/L/C/V into Ohlcv's
// barrier group (the canonical "OHLC from separate sources" case, C6), so Ohlcv
// emits one complete bar per timestamp. A downstream Sub binds field 1 (high)
// minus field 2 (low) == the bar range. Observing Sub proves field-wise binding
// routes the right columns. Nodes are [Ohlcv (0), Sub (1)]; the five sources are
// SourceSpec entries targeting Ohlcv's five input slots.
//   Edge { from: 0 /*Ohlcv*/, to: 1 /*Sub*/, slot: 0, from_field: 1 /*high*/ }
//   Edge { from: 0 /*Ohlcv*/, to: 1 /*Sub*/, slot: 1, from_field: 2 /*low*/  }

The must-fail half of the proof (wrong code must be rejected at bootstrap):

// from_field past the producer's output width -> BadIndex
Edge { from: 0, to: 1, slot: 0, from_field: 9 }   // Ohlcv has 5 fields

// a field whose kind mismatches the consumer slot -> KindMismatch
// (e.g. binding an i64 field into an f64 input slot)

Before → after implementation shapes (secondary)

aura-core/src/node.rs — the contract:

// added
pub struct FieldSpec {
    pub name: &'static str,
    pub kind: ScalarKind,
}

// before
pub struct NodeSchema {
    pub inputs: Vec<InputSpec>,
    pub output: ScalarKind,
}
pub trait Node {
    fn schema(&self) -> NodeSchema;
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<Scalar>;
}

// after
pub struct NodeSchema {
    pub inputs: Vec<InputSpec>,
    pub output: Vec<FieldSpec>, // ordered base columns; len 1 = scalar (degenerate)
}
pub trait Node {
    fn schema(&self) -> NodeSchema;
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]>; // borrowed K-field row
}

FieldSpec derives Clone, Copy, Debug, PartialEq, Eq (like InputSpec); NodeSchema keeps its derives.

aura-std/src/sma.rs — the degenerate migration (Sub is analogous):

// before
pub struct Sma { length: usize }
// schema output: ScalarKind::F64
fn eval(&mut self, ctx: Ctx<'_>) -> Option<Scalar> {
    let w = ctx.f64_in(0);
    if w.len() < self.length { return None; }
    let mut sum = 0.0;
    for k in 0..self.length { sum += w[k]; }
    Some(Scalar::F64(sum / self.length as f64))
}

// after
pub struct Sma { length: usize, out: [Scalar; 1] }
// new() initializes out: [Scalar::F64(0.0)]
// schema output: vec![FieldSpec { name: "value", kind: ScalarKind::F64 }]
fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> {
    let w = ctx.f64_in(0);
    if w.len() < self.length { return None; }
    let mut sum = 0.0;
    for k in 0..self.length { sum += w[k]; }
    self.out[0] = Scalar::F64(sum / self.length as f64);
    Some(&self.out)
}

aura-engine/src/harness.rs — the edge and the run loop:

// before
pub struct Edge { pub from: usize, pub to: usize, pub slot: usize }

// after
pub struct Edge { pub from: usize, pub to: usize, pub slot: usize, pub from_field: usize }

Bootstrap edge check — from a single from.output compare to a per-field one:

// after
for &e in &edges {
    let from = schemas.get(e.from).ok_or(BootstrapError::BadIndex)?;
    let to = schemas.get(e.to).ok_or(BootstrapError::BadIndex)?;
    let field = from.output.get(e.from_field).ok_or(BootstrapError::BadIndex)?;
    let slot = to.inputs.get(e.slot).ok_or(BootstrapError::BadIndex)?;
    if field.kind != slot.kind {
        return Err(BootstrapError::KindMismatch { producer: field.kind, consumer: slot.kind });
    }
    out_edges[e.from].push(e);
}

Run loop — the eval-and-forward, with result: Option<&[Scalar]> (which is Copy, since &[Scalar] is a shared ref) and one reused scratch row declared once before the loop:

// after (per evaluated node nidx)
let result: Option<&[Scalar]> = {
    let nb = &mut nodes[nidx];
    nb.node.eval(Ctx::new(&nb.inputs))
};
if nidx == observe {
    observed = result.map(|row| row.to_vec());
}
if let Some(row) = result {
    debug_assert_eq!(row.len(), schemas_out_len[nidx]); // node honours its schema width
    scratch.clear();
    scratch.extend_from_slice(row); // copy ends the producer borrow; no per-cycle alloc once warm
    for e in out_edges[nidx].iter() {
        let nb = &mut nodes[e.to];
        nb.inputs[e.slot].push(scratch[e.from_field]).expect("edge kind checked at wiring");
        nb.slots[e.slot] = SlotState { fresh_at: cycle_id, last_ts: ts };
    }
}

run's return type changes from Vec<Option<Scalar>> to Vec<Option<Vec<Scalar>>> — the observed node's full row per cycle. The per-fire to_vec() is in the materialization surface (the returned collection, already a Vec today), not the inter-node hot path C7's no-alloc rule governs.

Components

Component	Change
aura-core::FieldSpec	new — { name: &'static str, kind: ScalarKind }, an output column descriptor
aura-core::NodeSchema	output: ScalarKind → output: Vec<FieldSpec>
aura-core::Node::eval	-> Option<Scalar> → -> Option<&[Scalar]>
aura-core lib doc	update the "single output kind" line
aura-std::Sma, ::Sub	hold an out buffer; 1-field schema; return Some(&out); in-crate tests adjusted to the slice return
aura-engine::Edge	gains from_field: usize
aura-engine::Harness::bootstrap	per-field edge kind/range check
aura-engine::Harness::run	reused scratch row; field-indexed forward; -> Vec<Option<Vec<Scalar>>>
aura-engine harness doc	the output-forward framing (one scalar → one record field)
aura-engine tests	Ohlcv fixture; field-binding proof; must-fail (from_field OOB, field kind-mismatch); existing 0003/0004 tests adapted to from_field + the Vec return

Data flow

1. The k-way merge picks the next (timestamp, source-index) source head and pushes its value into target slots, stamping freshness — unchanged (C3/C4).
2. In topological order, a fired node's eval fills its owned buffer and returns Some(&row) (or None to hold/filter).
3. The engine copies row into the reused scratch buffer (ending the producer borrow), then for each out-edge pushes scratch[from_field] into the consumer slot, stamping that slot fresh at the current cycle_id/ts.
4. Because all K fields are written in one eval and forwarded in one cycle, every consumer of any field of the record sees the same timestamp — co-freshness is structural, needing no barrier among fields (C6 stays for cross-source sync).
5. The observed node's row is cloned into the run's output vector per cycle.

Error handling

• from_field out of range (from_field >= producer.output.len()) → BootstrapError::BadIndex, caught once at wiring (C7: type check paid at bootstrap).
• Per-field kind mismatch (producer.output[from_field].kind != consumer.inputs[slot].kind) → BootstrapError::KindMismatch.
• Node violates its declared width (returned row length ≠ schema.output.len()) → debug_assert_eq! in the run loop; a node-author bug, not a wiring fault, so it is a debug assertion, not a typed BootstrapError.
• Runtime per-field kind is enforced by AnyColumn::push as today (.expect("edge kind checked at wiring")) — the bootstrap check makes the expect unreachable for well-typed wiring.

Testing strategy

• Degenerate compat — the existing 0003 fan-out/join DAG and the 0004 firing rails (AsOfSum/BarrierSum/MixedSum, diamond rejoin) keep their expected vectors; every edge gains from_field: 0, every fixture returns a 1-field slice. Proves the scalar = 1-field-record identity preserves all prior behaviour.
• K > 1 output — Ohlcv (5 f64 inputs → 5-field record); observing it yields the full 5-field row per cycle.
• Field-wise binding — source → Ohlcv → Sub(high − low); observing Sub proves from_field routes field 1 and field 2 (not field 0). A second consumer binding different fields (e.g. close − open) on the same Ohlcv confirms several edges read distinct fields of one record.
• Co-freshness — the field-binding consumer fires on the bar's single timestamp (no barrier needed among the bound fields), confirming structural co-freshness.
• Determinism — a bit-identical re-run of the K > 1 proof (C1).
• Must-fail (rejection) — from_field past the output width → BadIndex; a field kind-mismatch → KindMismatch.
• Purity — grep -rnE 'RefCell|Rc<|dyn Any' crates/*/src stays empty (no fifth type, no heterogeneous payload).

Gates: cargo build/test/clippy --workspace --all-targets -- -D warnings, plus the purity grep.

Acceptance criteria

1. A node emits a record of K ≥ 1 named base columns through one output port; Sma/Sub are the K = 1 degenerate case with unchanged behaviour.
2. An Edge binds exactly one producer field (from_field) into one consumer slot; consuming a whole record is N such edges; there is no "bind whole record" mechanism.
3. The K fields of one record are co-fresh by construction; C6 is untouched.
4. C7 holds: no fifth scalar type, no dyn Any, no heterogeneous buffer, no per-cycle heap allocation on the inter-node forward path; the purity grep is clean.
5. Bootstrap rejects an out-of-range from_field (BadIndex) and a per-field kind mismatch (KindMismatch).
6. The neutral OHLCV proof (producer + field-binding consumer + must-fail cases) is green; all prior 0003/0004 tests pass adapted to from_field.
7. The design ledger records the C8 revision and the C7 sharpening (ledger edit is part of this cycle; the glossary composite/node record-reality pass is an audit-time follow-up).