# Cell as the hot-path value carrier — Design Spec

**Date:** 2026-06-16
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude
**Tracker:** closes #74

## Goal

Make the tag-free `Cell` the value carrier on aura's hot path, and narrow
`Scalar` (= `{ kind: ScalarKind, cell: Cell }`) to the *self-describing dynamic
boundaries* only. This is the deferred carrier swap the C7 realization note in
`docs/design/INDEX.md` (commits `cd3d1ca` / `049f22a`) explicitly flagged:

> `Cell` is **not yet** the hot-path carrier — `eval -> Option<&[Scalar]>` and
> the edges still flow `Scalar`; replacing the carrier with `Cell` is a larger
> step that will get its own note (and touch C8's `eval` contract).

The change is **behaviour-preserving** (C1): same input → same run,
byte-identical metrics and manifests. Only the carrier *type* narrows; the
streamed values are unchanged.

## Architecture

### The settled principle (the decision procedure for every value site)

C7 already holds that the scalar kind lives at the schema / column / port, not
in the value. The carrier swap makes that literal on the single-value carrier:

- **Kind structurally co-present at the site → bare `Cell`.** If the kind is
  already known from the surrounding context (a column's own kind, a declared
  output `FieldSpec.kind`, a bootstrap-verified edge), the value must not
  re-carry a redundant per-value tag.
- **Value travels detached from its kind → `Scalar`.** If a value must
  self-describe because it is serialized, rendered, bound by name, or merged
  heterogeneously, it stays the fatter `{ kind, cell }` form.

Applying that procedure partitions every current `Scalar` site into a
**convert-set** and a **keep-set**.

### Convert-set (→ `Cell`)

| Site | Why the kind is co-present |
|------|----------------------------|
| `Node::eval -> Option<&[Cell]>` (`crates/aura-core/src/node.rs`) | Each output field's kind is declared in the node's own `NodeSchema.output[i].kind`; the harness holds the schema. |
| Each node's output buffer `out: [Scalar; N]` → `[Cell; N]` | The node declares its own output kinds and constructs the values; it never needs to read its own buffer back through a tag. |
| Harness forward path (`crates/aura-engine/src/harness.rs`) — `scratch: Vec<Cell>` + the inter-node edge push | The consumer column's kind is fixed at bootstrap and the edge `from_field`→slot kind match is bootstrap-verified. |
| `AnyColumn::push_cell(Cell)` — **new**, branch-free, infallible | Bootstrap guarantees the kind match, so the runtime push needs no kind check and no `Result`. |

### Keep-set (stays `Scalar`)

| Site | Why it must self-describe |
|------|---------------------------|
| `AnyColumn::get -> Option<Scalar>` (`crates/aura-core/src/any.rs`) | The type-erased read for sinks / recorder / serialization / playground — the caller has no static kind, so the value must carry its own. |
| The whole param path: `PrimitiveBuilder::build(&[Scalar])` + build closure, `bind(slot, value: Scalar)`, `BoundParam.value`, `zip_params`, `Blueprint::compile_with_params(&[Scalar])`, the sweep / mc / walkforward param-point args, aura-cli `run_oos` / `scalar_as_param_f64` | A param point routinely travels **detached** from its co-indexed param-space — serialized into the `RunManifest`, passed to `run_one(seed, point)`. `bind` kind-checks the value against the slot. Self-description is load-bearing exactly where the manifest needs it. |
| Source ingestion: `Source::next -> Option<(Timestamp, Scalar)>` + the source-target push (`crates/aura-engine/src/harness.rs`) | The k-way-merged ingestion boundary (C3) is heterogeneous and self-describing by nature; the source *produces* a `Scalar`, so stripping it to a cell at the push would be churn at exactly the boundary whose job is to carry heterogeneous values. |
| The recorder's out-of-graph channel row `Vec<Scalar>` (`crates/aura-std/src/recorder.rs`) | An out-of-graph destination (C8 sink side effect) — the consumer at the other end of the `mpsc` has no schema, so the row self-describes. |

### The boundary, in one picture

```
  Source::next ──Scalar──▶ [push] ──▶ AnyColumn (typed) ──get──Scalar──▶ sinks/serde
   (ingestion, C3)                         │
                                           │ Ctx windows (native f64/i64/…)
                                           ▼
                                     Node::eval ──&[Cell]──▶ [push_cell] ──▶ AnyColumn (next node)
                                           ▲                                   (inter-node forward)
                                   out: [Cell; N]

  params:  &[Scalar] ─────────────────────────────────────▶ build / bind / manifest (self-describing)
```

The hot inter-node forward (`eval` → `push_cell`) carries bare cells; the two
*dynamic* edges (ingestion in, `get` / serde out) and the param plane carry
`Scalar`.

## Concrete code shapes

### Worked example — the node-author surface (the empirical evidence)

The authoring surface is the consumer here: every node author writes `eval` and
an output buffer. `Add` is representative.

**Before** (`crates/aura-std/src/add.rs`):

```rust
pub struct Add {
    out: [Scalar; 1],
}
impl Add {
    pub fn new() -> Self {
        Self { out: [Scalar::f64(0.0)] }
    }
}
impl Node for Add {
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Scalar]> {
        let a = ctx.f64_in(0);
        let b = ctx.f64_in(1);
        if a.is_empty() || b.is_empty() {
            return None;
        }
        self.out[0] = Scalar::f64(a[0] + b[0]);
        Some(&self.out)
    }
}
```

**After**:

```rust
pub struct Add {
    out: [Cell; 1],
}
impl Add {
    pub fn new() -> Self {
        Self { out: [Cell::from_f64(0.0)] }
    }
}
impl Node for Add {
    fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
        let a = ctx.f64_in(0);
        let b = ctx.f64_in(1);
        if a.is_empty() || b.is_empty() {
            return None;
        }
        self.out[0] = Cell::from_f64(a[0] + b[0]);
        Some(&self.out)
    }
}
```

The change is strictly a *simplification* for the author: the kind is named
once on the constructor (`Cell::from_f64`) and never re-asserted, because the
node already declared `FieldSpec { kind: F64 }` in its schema. There is no
authoring concept added — the read side (`ctx.f64_in`) is already native and is
untouched.

A node-author eval test migrates its output assertion from the `Scalar` carrier
to the `Cell` one (the read side and the param side stay `Scalar`):

```rust
// before
assert_eq!(add.eval(ctx), Some([Scalar::f64(14.0)].as_slice()));
// after
assert_eq!(add.eval(ctx), Some([Cell::from_f64(14.0)].as_slice()));
```

`Cell` derives `PartialEq`/`Eq` (bit-exact), so the assertion compares the raw
words — which for a deterministic f64 sum is exactly the intended check.

#### Test-migration rule (precise — what migrates vs. what stays)

A unit test touches `Scalar` in three distinct roles; only one migrates. The
rule keys on the role, not on "it is a test":

1. **Eval-output assertions → migrate to `Cell`.** The value a test reads back
   *out of `eval`* is now a cell: `Some([Scalar::f64(x)].as_slice())` →
   `Some([Cell::from_f64(x)].as_slice())`, and a value extractor on an eval row
   migrates too — the `sim_broker` helper `Some([s]) => s.as_f64()` →
   `Some([c]) => c.f64()`. This is the **only** role that changes.
2. **Test-fixture column *writes* → stay `Scalar` (`push`).** Every node test
   seeds inputs with `inputs[i].push(Scalar::f64(x)).unwrap()` (e.g.
   `exposure.rs`, `sma.rs`, all of aura-std + the harness fixtures, ~40 sites).
   A column write models a **source feed** into that column — it is the
   ingestion boundary in miniature, squarely in the keep-set. These lines are
   **left untouched**; they keep the fallible `push(Scalar)` (which is retained
   precisely for ingestion + tests). They are **not** rewritten to `push_cell`.
3. **Out-of-graph channel sends → stay `Scalar`.** A fixture that records to a
   channel sends a self-describing `Vec<Scalar>` row — the recorder
   (`recorder.rs`) and any dual-role test fixture that both forwards an `eval`
   output (convert) **and** sends a channel row, e.g. `TapForward`
   (`harness.rs`, which sends `vec![Scalar::f64(v)]`). For such a fixture the
   `eval` signature + out-buffer migrate to `Cell`, but its channel send stays
   `Scalar`. The two roles in one node migrate independently, by role.

In one line: **only what comes *out of* `eval` becomes a `Cell`; what goes
*into* a column or *out to* a channel stays a `Scalar`.**

### Engine-internal shape — `AnyColumn::push_cell` (secondary)

**Before** — the forward path pushes a tagged `Scalar` and unwraps a kind check
the code already trusts (`crates/aura-engine/src/harness.rs`):

```rust
// AnyColumn::push (kept, for the source-ingestion boundary + tests)
pub fn push(&mut self, v: Scalar) -> Result<(), KindMismatch> {
    match (self, v.kind()) {
        (AnyColumn::F64(c), ScalarKind::F64) => { c.push(v.as_f64()); Ok(()) }
        // … one arm per kind …
        (this, got) => Err(KindMismatch { expected: this.kind(), got }),
    }
}

// harness forward loop
let result: Option<&[Scalar]> = nb.node.eval(Ctx::new(&nb.inputs, ts));
if let Some(row) = result {
    scratch.clear();
    scratch.extend_from_slice(row);                 // scratch: Vec<Scalar>
    for e in out_edges[nidx].iter() {
        nb.inputs[e.slot]
            .push(scratch[e.from_field])
            .expect("edge kind checked at wiring");  // ← already treated infallible
    }
}
```

**After** — a branch-free, infallible cell push for the inter-node forward; the
fallible `Scalar` `push` stays for ingestion:

```rust
// AnyColumn — new method; the kind is the column's own, the cell is reinterpreted
pub fn push_cell(&mut self, c: Cell) {
    match self {
        AnyColumn::I64(col) => col.push(c.i64()),
        AnyColumn::F64(col) => col.push(c.f64()),
        AnyColumn::Bool(col) => col.push(c.bool()),
        AnyColumn::Ts(col) => col.push(c.ts()),
    }
}

// harness forward loop
let result: Option<&[Cell]> = nb.node.eval(Ctx::new(&nb.inputs, ts));
if let Some(row) = result {
    scratch.clear();
    scratch.extend_from_slice(row);                 // scratch: Vec<Cell>
    for e in out_edges[nidx].iter() {
        nb.inputs[e.slot].push_cell(scratch[e.from_field]); // infallible
    }
}
```

The kind check is not lost — it is paid once at bootstrap (`harness.rs`
verifies `from.output[from_field].kind == to.inputs[slot].kind` for every edge,
green tests `bootstrap_rejects_a_kind_mismatch`,
`bootstrap_rejects_per_field_kind_mismatch`,
`bootstrap_rejects_kind_mismatched_recorder_edge`). `push_cell` realizes C7's
"type check paid once at wiring, then direct dispatch".

The source-target forward keeps the `Scalar` path (ingestion is in the
keep-set):

```rust
// unchanged — source value is a self-describing Scalar
nb.inputs[t.slot].push(value).expect("source kind checked at wiring");
```

### `AnyColumn::get` stays `Scalar` (secondary)

`get` is the type-erased read for the boundary; it reconstructs the tag from
the column kind on the way out, exactly as today:

```rust
pub fn get(&self, k: usize) -> Option<Scalar> {
    match self {
        AnyColumn::I64(c) => c.get(k).map(Scalar::i64),
        // … unchanged …
    }
}
```

## Components

- **`crates/aura-core/src/node.rs`** — `Node::eval` return type
  `Option<&[Scalar]>` → `Option<&[Cell]>`. The param path in the same file
  (`PrimitiveBuilder::build`/`new`/`bind`, `BoundParam.value`, `zip_params`)
  stays `Scalar`. Test/fixture nodes (`Bare`, `Probe`) migrate their `eval`
  signature; `Probe`'s param-vector field stays `Vec<Scalar>` (it stores a
  param point).
- **`crates/aura-core/src/any.rs`** — add `push_cell(Cell)`; keep `push(Scalar)`
  and `get -> Option<Scalar>`. The `aura_core::Cell` re-export already exists.
- **`crates/aura-std/`** — `add`, `sub`, `sma`, `ema`, `exposure`, `lincomb`,
  `sim_broker`: `out: [Scalar; N]` → `[Cell; N]`, constructors and `eval` bodies
  use `Cell::from_*`; `eval` signature changes. `recorder`: `eval` signature
  changes (returns `None`); the channel row stays `Vec<Scalar>` and the
  per-kind `Scalar::*` construction from `get(0)?` stays. Each node's eval tests
  migrate output assertions to `Cell`; the `sim_broker` test helper
  `Some([s]) => s.as_f64()` becomes `Some([c]) => c.f64()`.
- **`crates/aura-engine/src/harness.rs`** — `scratch: Vec<Cell>`; the inter-node
  forward uses `push_cell`; the source forward keeps `push(Scalar)`. The
  in-file test/fixture nodes migrate their `eval` signatures and out-buffers per
  the test-migration rule above: a dual-role fixture such as `TapForward`
  migrates its `eval` output to `Cell` but keeps its out-of-graph channel send
  `Scalar`, and test column writes (`inputs[i].push(Scalar::f64(x))`) stay on
  the fallible `push`.
- **`crates/aura-engine/src/blueprint.rs`, `graph_model.rs`** — the
  test/fixture nodes that `impl Node` migrate their `eval` signatures + buffers.
  `graph_model.rs::scalar_str` (renders a `BoundParam.value`) stays `Scalar`.
  `blueprint.rs::compile_with_params(&[Scalar])` and the sweep entry points stay
  `Scalar` (param plane).
- **`crates/aura-engine/src/sweep.rs`, `mc.rs`, `walkforward.rs`** — the
  `&[Scalar]` param-point args and `scalar_as_f64` stay `Scalar` (param plane).
- **`crates/aura-cli/src/main.rs`** — `run_oos(&[Scalar])`,
  `scalar_as_param_f64(&Scalar)` stay `Scalar` (param plane).
- **`crates/aura-engine/src/report.rs`** — `f64_field(&[(Timestamp,
  Vec<Scalar>)])` stays `Scalar` (it reads recorder channel rows).
- **No change to** `crates/aura-core/src/ctx.rs` (the read side is already
  native — `f64_in`/`i64_in`/… return typed `Window`s), `column.rs`, `error.rs`,
  `scalar.rs`, `cell.rs`.

## Data flow

1. A source yields `(Timestamp, Scalar)`; the harness pushes the self-describing
   `Scalar` into its target column via the fallible `push` (kind verified at
   wiring). **[keep]**
2. The engine evaluates each fired node in topo order. `eval` reads native
   windows from `Ctx` and returns `Option<&[Cell]>` — a borrowed row of bare
   cells into the node's own buffer. **[convert]**
3. The harness copies the row into `scratch: Vec<Cell>` and, per out-edge,
   `push_cell`s one cell into the consumer column. No per-value kind check —
   bootstrap guaranteed it. **[convert]**
4. A sink/recorder reads `AnyColumn::get -> Scalar` (or native windows via
   `Ctx`) and emits a self-describing row out of the graph. **[keep]**
5. Params flow as `&[Scalar]` from the builder/sweep into `compile_with_params`
   and the `RunManifest`. **[keep]**

The values at every step are bit-identical to before; only steps 2–3 change the
*static type* of the carrier from `Scalar` to `Cell`.

## Error handling

- **`push_cell` is infallible by construction.** It cannot return
  `KindMismatch` because there is no second kind to compare — it pushes the cell
  into *this* column's concrete `Column<T>`, and the only way the wrong cell
  reaches the wrong column is a bootstrap-verified edge being wrong, which the
  bootstrap kind-check (green tests above) already forbids. `KindMismatch` and
  the fallible `push` remain for the ingestion boundary and the unit tests that
  exercise the guard directly.
- **The per-value runtime kind check on *node output* is removed.** Today, a
  node whose `eval` built a `Scalar` of the wrong kind for its declared output
  slot would be caught by the forward-path `push(...).expect(...)` (a panic).
  After the swap, a node that builds a cell of the wrong *kind* (e.g.
  `Cell::from_i64` in an `f64`-output node) pushes raw bits with no panic. This
  is an accepted, bounded trade:
  - It is the *same class* of authoring bug as a node returning the wrong row
    *width*, which C8 already guards only with a `debug_assert`
    (`row.len() == out_len`), not a hard error.
  - The node's own `eval` test asserts the exact output value; a wrong-kind
    cell yields a garbage value that fails that assertion at the node's own test
    layer.
  - It is consistent with C7: the output kind is the node's *declared* contract
    (`FieldSpec.kind`), honored by the node, not re-checked per value on the hot
    path. The check that *does* matter cross-node — that producer and consumer
    agree — stays at bootstrap.
- **No new panics or `Result`s on the hot path.** The change only *removes* a
  `Result` the code already `.expect`ed.

## Testing strategy

- **Behaviour preservation is the headline.** The existing suite is the
  regression net: every node `eval` test, the harness end-to-end forward tests,
  the sweep/mc/walk-forward/registry tests, and the manifest/metric tests must
  stay green after the carrier migration. Their *values* do not change; only the
  output-side assertions re-spell `Scalar::f64(x)` as `Cell::from_f64(x)`.
- **`push_cell` round-trip.** Add a focused `AnyColumn` test: `push_cell` of
  each kind round-trips through `get` to the same `Scalar` (mirrors the existing
  `same_kind_push_round_trips`), proving the cell is interpreted by the column's
  kind.
- **`push` guard retained.** The existing `wrong_kind_push_is_rejected` stays
  green (the fallible `Scalar` `push` is unchanged), proving the ingestion guard
  is intact.
- **Determinism spot-check.** A harness run's recorded equity/exposure rows and
  `RunManifest` are unchanged vs. before the swap — covered by the existing
  end-to-end report tests (`f64_field` projections + `summarize`); no new
  golden file needed because those tests already pin the exact numeric output.
- All four gates clean: `cargo build / clippy --all-targets -D warnings /
  test / doc --workspace`.

## Acceptance criteria

Judged against aura's domain invariants (the project's feature-acceptance
criterion):

1. **C7 made literal on the carrier.** `Node::eval` and every node out-buffer
   carry bare `Cell` (8 bytes, no tag, no per-value branch); the kind lives only
   at the schema/column. `Scalar` survives exactly on the param / `get` / source
   boundaries enumerated above — nowhere on the inter-node hot path.
2. **C1 preserved.** Same input → same run: every existing test green with only
   carrier-type spelling changes on `eval`-output assertions; recorded rows and
   manifests byte-identical.
3. **No reintroduced failure class.** The cross-node kind agreement stays
   enforced at bootstrap (the green bootstrap-reject tests); the only removed
   check is the redundant runtime one the code already `.expect`ed. The
   node-output-kind responsibility is the node author's, caught by the node's own
   test — the same posture C8 already takes for output width.
4. **Authoring surface no harder.** The worked `Add` before→after shows the node
   author writes *less* ceremony (one `Cell::from_f64` per output, no tag named
   twice), with no new concept on the read side.
5. All four workspace gates clean.
