Files
Aura/docs/specs/fieldtest-0010-aura-run.md
T
Brummel f68258b044 fieldtest: cycle-0010 — 3 examples, 6 findings
Public-interface-only field test of the walking-skeleton closing seam (#8):
the `aura run` CLI and the reusable aura-std::Recorder sink. A standalone
consumer crate (path-deps on the three engine crates, as a C16 project would)
drives the real binary as a subprocess and wires Recorder via the raw
Harness::bootstrap API.

Examples (all built from HEAD and ran green):
- c0010_1: drive `aura run` as a downstream CLI/jq consumer — single-line JSON
  report, byte-identical across two runs (C1), bad-args → exit 2 + exact usage on
  stderr with empty stdout.
- c0010_2: wire Recorder onto Sma(3) via raw bootstrap, drain the mpsc::Receiver
  — rows match the rustdoc warm-up model exactly.
- c0010_3: Recorder over [I64, Bool, Timestamp] — the four-kind claim (never
  exercised by the CLI's f64-only path) holds, verified from rustdoc alone.

Findings: 0 bugs, 4 working, 1 spec_gap, 1 friction.
- [spec_gap] `aura run <trailing-arg>` is accepted (exit 0), not rejected — the
  arg parse keys only on the first token being `run` and ignores the rest. The
  cycle_scope's "any other or missing subcommand -> exit 2" does not unambiguously
  cover `run <junk>`; the binary chose the lenient reading. Tracked for a
  ratify-or-tighten decision.
- [friction] verifying the documented {manifest, metrics} JSON nesting needs a
  hand-rolled string walker — the zero-dep consumer has only to_json() and no
  typed read-path. Low-priority tidy.

Both non-bug findings filed to the forward queue; neither blocks the cycle.
2026-06-04 20:55:28 +02:00

164 lines
9.4 KiB
Markdown

# Fieldtest — cycle-0010 — 2026-06-04
**Status:** Draft — awaiting orchestrator triage
**Author:** fieldtester (dispatched by fieldtest skill)
## Scope
Cycle 0010 is the Walking-skeleton milestone's closing seam (#8). It shipped two
public surfaces:
1. **`aura_std::Recorder`** — a reusable recording-sink node. A pure consumer
(`output: vec![]`) over `kinds.len()` input columns, holding an
`mpsc::Sender<(Timestamp, Vec<Scalar>)>` as its out-of-graph destination
(C7/C8/C22). Constructor: `Recorder::new(kinds: &[ScalarKind], firing: Firing,
tx) -> Self`. Each fired cycle it sends `(ctx.now(), row)` with the newest
value of every column; returns `None` and records nothing until all columns
are warm. Supports all four base scalar kinds.
2. **The `aura run` CLI** (crate `aura-cli`, bin `aura`) — bootstraps a built-in
sample harness (synthetic source → SMA(2)/SMA(4) → Sub → Exposure → SimBroker
→ two Recorder sinks), runs it deterministically (C1), and prints the
cycle-0009 `RunReport` as canonical single-line JSON to stdout (exit 0). Any
other or missing subcommand prints `aura: usage: aura run` to stderr, exit 2.
Tested from the public interface only (rustdoc, design ledger, glossary, project
layout, the observable behaviour of the `aura` binary) — never `crates/*/src`.
The CLI's own functions are `pub(crate)`, so axis (a) drove the real binary as a
subprocess. The build exercised: workspace `cargo build` from HEAD
(`target/debug/aura`) for the binary; `cargo run --manifest-path
fieldtests/cycle-0010-aura-run/Cargo.toml` for the two library-consumer
fixtures.
## Examples
### fieldtests/cycle-0010-aura-run/c0010_1_cli_report.rs — `aura run` as a downstream CLI consumer (axis a)
- Spawns the real `aura` binary as a subprocess (path via `$AURA_BIN`, default
the sibling workspace `target/debug/aura`); captures stdout/stderr/exit for
`run`, no-args, an unknown subcommand, and `run` with a trailing junk arg.
- Fits the cycle's headline C14 move: a shell/jq/registry consumer parses the
single-line JSON report and relies on the exit-code + determinism contract.
- Outcome: built (from HEAD) and ran; exit 0 with `{manifest:{commit,params,
window,seed,broker}, metrics:{total_pips,max_drawdown,exposure_sign_flips}}`,
byte-identical across two runs, clean stderr; bad-args → exit 2 + exact usage.
Matched expected. Surfaced one spec_gap (trailing-arg leniency) and one
friction (no JSON parser in the consumer).
### fieldtests/cycle-0010-aura-run/c0010_2_recorder_sink.rs — `aura_std::Recorder` as a reusable sink block (axis b)
- Wires the shipped `Recorder` onto an `Sma(3)` output via the raw
`Harness::bootstrap(nodes, sources, edges)` API (exactly as a C16 project
would), runs 5 ticks, drains the `mpsc::Receiver`, asserts the recorded
`(Timestamp, Vec<Scalar>)` rows against the rustdoc warm-up model.
- Fits the cycle's first deliverable: Recorder used the way a downstream project
consumes it, not the way the CLI's built-in harness does.
- Outcome: built and ran first try; rows `[(3,[20.0]),(4,[30.0]),(5,[40.0])]`
exactly matched the hand model (no warm-up rows, one row per warm cycle,
ts = `ctx.now()`). Matched expected.
### fieldtests/cycle-0010-aura-run/c0010_3_recorder_four_kind.rs — Recorder four-kind support (axis c)
- Authors a tiny custom 3-field producer node (i64 counter / bool even? /
timestamp now) in Rust (C17), wires a multi-column
`Recorder::new(&[I64, Bool, Timestamp], …)` over it via three field-wise edges,
runs, drains, asserts the recorded Scalars are `I64`/`Bool`/`Ts`.
- Fits the cycle's four-kind claim: the CLI only ever exercises the f64 path, so
the non-f64 kinds are verifiable from the public rustdoc alone only this way.
- Outcome: built (after one trivial authoring fix — see working finding) and
ran; rows matched `[(100,[I64(1),Bool(false),Ts(100)]),(200,[I64(2),Bool(true),
Ts(200)]),(300,[I64(3),Bool(false),Ts(300)])]` exactly. Matched expected.
## Findings
### [working] `aura run` end-to-end: clean single-line JSON, deterministic, jq-parseable
- Example: c0010_1.
- What happened: `aura run` → exit 0, one line of JSON on stdout, empty stderr.
The documented `{manifest, metrics}` nesting and all eight documented fields
(`commit, params, window, seed, broker`; `total_pips, max_drawdown,
exposure_sign_flips`) are present. Two runs are byte-identical (C1). Piped
through `jq`, `.metrics.total_pips` is a number, `.manifest.params | keys`
resolves — a downstream C18/shell consumer can author against it.
- Why working: the headline C14 "run a sim, emit structured metrics" move is
reachable from a real binary, from rustdoc alone, on the first try, with the
determinism and shape the ledger promises.
- Recommended action: carry-on.
### [working] Bad-args contract exact and stdout-clean
- Example: c0010_1.
- What happened: `aura` (no args) and `aura play` (unknown) each → exit 2,
stderr exactly `aura: usage: aura run\n`, stdout empty (0 bytes). A consumer
keying off exit code never sees a half-report on stdout.
- Why working: matches the cycle_scope contract precisely; the stdout/stderr
split is clean (errors never contaminate the machine-readable channel).
- Recommended action: carry-on.
### [working] Recorder as a reusable sink: warm-up + row shape match rustdoc
- Example: c0010_2.
- What happened: a downstream wiring of `Recorder` over `Sma(3)` via raw
`Harness::bootstrap`, drained from `mpsc::Receiver`, produced exactly the rows
the rustdoc model predicts — no warm-up rows, one one-element `Vec<Scalar>`
per warm cycle, each tagged `ctx.now()`. The constructor signature on the
surface (`&[ScalarKind]`, `Firing`, `Sender`) is enough to author against with
no source reading.
- Why working: the cycle's primary deliverable is reachable and correct from the
public interface as a standalone consumer block (not just inside the CLI).
- Recommended action: carry-on.
### [working] Recorder four-kind claim holds on a non-f64 column set
- Example: c0010_3.
- What happened: a `Recorder` over `[I64, Bool, Timestamp]` recorded
`Scalar::I64` / `Scalar::Bool` / `Scalar::Ts` values verbatim, matching the
rustdoc's four-kind claim — a path the CLI never exercises. The trivial
authoring miss (`FieldSpec.name` is `&'static str`, I wrote `.to_string()`)
was caught by the compiler with an exact fix-it; the public `FieldSpec`
rustdoc states `pub name: &'static str` plainly. No source reading needed.
- Why working: the four-kind claim, otherwise unverifiable from the CLI, is
empirically true; the supporting types (`FieldSpec`, `Scalar`, multi-edge
field-wise binding) author cleanly from rustdoc.
- Recommended action: carry-on.
### [spec_gap] `aura run <trailing-arg>` is accepted (exit 0), not rejected
- Example: c0010_1.
- What happened: `aura run extra-garbage` printed the full JSON report and exited
0 — trailing args after `run` are silently ignored. The cycle_scope states
"Any **other** or missing subcommand prints `aura: usage: aura run` to stderr
and exits 2."
- Why spec_gap: `run <junk>` is neither a clean `run` nor a *different*
subcommand, so the contract does not unambiguously cover it. The binary picked
the lenient reading (key only on the first arg being `run`, ignore the rest);
the strict reading (any extra/unexpected token → usage + exit 2) is equally
plausible and is what a user who typos `aura run --sweep` would expect (they'd
get a silent full run instead of a hint). I asserted the lenient reading the
binary exhibits and recorded the ambiguity rather than treating it as a bug.
- Recommended action: ratify (document the arg-parse contract as
"first-arg-keyed, trailing args ignored") **or** tighten — decide whether
unexpected trailing tokens should be a usage error. A one-line note on the
intended arg contract in the `aura` crate rustdoc closes it.
### [friction] Verifying the documented JSON nesting needs a hand-rolled walker
- Example: c0010_1.
- What happened: the consumer crate is zero-external-dependency by design (no
serde/serde_json), and `RunReport` exposes only `to_json() -> String` with no
typed accessor. To confirm the documented `{manifest, metrics}` nesting and
field presence, the fixture hand-walks the string (`find("\"key\":")`) — a
consumer authoring real assertions against the report shape either pulls in a
JSON dep or writes this dance.
- Why friction: the task (confirm the structured-face shape) completed, but the
surface offered no programmatic handle on the report's structure short of
re-parsing the string the engine just serialized. The cycle's C14 framing
("machine-readable") implies a consumer can *inspect* the structure, not only
re-stringify it.
- Recommended action: plan (tidy) — consider a minimal typed read-path
(e.g. `RunReport` already round-trips its own fields; exposing the structured
values, or shipping a tiny documented parse helper, would let consumers assert
without a serde dep). Low priority; the JSON is correct and jq-parseable for
shell consumers today.
## Recommendation summary
| Finding | Class | Action |
|---|---|---|
| `aura run` end-to-end JSON + determinism | working | carry-on |
| Bad-args contract exact + stdout-clean | working | carry-on |
| Recorder reusable sink matches rustdoc | working | carry-on |
| Recorder four-kind claim holds | working | carry-on |
| `aura run <trailing-arg>` accepted | spec_gap | ratify / tighten the design ledger |
| JSON nesting needs a hand-rolled walker | friction | plan (tidy) |