Files
Aura/docs/specs/fieldtest-0010-aura-run.md
T
Brummel f68258b044 fieldtest: cycle-0010 — 3 examples, 6 findings
Public-interface-only field test of the walking-skeleton closing seam (#8):
the `aura run` CLI and the reusable aura-std::Recorder sink. A standalone
consumer crate (path-deps on the three engine crates, as a C16 project would)
drives the real binary as a subprocess and wires Recorder via the raw
Harness::bootstrap API.

Examples (all built from HEAD and ran green):
- c0010_1: drive `aura run` as a downstream CLI/jq consumer — single-line JSON
  report, byte-identical across two runs (C1), bad-args → exit 2 + exact usage on
  stderr with empty stdout.
- c0010_2: wire Recorder onto Sma(3) via raw bootstrap, drain the mpsc::Receiver
  — rows match the rustdoc warm-up model exactly.
- c0010_3: Recorder over [I64, Bool, Timestamp] — the four-kind claim (never
  exercised by the CLI's f64-only path) holds, verified from rustdoc alone.

Findings: 0 bugs, 4 working, 1 spec_gap, 1 friction.
- [spec_gap] `aura run <trailing-arg>` is accepted (exit 0), not rejected — the
  arg parse keys only on the first token being `run` and ignores the rest. The
  cycle_scope's "any other or missing subcommand -> exit 2" does not unambiguously
  cover `run <junk>`; the binary chose the lenient reading. Tracked for a
  ratify-or-tighten decision.
- [friction] verifying the documented {manifest, metrics} JSON nesting needs a
  hand-rolled string walker — the zero-dep consumer has only to_json() and no
  typed read-path. Low-priority tidy.

Both non-bug findings filed to the forward queue; neither blocks the cycle.
2026-06-04 20:55:28 +02:00

9.4 KiB

Fieldtest — cycle-0010 — 2026-06-04

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill)

Scope

Cycle 0010 is the Walking-skeleton milestone's closing seam (#8). It shipped two public surfaces:

  1. aura_std::Recorder — a reusable recording-sink node. A pure consumer (output: vec![]) over kinds.len() input columns, holding an mpsc::Sender<(Timestamp, Vec<Scalar>)> as its out-of-graph destination (C7/C8/C22). Constructor: Recorder::new(kinds: &[ScalarKind], firing: Firing, tx) -> Self. Each fired cycle it sends (ctx.now(), row) with the newest value of every column; returns None and records nothing until all columns are warm. Supports all four base scalar kinds.
  2. The aura run CLI (crate aura-cli, bin aura) — bootstraps a built-in sample harness (synthetic source → SMA(2)/SMA(4) → Sub → Exposure → SimBroker → two Recorder sinks), runs it deterministically (C1), and prints the cycle-0009 RunReport as canonical single-line JSON to stdout (exit 0). Any other or missing subcommand prints aura: usage: aura run to stderr, exit 2.

Tested from the public interface only (rustdoc, design ledger, glossary, project layout, the observable behaviour of the aura binary) — never crates/*/src. The CLI's own functions are pub(crate), so axis (a) drove the real binary as a subprocess. The build exercised: workspace cargo build from HEAD (target/debug/aura) for the binary; cargo run --manifest-path fieldtests/cycle-0010-aura-run/Cargo.toml for the two library-consumer fixtures.

Examples

fieldtests/cycle-0010-aura-run/c0010_1_cli_report.rs — aura run as a downstream CLI consumer (axis a)

  • Spawns the real aura binary as a subprocess (path via $AURA_BIN, default the sibling workspace target/debug/aura); captures stdout/stderr/exit for run, no-args, an unknown subcommand, and run with a trailing junk arg.
  • Fits the cycle's headline C14 move: a shell/jq/registry consumer parses the single-line JSON report and relies on the exit-code + determinism contract.
  • Outcome: built (from HEAD) and ran; exit 0 with {manifest:{commit,params, window,seed,broker}, metrics:{total_pips,max_drawdown,exposure_sign_flips}}, byte-identical across two runs, clean stderr; bad-args → exit 2 + exact usage. Matched expected. Surfaced one spec_gap (trailing-arg leniency) and one friction (no JSON parser in the consumer).

fieldtests/cycle-0010-aura-run/c0010_2_recorder_sink.rs — aura_std::Recorder as a reusable sink block (axis b)

  • Wires the shipped Recorder onto an Sma(3) output via the raw Harness::bootstrap(nodes, sources, edges) API (exactly as a C16 project would), runs 5 ticks, drains the mpsc::Receiver, asserts the recorded (Timestamp, Vec<Scalar>) rows against the rustdoc warm-up model.
  • Fits the cycle's first deliverable: Recorder used the way a downstream project consumes it, not the way the CLI's built-in harness does.
  • Outcome: built and ran first try; rows [(3,[20.0]),(4,[30.0]),(5,[40.0])] exactly matched the hand model (no warm-up rows, one row per warm cycle, ts = ctx.now()). Matched expected.

fieldtests/cycle-0010-aura-run/c0010_3_recorder_four_kind.rs — Recorder four-kind support (axis c)

  • Authors a tiny custom 3-field producer node (i64 counter / bool even? / timestamp now) in Rust (C17), wires a multi-column Recorder::new(&[I64, Bool, Timestamp], …) over it via three field-wise edges, runs, drains, asserts the recorded Scalars are I64/Bool/Ts.
  • Fits the cycle's four-kind claim: the CLI only ever exercises the f64 path, so the non-f64 kinds are verifiable from the public rustdoc alone only this way.
  • Outcome: built (after one trivial authoring fix — see working finding) and ran; rows matched [(100,[I64(1),Bool(false),Ts(100)]),(200,[I64(2),Bool(true), Ts(200)]),(300,[I64(3),Bool(false),Ts(300)])] exactly. Matched expected.

Findings

[working] aura run end-to-end: clean single-line JSON, deterministic, jq-parseable

  • Example: c0010_1.
  • What happened: aura run → exit 0, one line of JSON on stdout, empty stderr. The documented {manifest, metrics} nesting and all eight documented fields (commit, params, window, seed, broker; total_pips, max_drawdown, exposure_sign_flips) are present. Two runs are byte-identical (C1). Piped through jq, .metrics.total_pips is a number, .manifest.params | keys resolves — a downstream C18/shell consumer can author against it.
  • Why working: the headline C14 "run a sim, emit structured metrics" move is reachable from a real binary, from rustdoc alone, on the first try, with the determinism and shape the ledger promises.
  • Recommended action: carry-on.

[working] Bad-args contract exact and stdout-clean

  • Example: c0010_1.
  • What happened: aura (no args) and aura play (unknown) each → exit 2, stderr exactly aura: usage: aura run\n, stdout empty (0 bytes). A consumer keying off exit code never sees a half-report on stdout.
  • Why working: matches the cycle_scope contract precisely; the stdout/stderr split is clean (errors never contaminate the machine-readable channel).
  • Recommended action: carry-on.

[working] Recorder as a reusable sink: warm-up + row shape match rustdoc

  • Example: c0010_2.
  • What happened: a downstream wiring of Recorder over Sma(3) via raw Harness::bootstrap, drained from mpsc::Receiver, produced exactly the rows the rustdoc model predicts — no warm-up rows, one one-element Vec<Scalar> per warm cycle, each tagged ctx.now(). The constructor signature on the surface (&[ScalarKind], Firing, Sender) is enough to author against with no source reading.
  • Why working: the cycle's primary deliverable is reachable and correct from the public interface as a standalone consumer block (not just inside the CLI).
  • Recommended action: carry-on.

[working] Recorder four-kind claim holds on a non-f64 column set

  • Example: c0010_3.
  • What happened: a Recorder over [I64, Bool, Timestamp] recorded Scalar::I64 / Scalar::Bool / Scalar::Ts values verbatim, matching the rustdoc's four-kind claim — a path the CLI never exercises. The trivial authoring miss (FieldSpec.name is &'static str, I wrote .to_string()) was caught by the compiler with an exact fix-it; the public FieldSpec rustdoc states pub name: &'static str plainly. No source reading needed.
  • Why working: the four-kind claim, otherwise unverifiable from the CLI, is empirically true; the supporting types (FieldSpec, Scalar, multi-edge field-wise binding) author cleanly from rustdoc.
  • Recommended action: carry-on.

[spec_gap] aura run <trailing-arg> is accepted (exit 0), not rejected

  • Example: c0010_1.
  • What happened: aura run extra-garbage printed the full JSON report and exited 0 — trailing args after run are silently ignored. The cycle_scope states "Any other or missing subcommand prints aura: usage: aura run to stderr and exits 2."
  • Why spec_gap: run <junk> is neither a clean run nor a different subcommand, so the contract does not unambiguously cover it. The binary picked the lenient reading (key only on the first arg being run, ignore the rest); the strict reading (any extra/unexpected token → usage + exit 2) is equally plausible and is what a user who typos aura run --sweep would expect (they'd get a silent full run instead of a hint). I asserted the lenient reading the binary exhibits and recorded the ambiguity rather than treating it as a bug.
  • Recommended action: ratify (document the arg-parse contract as "first-arg-keyed, trailing args ignored") or tighten — decide whether unexpected trailing tokens should be a usage error. A one-line note on the intended arg contract in the aura crate rustdoc closes it.

[friction] Verifying the documented JSON nesting needs a hand-rolled walker

  • Example: c0010_1.
  • What happened: the consumer crate is zero-external-dependency by design (no serde/serde_json), and RunReport exposes only to_json() -> String with no typed accessor. To confirm the documented {manifest, metrics} nesting and field presence, the fixture hand-walks the string (find("\"key\":")) — a consumer authoring real assertions against the report shape either pulls in a JSON dep or writes this dance.
  • Why friction: the task (confirm the structured-face shape) completed, but the surface offered no programmatic handle on the report's structure short of re-parsing the string the engine just serialized. The cycle's C14 framing ("machine-readable") implies a consumer can inspect the structure, not only re-stringify it.
  • Recommended action: plan (tidy) — consider a minimal typed read-path (e.g. RunReport already round-trips its own fields; exposing the structured values, or shipping a tiny documented parse helper, would let consumers assert without a serde dep). Low priority; the JSON is correct and jq-parseable for shell consumers today.

Recommendation summary

Finding Class Action
aura run end-to-end JSON + determinism working carry-on
Bad-args contract exact + stdout-clean working carry-on
Recorder reusable sink matches rustdoc working carry-on
Recorder four-kind claim holds working carry-on
aura run <trailing-arg> accepted spec_gap ratify / tighten the design ledger
JSON nesting needs a hand-rolled walker friction plan (tidy)