Aura/docs/specs/fieldtest-0009-run-metrics.md

Fieldtest — cycle-0009 (run metrics + manifest) — 2026-06-04

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill)

Scope

Cycle 0009 (Walking-skeleton milestone, issue #6) shipped a pure-additive report module in aura-engine, re-exported at the crate root: summarize(equity, exposure) -> RunMetrics (fields total_pips, max_drawdown, exposure_sign_flips); f64_field(rows, field) bridging a recording sink's Vec<Scalar> rows to summarize; RunManifest (commit, params, window, seed, broker); RunReport { manifest, metrics } with to_json() -> String (the structured C14 face). The intended downstream move: after Harness::run, drain recording sinks, project f64 columns with f64_field, summarize the pip-equity + exposure streams, pair with a caller-built RunManifest, render to_json. The aura run CLI that prints this is deferred to #8 and out of scope. Worked from the public interface only (rustdoc via cargo doc, docs/design/INDEX.md, docs/glossary.md, project layout, public re-exports); never crates/*/src.

Build/run: workspace built clean from HEAD (cargo build --workspace); each example run via cargo run --manifest-path fieldtests/cycle-0009-run-metrics/Cargo.toml --bin <name>, so HEAD source is always what compiled and ran (a standalone non-workspace consumer crate path-depending on the three engine crates, exactly as a C16 project would).

Examples

fieldtests/cycle-0009-run-metrics/c0009_1_run_to_report.rs — north-star run-to-report

• What it does. Bootstraps the full SMA(2)/SMA(4) → Sub → Exposure(4) → SimBroker harness with TWO recording sinks (equity tapped on the broker, exposure tapped on the Exposure node), runs a 7-tick rising-price stream, drains both sinks, f64_field-projects each, summarizes, builds a RunManifest, and renders to_json.
• Why it fits. This is the carrier's primary axis verbatim — the end-to-end research-loop move the whole module exists to enable, reaching every new surface element once.
• Outcome. Built, ran, matched the hand model exactly: equity [0,0,0,0,1,2,3] → total_pips=3.0, max_drawdown=0.0; exposure all +0.5 → exposure_sign_flips=0. to_json emitted a single well-formed object.

fieldtests/cycle-0009-run-metrics/c0009_2_compare_two_runs.rs — comparison + determinism

• What it does. Runs the same harness twice at Exposure scale=4 (asserts bit-identical metrics AND bit-identical to_json across the two runs), then a third run at scale=2, and compares total_pips.
• Why it fits. The carrier's second axis: determinism (C1) plus a smallest-slice tuning-sweep comparison (C12/C21) read off RunMetrics.
• Outcome. Built, ran, matched expected: scale=4 deterministic (total_pips=3), scale=2 = total_pips=6 (exactly double — exposure saturates 0.5→1.0), both drawdown-free, no sign flips.

fieldtests/cycle-0009-run-metrics/c0009_3_degenerate_streams.rs — degenerate summarize semantics

• What it does. Calls summarize with six hand-built &[(Timestamp,f64)] argument literals: empty, monotonic-up, dip-and-recover, long/short/flat sign-flips, same-sign varying magnitude, and an all-negative curve.
• Why it fits. The carrier's third axis: verify the documented metric definitions from docs alone. The slices are the public function's direct argument type — calling the function with literals, not authoring an intermediate representation.
• Outcome. Built, ran, every documented definition held exactly: empty→ zeros; last-value total_pips (incl. negative); running-peak drawdown (incl. on a negative curve, peak −1 → trough −5 = 4); sign-flips counting flat (sign 0) as distinct from long/short (3 flips), magnitude changes not counted.

fieldtests/cycle-0009-run-metrics/c0009_4_f64field_and_json.rs — f64_field bridge + JSON shape

• What it does. Projects a non-zero column from a multi-column recorded row; confirms (via catch_unwind) the documented panics on a non-f64 field and an out-of-range field index; inspects the exact to_json byte shape under a realistic FX pip_size=0.0001, fractional/negative metrics, and ns-scale timestamps.
• Why it fits. Exercises the f64_field "checked at wiring" contract and the to_json round-trippability claim — the corners of the north-star surface a generic sink-drainer would hit.
• Outcome. Built, ran, matched: column-pick correct; both panics fired with precise messages; JSON kept ns timestamps as bare integers (no sci-notation), params in insertion order as a nested object.

Findings

[working] North-star run-to-report is reachable from rustdoc alone

• Example(s): c0009_1, c0009_2.
• What happened. The full drain → f64_field → summarize → RunManifest → to_json chain was authored entirely from the crate-root rustdoc (fn.summarize, fn.f64_field, the three structs, to_json) plus the prior fieldtest corpus for the harness wiring. Metrics matched the hand model on the first run; to_json produced a clean object.
• Why working. The cycle's headline move is reachable and correct without reading src — the module rustdoc names the post-run drain-and-fold workflow explicitly, and the field docs are precise enough to predict every value.
• Recommended action: carry-on.

[working] SimBroker firing/warm-up/slot-order docs (a resolved 0007 spec_gap) now carry the example

• Example(s): c0009_1.
• What happened. Predicting the equity sink's [0,0,0,0,1,2,3] shape required knowing slot 0 = exposure / slot 1 = price, both Firing::Any, and the "leading 0.0 rows, one row per price cycle" warm-up. All three are now on struct.SimBroker rustdoc ("Input slots" / "Firing and warm-up"). The 0007 fieldtest had to recover these from the ledger + commit body and recorded them as a spec_gap; they are now on the surface.
• Why working. A prior gap is closed and directly enabled this cycle's north-star example to be authored from rustdoc.
• Recommended action: carry-on.

[working] summarize metric definitions hold exactly on all degenerate inputs

• Example(s): c0009_3.
• What happened. All six hand-built cases matched the rustdoc field definitions to the bit, including the two subtle ones: drawdown measured from the running peak even when the whole curve is negative, and exposure_sign_flips treating flat (0.0 → sign 0) as distinct from long/short.
• Why working. The RunMetrics field docs are unambiguous and empirically exact; the "flat is sign 0" and "running-peak" subtleties are spelled out and behave as written.
• Recommended action: carry-on.

[working] f64_field panics are precise and well-located

• Example(s): c0009_4.
• What happened. f64_field on an I64 field panicked with f64_field: field 0 is not an f64 scalar: I64(7); on an out-of-range index with f64_field: row has no field 5 (row width 2). Both match the documented "checked at wiring" panic contract and name the exact offending field/width.
• Why working. The diagnostic is actionable and matches the rustdoc promise; a downstream consumer mis-wiring a sink column gets a precise message.
• Recommended action: carry-on.

[spec_gap] to_json key names / nesting are not on the public surface

• Example(s): c0009_1, c0009_4.
• What happened. RunReport::to_json rustdoc promises "field order is fixed", "params renders as a JSON object in insertion order", "f64 uses the round-trippable {} shortest form" — but it does not state the actual JSON key names or the {manifest:{...},metrics:{...}} nesting. The observed shape is {"manifest":{"commit":..,"params":{..},"window":[from,to],"seed":.., "broker":..},"metrics":{"total_pips":..,"max_drawdown":..,"exposure_sign_flips":..}}. I had to read the runtime output to learn the schema; a consumer parsing the JSON (the C18 registry, the deferred aura run printer) cannot author against it from rustdoc alone and would couple to an undocumented contract.
• Why spec_gap. The reading I guessed (keys = struct field names, window as a 2-array, params as an object) was the natural one and proved correct, but it is unconstrained by the public surface; another reading (e.g. window as {from,to}, or top-level flattening) was equally plausible. For a "machine-readable" (C14) face this schema is part of the contract.
• Recommended action: tighten the design ledger / rustdoc — add a documented JSON schema example to to_json rustdoc (a one-line sample object suffices), or ratify the field-name-mirroring as the stated contract.

[friction] to_json renders whole-valued f64 without a decimal point (3.0 → 3)

• Example(s): c0009_1, c0009_2.
• What happened. total_pips: 3.0_f64 serializes as "total_pips":3 (and 4.0 param → 4), whereas -12.5/2.5 keep the point. This is the documented round-trippable {} shortest form and is valid JSON (a number is a number), so it is not a bug. But a downstream consumer string-matching or schema-typing the field could trip: the same logical f64 field appears sometimes as an integer token, sometimes as a decimal token, within one schema. My example had to substring-match "3" rather than "3.0".
• Why friction. The task completed, but the mixed integer/decimal rendering of a single f64 field is a small surprise for a consumer expecting stable per-field token shape; it interacts with the spec_gap above (no documented schema to set the expectation).
• Recommended action: plan (tidy) — either document explicitly in the to_json rustdoc that f64 fields may render without a fractional part, or (if a stable decimal shape is wanted for the registry) normalize. Low urgency; fold into the same doc pass as the spec_gap.

Recommendation summary

Finding	Class	Action
North-star reachable from rustdoc	working	carry-on
SimBroker firing/slot docs now carry the example	working	carry-on
summarize definitions exact on degenerate inputs	working	carry-on
f64_field panics precise + well-located	working	carry-on
to_json key names / nesting not on public surface	spec_gap	tighten the design ledger / rustdoc (or ratify)
to_json renders whole f64 as integer token	friction	plan (tidy doc pass)