Aura/docs/specs/0033-unify-runreport-json-serde.md

Unify RunReport JSON — one serde shape for stored and printed records — Design Spec

Date: 2026-06-11 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Source: Gitea issue #54 (settled design, direct specify entry — no brainstorm). Closes #54.

Goal

A single RunReport produces one JSON shape, whether it is stored to the run registry (runs/runs.jsonl) or printed to stdout (aura run / aura sweep / aura runs list / aura runs rank). Today it produces two divergent shapes: the registry serializes with serde_json (params as an array-of-pairs, floats as 2.0), while stdout renders through a hand-rolled RunReport::to_json (params as an object, whole floats as 2). The same record is therefore a structurally different JSON document on disk than on the wire, and no test pins their relationship — most visibly, aura runs list reads a record back from disk (via serde) and reprints it (via the hand-rolled writer) in a different shape than the line it just read.

This cycle retires the hand-rolled writer and renders stdout through serde too, so disk and stdout are byte-identical for the same record, and pins that identity with a test. It removes the last hand-rolled JSON writer for RunReport, realizing the amended C16 ("don't hand-roll what a vetted standard crate does").

Non-goals. The on-disk shape does not change (the registry already uses serde; this cycle makes stdout match it, nothing on the disk side moves). The graph-model JSON (model_to_json, the WASM viewer's sample-model.json) is a separate hand-rolled writer (C14) and is out of scope — it is #58/#28 territory, not touched here.

Architecture

RunReport and its nested types (RunManifest, RunMetrics) already #[derive(serde::Serialize, serde::Deserialize)] (added cycle 0029 for the registry's typed read-path). The registry already writes each record with serde_json::to_string(report) (aura-registry/src/lib.rs:35). The only thing still producing a different shape is RunReport::to_json — a hand-rolled format!-based writer plus a json_str helper in aura-engine/src/report.rs.

The change is therefore small and local: RunReport::to_json keeps its name and signature (it is the single canonical stdout renderer — the CLI calls it at every print site, and the registry header documents it as "the CLI prints via RunReport::to_json"), but its body is replaced by a delegation to serde_json::to_string(self). The hand-rolled json_str helper is deleted (it becomes unused — "the last hand-rolled JSON writer" is gone). Keeping the one canonical render method means the CLI call sites and the engine's own determinism tests are untouched at the call boundary; only the bytes they emit change shape.

Dependency consequence (load-bearing, named for the C16 per-case review). serde_json is today a dev-dependency of aura-engine (serde is a normal dep for the derive; serde_json is only pulled in for the test module). Because to_json is a non-test method, it must move serde_json from [dev-dependencies] to [dependencies] in aura-engine/Cargo.toml. Under the amended C16 per-case dependency policy this passes: serde_json is a vetted standard crate already in the workspace (used by aura-registry and aura-ingest), its output is deterministic (C1-safe — already relied upon for the disk path), and it is precisely "the vetted standard crate doing what the code would otherwise hand-roll". The frozen-bot closure gains serde_json, which is justified: the bot emits RunReport JSON as its structured output, so the canonical JSON serializer is a legitimate inclusion, not redundant weight.

Concrete code shapes

The observable change (consumer surface) — the same record, one shape

The criterion's empirical evidence is what a consumer reading aura's stdout sees. A consumer runs aura sweep (or aura run / aura runs list) and gets one JSON line per record. The only change a consumer observes is that stdout now matches the disk shape: params is an array-of-pairs, and finite floats carry a fractional part.

Before (hand-rolled to_json) — one aura sweep line, abbreviated to the manifest's params:

…"params":{"sma_cross.fast.length":2,"sma_cross.slow.length":4,"exposure.scale":0.5}…

After (serde) — the identical record, now byte-identical to its runs.jsonl line:

…"params":[["sma_cross.fast.length",2.0],["sma_cross.slow.length",4.0],["exposure.scale",0.5]]…

The full canonical-form record (the to_json_renders_the_canonical_form fixture) changes shape as follows.

Before:

{"manifest":{"commit":"abc123","params":{"sma_fast":2,"sma_slow":4,"exposure_scale":1},"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12,"max_drawdown":1,"exposure_sign_flips":1}}

After:

{"manifest":{"commit":"abc123","params":[["sma_fast",2.0],["sma_slow",4.0],["exposure_scale",1.0]],"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12.0,"max_drawdown":1.0,"exposure_sign_flips":1}}

What changed: params object → array-of-pairs; the finite-float fields render with a fractional part (2 → 2.0, 12 → 12.0, 1 → 1.0). What is unchanged: field order (serde follows struct-declaration order, which already matches the hand-rolled order), the nested manifest / metrics envelope, window as a 2-element integer array [1,6], and the integer-valued seed / exposure_sign_flips (they are u64, not f64, so they stay 0 / 1).

The unifying pin (the new load-bearing test)

The relationship the cycle exists to guarantee — stdout shape equals disk shape for the same record — is pinned directly:

// aura-engine/src/report.rs (tests)
#[test]
fn to_json_equals_serde_disk_shape() {
    // the same RunReport value the `to_json_renders_the_canonical_form`
    // test constructs (commit "abc123", three params, window (1,6), …).
    let report = RunReport { /* canonical-form fixture, as in that test */ };
    // stdout (to_json) and disk (serde_json::to_string) are now the same bytes.
    assert_eq!(report.to_json(), serde_json::to_string(&report).unwrap());
}

Implementation shape (before → after)

Before (aura-engine/src/report.rs:80):

pub fn to_json(&self) -> String {
    let m = &self.manifest;
    let mut params = String::from("{");
    for (i, (name, value)) in m.params.iter().enumerate() {
        if i > 0 { params.push(','); }
        params.push_str(&json_str(name));
        params.push(':');
        params.push_str(&value.to_string());
    }
    params.push('}');
    format!(
        "{{\"manifest\":{{\"commit\":{commit},\"params\":{params},…}}}}",
        /* … hand-rolled field interpolation … */
    )
}
// … plus `fn json_str(s: &str) -> String { … }` lower in the file

After:

/// Render the canonical, machine-readable JSON (C14) via serde — the same
/// encoder the run registry uses on disk, so a record's stdout shape and its
/// `runs.jsonl` shape are byte-identical. `params` is an array of
/// `[name, value]` pairs; finite `f64` fields carry a fractional part (`2.0`).
pub fn to_json(&self) -> String {
    serde_json::to_string(self).expect("a finite RunReport always serializes")
}
// `json_str` is deleted (unused once the hand-rolled writer is gone).

Cargo manifest (before → after) for aura-engine/Cargo.toml:

# before: serde_json only under [dev-dependencies]
# after:  serde_json moved to [dependencies] (vetted, workspace-pinned, C1-safe)
[dependencies]
serde = { workspace = true }
serde_json = { workspace = true }

Components

• aura-engine/src/report.rs — RunReport::to_json body swapped to serde; json_str deleted; the method's doc comment rewritten to describe the serde shape (drop the "hand-rolled" / params-as-object / whole-float-token prose).
• aura-engine/Cargo.toml — serde_json promoted dev-dep → normal dep.
• No CLI code change. The report.to_json() call sites in aura-cli/src/main.rs (run / sweep / sweep-point / runs list / runs rank / the run and run --macd dispatch) are unchanged — they keep calling the same method, which now renders via serde.

Data flow

Unchanged structurally. A run produces a RunReport; the World drains its recording sinks and folds metrics (as today). On the print path the CLI calls report.to_json(), which now defers to serde_json::to_string. On the store path the registry calls serde_json::to_string (as today). Both paths now go through serde, so they emit identical bytes. aura runs list / aura runs rank read records back from disk via serde_json::from_str and reprint via the same serde encoder — read-shape and reprint-shape now coincide.

Error handling

serde_json::to_string on a RunReport is infallible in practice: the type is a flat composition of String, f64, u64, i64 (via Timestamp), Vec, and tuples — no map with non-string keys, no custom serializer that can error. The .expect("a finite RunReport always serializes") mirrors the registry's existing serde_json::to_string(report).expect("a finite RunReport serializes") (aura-registry/src/lib.rs:35) — the same justified expectation, consistent across the two render sites.

Determinism (C1) is preserved: serde follows struct-declaration order, Vec and tuple order are positional, and there is no HashMap anywhere in the report types, so the serialization is a pure deterministic function of the value — the same property the registry has relied on since cycle 0029.

Testing strategy

1. Update the engine canonical-form golden. to_json_renders_the_canonical_form (report.rs) asserts the full string; its expected value changes to the serde shape shown above (params array-of-pairs, finite floats with .0). RED before the body swap (old golden ≠ serde output), GREEN after.
2. Update the CLI sweep golden. aura-cli/tests/cli_run.rs (the "params":{…} substring at ~line 214) changes from the params-object form to the serde array-of-pairs form "params":[["sma_cross.fast.length",2.0],["sma_cross.slow.length",4.0],["exposure.scale",0.5]]. RED before, GREEN after.
3. Add the unifying pin to_json_equals_serde_disk_shape (shown above): report.to_json() == serde_json::to_string(&report). This is the test the issue says is missing — it pins the stdout↔disk relationship so the two encoders can never silently diverge again.
4. Determinism asserts stay green, unchanged. report_is_deterministic_end_to_end (report.rs), sweep_report_is_deterministic, run_macd_compiles_…_is_deterministic, run_sample_is_deterministic_and_non_trivial (aura-cli/src/main.rs) all assert r1.to_json() == r2.to_json(); serde is deterministic, so they remain green with no edit.
5. Structural CLI asserts stay green, unchanged. The run_prints_json_and_exits_zero structural checks (cli_run.rs): starts_with("{\"manifest\":{\"commit\":\""), contains("\"window\":[1,7]"), contains("\"metrics\":{\"total_pips\":"), ends_with("\"exposure_sign_flips\":1}}") — none touches the params shape or a whole-float value, so all stay green (verified against the serde shape).
6. The existing runreport_serde_round_trips stays green, unchanged — it already exercises the serde path the stdout side now shares.

Final gate: cargo build --workspace + cargo test --workspace green + cargo clippy --workspace --all-targets -- -D warnings clean.

Acceptance criteria

1. Removes redundancy (criterion: measurably removes redundancy). The hand-rolled RunReport JSON writer (to_json's old body + json_str) is gone; one encoder (serde) produces the report's JSON for both store and print. This is the last hand-rolled RunReport JSON writer.
2. Improves correctness (criterion: measurably improves correctness). A new test pins to_json() == serde_json::to_string(&report), closing the previously-unpinned divergence where the same record was a different document on disk vs. on stdout (visible in aura runs list).
3. Reintroduces no failure class (criterion: no regression of a core constraint). C1 determinism is preserved (serde is a deterministic pure function of the value, no HashMap); no new external crate enters the workspace (serde_json is already vetted and present), only a dev-dep → normal-dep promotion within aura-engine, justified under the amended C16.
4. Disk shape is unchanged. runs.jsonl records are byte-identical before and after this cycle; only stdout moves to match disk.
5. Out-of-scope surfaces untouched. model_to_json / sample-model.json (the graph-model JSON, C14) is not modified.

Minor follow-up (not this cycle): the design ledger cross-reference at docs/design/INDEX.md:729 ("hand-rolled JSON model (C14, golden-tested like RunReport::to_json)") becomes a stale comparison once RunReport::to_json is serde-backed. It refers to model_to_json (still hand-rolled); the architect may touch up the prose at cycle-close audit.