Brummel/Aura
1
0
Fork 0
Code Issues 33 Pull Requests Actions Packages Projects Releases Wiki Activity

spec: 0033 unify-runreport-json-serde (boss-signed)

Browse Source 
Settled-source direct entry from issue #54 (no brainstorm). Retire the
hand-rolled RunReport::to_json writer and render stdout through serde, so a
record's stdout JSON is byte-identical to its runs.jsonl line, and pin that
identity with a test. Removes the last hand-rolled RunReport JSON writer
(amended C16). Disk shape unchanged; graph-model JSON (model_to_json) out of
scope (#58/#28).

Auto-signed under /boss spec_auto_sign: objective gates green (precondition
clean, self-review clean, grounding-check PASS) + unanimous 5-lens spec-skeptic
panel (criterion/grounding/scope-fork/ambiguity/plan-readiness all SOUND). The
grounding juror re-ran serde_json 1.0.150 against the canonical-form fixture and
confirmed the spec's "After" bytes. The one C16-scrutinized consequence
(serde_json dev-dep -> normal dep in aura-engine) is named and justified in the
spec; scope-fork juror confirmed it is settled by amended C16, not a silent pick.

refs #54
This commit is contained in:
Brummel
2026-06-11 19:33:33 +02:00
parent 065cf1df1e
commit c8634d7fa9
1 changed files with 267 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/0033-unify-runreport-json-serde.md
+267
View File
@@ -0,0 +1,267 @@
# Unify RunReport JSON — one serde shape for stored and printed records — Design Spec
**Date:** 2026-06-11
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude
> **Source:** Gitea issue #54 (settled design, direct `specify` entry — no
> brainstorm). Closes #54.
## Goal
A single `RunReport` produces **one** JSON shape, whether it is stored to the
run registry (`runs/runs.jsonl`) or printed to stdout (`aura run` / `aura sweep`
/ `aura runs list` / `aura runs rank`). Today it produces **two divergent
shapes**: the registry serializes with `serde_json` (params as an array-of-pairs,
floats as `2.0`), while stdout renders through a hand-rolled
`RunReport::to_json` (params as an object, whole floats as `2`). The same record
is therefore a structurally different JSON document on disk than on the wire, and
no test pins their relationship — most visibly, `aura runs list` reads a record
back from disk (via serde) and reprints it (via the hand-rolled writer) in a
*different* shape than the line it just read.
This cycle retires the hand-rolled writer and renders stdout through serde too,
so disk and stdout are byte-identical for the same record, and pins that identity
with a test. It removes the last hand-rolled JSON writer for `RunReport`,
realizing the amended C16 ("don't hand-roll what a vetted standard crate does").
**Non-goals.** The on-disk shape does **not** change (the registry already uses
serde; this cycle makes stdout *match* it, nothing on the disk side moves). The
graph-model JSON (`model_to_json`, the WASM viewer's `sample-model.json`) is a
*separate* hand-rolled writer (C14) and is out of scope — it is #58/#28
territory, not touched here.
## Architecture
`RunReport` and its nested types (`RunManifest`, `RunMetrics`) already
`#[derive(serde::Serialize, serde::Deserialize)]` (added cycle 0029 for the
registry's typed read-path). The registry already writes each record with
`serde_json::to_string(report)` (`aura-registry/src/lib.rs:35`). The only thing
still producing a *different* shape is `RunReport::to_json` — a hand-rolled
`format!`-based writer plus a `json_str` helper in `aura-engine/src/report.rs`.
The change is therefore small and local: `RunReport::to_json` keeps its name and
signature (it is the single canonical stdout renderer — the CLI calls it at
every print site, and the registry header documents it as "the CLI prints via
`RunReport::to_json`"), but its **body** is replaced by a delegation to
`serde_json::to_string(self)`. The hand-rolled `json_str` helper is deleted (it
becomes unused — "the last hand-rolled JSON writer" is gone). Keeping the one
canonical render method means the CLI call sites and the engine's own
determinism tests are untouched at the call boundary; only the bytes they emit
change shape.
**Dependency consequence (load-bearing, named for the C16 per-case review).**
`serde_json` is today a **dev-dependency** of `aura-engine` (`serde` is a normal
dep for the derive; `serde_json` is only pulled in for the test module). Because
`to_json` is a non-test method, it must move `serde_json` from
`[dev-dependencies]` to `[dependencies]` in `aura-engine/Cargo.toml`. Under the
amended C16 per-case dependency policy this passes: `serde_json` is a vetted
standard crate already in the workspace (used by `aura-registry` and
`aura-ingest`), its output is deterministic (C1-safe — already relied upon for
the disk path), and it is precisely "the vetted standard crate doing what the
code would otherwise hand-roll". The frozen-bot closure gains `serde_json`, which
is justified: the bot emits `RunReport` JSON as its structured output, so the
canonical JSON serializer is a legitimate inclusion, not redundant weight.
## Concrete code shapes
### The observable change (consumer surface) — the same record, one shape
The criterion's empirical evidence is what a consumer reading aura's stdout sees.
A consumer runs `aura sweep` (or `aura run` / `aura runs list`) and gets one JSON
line per record. The **only** change a consumer observes is that stdout now
matches the disk shape: `params` is an array-of-pairs, and finite floats carry a
fractional part.
Before (hand-rolled `to_json`) — one `aura sweep` line, abbreviated to the
manifest's `params`:
```text
…"params":{"sma_cross.fast.length":2,"sma_cross.slow.length":4,"exposure.scale":0.5}…
```
After (serde) — the identical record, now byte-identical to its `runs.jsonl`
line:
```text
…"params":[["sma_cross.fast.length",2.0],["sma_cross.slow.length",4.0],["exposure.scale",0.5]]…
```
The full canonical-form record (the `to_json_renders_the_canonical_form` fixture)
changes shape as follows.
Before:
```text
{"manifest":{"commit":"abc123","params":{"sma_fast":2,"sma_slow":4,"exposure_scale":1},"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12,"max_drawdown":1,"exposure_sign_flips":1}}
```
After:
```text
{"manifest":{"commit":"abc123","params":[["sma_fast",2.0],["sma_slow",4.0],["exposure_scale",1.0]],"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12.0,"max_drawdown":1.0,"exposure_sign_flips":1}}
```
What changed: `params` object → array-of-pairs; the finite-float fields render
with a fractional part (`2` → `2.0`, `12` → `12.0`, `1` → `1.0`). What is
unchanged: field order (serde follows struct-declaration order, which already
matches the hand-rolled order), the nested `manifest` / `metrics` envelope,
`window` as a 2-element integer array `[1,6]`, and the integer-valued `seed` /
`exposure_sign_flips` (they are `u64`, not `f64`, so they stay `0` / `1`).
### The unifying pin (the new load-bearing test)
The relationship the cycle exists to guarantee — stdout shape **equals** disk
shape for the same record — is pinned directly:
```rust
// aura-engine/src/report.rs (tests)
#[test]
fn to_json_equals_serde_disk_shape() {
// the same RunReport value the `to_json_renders_the_canonical_form`
// test constructs (commit "abc123", three params, window (1,6), …).
let report = RunReport { /* canonical-form fixture, as in that test */ };
// stdout (to_json) and disk (serde_json::to_string) are now the same bytes.
assert_eq!(report.to_json(), serde_json::to_string(&report).unwrap());
}
```
### Implementation shape (before → after)
Before (`aura-engine/src/report.rs:80`):
```rust
pub fn to_json(&self) -> String {
let m = &self.manifest;
let mut params = String::from("{");
for (i, (name, value)) in m.params.iter().enumerate() {
if i > 0 { params.push(','); }
params.push_str(&json_str(name));
params.push(':');
params.push_str(&value.to_string());
}
params.push('}');
format!(
"{{\"manifest\":{{\"commit\":{commit},\"params\":{params},…}}}}",
/* … hand-rolled field interpolation … */
)
}
// … plus `fn json_str(s: &str) -> String { … }` lower in the file
```
After:
```rust
/// Render the canonical, machine-readable JSON (C14) via serde — the same
/// encoder the run registry uses on disk, so a record's stdout shape and its
/// `runs.jsonl` shape are byte-identical. `params` is an array of
/// `[name, value]` pairs; finite `f64` fields carry a fractional part (`2.0`).
pub fn to_json(&self) -> String {
serde_json::to_string(self).expect("a finite RunReport always serializes")
}
// `json_str` is deleted (unused once the hand-rolled writer is gone).
```
Cargo manifest (before → after) for `aura-engine/Cargo.toml`:
```toml
# before: serde_json only under [dev-dependencies]
# after: serde_json moved to [dependencies] (vetted, workspace-pinned, C1-safe)
[dependencies]
serde = { workspace = true }
serde_json = { workspace = true }
```
## Components
- **`aura-engine/src/report.rs`** — `RunReport::to_json` body swapped to serde;
`json_str` deleted; the method's doc comment rewritten to describe the serde
shape (drop the "hand-rolled" / params-as-object / whole-float-token prose).
- **`aura-engine/Cargo.toml`** — `serde_json` promoted dev-dep → normal dep.
- **No CLI code change.** The `report.to_json()` call sites in
`aura-cli/src/main.rs` (run / sweep / sweep-point / runs list / runs rank /
the `run` and `run --macd` dispatch) are unchanged — they keep calling the
same method, which now renders via serde.
## Data flow
Unchanged structurally. A run produces a `RunReport`; the World drains its
recording sinks and folds metrics (as today). On the print path the CLI calls
`report.to_json()`, which now defers to `serde_json::to_string`. On the store
path the registry calls `serde_json::to_string` (as today). Both paths now go
through serde, so they emit identical bytes. `aura runs list` / `aura runs rank`
read records back from disk via `serde_json::from_str` and reprint via the same
serde encoder — read-shape and reprint-shape now coincide.
## Error handling
`serde_json::to_string` on a `RunReport` is infallible in practice: the type is a
flat composition of `String`, `f64`, `u64`, `i64` (via `Timestamp`), `Vec`, and
tuples — no map with non-string keys, no custom serializer that can error. The
`.expect("a finite RunReport always serializes")` mirrors the registry's existing
`serde_json::to_string(report).expect("a finite RunReport serializes")`
(`aura-registry/src/lib.rs:35`) — the same justified expectation, consistent
across the two render sites.
Determinism (C1) is preserved: serde follows struct-declaration order, `Vec` and
tuple order are positional, and there is no `HashMap` anywhere in the report
types, so the serialization is a pure deterministic function of the value — the
same property the registry has relied on since cycle 0029.
## Testing strategy
1. **Update the engine canonical-form golden.**
`to_json_renders_the_canonical_form` (`report.rs`) asserts the full string;
its expected value changes to the serde shape shown above (params
array-of-pairs, finite floats with `.0`). RED before the body swap (old golden
≠ serde output), GREEN after.
2. **Update the CLI sweep golden.** `aura-cli/tests/cli_run.rs` (the
`"params":{…}` substring at ~line 214) changes from the params-object form to
the serde array-of-pairs form
`"params":[["sma_cross.fast.length",2.0],["sma_cross.slow.length",4.0],["exposure.scale",0.5]]`.
RED before, GREEN after.
3. **Add the unifying pin** `to_json_equals_serde_disk_shape` (shown above):
`report.to_json() == serde_json::to_string(&report)`. This is the test the
issue says is missing — it pins the stdout↔disk relationship so the two
encoders can never silently diverge again.
4. **Determinism asserts stay green, unchanged.** `report_is_deterministic_end_to_end`
(`report.rs`), `sweep_report_is_deterministic`,
`run_macd_compiles_…_is_deterministic`, `run_sample_is_deterministic_and_non_trivial`
(`aura-cli/src/main.rs`) all assert `r1.to_json() == r2.to_json()`; serde is
deterministic, so they remain green with no edit.
5. **Structural CLI asserts stay green, unchanged.** The `run_prints_json_and_exits_zero`
structural checks (`cli_run.rs`): `starts_with("{\"manifest\":{\"commit\":\"")`,
`contains("\"window\":[1,7]")`, `contains("\"metrics\":{\"total_pips\":")`,
`ends_with("\"exposure_sign_flips\":1}}")` — none touches the `params` shape or
a whole-float value, so all stay green (verified against the serde shape).
6. **The existing `runreport_serde_round_trips` stays green, unchanged** — it
already exercises the serde path the stdout side now shares.
Final gate: `cargo build --workspace` + `cargo test --workspace` green +
`cargo clippy --workspace --all-targets -- -D warnings` clean.
## Acceptance criteria
1. **Removes redundancy (criterion: measurably removes redundancy).** The
hand-rolled `RunReport` JSON writer (`to_json`'s old body + `json_str`) is
gone; one encoder (serde) produces the report's JSON for both store and print.
This is the last hand-rolled `RunReport` JSON writer.
2. **Improves correctness (criterion: measurably improves correctness).** A new
test pins `to_json() == serde_json::to_string(&report)`, closing the
previously-unpinned divergence where the same record was a different document
on disk vs. on stdout (visible in `aura runs list`).
3. **Reintroduces no failure class (criterion: no regression of a core
constraint).** C1 determinism is preserved (serde is a deterministic pure
function of the value, no `HashMap`); no *new* external crate enters the
workspace (`serde_json` is already vetted and present), only a dev-dep → normal-dep
promotion within `aura-engine`, justified under the amended C16.
4. **Disk shape is unchanged.** `runs.jsonl` records are byte-identical before
and after this cycle; only stdout moves to match disk.
5. **Out-of-scope surfaces untouched.** `model_to_json` / `sample-model.json`
(the graph-model JSON, C14) is not modified.
> **Minor follow-up (not this cycle):** the design ledger cross-reference at
> `docs/design/INDEX.md:729` ("hand-rolled JSON model (C14, golden-tested like
> `RunReport::to_json`)") becomes a stale comparison once `RunReport::to_json` is
> serde-backed. It refers to `model_to_json` (still hand-rolled); the architect
> may touch up the prose at cycle-close audit.