diff --git a/docs/specs/0095-mc-over-a-loaded-blueprint.md b/docs/specs/0095-mc-over-a-loaded-blueprint.md new file mode 100644 index 0000000..85d6e26 --- /dev/null +++ b/docs/specs/0095-mc-over-a-loaded-blueprint.md @@ -0,0 +1,305 @@ +# Monte-Carlo over a Loaded Blueprint — Design Spec + +**Date:** 2026-07-01 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +**Reference issue:** #170 (World/C21 milestone). This cycle ships the **Monte-Carlo +half** of #170; the walk-forward half is deferred (see Scope below). Design +decisions, the grounding correction, and the two-gate narrowing are recorded on +#170. Follow-on tracks: #172 (real-data block-bootstrap MC, deps #124) and #173 +(walk-forward over a loaded blueprint, deps #169). + +## Goal + +Add a `.json`-discriminator dispatch arm — `aura mc --seeds N` — +that builds a **Monte-Carlo family** from a loaded (serialized) **closed** blueprint, +mirroring cycle 0093's `aura sweep `. It runs the fixed blueprint +across N seeds, each seed drawing a distinct synthetic price walk (`SyntheticSpec:: +source(seed)`, the `mc_family` pattern), persists the family under +`FamilyKind::MonteCarlo` with the members' shared `topology_hash`, and writes the +canonical blueprint once to the content-addressed store — so a persisted +`aura mc` family reproduces bit-identically via `aura reproduce` (the concrete +proof the store hook fired). + +## Scope — MC only this cycle; walk-forward deferred + +#170 asked for both `mc` and `walkforward` over a loaded blueprint. Two independent +review gates on the draft narrowed this cycle to **MC only**: + +- **Verb-name collision (walk-forward).** The hard-wired `aura walkforward` does a + true per-window **in-sample re-optimization** (`sweep_over(is) → select_winner → + run_oos`, main.rs:2010-2016). A loaded blueprint declares no sweepable axes yet + (that is #169), so a loaded-blueprint `walkforward` could only be a *fixed-blueprint + windowed out-of-sample evaluation* — no IS refit. Shipping that under the identical + verb name (split only by a `.json` suffix) would let one trading term name two + categorically different guarantees. The honest walk-forward over a loaded blueprint + is the IS-refit form, which needs axis-discovery (#169) first; it is tracked by + #173, not forced into this cycle under a colliding name. +- The MC half carries **no** such collision: `aura mc` over a fixed closed blueprint, + varying the data realization by seed, is a Monte-Carlo in the plain sense. + +## Architecture + +### The grounding correction (foundation) + +`run_blueprint_member(signal, point, space, sources, window, seed, pip, topo)` +(main.rs:3109) runs the member over the **`sources` argument as given** and uses +`seed` / `window` only to stamp the manifest (`sim_optimal_manifest`). The +stage-1-R graph is deterministic (C1). Therefore Monte-Carlo variation cannot come +from the `seed` label alone — it must come from **re-drawing the input**, exactly +as the hard-wired `mc_family` does (`SyntheticSpec::source(seed)` → a distinct price +walk per seed, main.rs:2212). `aura mc` over a loaded blueprint therefore supplies +**seed-derived synthetic sources** to `run_blueprint_member` per member: + +| verb | facet | per-member sources | window | seed | +|------|-------|--------------------|--------|------| +| sweep (0093) | param point | showcase (full), fixed | full | 0 | +| **mc** (new) | seed | `SyntheticSpec::source(seed)` walk | that walk's window | the seed | + +`run_blueprint_member`'s signature is **unchanged** — MC only passes a different +`(sources, window, seed)` triple than the sweep's `(showcase, full, 0)`. The shared +reduce-mode member path makes reproduction bit-identical by construction (C1) — the +0094 keystone. + +### Closed blueprint only (grounded constraint) + +MC declares **no** parameter axis (only the seed varies), so it runs the blueprint at +its **fixed, fully-bound** configuration — a **closed** blueprint (an empty wrapped +`param_space`). This is exactly the case `aura run ` already supports: +it passes empty params and bootstraps only when the blueprint has zero free knobs (the +closed `stage1_signal.json` fixture). Grounding confirmed there is no per-knob +"declared default" mechanism — `ParamSpec { name, kind }` carries no default and +`compile_with_params` requires `params.len() == space.len()`. So MC uses the **empty +point** (`&[]`), never a synthesized default point. + +An **open** blueprint (a non-empty wrapped `param_space` — free knobs) is **rejected** +with a named error + exit 2 *before* any run: MC has no axis to bind those knobs. This +also fixes the boundary honestly — the raw `bootstrap`/`compile_with_params` path +*panics* on an arity mismatch (main.rs:3084), so MC guards against it up front rather +than letting a panic escape. (To vary a knob, the user reaches for `aura sweep + --axis`; to Monte-Carlo it, the blueprint must be closed.) + +### The persist seam + +`run_blueprint_mc` replicates the two things the 0093 sweep persist seam does +(`run_blueprint_sweep`, main.rs:3210): + +1. `reg.put_blueprint(&topo, &canonical)` — the content-addressed store write (the + 0094 hook), keyed by the family's shared `topology_hash`, canonical bytes + `blueprint_to_json(blueprint_from_json(doc))`. **Load-bearing:** without it, + `aura reproduce` reports the family's members as "blueprint `` missing from + store" (exit 2). (#170 reconciliation comment.) +2. `reg.append_family(name, FamilyKind::MonteCarlo, &member_reports)`. + +Then print each member line (`mc_member_line`, carrying the seed) plus the aggregate +line — matching the hard-wired `run_mc` output shape. + +### Reproduce becomes realization-aware (the one change to 0094 code) + +`reproduce_family_in` (main.rs:2503) currently reconstructs every member over +`data.run_sources()` (showcase) at `data.full_window()` with the stored seed — +correct for `Sweep`, wrong for MC (whose data is seed-derived). It gains a per-member +branch on the **family kind** that computes the member's `(sources, window, seed)`: + +- `MonteCarlo` → `synthetic_walk_sources(manifest.seed)`, that walk's window, the seed. +- everything else (`Sweep`, `CrossInstrument`) → `data.run_sources()`, + `data.full_window()`, `manifest.seed` (unchanged; keeps 0094 green). + +The grouped `Family` struct exposes `.kind: FamilyKind` (aura-registry lineage.rs:56); +the manifest stores `seed` recoverably (compat.rs:30); `synthetic_walk_sources` is a +fixed constant shared by the persist and reproduce paths, so the seed→walk +reconstruction is bit-exact (C1). + +## Concrete code shapes + +### User-facing program (the acceptance evidence) + +Monte-Carlo over a loaded closed blueprint, then prove it reproduces: + +```console +$ aura mc crates/aura-cli/tests/fixtures/stage1_signal.json --seeds 8 --name mc1 +mc1 {"family_id":"mc1-0","seed":1,"report":{...topology_hash:""...}} +mc1 {"family_id":"mc1-0","seed":2,"report":{...topology_hash:""...}} +... (8 members, one per seed; same topology_hash , differing metrics) +{"mc_aggregate":{...}} + +$ aura reproduce mc1-0 +mc1-0 member reproduced: bit-identical +... (8 members) +reproduced 8/8 members bit-identically # exit 0 +``` + +Rejecting an open blueprint (free knobs) — a named error, not a panic: + +```console +$ aura mc crates/aura-cli/tests/fixtures/stage1_signal_open.json --seeds 4 +aura: mc requires a closed blueprint (no free parameters); stage1_signal_open.json + has 2 free knobs — bind them or use `aura sweep --axis` # exit 2 +``` + +All members carry the shared `topology_hash `; a single `runs/blueprints/.json` +is written per family (one blueprint per family, C11/C12). The synthetic-walk +realization is the deliverable's *machinery*, not a trader-grade Monte-Carlo — a +real-data block-bootstrap MC is #172. + +### Dispatch arm (before → after) + +Before (main.rs:4006) — the hard-wired verb only: + +```rust +["mc", rest @ ..] => match parse_mc_args(rest) { /* Synthetic | RealR */ }, +``` + +After — a `.json`-file first positional selects the loaded-blueprint path (mirror of +the 0093 `["sweep", rest @ ..]` arm at 3952), else fall through unchanged: + +```rust +["mc", rest @ ..] => { + if let Some(path) = rest.first() + .filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) + { + let doc = read_to_string(path)/* exit 2 on io err */; + blueprint_from_json(&doc, &|t| std_vocabulary(t))/* exit 2 on parse err, file-path ctx */; + let McBlueprintArgs { n_seeds, name, data } = + parse_mc_blueprint_args(&rest[1..])/* exit 2 on usage */; + run_blueprint_mc(&doc, n_seeds, &name, DataSource::from_choice(data)); + return; + } + match parse_mc_args(rest) { /* unchanged */ } +} +``` + +### Family builder + closed-blueprint guard + +```rust +fn blueprint_mc_family(doc: &str, n_seeds: u64, data: &DataSource) -> SweepFamily { + let reload = |d: &str| blueprint_from_json(d, &|t| std_vocabulary(t)) + .expect("doc parse-validated at the dispatch boundary"); + let topo = topology_hash(&reload(doc)); + let pip = data.pip_size(); + let space = /* wrapped param_space probe, as blueprint_sweep_family does (main.rs:3177) */; + // closed-blueprint guard: MC binds no axis, so a free knob has no binder. + if !space.is_empty() { + eprintln!("aura: mc requires a closed blueprint (no free parameters); \ + {n} free knob(s) — bind them or use `aura sweep --axis`", n = space.len()); + std::process::exit(2); + } + let point: Vec = Vec::new(); // closed → empty point (as `aura run`) + let points = (1..=n_seeds).map(|seed| { + let sources = synthetic_walk_sources(seed); // fixed SyntheticSpec, seeded + let window = window_of(&sources).expect("non-empty synthetic walk"); + run_blueprint_member(reload(doc), &point, &space, sources, window, seed, pip, &topo) + }).collect(); + SweepFamily { points /* SweepPoint { report } */ } +} +``` + +### Reproduce realization branch (before → after) + +Before (main.rs:2514-2570) — one path for all members: + +```rust +let window = data.full_window(); +for member in &family.members { + // ... run_blueprint_member(reload(), &point, &space, data.run_sources(), + // window, stored.manifest.seed, pip, &hash) +} +``` + +After — the `(sources, window, seed)` triple is chosen by the family kind: + +```rust +for member in &family.members { + let (sources, window, seed) = match family.kind { + FamilyKind::MonteCarlo => { + let s = synthetic_walk_sources(stored.manifest.seed); + let w = window_of(&s).expect("non-empty walk"); + (s, w, stored.manifest.seed) + } + _ /* Sweep, CrossInstrument, (WalkForward → #173) */ => + (data.run_sources(), data.full_window(), stored.manifest.seed), + }; + let rerun = run_blueprint_member(reload(), &point, &space, sources, window, seed, pip, &hash); + outcomes.push((label, rerun.metrics == stored.metrics)); +} +``` + +## Components + +All in aura-cli `main.rs` (invariant 8: the frozen engine is untouched; invariant 9: +the registry stays a dumb bytes/records store): + +- `parse_mc_blueprint_args(&[&str]) -> Result` — pure arg + parser (mirror `parse_sweep_blueprint_args`): `--seeds ` (required, ≥1), + `--name ` (default `"mc"`); synthetic-only (real data is #172). +- `blueprint_mc_family` — the builder above, with the closed-blueprint guard. +- `run_blueprint_mc(doc, n_seeds, name, data)` — the persist seam (put_blueprint + + append_family(MonteCarlo) + print member/aggregate lines), mirroring + `run_blueprint_sweep` / `run_mc`. +- `synthetic_walk_sources(seed) -> Vec>` — the fixed seeded + `SyntheticSpec` walk (sized to warm the loaded graph — showcase-length regime), + shared by the persist and reproduce paths. +- `reproduce_family_in` — gains the `MonteCarlo` realization branch. +- one new dispatch arm. + +## Data flow + +`aura mc --seeds N`: read + parse-validate doc → `blueprint_mc_family` +(closed-guard; N members, each `run_blueprint_member` over its seed's synthetic walk) +→ `put_blueprint(topo, canonical)` → `append_family(name, MonteCarlo, reports)` → +print member lines + aggregate. + +`aura reproduce ` (MC family): load families → find `id` → for each member, +reconstruct `synthetic_walk_sources(manifest.seed)` + its window → `run_blueprint_member` +→ compare `metrics` → print bit-identical/DIVERGED → exit 0 (all identical) / 1 (any +diverged). + +## Error handling + +Mirrors the 0093/0094 boundary discipline (stderr `aura: ` + exit 2, never a +panic): + +- io error reading the `.json` file → `aura: : `, exit 2. +- blueprint parse error → `aura: : `, exit 2 (validated once at the boundary, + so per-member reloads are infallible). +- **open blueprint (non-empty wrapped `param_space`)** → the closed-blueprint guard's + named error, exit 2 (before any run — pre-empts the arity panic at main.rs:3084). +- arg usage error (missing/`0` `--seeds`, unknown flag) → the parser's usage string, exit 2. +- `aura reproduce` on an MC family: unknown id / member without `topology_hash` / + blueprint missing from store → exit 2 (unchanged 0094 behaviour); any member + diverges → named `DIVERGED` + exit 1. + +## Testing strategy + +- **Unit (`cargo test -p aura-cli --bin aura`)**: + - `blueprint_mc_family` over the **closed** fixture with `--seeds 3`: 3 members, all + sharing one `topology_hash`, with **differing** metrics — proves the seed reaches + the data (the anti-degenerate guard; a regression to seed-as-label-only would make + the members identical and fail this test). + - `parse_mc_blueprint_args`: `--seeds` required/parsed/`≥1`, default name, + unknown-flag rejection. +- **E2E (`cargo test -p aura-cli --test cli_run`)** driving the built binary: + - `aura mc --seeds 4 --name mcX` then `aura reproduce mcX-0` → + asserts `reproduced 4/4 members bit-identically` + exit 0, and exactly one + `runs/blueprints/.json`. **This is acceptance criterion (d)** — the + store-hook proof. + - `aura mc --seeds 4` → asserts the closed-blueprint named error on + stderr + exit 2 (NOT a panic / exit 101) — the boundary guard. +- No new registry code (the `append_family` MonteCarlo round-trip is already green). + +## Acceptance criteria + +1. `aura mc --seeds N` builds an N-member + `FamilyKind::MonteCarlo` family from the loaded blueprint, members sharing one + `topology_hash` and **differing** in metrics (seed → synthetic data), and writes + one stored blueprint. +2. An **open** blueprint is rejected with a named `aura:` error + exit 2 (never a + panic); the hard-wired `aura mc` path stays unchanged (the `.json` discriminator + keeps them disjoint). +3. **Reproduce compatibility (the store-hook proof):** a persisted `aura mc` family + `aura reproduce`s **bit-identically** (exit 0); the 0094 `Sweep` reproduce path + stays green. +4. Engine untouched (invariant 8); the registry gains no domain logic (invariant 9); + `cargo clippy --workspace --all-targets -- -D warnings` clean; determinism (C1) + holds — reproduction is bit-exact via the shared `run_blueprint_member` path.