diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index 2dcb5f0..ac4a5e1 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -1313,7 +1313,10 @@ content-addressing, and whether the id should exclude the composite **debug-name non-load-bearing symbol, invariant 11 — an op-script and the Rust builder produce different canonical forms today), remain deferred (#171, #170). Reproduction is proven on synthetic (deterministic) data; recorded-dataset reproduction rides the DataServer seam -(#124). +(#124). **Cycle 0095 (#170):** `aura reproduce` now also spans `FamilyKind::MonteCarlo` — +branching on the family kind, it reconstructs each member's seed-driven synthetic walk from +the recorded `manifest.seed` (the `Sweep` arm unchanged), so an `aura mc` family re-derives +bit-identically through the same `run_blueprint_member` path. ### C19 — Bootstrap: blueprint → instance (recursive) **Guarantee.** Construction is a distinct phase, recursive at every level. Each @@ -1886,9 +1889,21 @@ reproducible per C18. Two design facts: a sweep needs an **open** blueprint (a f has an empty `param_space`, distinct from cycle-1's bound run fixture), and the signal is **re-loaded per member** from its serialized doc (the `Composite` is `!Clone` — its `PrimitiveBuilder`s hold `Box` build closures, #164 — and `bootstrap_with_cells` -consumes the graph). Monte-Carlo / walk-forward over a loaded blueprint are the same machinery -per verb (a tracked follow-on); per-member trace-writing on `--trace` and an axis-name -discovery surface are tracked debt. +consumes the graph). **Monte-Carlo over a loaded blueprint shipped (cycle 0095, #170).** `aura mc + --seeds N` runs a **closed** blueprint across N seeds — each seed a distinct +synthetic price walk (`synthetic_walk_sources`, the `mc_family` `SyntheticSpec` pattern), the +draws disjoint-parallel via the engine `monte_carlo` seam (invariant 1) — and aggregates to a +`FamilyKind::MonteCarlo` family with one stored blueprint per family (the C18 hook), so it +`aura reproduce`s bit-identically (C1). MC binds no axis, so it needs a **closed** blueprint +(the sweep's open/closed distinction inverted); an open one returns a named `Err` (rendered +exit-2 at the `run_blueprint_mc` boundary, the sweep sibling's contract — no hidden exit in the +pure builder). **Walk-forward is deferred (#173, dep #169):** a loaded blueprint declares no +axes to re-fit per window, so it could only be a non-IS-refit windowed eval — which would +collide with the hard-wired `walkforward` verb's IS-optimizing guarantee. The synthetic-walk DGP +is the machinery, not trader-grade MC statistics; a real-data block-bootstrap — and the +`synthetic_walk_sources` `len:60`↔warm-up coupling it retires (a deep-lookback closed blueprint +warms poorly → silent-vacuous draws today) — ride #172. Per-member trace-writing on `--trace` and +an axis-name discovery surface are tracked debt. **Content-addressed reproduction shipped (cycle 0094, #158, C18)**: `topology_hash` landed cycle 0092; re-deriving a member's FlatGraph from a stored manifest + the diff --git a/docs/plans/0095-mc-over-a-loaded-blueprint.md b/docs/plans/0095-mc-over-a-loaded-blueprint.md deleted file mode 100644 index 427ad13..0000000 --- a/docs/plans/0095-mc-over-a-loaded-blueprint.md +++ /dev/null @@ -1,565 +0,0 @@ -# Monte-Carlo over a Loaded Blueprint — Implementation Plan - -> **Parent spec:** `docs/specs/0095-mc-over-a-loaded-blueprint.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run -> this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** Add `aura mc --seeds N` — a Monte-Carlo family from a -loaded CLOSED blueprint, each seed drawing a distinct synthetic walk — persisted as -`FamilyKind::MonteCarlo` with the content-addressed store hook, `aura reproduce` -bit-identical. - -**Architecture:** All code lands in `crates/aura-cli/src/main.rs` (engine + registry -untouched — invariants 8/9). A new `.json`-discriminator on the `["mc", ..]` dispatch -arm (mirroring the 0093 sweep arm) routes to `blueprint_mc_family` (builds an -`McFamily` by running the fixed closed blueprint over N seed-driven synthetic walks -through the shared `run_blueprint_member` path) → `run_blueprint_mc` (the persist seam: -`put_blueprint` + `append_family(MonteCarlo)` + print). `reproduce_family_in` gains a -`MonteCarlo` realization branch that reconstructs each member's seed-driven walk. - -**Tech Stack:** Rust; `SyntheticSpec::source(seed)` (aura-engine), `McFamily`/`McDraw`/ -`McAggregate` (aura-engine), `run_blueprint_member` (unchanged), `Registry::put_blueprint` -+ `append_family` (aura-registry), `topology_hash`/`content_id` (local). - ---- - -**Files this plan creates or modifies:** - -- Modify: `crates/aura-cli/src/main.rs` — new `parse_mc_blueprint_args` + `McBlueprintArgs` - (beside `parse_sweep_blueprint_args` ~3797-3835); new `synthetic_walk_sources` + - `blueprint_mc_family` + `run_blueprint_mc` (beside `blueprint_sweep_family`/`run_blueprint_sweep` - ~3154-3246); the `MonteCarlo` branch in `reproduce_family_in` (~2503-2574, the run at ~2561); - the `["mc", rest @ ..]` dispatch arm (~4006-4015). -- Test: `crates/aura-cli/src/main.rs` — unit tests beside the sweep units (~5144-5230): - `parse_mc_blueprint_args_grammar`, `blueprint_mc_family_seeds_differ`, - `reproduce_family_re_derives_every_mc_member_bit_identically`. -- Test: `crates/aura-cli/tests/cli_run.rs` — E2E beside the sweep/reproduce E2Es (~3296-3489): - `aura_mc_over_a_blueprint_reproduces_bit_identically`, `aura_mc_rejects_an_open_blueprint`. - -Facts pinned by plan-recon (line refs are current; re-anchor by symbol if drifted): -- `run_blueprint_member(signal: Composite, point: &[Cell], space: &[ParamSpec], sources: - Vec>, window: (Timestamp,Timestamp), seed: u64, pip: f64, - topo: &str) -> RunReport` — **unchanged**; stamps `manifest.seed`/`manifest.topology_hash`. -- `McDraw { pub seed: u64, pub report: RunReport }`, `McFamily { pub draws, pub aggregate }`, - `McAggregate::from_draws(&[McDraw]) -> McAggregate` (all pub; `from_draws` needs draws non-empty). -- `SyntheticSpec { start: f64, len: usize, step: i64 }`; `.source(seed) -> impl Source`; - `window_of(&[Box]) -> Option<(Timestamp,Timestamp)>`. -- `mc_member_line(id, seed, report) -> String`; `mc_aggregate_json(&McAggregate) -> String`; - `mc_member_reports(&McFamily) -> Vec`. -- `Family { pub id, pub kind: FamilyKind, pub members: Vec }`; - `FamilyKind::MonteCarlo` is a variant. `stage1_signal(fast: Option, slow: Option) - -> Composite` (Some binds → closed). `STAGE1_R_STOP_LENGTH = 3`, fixture SMA slow=4 → the - graph warms in ~4 bars, so a len-60 walk yields many trades that differ across seeds. -- All needed imports (`Cell`, `ParamSpec`, `Composite`, `McFamily`, `McDraw`, `McAggregate`, - `SyntheticSpec`, `window_of`, `blueprint_from_json`, `blueprint_to_json`, `FamilyKind`, - `mc_member_reports`, `std_vocabulary`) are already in `main.rs` — no `use` edits. - ---- - -### Task 1: `parse_mc_blueprint_args` + `McBlueprintArgs` - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (beside `parse_sweep_blueprint_args` ~3797-3835) -- Test: `crates/aura-cli/src/main.rs` (beside the sweep-arg unit `parse_sweep_blueprint_args_grammar` ~5107) - -- [ ] **Step 1: Write the failing test** - -Add to the `#[cfg(test)] mod tests` block, beside `parse_sweep_blueprint_args_grammar`: - -```rust -#[test] -fn parse_mc_blueprint_args_grammar() { - // --seeds required and parsed; default name "mc"; seeds >= 1; unknown/real flags rejected. - let ok = parse_mc_blueprint_args(&["--seeds", "8", "--name", "mc1"]).expect("valid"); - assert_eq!(ok.n_seeds, 8); - assert_eq!(ok.name, "mc1"); - let def = parse_mc_blueprint_args(&["--seeds", "3"]).expect("default name"); - assert_eq!(def.name, "mc"); - assert!(parse_mc_blueprint_args(&[]).is_err(), "--seeds is required"); - assert!(parse_mc_blueprint_args(&["--seeds", "0"]).is_err(), "seeds must be >= 1"); - assert!(parse_mc_blueprint_args(&["--seeds", "x"]).is_err(), "seeds must be a number"); - assert!( - parse_mc_blueprint_args(&["--seeds", "4", "--real", "EURUSD"]).is_err(), - "synthetic-only this cycle: --real is rejected (real-data MC is #172)" - ); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --bin aura parse_mc_blueprint_args_grammar` -Expected: FAIL — compile error `cannot find function parse_mc_blueprint_args` (the fn/struct do not exist yet). - -- [ ] **Step 3: Write minimal implementation** - -Add beside `parse_sweep_blueprint_args` (~3797): - -```rust -/// Parsed tail of `aura mc …`: the seed count + the family name. -/// Synthetic-only this cycle (real-data MC rides #172), so `--real`/`--from`/`--to` -/// are rejected as unknown flags. Mirrors `parse_sweep_blueprint_args`' shape. -struct McBlueprintArgs { - n_seeds: u64, - name: String, -} - -fn parse_mc_blueprint_args(rest: &[&str]) -> Result { - let usage = || "usage: mc --seeds [--name ]".to_string(); - let mut n_seeds: Option = None; - let mut name: Option = None; - let mut tail = rest; - while let Some((flag, t)) = tail.split_first() { - let (value, t) = t.split_first().ok_or_else(usage)?; - match *flag { - "--seeds" => { - let n: u64 = value.parse().map_err(|_| usage())?; - if n == 0 { - return Err(usage()); - } - n_seeds = Some(n); - } - "--name" if name.is_none() => name = Some((*value).to_string()), - _ => return Err(usage()), - } - tail = t; - } - let n_seeds = n_seeds.ok_or_else(usage)?; // --seeds is required - Ok(McBlueprintArgs { n_seeds, name: name.unwrap_or_else(|| "mc".to_string()) }) -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cargo test -p aura-cli --bin aura parse_mc_blueprint_args_grammar` -Expected: PASS. - ---- - -### Task 2: `synthetic_walk_sources` + `blueprint_mc_family` (closed-blueprint guard) - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (beside `blueprint_sweep_family` ~3154; `mc_family` pattern ~2210) -- Test: `crates/aura-cli/src/main.rs` (beside `blueprint_sweep_member_equals_single_run_and_shares_topology_hash` ~5151) - -- [ ] **Step 1: Write the failing test** - -```rust -#[test] -fn blueprint_mc_family_seeds_differ() { - // MC over a CLOSED signal (both SMA knobs bound): 3 seeds -> 3 draws, one shared - // topology_hash, and DIFFERING metrics — the seed reaches the DATA (a distinct - // synthetic walk per draw), not just the manifest label. The anti-degenerate guard: - // a regression to seed-as-label-only would make the three draws identical. - let closed = stage1_signal(Some(2), Some(4)); - let doc = blueprint_to_json(&closed).expect("serializes"); - let family = blueprint_mc_family(&doc, 3, &DataSource::Synthetic); - - assert_eq!(family.draws.len(), 3, "one draw per seed"); - assert_eq!(family.draws.iter().map(|d| d.seed).collect::>(), vec![1, 2, 3]); - let topo = family.draws[0].report.manifest.topology_hash.clone(); - assert!(topo.is_some(), "members carry a topology_hash"); - assert!( - family.draws.iter().all(|d| d.report.manifest.topology_hash == topo), - "all members share one topology_hash" - ); - let m: Vec<_> = family.draws.iter().map(|d| &d.report.metrics).collect(); - assert!(m[0] != m[1] || m[1] != m[2], "seeds must yield differing realizations"); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --bin aura blueprint_mc_family_seeds_differ` -Expected: FAIL — compile error `cannot find function blueprint_mc_family` (fn does not exist yet). - -- [ ] **Step 3: Write minimal implementation** - -Add beside `blueprint_sweep_family` (~3154): - -```rust -/// A fresh seeded synthetic price walk for one Monte-Carlo draw — the `mc_family` -/// pattern (a distinct realization per seed). A FIXED `SyntheticSpec` shared by the -/// `aura mc ` persist path AND the reproduce MonteCarlo branch, so the -/// seed->walk reconstruction is bit-exact (C1). Length 60 comfortably warms the loaded -/// stage1-r graph (SMA slow=4 + the len-3 vol stop) so draws produce differing trades. -fn synthetic_walk_sources(seed: u64) -> Vec> { - let spec = SyntheticSpec { start: 1.0, len: 60, step: 1 }; - vec![Box::new(spec.source(seed))] -} - -/// Build a Monte-Carlo family from a loaded CLOSED signal blueprint: run the fixed -/// blueprint across `n_seeds` seeds, each seed drawing a distinct synthetic walk. The -/// blueprint must be CLOSED (empty wrapped `param_space`) — MC binds no axis, so a free -/// knob has no binder; an OPEN blueprint is rejected with a named error (exit 2) before -/// any run, pre-empting the `compile_with_params` arity panic. Each draw runs the shared -/// reduce-mode member path (`run_blueprint_member`, the same fn reproduce re-runs), so -/// reproduction is bit-identical (C1); every member carries the shared `topology_hash`. -fn blueprint_mc_family(doc: &str, n_seeds: u64, data: &DataSource) -> McFamily { - let reload = |d: &str| { - blueprint_from_json(d, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary; reload is infallible") - }; - let topo = topology_hash(&reload(doc)); - let pip = data.pip_size(); - // probe the wrapped param_space (as blueprint_sweep_family does); MC needs it empty. - let (tx_eq, _) = mpsc::channel(); - let (tx_ex, _) = mpsc::channel(); - let (tx_r, _) = mpsc::channel(); - let (tx_req, _) = mpsc::channel(); - let space = - wrap_stage1r(reload(doc), tx_eq, tx_ex, tx_r, tx_req, false, true, None).param_space(); - if !space.is_empty() { - eprintln!( - "aura: mc requires a closed blueprint (no free parameters); {} free knob(s) — \ - bind them or use `aura sweep --axis`", - space.len() - ); - std::process::exit(2); - } - let point: Vec = Vec::new(); // closed -> empty point (as `aura run `) - let draws: Vec = (1..=n_seeds) - .map(|seed| { - let sources = synthetic_walk_sources(seed); - let window = window_of(&sources).expect("non-empty synthetic walk"); - let report = - run_blueprint_member(reload(doc), &point, &space, sources, window, seed, pip, &topo); - McDraw { seed, report } - }) - .collect(); - let aggregate = McAggregate::from_draws(&draws); - McFamily { draws, aggregate } -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cargo test -p aura-cli --bin aura blueprint_mc_family_seeds_differ` -Expected: PASS. (If the three draws come back identical, the walk did not warm the -stop — increase `SyntheticSpec.len`; 60 should suffice for slow=4 + stop-len 3.) - ---- - -### Task 3: `reproduce_family_in` MonteCarlo realization branch - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (`reproduce_family_in` ~2503-2574; the run at ~2561-2570) -- Test: `crates/aura-cli/src/main.rs` (beside `reproduce_family_re_derives_every_member_bit_identically` ~5194) - -- [ ] **Step 1: Write the failing test** - -```rust -#[test] -fn reproduce_family_re_derives_every_mc_member_bit_identically() { - let dir = std::env::temp_dir().join(format!("aura-repro-mc-{}", std::process::id())); - let _ = std::fs::remove_dir_all(&dir); - std::fs::create_dir_all(&dir).expect("temp dir"); - let reg = Registry::open(dir.join("runs.jsonl")); - - // a CLOSED signal (both SMA knobs bound) — MC binds no axis. - let closed = stage1_signal(Some(2), Some(4)); - let doc = blueprint_to_json(&closed).expect("serializes"); - let data = DataSource::Synthetic; - let family = blueprint_mc_family(&doc, 3, &data); - - // persist exactly as run_blueprint_mc does: store the blueprint, append the MC family. - let topo = family.draws[0].report.manifest.topology_hash.clone().expect("topo"); - let canonical = - blueprint_to_json(&blueprint_from_json(&doc, &|t| std_vocabulary(t)).unwrap()).unwrap(); - reg.put_blueprint(&topo, &canonical).expect("store blueprint"); - let id = reg - .append_family("mcrepro", FamilyKind::MonteCarlo, &mc_member_reports(&family)) - .expect("append"); - - // reproduce: every MC member re-derives bit-identically (its seed-driven walk is - // reconstructed from manifest.seed — the realization branch). - let rep = reproduce_family_in(®, &id, &data); - assert_eq!(rep.outcomes.len(), 3, "three MC members reproduced"); - assert!( - rep.outcomes.iter().all(|(_, ok)| *ok), - "every MC member re-derives bit-identically: {:?}", - rep.outcomes - ); - let _ = std::fs::remove_dir_all(&dir); -} -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `cargo test -p aura-cli --bin aura reproduce_family_re_derives_every_mc_member_bit_identically` -Expected: FAIL — the assertion `every MC member re-derives bit-identically` fails: without -the branch, reproduce re-runs MC members over `data.run_sources()` (the showcase series), -not the seed-driven walk, so `rerun.metrics != stored.metrics` for every member. - -- [ ] **Step 3: Write minimal implementation** - -In `reproduce_family_in`, replace the single member run (currently ~2561-2570): - -```rust - let rerun = run_blueprint_member( - reload(), - &point, - &space, - data.run_sources(), - window, - stored.manifest.seed, - pip, - &hash, - ); -``` - -with a realization-aware version that reconstructs the member's data from the facet its -family kind recorded: - -```rust - // Realization-aware: a MonteCarlo member ran over a seed-driven synthetic walk, - // not the showcase — reconstruct it from manifest.seed so the re-run matches (C1). - let (sources, member_window, seed) = match family.kind { - FamilyKind::MonteCarlo => { - let s = synthetic_walk_sources(stored.manifest.seed); - let w = window_of(&s).expect("non-empty synthetic walk"); - (s, w, stored.manifest.seed) - } - _ => (data.run_sources(), window, stored.manifest.seed), - }; - let rerun = run_blueprint_member( - reload(), - &point, - &space, - sources, - member_window, - seed, - pip, - &hash, - ); -``` - -(`window` is the pre-loop `data.full_window()` at ~2515, used unchanged for the `_` arm; -`family.kind` is on the grouped `Family` the loop iterates.) - -- [ ] **Step 4: Run test to verify it passes** - -Run: `cargo test -p aura-cli --bin aura reproduce_family_re_derives_every_mc_member_bit_identically` -Expected: PASS. - -- [ ] **Step 5: Verify the Sweep reproduce path stays green** - -Run: `cargo test -p aura-cli --bin aura reproduce_family_re_derives_every_member_bit_identically` -Expected: PASS (the `_` arm is byte-unchanged for Sweep members). - ---- - -### Task 4: `run_blueprint_mc` + the `["mc", ..]` dispatch arm + E2E - -**Files:** -- Modify: `crates/aura-cli/src/main.rs` (`run_blueprint_mc` beside `run_blueprint_sweep` ~3210; the - `["mc", rest @ ..]` dispatch arm ~4006-4015) -- Test: `crates/aura-cli/tests/cli_run.rs` (beside `aura_reproduce_re_derives_a_persisted_sweep_bit_identically` ~3456) - -- [ ] **Step 1: Write the failing E2E tests** - -Add to `crates/aura-cli/tests/cli_run.rs`: - -```rust -/// E2E (acc 1+3+4): `aura mc --seeds N` builds a MonteCarlo -/// family (one member per seed), writes exactly one content-addressed blueprint, and -/// `aura reproduce ` re-derives every member bit-identically (exit 0). -#[test] -fn aura_mc_over_a_blueprint_reproduces_bit_identically() { - let cwd = temp_cwd("mc-blueprint-reproduce"); - let fixture = format!("{}/tests/fixtures/stage1_signal.json", env!("CARGO_MANIFEST_DIR")); - - let mc = Command::new(BIN) - .args(["mc", &fixture, "--seeds", "4", "--name", "mcx"]) - .current_dir(&cwd) - .output() - .expect("spawn aura mc blueprint"); - assert!(mc.status.success(), "mc exit: {:?} stderr={}", mc.status, String::from_utf8_lossy(&mc.stderr)); - let mc_out = String::from_utf8(mc.stdout).expect("utf-8 stdout"); - // one member line per seed (each carries "seed":n) + one aggregate line. - assert_eq!( - mc_out.lines().filter(|l| l.contains("\"seed\":")).count(), - 4, - "one member line per seed: {mc_out}" - ); - assert!(mc_out.contains("mc_aggregate"), "prints the aggregate line: {mc_out}"); - - // exactly one blueprint stored for the whole family (one topology per family). - let store = cwd.join("runs/blueprints"); - let entries: Vec<_> = std::fs::read_dir(&store) - .unwrap_or_else(|e| panic!("mc must create the blueprint store {store:?}: {e}")) - .map(|e| e.expect("dir entry").path()) - .collect(); - assert_eq!(entries.len(), 1, "one stored topology per family: {entries:?}"); - - let repro = Command::new(BIN) - .args(["reproduce", "mcx-0"]) - .current_dir(&cwd) - .output() - .expect("spawn aura reproduce"); - assert!(repro.status.success(), "reproduce exit: {:?} stderr={}", repro.status, String::from_utf8_lossy(&repro.stderr)); - let repro_out = String::from_utf8(repro.stdout).expect("utf-8 stdout"); - assert!( - repro_out.contains("reproduced 4/4 members bit-identically"), - "every MC member re-derives: {repro_out}" - ); - let _ = std::fs::remove_dir_all(&cwd); -} - -/// E2E (acc 2): `aura mc ` is rejected with a named error + exit 2 -/// (NOT a panic / exit 101) — MC binds no axis, so a free knob has no binder. -#[test] -fn aura_mc_rejects_an_open_blueprint() { - let cwd = temp_cwd("mc-blueprint-open-reject"); - let fixture = format!("{}/tests/fixtures/stage1_signal_open.json", env!("CARGO_MANIFEST_DIR")); - let out = Command::new(BIN) - .args(["mc", &fixture, "--seeds", "4"]) - .current_dir(&cwd) - .output() - .expect("spawn aura mc open-blueprint"); - assert_eq!(out.status.code(), Some(2), "an open blueprint fails clean (exit 2, not a panic): stderr={}", String::from_utf8_lossy(&out.stderr)); - let stderr = String::from_utf8(out.stderr).expect("utf-8"); - assert!(stderr.contains("closed blueprint"), "names the closed-blueprint requirement: {stderr}"); - let _ = std::fs::remove_dir_all(&cwd); -} -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `cargo test -p aura-cli --test cli_run aura_mc_over_a_blueprint_reproduces_bit_identically` -Expected: FAIL — `aura mc .json` currently falls into `parse_mc_args`, which does -not accept a `.json` first positional, so the command errors (no `.json` discriminator yet). - -- [ ] **Step 3: Write `run_blueprint_mc`** - -Add beside `run_blueprint_sweep` (~3210): - -```rust -/// `aura mc --seeds N`: build a Monte-Carlo family from a loaded CLOSED -/// blueprint (the World/C21 verb), store the canonical blueprint ONCE keyed by the shared -/// `topology_hash` (the 0094 hook, so `aura reproduce` re-derives it), record it as a -/// `FamilyKind::MonteCarlo` family (C18/C21 lineage), and print each draw's member line -/// (carrying the seed) plus the aggregate — mirroring `run_mc` / `run_blueprint_sweep`. -fn run_blueprint_mc(doc: &str, n_seeds: u64, name: &str, data: DataSource) { - let family = blueprint_mc_family(doc, n_seeds, &data); - let reg = default_registry(); - // Store the canonical blueprint ONCE, keyed by the family's shared topology_hash. - let topo = family.draws[0] - .report - .manifest - .topology_hash - .clone() - .expect("a blueprint mc stamps every member's topology_hash"); - let canonical = blueprint_to_json( - &blueprint_from_json(doc, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary"), - ) - .expect("a loaded blueprint re-serializes"); - reg.put_blueprint(&topo, &canonical).unwrap_or_else(|e| { - eprintln!("aura: {e}"); - std::process::exit(2); - }); - let id = match reg.append_family(name, FamilyKind::MonteCarlo, &mc_member_reports(&family)) { - Ok(id) => id, - Err(e) => { - eprintln!("aura: {e}"); - std::process::exit(2); - } - }; - for draw in &family.draws { - println!("{}", mc_member_line(&id, draw.seed, &draw.report)); - } - println!("{}", mc_aggregate_json(&family.aggregate)); -} -``` - -- [ ] **Step 4: Grow the `["mc", ..]` dispatch arm** - -Replace the current arm (~4006-4015): - -```rust - ["mc", rest @ ..] => match parse_mc_args(rest) { - Ok(McArgs::Synthetic { name, persist }) => run_mc(&name, persist), - Ok(McArgs::RealR { choice, grid, block_len, n_resamples, seed }) => { - run_mc_r_bootstrap(DataSource::from_choice(choice), &grid, block_len, n_resamples, seed) - } - Err(msg) => { - eprintln!("aura: {msg}"); - std::process::exit(2); - } - }, -``` - -with the `.json`-discriminator arm (mirroring the `["sweep", ..]` arm ~3952): - -```rust - ["mc", rest @ ..] => { - // A `.json`-file first argument selects the loaded-blueprint Monte-Carlo (the - // World/C21 verb), mirroring the `["sweep", ..]` discriminator; a bare - // `--strategy`/`--name` token never ends in `.json`, so the paths stay disjoint. - if let Some(path) = - rest.first().filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) - { - let doc = std::fs::read_to_string(path).unwrap_or_else(|e| { - eprintln!("aura: {path}: {e}"); - std::process::exit(2); - }); - if let Err(e) = blueprint_from_json(&doc, &|t| std_vocabulary(t)) { - eprintln!("aura: {path}: {e:?}"); - std::process::exit(2); - } - let McBlueprintArgs { n_seeds, name } = - parse_mc_blueprint_args(&rest[1..]).unwrap_or_else(|msg| { - eprintln!("aura: {msg}"); - std::process::exit(2); - }); - run_blueprint_mc(&doc, n_seeds, &name, DataSource::Synthetic); - return; - } - match parse_mc_args(rest) { - Ok(McArgs::Synthetic { name, persist }) => run_mc(&name, persist), - Ok(McArgs::RealR { choice, grid, block_len, n_resamples, seed }) => { - run_mc_r_bootstrap(DataSource::from_choice(choice), &grid, block_len, n_resamples, seed) - } - Err(msg) => { - eprintln!("aura: {msg}"); - std::process::exit(2); - } - } - } -``` - -- [ ] **Step 5: Run the E2E tests to verify they pass** - -Run: `cargo test -p aura-cli --test cli_run aura_mc_over_a_blueprint_reproduces_bit_identically` -Expected: PASS. - -Run: `cargo test -p aura-cli --test cli_run aura_mc_rejects_an_open_blueprint` -Expected: PASS. - -- [ ] **Step 6: Verify the hard-wired `aura mc` path is undisturbed** - -Run: `cargo test -p aura-cli --test cli_run mc` -Expected: PASS (the hard-wired-`mc` E2Es still green — the `.json` discriminator keeps the -non-`.json` path falling through to `parse_mc_args`). - ---- - -### Task 5: Full-workspace gate - -**Files:** none (verification only) - -- [ ] **Step 1: Build the workspace** - -Run: `cargo build --workspace` -Expected: builds clean, 0 errors. - -- [ ] **Step 2: Run the full test suite** - -Run: `cargo test --workspace` -Expected: all suites pass (the 3 new units + 2 new E2Es green; no regressions). - -- [ ] **Step 3: Lint** - -Run: `cargo clippy --workspace --all-targets -- -D warnings` -Expected: 0 warnings. diff --git a/docs/specs/0095-mc-over-a-loaded-blueprint.md b/docs/specs/0095-mc-over-a-loaded-blueprint.md deleted file mode 100644 index 85d6e26..0000000 --- a/docs/specs/0095-mc-over-a-loaded-blueprint.md +++ /dev/null @@ -1,305 +0,0 @@ -# Monte-Carlo over a Loaded Blueprint — Design Spec - -**Date:** 2026-07-01 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude - -**Reference issue:** #170 (World/C21 milestone). This cycle ships the **Monte-Carlo -half** of #170; the walk-forward half is deferred (see Scope below). Design -decisions, the grounding correction, and the two-gate narrowing are recorded on -#170. Follow-on tracks: #172 (real-data block-bootstrap MC, deps #124) and #173 -(walk-forward over a loaded blueprint, deps #169). - -## Goal - -Add a `.json`-discriminator dispatch arm — `aura mc --seeds N` — -that builds a **Monte-Carlo family** from a loaded (serialized) **closed** blueprint, -mirroring cycle 0093's `aura sweep `. It runs the fixed blueprint -across N seeds, each seed drawing a distinct synthetic price walk (`SyntheticSpec:: -source(seed)`, the `mc_family` pattern), persists the family under -`FamilyKind::MonteCarlo` with the members' shared `topology_hash`, and writes the -canonical blueprint once to the content-addressed store — so a persisted -`aura mc` family reproduces bit-identically via `aura reproduce` (the concrete -proof the store hook fired). - -## Scope — MC only this cycle; walk-forward deferred - -#170 asked for both `mc` and `walkforward` over a loaded blueprint. Two independent -review gates on the draft narrowed this cycle to **MC only**: - -- **Verb-name collision (walk-forward).** The hard-wired `aura walkforward` does a - true per-window **in-sample re-optimization** (`sweep_over(is) → select_winner → - run_oos`, main.rs:2010-2016). A loaded blueprint declares no sweepable axes yet - (that is #169), so a loaded-blueprint `walkforward` could only be a *fixed-blueprint - windowed out-of-sample evaluation* — no IS refit. Shipping that under the identical - verb name (split only by a `.json` suffix) would let one trading term name two - categorically different guarantees. The honest walk-forward over a loaded blueprint - is the IS-refit form, which needs axis-discovery (#169) first; it is tracked by - #173, not forced into this cycle under a colliding name. -- The MC half carries **no** such collision: `aura mc` over a fixed closed blueprint, - varying the data realization by seed, is a Monte-Carlo in the plain sense. - -## Architecture - -### The grounding correction (foundation) - -`run_blueprint_member(signal, point, space, sources, window, seed, pip, topo)` -(main.rs:3109) runs the member over the **`sources` argument as given** and uses -`seed` / `window` only to stamp the manifest (`sim_optimal_manifest`). The -stage-1-R graph is deterministic (C1). Therefore Monte-Carlo variation cannot come -from the `seed` label alone — it must come from **re-drawing the input**, exactly -as the hard-wired `mc_family` does (`SyntheticSpec::source(seed)` → a distinct price -walk per seed, main.rs:2212). `aura mc` over a loaded blueprint therefore supplies -**seed-derived synthetic sources** to `run_blueprint_member` per member: - -| verb | facet | per-member sources | window | seed | -|------|-------|--------------------|--------|------| -| sweep (0093) | param point | showcase (full), fixed | full | 0 | -| **mc** (new) | seed | `SyntheticSpec::source(seed)` walk | that walk's window | the seed | - -`run_blueprint_member`'s signature is **unchanged** — MC only passes a different -`(sources, window, seed)` triple than the sweep's `(showcase, full, 0)`. The shared -reduce-mode member path makes reproduction bit-identical by construction (C1) — the -0094 keystone. - -### Closed blueprint only (grounded constraint) - -MC declares **no** parameter axis (only the seed varies), so it runs the blueprint at -its **fixed, fully-bound** configuration — a **closed** blueprint (an empty wrapped -`param_space`). This is exactly the case `aura run ` already supports: -it passes empty params and bootstraps only when the blueprint has zero free knobs (the -closed `stage1_signal.json` fixture). Grounding confirmed there is no per-knob -"declared default" mechanism — `ParamSpec { name, kind }` carries no default and -`compile_with_params` requires `params.len() == space.len()`. So MC uses the **empty -point** (`&[]`), never a synthesized default point. - -An **open** blueprint (a non-empty wrapped `param_space` — free knobs) is **rejected** -with a named error + exit 2 *before* any run: MC has no axis to bind those knobs. This -also fixes the boundary honestly — the raw `bootstrap`/`compile_with_params` path -*panics* on an arity mismatch (main.rs:3084), so MC guards against it up front rather -than letting a panic escape. (To vary a knob, the user reaches for `aura sweep - --axis`; to Monte-Carlo it, the blueprint must be closed.) - -### The persist seam - -`run_blueprint_mc` replicates the two things the 0093 sweep persist seam does -(`run_blueprint_sweep`, main.rs:3210): - -1. `reg.put_blueprint(&topo, &canonical)` — the content-addressed store write (the - 0094 hook), keyed by the family's shared `topology_hash`, canonical bytes - `blueprint_to_json(blueprint_from_json(doc))`. **Load-bearing:** without it, - `aura reproduce` reports the family's members as "blueprint `` missing from - store" (exit 2). (#170 reconciliation comment.) -2. `reg.append_family(name, FamilyKind::MonteCarlo, &member_reports)`. - -Then print each member line (`mc_member_line`, carrying the seed) plus the aggregate -line — matching the hard-wired `run_mc` output shape. - -### Reproduce becomes realization-aware (the one change to 0094 code) - -`reproduce_family_in` (main.rs:2503) currently reconstructs every member over -`data.run_sources()` (showcase) at `data.full_window()` with the stored seed — -correct for `Sweep`, wrong for MC (whose data is seed-derived). It gains a per-member -branch on the **family kind** that computes the member's `(sources, window, seed)`: - -- `MonteCarlo` → `synthetic_walk_sources(manifest.seed)`, that walk's window, the seed. -- everything else (`Sweep`, `CrossInstrument`) → `data.run_sources()`, - `data.full_window()`, `manifest.seed` (unchanged; keeps 0094 green). - -The grouped `Family` struct exposes `.kind: FamilyKind` (aura-registry lineage.rs:56); -the manifest stores `seed` recoverably (compat.rs:30); `synthetic_walk_sources` is a -fixed constant shared by the persist and reproduce paths, so the seed→walk -reconstruction is bit-exact (C1). - -## Concrete code shapes - -### User-facing program (the acceptance evidence) - -Monte-Carlo over a loaded closed blueprint, then prove it reproduces: - -```console -$ aura mc crates/aura-cli/tests/fixtures/stage1_signal.json --seeds 8 --name mc1 -mc1 {"family_id":"mc1-0","seed":1,"report":{...topology_hash:""...}} -mc1 {"family_id":"mc1-0","seed":2,"report":{...topology_hash:""...}} -... (8 members, one per seed; same topology_hash , differing metrics) -{"mc_aggregate":{...}} - -$ aura reproduce mc1-0 -mc1-0 member reproduced: bit-identical -... (8 members) -reproduced 8/8 members bit-identically # exit 0 -``` - -Rejecting an open blueprint (free knobs) — a named error, not a panic: - -```console -$ aura mc crates/aura-cli/tests/fixtures/stage1_signal_open.json --seeds 4 -aura: mc requires a closed blueprint (no free parameters); stage1_signal_open.json - has 2 free knobs — bind them or use `aura sweep --axis` # exit 2 -``` - -All members carry the shared `topology_hash `; a single `runs/blueprints/.json` -is written per family (one blueprint per family, C11/C12). The synthetic-walk -realization is the deliverable's *machinery*, not a trader-grade Monte-Carlo — a -real-data block-bootstrap MC is #172. - -### Dispatch arm (before → after) - -Before (main.rs:4006) — the hard-wired verb only: - -```rust -["mc", rest @ ..] => match parse_mc_args(rest) { /* Synthetic | RealR */ }, -``` - -After — a `.json`-file first positional selects the loaded-blueprint path (mirror of -the 0093 `["sweep", rest @ ..]` arm at 3952), else fall through unchanged: - -```rust -["mc", rest @ ..] => { - if let Some(path) = rest.first() - .filter(|a| a.ends_with(".json") && std::path::Path::new(a).is_file()) - { - let doc = read_to_string(path)/* exit 2 on io err */; - blueprint_from_json(&doc, &|t| std_vocabulary(t))/* exit 2 on parse err, file-path ctx */; - let McBlueprintArgs { n_seeds, name, data } = - parse_mc_blueprint_args(&rest[1..])/* exit 2 on usage */; - run_blueprint_mc(&doc, n_seeds, &name, DataSource::from_choice(data)); - return; - } - match parse_mc_args(rest) { /* unchanged */ } -} -``` - -### Family builder + closed-blueprint guard - -```rust -fn blueprint_mc_family(doc: &str, n_seeds: u64, data: &DataSource) -> SweepFamily { - let reload = |d: &str| blueprint_from_json(d, &|t| std_vocabulary(t)) - .expect("doc parse-validated at the dispatch boundary"); - let topo = topology_hash(&reload(doc)); - let pip = data.pip_size(); - let space = /* wrapped param_space probe, as blueprint_sweep_family does (main.rs:3177) */; - // closed-blueprint guard: MC binds no axis, so a free knob has no binder. - if !space.is_empty() { - eprintln!("aura: mc requires a closed blueprint (no free parameters); \ - {n} free knob(s) — bind them or use `aura sweep --axis`", n = space.len()); - std::process::exit(2); - } - let point: Vec = Vec::new(); // closed → empty point (as `aura run`) - let points = (1..=n_seeds).map(|seed| { - let sources = synthetic_walk_sources(seed); // fixed SyntheticSpec, seeded - let window = window_of(&sources).expect("non-empty synthetic walk"); - run_blueprint_member(reload(doc), &point, &space, sources, window, seed, pip, &topo) - }).collect(); - SweepFamily { points /* SweepPoint { report } */ } -} -``` - -### Reproduce realization branch (before → after) - -Before (main.rs:2514-2570) — one path for all members: - -```rust -let window = data.full_window(); -for member in &family.members { - // ... run_blueprint_member(reload(), &point, &space, data.run_sources(), - // window, stored.manifest.seed, pip, &hash) -} -``` - -After — the `(sources, window, seed)` triple is chosen by the family kind: - -```rust -for member in &family.members { - let (sources, window, seed) = match family.kind { - FamilyKind::MonteCarlo => { - let s = synthetic_walk_sources(stored.manifest.seed); - let w = window_of(&s).expect("non-empty walk"); - (s, w, stored.manifest.seed) - } - _ /* Sweep, CrossInstrument, (WalkForward → #173) */ => - (data.run_sources(), data.full_window(), stored.manifest.seed), - }; - let rerun = run_blueprint_member(reload(), &point, &space, sources, window, seed, pip, &hash); - outcomes.push((label, rerun.metrics == stored.metrics)); -} -``` - -## Components - -All in aura-cli `main.rs` (invariant 8: the frozen engine is untouched; invariant 9: -the registry stays a dumb bytes/records store): - -- `parse_mc_blueprint_args(&[&str]) -> Result` — pure arg - parser (mirror `parse_sweep_blueprint_args`): `--seeds ` (required, ≥1), - `--name ` (default `"mc"`); synthetic-only (real data is #172). -- `blueprint_mc_family` — the builder above, with the closed-blueprint guard. -- `run_blueprint_mc(doc, n_seeds, name, data)` — the persist seam (put_blueprint + - append_family(MonteCarlo) + print member/aggregate lines), mirroring - `run_blueprint_sweep` / `run_mc`. -- `synthetic_walk_sources(seed) -> Vec>` — the fixed seeded - `SyntheticSpec` walk (sized to warm the loaded graph — showcase-length regime), - shared by the persist and reproduce paths. -- `reproduce_family_in` — gains the `MonteCarlo` realization branch. -- one new dispatch arm. - -## Data flow - -`aura mc --seeds N`: read + parse-validate doc → `blueprint_mc_family` -(closed-guard; N members, each `run_blueprint_member` over its seed's synthetic walk) -→ `put_blueprint(topo, canonical)` → `append_family(name, MonteCarlo, reports)` → -print member lines + aggregate. - -`aura reproduce ` (MC family): load families → find `id` → for each member, -reconstruct `synthetic_walk_sources(manifest.seed)` + its window → `run_blueprint_member` -→ compare `metrics` → print bit-identical/DIVERGED → exit 0 (all identical) / 1 (any -diverged). - -## Error handling - -Mirrors the 0093/0094 boundary discipline (stderr `aura: ` + exit 2, never a -panic): - -- io error reading the `.json` file → `aura: : `, exit 2. -- blueprint parse error → `aura: : `, exit 2 (validated once at the boundary, - so per-member reloads are infallible). -- **open blueprint (non-empty wrapped `param_space`)** → the closed-blueprint guard's - named error, exit 2 (before any run — pre-empts the arity panic at main.rs:3084). -- arg usage error (missing/`0` `--seeds`, unknown flag) → the parser's usage string, exit 2. -- `aura reproduce` on an MC family: unknown id / member without `topology_hash` / - blueprint missing from store → exit 2 (unchanged 0094 behaviour); any member - diverges → named `DIVERGED` + exit 1. - -## Testing strategy - -- **Unit (`cargo test -p aura-cli --bin aura`)**: - - `blueprint_mc_family` over the **closed** fixture with `--seeds 3`: 3 members, all - sharing one `topology_hash`, with **differing** metrics — proves the seed reaches - the data (the anti-degenerate guard; a regression to seed-as-label-only would make - the members identical and fail this test). - - `parse_mc_blueprint_args`: `--seeds` required/parsed/`≥1`, default name, - unknown-flag rejection. -- **E2E (`cargo test -p aura-cli --test cli_run`)** driving the built binary: - - `aura mc --seeds 4 --name mcX` then `aura reproduce mcX-0` → - asserts `reproduced 4/4 members bit-identically` + exit 0, and exactly one - `runs/blueprints/.json`. **This is acceptance criterion (d)** — the - store-hook proof. - - `aura mc --seeds 4` → asserts the closed-blueprint named error on - stderr + exit 2 (NOT a panic / exit 101) — the boundary guard. -- No new registry code (the `append_family` MonteCarlo round-trip is already green). - -## Acceptance criteria - -1. `aura mc --seeds N` builds an N-member - `FamilyKind::MonteCarlo` family from the loaded blueprint, members sharing one - `topology_hash` and **differing** in metrics (seed → synthetic data), and writes - one stored blueprint. -2. An **open** blueprint is rejected with a named `aura:` error + exit 2 (never a - panic); the hard-wired `aura mc` path stays unchanged (the `.json` discriminator - keeps them disjoint). -3. **Reproduce compatibility (the store-hook proof):** a persisted `aura mc` family - `aura reproduce`s **bit-identically** (exit 0); the 0094 `Sweep` reproduce path - stays green. -4. Engine untouched (invariant 8); the registry gains no domain logic (invariant 9); - `cargo clippy --workspace --all-targets -- -D warnings` clean; determinism (C1) - holds — reproduction is bit-exact via the shared `run_blueprint_member` path.