# Monte-Carlo Family — Implementation Plan

> **Parent spec:** `docs/specs/0043-monte-carlo-family.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
> this plan. Steps use `- [ ]` checkboxes for tracking.

**Goal:** Add `monte_carlo` — the Monte-Carlo orchestration family (C12 axis 4),
an `McFamily` over a seed set analog to `SweepFamily` — reusing the existing
disjoint-parallel executor (extracted into a shared `run_indexed<T>` core) rather
than forking a new run loop.

**Architecture:** Task 1 extracts the index-parallel loop out of
`sweep_with_threads` into a `pub(crate) run_indexed<T>` in `sweep.rs` and rewires
`sweep_with_threads` as a thin adapter — a behaviour-preserving refactor whose
green guard is the three pre-existing `sweep` tests. Task 2 adds the `mc.rs`
module (the five MC types, the private type-7 `quantile`, `McAggregate::from_draws`,
and `monte_carlo`/`monte_carlo_with_threads` driving `run_indexed`). Task 3 wires
the module + public exports into `lib.rs`.

**Tech Stack:** `crates/aura-engine` — `sweep.rs` (shared executor),
`mc.rs` (new module), `lib.rs` (module decl + exports), reusing `report.rs`
(`RunReport`/`RunMetrics`) and the `pub(crate)` test fixtures in
`test_fixtures.rs`.

---

**Files this plan creates or modifies:**

- Modify: `crates/aura-engine/src/sweep.rs:131-167` — extract `run_indexed<T>`; rewire `sweep_with_threads` as adapter
- Create: `crates/aura-engine/src/mc.rs` — MC family types, `monte_carlo`, `quantile`, `from_draws`, `#[cfg(test)] mod tests`
- Modify: `crates/aura-engine/src/lib.rs:36-53` — add `mod mc;` and the five public exports
- Test: `crates/aura-engine/src/mc.rs` (`#[cfg(test)] mod tests`) — the 7 RED tests per spec §Testing 1–7
- Test (guard, unchanged): `crates/aura-engine/src/sweep.rs:292,314,325` — the three pre-existing sweep tests must stay green

---

## Task 1: Extract the shared `run_indexed<T>` disjoint-parallel core

**Files:**
- Modify: `crates/aura-engine/src/sweep.rs:131-167`

This is a **behaviour-preserving** refactor: the existing `sweep.rs` tests
(`family_is_deterministic_across_thread_counts`, `sweep_equals_n_independent_runs`,
`distinct_points_produce_distinct_metrics`) are the green guard — they must keep
passing with no edit. No new test is added in this task; the guard tests already
exist and exercise the new code path through the rewired `sweep_with_threads`.

- [ ] **Step 1: Run the guard tests first to confirm the green baseline**

Run: `cargo test -p aura-engine --lib sweep_equals_n_independent_runs`
Expected: PASS (`test result: ok. 1 passed`)

- [ ] **Step 2: Add `run_indexed<T>` and rewire `sweep_with_threads`**

In `crates/aura-engine/src/sweep.rs`, replace the entire `sweep_with_threads`
function body (lines 131–167, the function whose signature is
`fn sweep_with_threads<F>(space: &GridSpace, nthreads: usize, run_one: F) -> SweepFamily`)
with the two functions below. `run_indexed` is the generalized core (the old loop,
generic over the per-job result `T`); `sweep_with_threads` becomes a thin adapter.
The `use std::sync::atomic::{AtomicUsize, Ordering};` import already present at
`sweep.rs:9` is reused unchanged.

```rust
/// Run `n` disjoint jobs in parallel and collect their results in **job-index
/// order** — the shared disjoint-parallel core both `sweep` (over grid points)
/// and `monte_carlo` (over seeds) drive (C1: order is the input order, not the
/// completion order). Workers pull job indices from a shared atomic cursor
/// (work-stealing load-balances uneven per-job cost); each tags its result with
/// the index, and the results are sorted on that index after the scope joins.
/// Only the cursor is shared; the results side is lock-free. `nthreads` is
/// clamped to `[1, n.max(1)]` (a 0-job call yields an empty vec; a 0 thread
/// count coerces to 1).
pub(crate) fn run_indexed<T, F>(n: usize, nthreads: usize, run_one: F) -> Vec<T>
where
    T: Send,
    F: Fn(usize) -> T + Sync,
{
    let nthreads = nthreads.clamp(1, n.max(1));
    let cursor = AtomicUsize::new(0);

    let mut results: Vec<(usize, T)> = std::thread::scope(|scope| {
        let handles: Vec<_> = (0..nthreads)
            .map(|_| {
                scope.spawn(|| {
                    let mut local: Vec<(usize, T)> = Vec::new();
                    loop {
                        let i = cursor.fetch_add(1, Ordering::Relaxed);
                        if i >= n {
                            break;
                        }
                        local.push((i, run_one(i)));
                    }
                    local
                })
            })
            .collect();
        handles.into_iter().flat_map(|h| h.join().unwrap()).collect()
    });

    results.sort_by_key(|&(i, _)| i);
    results.into_iter().map(|(_, t)| t).collect()
}

/// The thread-count-explicit core of [`sweep`]. Module-private: the public
/// `sweep` derives the count, while the tests drive it at 1 and at N to pin
/// determinism under concurrency (C1). A thin adapter over [`run_indexed`]: it
/// enumerates the grid points, runs each disjointly, and zips the reports back
/// onto their points in enumeration (odometer) order.
fn sweep_with_threads<F>(space: &GridSpace, nthreads: usize, run_one: F) -> SweepFamily
where
    F: Fn(&[Scalar]) -> RunReport + Sync,
{
    let points = space.points();
    let reports = run_indexed(points.len(), nthreads, |i| run_one(&points[i]));
    SweepFamily {
        points: points
            .into_iter()
            .zip(reports)
            .map(|(params, report)| SweepPoint { params, report })
            .collect(),
    }
}
```

- [ ] **Step 3: Build the crate**

Run: `cargo build -p aura-engine`
Expected: builds with 0 errors (the `run_indexed` extraction compiles; no caller
signature changed).

- [ ] **Step 4: Run the refactor-guard tests to verify behaviour preserved**

Run: `cargo test -p aura-engine --lib sweep`
Expected: PASS — all `sweep::tests` green, including
`family_is_deterministic_across_thread_counts`, `sweep_equals_n_independent_runs`,
`distinct_points_produce_distinct_metrics` (the proof the refactor preserved
behaviour, C1/C11/C23). `test result: ok` with 0 failed.

---

## Task 2: The `mc.rs` module — types, quantile, aggregate, `monte_carlo`

**Files:**
- Create: `crates/aura-engine/src/mc.rs`

The module is not yet wired into `lib.rs` (Task 3 does that), so its tests will
not run until Task 3. This task creates the full module incl. its `#[cfg(test)]
mod tests`; the RED→GREEN cycle is observed end-to-end in Task 3 Step 2/3 once
`mod mc;` is declared. The implementation is written here in full (no
placeholder), with the failing-first ordering preserved by Task 3's gate.

- [ ] **Step 1: Create `crates/aura-engine/src/mc.rs` with the module + types + impl**

Create `crates/aura-engine/src/mc.rs` with exactly:

```rust
//! Monte-Carlo orchestration family (C12 axis 4): Monte-Carlo as a sweep over
//! seeds. `monte_carlo(base_point, seeds, run_one)` runs a fixed base point over
//! a seed set — each seed a disjoint C1 realization — and collects an
//! [`McFamily`]: the per-seed [`McDraw`]s in seed-input order plus a stored
//! [`McAggregate`] (mean + quantiles of all three run metrics across the
//! realizations). It reuses the disjoint-parallel [`run_indexed`](crate::sweep)
//! core `sweep` drives — the varying dimension is the *seed*, not a tuning param.
//! Eager-agnostic (C12/#71): the API takes seeds + a per-draw closure, never a
//! materialized stream `Vec`; the seed -> `Source` construction is a closure-body
//! concern.

use crate::sweep::run_indexed;
use crate::{RunMetrics, RunReport, Scalar};

/// One Monte-Carlo realization: the seed that drove it and the full `RunReport`.
/// Self-describing, analog to [`SweepPoint`](crate::SweepPoint).
#[derive(Clone, Debug, PartialEq)]
pub struct McDraw {
    pub seed: u64,
    pub report: RunReport,
}

/// The result family of a Monte-Carlo run — one [`McDraw`] per seed, in
/// seed-**input** order (independent of thread completion), plus the stored
/// [`McAggregate`]. Analog to [`SweepFamily`](crate::SweepFamily).
#[derive(Clone, Debug, PartialEq)]
pub struct McFamily {
    pub draws: Vec<McDraw>,
    pub aggregate: McAggregate,
}

/// Distribution summary of all three run metrics across the realizations: covers
/// every metric (not a single "chosen" one). A pure post-run reduction over the
/// draws — stored for the common robustness case; custom statistics read the raw
/// draws directly.
#[derive(Clone, Debug, PartialEq)]
pub struct McAggregate {
    pub total_pips: MetricStats,
    pub max_drawdown: MetricStats,
    pub exposure_sign_flips: MetricStats,
}

/// Mean + a fixed quantile set of one metric across the realizations. `p50` is
/// the median (the two coincide by definition), so "mean/median/quantiles" is
/// `mean` + `p50` + the surrounding quantiles, with no redundant field.
#[derive(Clone, Debug, PartialEq)]
pub struct MetricStats {
    pub mean: f64,
    pub p5: f64,
    pub p25: f64,
    pub p50: f64, // == median
    pub p75: f64,
    pub p95: f64,
}

impl McAggregate {
    /// Pure reduction over the realizations — recomputable from `draws` alone (it
    /// is exactly what [`monte_carlo`] stored). `draws` must be non-empty (a
    /// Monte-Carlo over zero realizations has no defined mean/quantile); on a
    /// non-empty slice every field is finite.
    pub fn from_draws(draws: &[McDraw]) -> McAggregate {
        let pick = |f: fn(&RunMetrics) -> f64| -> MetricStats {
            let mut xs: Vec<f64> = draws.iter().map(|d| f(&d.report.metrics)).collect();
            xs.sort_by(|a, b| a.partial_cmp(b).expect("run metrics are finite"));
            MetricStats {
                mean: xs.iter().sum::<f64>() / xs.len() as f64,
                p5: quantile(&xs, 0.05),
                p25: quantile(&xs, 0.25),
                p50: quantile(&xs, 0.50),
                p75: quantile(&xs, 0.75),
                p95: quantile(&xs, 0.95),
            }
        };
        McAggregate {
            total_pips: pick(|m| m.total_pips),
            max_drawdown: pick(|m| m.max_drawdown),
            exposure_sign_flips: pick(|m| m.exposure_sign_flips as f64),
        }
    }
}

/// Linear-interpolation quantile (the numpy/Excel "type 7" default) over a
/// pre-sorted, non-empty slice. `p` in `[0, 1]`. `rank = p * (n - 1)`;
/// interpolate between the two bracketing order statistics. `n == 1` returns the
/// sole value.
fn quantile(sorted: &[f64], p: f64) -> f64 {
    let n = sorted.len();
    if n == 1 {
        return sorted[0];
    }
    let rank = p * (n - 1) as f64;
    let lo = rank.floor() as usize;
    let frac = rank - lo as f64;
    if lo + 1 < n {
        sorted[lo] + frac * (sorted[lo + 1] - sorted[lo])
    } else {
        sorted[n - 1]
    }
}

/// Run `run_one(seed, base_point)` over every seed, disjointly in parallel (C1),
/// and collect the family in seed-input order plus the aggregate. The varying
/// dimension is the *seed* (C12 axis 4: MC = sweep over seeds), not a tuning
/// param; `base_point` is constant across draws. Eager-agnostic: the seed ->
/// `Source` construction lives inside `run_one`, never a materialized stream
/// `Vec` in this API (#71).
///
/// Precondition: `seeds` is non-empty (a Monte-Carlo over zero realizations has
/// no defined aggregate). A `debug_assert!` guards it; the `-> McFamily`
/// signature is preserved (no `Result`), matching [`sweep`](crate::sweep).
pub fn monte_carlo<F>(base_point: &[Scalar], seeds: &[u64], run_one: F) -> McFamily
where
    F: Fn(u64, &[Scalar]) -> RunReport + Sync,
{
    let nthreads = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
    monte_carlo_with_threads(base_point, seeds, nthreads, run_one)
}

/// The thread-count-explicit core of [`monte_carlo`]. Module-private: the public
/// `monte_carlo` derives the count, while the tests drive it at 1 and at N to pin
/// determinism under concurrency (C1). A thin adapter over
/// [`run_indexed`](crate::sweep): it runs each seed disjointly and zips the
/// reports back onto their seeds in seed-input order.
fn monte_carlo_with_threads<F>(
    base_point: &[Scalar],
    seeds: &[u64],
    nthreads: usize,
    run_one: F,
) -> McFamily
where
    F: Fn(u64, &[Scalar]) -> RunReport + Sync,
{
    debug_assert!(!seeds.is_empty(), "monte_carlo requires a non-empty seed set");
    let reports = run_indexed(seeds.len(), nthreads, |i| run_one(seeds[i], base_point));
    let draws: Vec<McDraw> = seeds
        .iter()
        .zip(reports)
        .map(|(&seed, report)| McDraw { seed, report })
        .collect();
    let aggregate = McAggregate::from_draws(&draws);
    McFamily { draws, aggregate }
}
```

- [ ] **Step 2: Append the test module to `crates/aura-engine/src/mc.rs`**

Append the following `#[cfg(test)] mod tests` to the end of `mc.rs`. The
`run_draw` helper is the MC analog of `sweep.rs`'s `run_point` (a free `fn`, so
`Copy + Sync`): it builds + bootstraps + runs one `(seed, point)` and folds the
sinks into a `RunReport` whose stream is seeded via `SyntheticSpec::source(seed)`.

```rust
#[cfg(test)]
mod tests {
    use super::*;
    use crate::test_fixtures::composite_sma_cross_harness;
    use crate::{f64_field, summarize, RunManifest, SyntheticSpec, Timestamp};
    use aura_core::Scalar;

    /// Build + bootstrap + run + drain + summarize one (seed, point) into a
    /// `RunReport`. A free `fn` (Copy + Sync) so it serves as the `monte_carlo`
    /// closure AND a direct reference for the "draw == independent run"
    /// comparison. The stream is generated from `seed` (seed-as-input, #66), and
    /// `seed` is recorded into the manifest.
    fn run_draw(seed: u64, point: &[Scalar]) -> RunReport {
        let (bp, rx_eq, rx_ex) = composite_sma_cross_harness();
        let mut h = bp
            .bootstrap_with_params(point.to_vec())
            .expect("base point is kind-checked against param_space");
        let spec = SyntheticSpec { start: 1.0, len: 64, step: 1 };
        let window = (Timestamp(1), Timestamp((spec.len as i64 - 1) * spec.step + 1));
        h.run(vec![Box::new(spec.source(seed))]);
        let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
        let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
        RunReport {
            manifest: RunManifest {
                commit: "test".to_string(),
                params: Vec::new(),
                window,
                seed,
                broker: "test".to_string(),
            },
            metrics: summarize(&equity, &exposure),
        }
    }

    fn base_point() -> Vec<Scalar> {
        // matches the composite_sma_cross param_space order: fast, slow, scale
        vec![Scalar::I64(2), Scalar::I64(4), Scalar::F64(0.5)]
    }

    fn draws_with_metrics(values: &[f64]) -> Vec<McDraw> {
        values
            .iter()
            .enumerate()
            .map(|(i, &v)| McDraw {
                seed: i as u64,
                report: RunReport {
                    manifest: RunManifest {
                        commit: "t".to_string(),
                        params: Vec::new(),
                        window: (Timestamp(0), Timestamp(0)),
                        seed: i as u64,
                        broker: "t".to_string(),
                    },
                    metrics: RunMetrics {
                        total_pips: v,
                        max_drawdown: v,
                        exposure_sign_flips: v as u64,
                    },
                },
            })
            .collect()
    }

    #[test]
    fn monte_carlo_runs_one_draw_per_seed_in_input_order() {
        // spec §Testing 1: N seeds -> N draws, seeds carried in INPUT order.
        let family = monte_carlo(&base_point(), &[1, 2, 3], run_draw);
        assert_eq!(family.draws.len(), 3);
        assert_eq!(
            family.draws.iter().map(|d| d.seed).collect::<Vec<_>>(),
            vec![1, 2, 3],
        );
    }

    #[test]
    fn family_is_deterministic_across_thread_counts() {
        // spec §Testing 2: order = seed input, not completion (C1).
        let point = base_point();
        let seeds: Vec<u64> = (1..=8).collect();
        let one = monte_carlo_with_threads(&point, &seeds, 1, run_draw);
        let many = monte_carlo_with_threads(&point, &seeds, 8, run_draw);
        assert_eq!(one, many);
        assert_eq!(one, monte_carlo(&point, &seeds, run_draw));
    }

    #[test]
    fn draw_equals_independent_seeded_run() {
        // spec §Testing 3: each draw == an independent run of that (seed, point)
        // — MC adds enumeration + execution, never a metrics change (C1).
        let point = base_point();
        let family = monte_carlo(&point, &[10, 11, 12], run_draw);
        for draw in &family.draws {
            assert_eq!(draw.report, run_draw(draw.seed, &point));
            assert!(draw.report.metrics.total_pips.is_finite());
        }
    }

    #[test]
    fn distinct_seeds_produce_distinct_draws() {
        // spec §Testing 4: the seed actually perturbs each run — the family does
        // not collapse to one metric.
        let family = monte_carlo(&base_point(), &[1, 2, 3, 4, 5], run_draw);
        let first = family.draws[0].report.metrics.total_pips;
        assert!(
            family.draws.iter().any(|d| d.report.metrics.total_pips != first),
            "a multi-seed family must not collapse to one metric",
        );
    }

    #[test]
    fn aggregate_is_a_pure_reduction() {
        // spec §Testing 5: the stored aggregate is recomputable from the draws.
        let family = monte_carlo(&base_point(), &[1, 2, 3, 4], run_draw);
        assert_eq!(McAggregate::from_draws(&family.draws), family.aggregate);
    }

    #[test]
    fn aggregate_stats_on_known_fixture() {
        // spec §Testing 6: type-7 quantile + mean over known metric values
        // [0,1,2,3,4]: mean=2.0, p50=2.0, p5≈0.2, p95≈3.8.
        let agg = McAggregate::from_draws(&draws_with_metrics(&[0.0, 1.0, 2.0, 3.0, 4.0]));
        assert_eq!(agg.total_pips.mean, 2.0);
        assert_eq!(agg.total_pips.p50, 2.0);
        assert!((agg.total_pips.p5 - 0.2).abs() < 1e-9, "p5 = {}", agg.total_pips.p5);
        assert!((agg.total_pips.p95 - 3.8).abs() < 1e-9, "p95 = {}", agg.total_pips.p95);
    }

    #[test]
    fn quantile_endpoints_and_singleton() {
        // spec §Testing 7: singleton returns the sole value; p=0 -> min, p=1 -> max.
        assert_eq!(quantile(&[42.0], 0.0), 42.0);
        assert_eq!(quantile(&[42.0], 0.5), 42.0);
        assert_eq!(quantile(&[42.0], 1.0), 42.0);
        let xs = [1.0, 2.0, 3.0, 4.0, 5.0];
        assert_eq!(quantile(&xs, 0.0), 1.0);
        assert_eq!(quantile(&xs, 1.0), 5.0);
    }
}
```

- [ ] **Step 3: Verify `mc.rs` does not yet compile into the crate (not declared)**

Run: `cargo build -p aura-engine`
Expected: builds 0 errors — `mc.rs` is a file on disk not yet referenced by any
`mod` (so the compiler ignores it). This step confirms the module file is
syntactically isolated until Task 3 wires it; the tests inside it do NOT run yet.

---

## Task 3: Wire the module + public exports into `lib.rs`

**Files:**
- Modify: `crates/aura-engine/src/lib.rs:36-53`

- [ ] **Step 1: Declare the module and add the public exports**

In `crates/aura-engine/src/lib.rs`, the module-declaration block is at lines
36–41 and currently reads:

```rust
mod blueprint;
mod builder;
mod graph_model;
mod harness;
mod report;
mod sweep;
```

Add `mod mc;` so the block reads (keep alphabetical placement after `harness`):

```rust
mod blueprint;
mod builder;
mod graph_model;
mod harness;
mod mc;
mod report;
mod sweep;
```

Then, immediately after the existing `pub use sweep::{sweep, GridSpace,
SweepError, SweepFamily, SweepPoint};` line (`lib.rs:53`), add the five new
public exports:

```rust
pub use mc::{monte_carlo, McAggregate, McDraw, McFamily, MetricStats};
```

(`quantile` and `monte_carlo_with_threads` stay private — not exported.)

- [ ] **Step 2: Run the new MC tests to verify they pass (now that the module is wired)**

Run: `cargo test -p aura-engine --lib mc::`
Expected: PASS — all 7 `mc::tests` green:
`monte_carlo_runs_one_draw_per_seed_in_input_order`,
`family_is_deterministic_across_thread_counts`, `draw_equals_independent_seeded_run`,
`distinct_seeds_produce_distinct_draws`, `aggregate_is_a_pure_reduction`,
`aggregate_stats_on_known_fixture`, `quantile_endpoints_and_singleton`.
`test result: ok. 7 passed`.

- [ ] **Step 3: Run the full workspace test suite (regression gate)**

Run: `cargo test --workspace`
Expected: PASS — all tests green, including the unchanged `sweep::tests` guard
(behaviour-preserving refactor) and the new `mc::tests`. 0 failed.

- [ ] **Step 4: Clippy + doc gate**

Run: `cargo clippy --workspace --all-targets -- -D warnings`
Expected: clean, 0 warnings.

Run: `cargo doc -p aura-engine --no-deps 2>&1`
Expected: builds clean — the new intra-doc links (`McDraw`, `McFamily`,
`McAggregate`, `run_indexed`, `sweep`, `SweepPoint`, `SweepFamily`) resolve, 0
warnings.
