Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

Real-data block-bootstrap Monte-Carlo over a loaded blueprint #172

New Issue
Closed
opened 2026-07-01 11:36:30 +02:00 by Brummel · 5 comments
Brummel commented 2026-07-01 11:36:30 +02:00
Owner
Copy Link
Copy Source

Cycle 0095 (#170) shipped aura mc <blueprint.json> --seeds N as a per-seed synthetic price walk (SyntheticSpec::source(seed), the mc_family precedent) — the machinery form (FamilyKind::MonteCarlo + content-addressed store + reproduce). The statistically stronger form for a deployed candidate is a moving-block bootstrap of a real recorded price/return series (cf. run_mc_r_bootstrap, which bootstraps pooled OOS R-rows). That rides the DataServer recorded-data seam.

depends on: #124 (recorded dataset-metadata / DataServer seam)
context: deferred from #170 cycle 0095 — synthetic-walk MC ships the machinery; real-data resample is the trader-grade refinement.

Cycle 0095 (#170) shipped `aura mc <blueprint.json> --seeds N` as a per-seed **synthetic price walk** (SyntheticSpec::source(seed), the mc_family precedent) — the machinery form (FamilyKind::MonteCarlo + content-addressed store + reproduce). The statistically stronger form for a *deployed* candidate is a moving-block bootstrap of a **real recorded** price/return series (cf. run_mc_r_bootstrap, which bootstraps pooled OOS R-rows). That rides the DataServer recorded-data seam. depends on: #124 (recorded dataset-metadata / DataServer seam) context: deferred from #170 cycle 0095 — synthetic-walk MC ships the machinery; real-data resample is the trader-grade refinement.
Brummel added the idea label 2026-07-01 11:36:30 +02:00
Brummel commented 2026-07-01 13:24:25 +02:00
Author
Owner
Copy Link
Copy Source

Fold in: the synthetic_walk_sources length↔warm-up coupling (from cycle 0095 audit)

The cycle-0095 audit (architect) flagged a coupling this issue's real-data DGP work
should retire, named here so it is not lost:

synthetic_walk_sources (aura-cli, blueprint_mc_family) uses a FIXED
SyntheticSpec { start: 1.0, len: 60, step: 1 }, hand-calibrated to the shallow
stage1 SMA(4) + vol-3 warm-up. But aura mc <blueprint.json> accepts ANY closed
blueprint. A deeper-lookback loaded blueprint warms poorly over a 60-bar walk, so its
draws can collapse to identical/empty members with no error — a silent-vacuous
Monte-Carlo. The anti-degenerate guard (blueprint_mc_family_seeds_differ) only
exercises the shallow fixture, so the coupling is untested at depth.

A real recorded series (this issue) sidesteps the synthetic-walk length calibration
entirely, which is why the architect recommended folding this in here rather than
opening a separate thread. If real data is not the near-term path, the cheap interim
guard is a silent-vacuous detector (reject an MC whose draws are all bit-identical, or
size the walk to the loaded blueprint's declared lookback).

## Fold in: the `synthetic_walk_sources` length↔warm-up coupling (from cycle 0095 audit) The cycle-0095 audit (architect) flagged a coupling this issue's real-data DGP work should retire, named here so it is not lost: `synthetic_walk_sources` (aura-cli, `blueprint_mc_family`) uses a FIXED `SyntheticSpec { start: 1.0, len: 60, step: 1 }`, hand-calibrated to the shallow stage1 `SMA(4)` + vol-3 warm-up. But `aura mc <blueprint.json>` accepts ANY closed blueprint. A deeper-lookback loaded blueprint warms poorly over a 60-bar walk, so its draws can collapse to identical/empty members **with no error** — a silent-vacuous Monte-Carlo. The anti-degenerate guard (`blueprint_mc_family_seeds_differ`) only exercises the shallow fixture, so the coupling is untested at depth. A real recorded series (this issue) sidesteps the synthetic-walk length calibration entirely, which is why the architect recommended folding this in here rather than opening a separate thread. If real data is not the near-term path, the cheap interim guard is a silent-vacuous detector (reject an MC whose draws are all bit-identical, or size the walk to the loaded blueprint's declared lookback).
Brummel commented 2026-07-01 18:19:36 +02:00
Author
Owner
Copy Link
Copy Source

Interim silent-vacuous-MC guard — reject over auto-size (2026-07-01)

The folded-in cycle-0095 audit item (the synthetic_walk_sources length↔warm-up
coupling) forces one design fork, settled here so the interim guard can land
independent of the real-data path:

Fork. When a loaded blueprint warms poorly over the fixed 60-bar synthetic
walk (SyntheticSpec { start: 1.0, len: 60, step: 1 }), every per-seed draw can
collapse to a bit-identical member and the Monte-Carlo silently returns Ok with
an indistinguishable family — a silent-vacuous MC. Two interim remedies:

  • (reject) — detect that all draws are bit-identical and refuse with a named
    error, so a vacuous MC is never returned as if it were a valid family.
  • (auto-size) — grow the synthetic walk to the loaded blueprint's declared
    lookback so the draws warm and differ.

Decision: reject. Rationale (semantics, not effort): auto-size needs a
per-blueprint declared-lookback contract that does not cleanly exist — an SMA
length is a bound param, not a surfaced lookback the harness can read — so
auto-size would have to guess the warm-up depth, which the refuse-don't-guess
discipline (C10) forbids; and its motivation is largely subsumed by the real-data
resample path (the headline of this issue), which sidesteps synthetic-walk
calibration entirely. Reject is the correct interim floor: it converts a
silent-vacuous result into an explicit refusal, closing the untested-at-depth gap
the audit flagged (blueprint_mc_family_seeds_differ only exercises the shallow
SMA(2,4) fixture).

Scope. This is the #124-independent interim guard only. The headline —
a moving-block bootstrap of a real recorded series — remains gated on #124
(the recorded dataset-metadata / DataServer seam) and is unchanged by this guard.

## Interim silent-vacuous-MC guard — reject over auto-size (2026-07-01) The folded-in cycle-0095 audit item (the `synthetic_walk_sources` length↔warm-up coupling) forces one design fork, settled here so the interim guard can land independent of the real-data path: **Fork.** When a loaded blueprint warms poorly over the fixed 60-bar synthetic walk (`SyntheticSpec { start: 1.0, len: 60, step: 1 }`), every per-seed draw can collapse to a bit-identical member and the Monte-Carlo silently returns `Ok` with an indistinguishable family — a silent-vacuous MC. Two interim remedies: - **(reject)** — detect that all draws are bit-identical and refuse with a named error, so a vacuous MC is never returned as if it were a valid family. - **(auto-size)** — grow the synthetic walk to the loaded blueprint's declared lookback so the draws warm and differ. **Decision: reject.** Rationale (semantics, not effort): auto-size needs a per-blueprint *declared-lookback* contract that does not cleanly exist — an SMA length is a bound param, not a surfaced lookback the harness can read — so auto-size would have to guess the warm-up depth, which the refuse-don't-guess discipline (C10) forbids; and its motivation is largely subsumed by the real-data resample path (the headline of this issue), which sidesteps synthetic-walk calibration entirely. Reject is the correct interim floor: it converts a silent-vacuous result into an explicit refusal, closing the untested-at-depth gap the audit flagged (`blueprint_mc_family_seeds_differ` only exercises the shallow SMA(2,4) fixture). **Scope.** This is the `#124`-independent interim guard only. The headline — a moving-block bootstrap of a real recorded series — remains gated on `#124` (the recorded dataset-metadata / DataServer seam) and is unchanged by this guard.
Brummel commented 2026-07-09 17:31:53 +02:00
Author
Owner
Copy Link
Copy Source

Triage 2026-07-09 (tree at 68317ec) — status split: the interim guard shipped, the headline is unbuilt, and the stated dependency has softened:

  • Shipped: the silent-vacuous-MC guard (8b2defa) refuses cross-seed bit-identical metrics (crates/aura-cli/src/main.rs:1655-1670) — the fixed-60-bar audit finding folded in by the first comment is resolved as decided (reject, not auto-size).
  • Unbuilt: no code resamples recorded bars into source streams; resample_block/r_bootstrap operate on per-trade R rows only (crates/aura-engine/src/mc.rs:156-215), and synthetic_walk_sources remains the only per-seed DGP for the blueprint MC family. Today's aura mc <bp> --real SYMBOL is the R-row bootstrap — a different statistical object (the glossary keeps the input-perturbation axis distinct from the std::monte_carlo R-bootstrap stage).
  • Softened: the body's hard gate on #124 no longer holds for raw bar access — real M1 bars are CLI-loadable via RunData::Real/open_real_source (crates/aura-cli/src/main.rs:1284-1310, aura-ingest); #124 still covers the instrument-metadata/cost tier. A price-path block bootstrap can ride the existing seam and surface as a campaign/process stage block.
Triage 2026-07-09 (tree at 68317ec) — status split: the interim guard shipped, the headline is unbuilt, and the stated dependency has softened: - Shipped: the silent-vacuous-MC guard (8b2defa) refuses cross-seed bit-identical metrics (crates/aura-cli/src/main.rs:1655-1670) — the fixed-60-bar audit finding folded in by the first comment is resolved as decided (reject, not auto-size). - Unbuilt: no code resamples recorded bars into source streams; resample_block/r_bootstrap operate on per-trade R rows only (crates/aura-engine/src/mc.rs:156-215), and synthetic_walk_sources remains the only per-seed DGP for the blueprint MC family. Today's `aura mc <bp> --real SYMBOL` is the R-row bootstrap — a different statistical object (the glossary keeps the input-perturbation axis distinct from the std::monte_carlo R-bootstrap stage). - Softened: the body's hard gate on #124 no longer holds for raw bar access — real M1 bars are CLI-loadable via RunData::Real/open_real_source (crates/aura-cli/src/main.rs:1284-1310, aura-ingest); #124 still covers the instrument-metadata/cost tier. A price-path block bootstrap can ride the existing seam and surface as a campaign/process stage block.
Brummel commented 2026-07-11 12:55:24 +02:00
Author
Owner
Copy Link
Copy Source

Demarcation note (2026-07-11, from the #230 resolution): this issue is the home of the "real Monte-Carlo family" capability — members are full backtests over block-bootstrapped/permuted real price series (the overfitting-falsification use of MC: does the edge survive on resampled data realizations?), a genuine FamilyKind::MonteCarlo family in the C1 sense with the same deep reproduce semantics as Sweep/WalkForward (every member re-derived from the manifest, compared bit-identically).

#230 was closed by design: the campaign std::monte_carlo stage is a trade-sequence bootstrap — a statistic annotating an already-produced OOS trade-R series, not a family producer — and its deep reproduction is the campaign re-run (document + recorded data are deterministic). The option of persisting an addressable MC family out of that annotator was declined there in favor of this issue.

Delivery note for when this is built: the family manifest needs the real instrument + window, the resample scheme + knobs (block length, resample count), and the resample seed as first-class fields. The synthetic MC family manifest's existing seed field is the member-generating price-walk seed; for this family kind the member-generating seed IS the resample seed, so the field carries the right meaning — it is the source description that changes from synthetic spec to instrument + window + scheme.

Demarcation note (2026-07-11, from the #230 resolution): this issue is the home of the "real Monte-Carlo family" capability — members are full backtests over block-bootstrapped/permuted *real* price series (the overfitting-falsification use of MC: does the edge survive on resampled data realizations?), a genuine `FamilyKind::MonteCarlo` family in the C1 sense with the same deep reproduce semantics as Sweep/WalkForward (every member re-derived from the manifest, compared bit-identically). #230 was closed by design: the campaign `std::monte_carlo` stage is a trade-sequence bootstrap — a statistic annotating an already-produced OOS trade-R series, not a family producer — and its deep reproduction is the campaign re-run (document + recorded data are deterministic). The option of persisting an addressable MC family out of that annotator was declined there in favor of this issue. Delivery note for when this is built: the family manifest needs the real instrument + window, the resample scheme + knobs (block length, resample count), and the resample seed as first-class fields. The synthetic MC family manifest's existing seed field is the member-generating price-walk seed; for this family kind the member-generating seed IS the resample seed, so the field carries the right meaning — it is the source description that changes from synthetic spec to instrument + window + scheme.
claude commented 2026-07-21 10:36:43 +02:00
Collaborator
Copy Link
Copy Source

The real-data moving-block bootstrap Monte-Carlo this issue asks for has shipped. aura mc <blueprint.json> --real <SYMBOL> --axis <name>=<csv> [--stop-length <n>] [--stop-k <x>] [--block-len <n>] [--resamples <n>] [--seed <n>] [--from <ms>] [--to <ms>] is a fully implemented second dispatch arm alongside the original synthetic-seed --seeds path — see dispatch_mc in crates/aura-cli/src/main.rs (the a.real.is_some() branch), which routes through verb_sugar::run_mc_sugar with McKnobs { resamples, block_len, seed }, over a real archive resolved via aura_ingest::DataServer.

The moving-block bootstrap mechanic the issue refers to as run_mc_r_bootstrap is r_bootstrap in crates/aura-backtest/src/mc.rs: a moving-block bootstrap of the pooled out-of-sample per-trade R series (the walk-forward winners' OOS R-rows), which is the domain-correct unit per this repo's invariant that signal quality is measured in R, not raw price/equity — a stronger form than the price-walk resample the issue's phrasing suggested. End-to-end coverage lives in crates/aura-cli/tests/cli_run.rs (e.g. the mc --real GER40 --stop-length --stop-k cases).

This landed under #220 (blueprint-generic generalize/walkforward/mc), which post-dates this issue's filing. The depends on: #124 link did not end up gating delivery — the --real path resolves data directly through DataServer without needing #124's full authored/recorded/fallback resolution hierarchy; #124 remains open as an independent concern (instrument-spec/cost resolution), not a blocker for this feature.

One residual thread: the decided document-first direction (#300) migrates exactly this feature's flags (--axis, --from/--to, --stop-length/--stop-k, --block-len, --resamples, --seed) from CLI flags into document vocabulary. That is forward work tracked under #300 and does not require reopening this issue — closing as delivered.

The real-data moving-block bootstrap Monte-Carlo this issue asks for has shipped. `aura mc <blueprint.json> --real <SYMBOL> --axis <name>=<csv> [--stop-length <n>] [--stop-k <x>] [--block-len <n>] [--resamples <n>] [--seed <n>] [--from <ms>] [--to <ms>]` is a fully implemented second dispatch arm alongside the original synthetic-seed `--seeds` path — see `dispatch_mc` in `crates/aura-cli/src/main.rs` (the `a.real.is_some()` branch), which routes through `verb_sugar::run_mc_sugar` with `McKnobs { resamples, block_len, seed }`, over a real archive resolved via `aura_ingest::DataServer`. The moving-block bootstrap mechanic the issue refers to as `run_mc_r_bootstrap` is `r_bootstrap` in `crates/aura-backtest/src/mc.rs`: a moving-block bootstrap of the pooled out-of-sample per-trade R series (the walk-forward winners' OOS R-rows), which is the domain-correct unit per this repo's invariant that signal quality is measured in R, not raw price/equity — a stronger form than the price-walk resample the issue's phrasing suggested. End-to-end coverage lives in `crates/aura-cli/tests/cli_run.rs` (e.g. the `mc --real GER40 --stop-length --stop-k` cases). This landed under #220 (blueprint-generic generalize/walkforward/mc), which post-dates this issue's filing. The `depends on: #124` link did not end up gating delivery — the `--real` path resolves data directly through `DataServer` without needing #124's full authored/recorded/fallback resolution hierarchy; #124 remains open as an independent concern (instrument-spec/cost resolution), not a blocker for this feature. One residual thread: the decided document-first direction (#300) migrates exactly this feature's flags (--axis, --from/--to, --stop-length/--stop-k, --block-len, --resamples, --seed) from CLI flags into document vocabulary. That is forward work tracked under #300 and does not require reopening this issue — closing as delivered.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label idea
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#172
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?