Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

Validate walkforward axes without executing the first IS-window sweep #253

New Issue
Closed
opened 2026-07-13 14:56:04 +02:00 by claude · 1 comment
claude commented 2026-07-13 14:56:04 +02:00
Collaborator
Copy Link
Copy Source

The walkforward campaign path validates the axis grid by executing it: dispatch's pre-flight calls blueprint_sweep_over(doc, axes, first_is_window) (crates/aura-cli/src/main.rs:2130-2138 at d7874e2), and blueprint_sweep_over (main.rs:2019-2049) runs every grid member end-to-end (sweep_with_lattice(|point| run_blueprint_member(...))). The result is discarded; the walk_forward closure (main.rs:2139-2141) then re-computes the same first IS window as part of the real run. Landed with 6a775f7 (2026-07-01).

Cost: every real-data walkforward pays roughly one extra IS window of full sim compute; with #239's fit-scaled short windows a single-roll walkforward pays ~2× its total IS compute. Context and measured per-window magnitudes: measured breakdown on #250 (a year-2025, 9-window, 4-cell walkforward is ~4.4 s debug; the discarded pre-flight is one of those windows across the whole grid).

The pre-flight's purpose is axis validation (unknown axis names, arity, bounds), which is resolvable without running members — the same predicate set the construction gate splits already share (edge_kind_check/resolve_*, crates/aura-engine construction). A validation-only pre-flight keeps the refusal semantics and drops the duplicated compute.

The walkforward campaign path validates the axis grid by executing it: `dispatch`'s pre-flight calls `blueprint_sweep_over(doc, axes, first_is_window)` (crates/aura-cli/src/main.rs:2130-2138 at `d7874e2`), and `blueprint_sweep_over` (main.rs:2019-2049) runs every grid member end-to-end (`sweep_with_lattice(|point| run_blueprint_member(...))`). The result is discarded; the `walk_forward` closure (main.rs:2139-2141) then re-computes the same first IS window as part of the real run. Landed with `6a775f7` (2026-07-01). Cost: every real-data walkforward pays roughly one extra IS window of full sim compute; with #239's fit-scaled short windows a single-roll walkforward pays ~2× its total IS compute. Context and measured per-window magnitudes: [measured breakdown on #250](issues/250#issuecomment-3433) (a year-2025, 9-window, 4-cell walkforward is ~4.4 s debug; the discarded pre-flight is one of those windows across the whole grid). The pre-flight's purpose is axis validation (unknown axis names, arity, bounds), which is resolvable without running members — the same predicate set the construction gate splits already share (`edge_kind_check`/`resolve_*`, crates/aura-engine construction). A validation-only pre-flight keeps the refusal semantics and drops the duplicated compute.
claude commented 2026-07-13 15:51:15 +02:00
Author
Collaborator
Copy Link
Copy Source

Attribution correction before close. The pre-flight this issue describes exists, is real, and is now fixed — but it serves only the synthetic walkforward path, so the issue's cost claim ("every real-data walkforward pays ~one extra IS window") was wrong.

Evidence:

  • blueprint_sweep_over / blueprint_walkforward_family each have exactly one call site, gated on DataSource::Synthetic (main.rs ~3832 at d7874e2). The --real branch builds a SugarInvocation and returns via verb_sugar::run_walkforward_sugar — it never reaches this pre-flight.
  • Benchmark of the fix on a real-data walkforward (year-2025 GER40, 2×2 grid, 3 reps each): before 6.09/6.22/6.10 s, after 6.16/6.15/6.09 s — identical within noise, as the call-graph predicts.

The fix that landed (validate_axis_grid): the synthetic pre-flight now drives the sweep terminal's own resolve_axes/arity/kind checks with a constant placeholder report — same single-sourced rejection wording (the #177 exactly-once property stays pinned by aura_walkforward_emits_an_unknown_axis_rejection_once), no member execution, no roller derivation.

The actual discarded computation for --real traffic lives in aura-campaign/src/exec.rs::run_cell (~lines 259-333): the dissolved walkforward/mc process is [Sweep{selection: argmax}, WalkForward], the leading Sweep stage executes every grid member over the entire campaign window, its argmax winner is (per translate_walkforward's own doc) "not part of the summary", and with no Gate stage between them the survivors flow on as the whole family regardless — the WalkForward stage reads only the grid points. Filed with the design fork it raises as a separate issue.

**Attribution correction before close.** The pre-flight this issue describes exists, is real, and is now fixed — but it serves only the **synthetic** walkforward path, so the issue's cost claim ("every real-data walkforward pays ~one extra IS window") was wrong. Evidence: - `blueprint_sweep_over` / `blueprint_walkforward_family` each have exactly one call site, gated on `DataSource::Synthetic` (main.rs ~3832 at `d7874e2`). The `--real` branch builds a `SugarInvocation` and returns via `verb_sugar::run_walkforward_sugar` — it never reaches this pre-flight. - Benchmark of the fix on a real-data walkforward (year-2025 GER40, 2×2 grid, 3 reps each): before 6.09/6.22/6.10 s, after 6.16/6.15/6.09 s — identical within noise, as the call-graph predicts. The fix that landed (`validate_axis_grid`): the synthetic pre-flight now drives the sweep terminal's own `resolve_axes`/arity/kind checks with a constant placeholder report — same single-sourced rejection wording (the #177 exactly-once property stays pinned by `aura_walkforward_emits_an_unknown_axis_rejection_once`), no member execution, no roller derivation. The **actual** discarded computation for `--real` traffic lives in `aura-campaign/src/exec.rs::run_cell` (~lines 259-333): the dissolved walkforward/mc process is `[Sweep{selection: argmax}, WalkForward]`, the leading Sweep stage executes every grid member over the **entire campaign window**, its argmax winner is (per `translate_walkforward`'s own doc) "not part of the summary", and with no Gate stage between them the survivors flow on as the whole family regardless — the WalkForward stage reads only the grid points. Filed with the design fork it raises as a separate issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
BLOCKER
Core functionality blocked; preempts other selection
 bug
Observable misbehaviour
 feature
Substantial extension; no full spec required
 idea
Droppable; no commitment
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude (Claude) Brummel
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Brummel/Aura#253
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?