d3b1a1aead
closes #272 A member fault (no-data, bind, run, or a caught panic) is now a recorded per-cell outcome instead of aborting the whole campaign and discarding every already-computed cell. The incident that motivated this (a 22-instrument campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap) now completes: the healthy cells persist, the gap cell is recorded as failed, and the run exits 3. Direction (owner decision 2026-07-14): run to completion and report compromised results; no coverage preflight, no window synthesis. Containment granularity: - The CELL for a sweep-stage member fault (a grid hole structurally compromises winner selection, so the whole cell fails). - The FOLD for a walk_forward member fault (independent time windows): the surviving folds pool into the family, failed folds are recorded as StageRealization.window_faults, and the summary names the ratio. - aura-registry: additive CellFault / CellFaultKind (closed: no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus fault/coverage fields on CellRealization and window_faults on StageRealization — all serde-default-skipped, so pre-#272 campaign_runs lines parse and round-trip byte-identical. - aura-campaign: run_cell returns a fault-annotated CellRealization instead of Err (execute's accumulate-then-append-once tail is unchanged and now persists every healthy cell + the one run record); a `contain` split keeps ExecFault::Registry and doc-shape preflight faults global while Member/Window become per-cell/per-fold. Member panics are caught with catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS) and recorded as MemberFault::Panic — a member panic no longer aborts the process. The wf stage partitions Registry faults (global) from Member/Window (per-fold) and filters faulted-fold placeholders (the "faulted-member-placeholder" broker sentinel) out of the persisted family. - aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage derives effective bounds + interior gap months from the #264 archive primitives; present_campaign prints per-cell failure notes + a completion summary and threads the failed-cell count; a run with >=1 failed cell exits 3 ("completed with failed cells") uniformly across `aura campaign run` and the dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result). Usage stays 2, refused-before-running stays 1, clean stays 0. Tests: the global-abort pins flip to containment (execute + the two wf fault tests → fold-containment + all-folds-fail-the-cell); new panic-containment tests on both the sweep path (PanicRunner) and the wf path (this commit adds the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3 register; a pre-#272-line byte-identical round-trip guard. Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean. Decision log: #272 comments (fork rationale, the fold Registry/Member split, the placeholder sentinel, uniform exit-3). Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry though the additive fields also touch aura-campaign's exec.rs literals — the loop absorbed it mechanically; a future plan for a cross-crate additive-field change should scope every crate's construction sites in the first task.
139 lines
5.8 KiB
Rust
139 lines
5.8 KiB
Rust
//! End-to-end coverage for the campaign-run store (cycle 0107 task 3): the
|
|
//! `campaign_runs.jsonl` sibling store's persistence and isolation properties,
|
|
//! driven through the PUBLIC `Registry` surface only (`append_family` /
|
|
//! `append_campaign_run` / `load_family_members` / `load_campaign_runs`) — the
|
|
//! in-crate unit tests in `lib.rs`/`lineage.rs` pin the counter and round-trip
|
|
//! shape on one live `Registry` instance; these tests pin the two properties a
|
|
//! real multi-invocation CLI depends on: the store survives a fresh `Registry`
|
|
//! handle (it is a real on-disk file, not an in-memory cache), and writing to
|
|
//! it never perturbs the existing family store it sits beside.
|
|
|
|
use std::fs;
|
|
use std::path::PathBuf;
|
|
|
|
use aura_core::{Scalar, Timestamp};
|
|
use aura_engine::{FamilySelection, RunManifest, RunMetrics, RunReport, SelectionMode};
|
|
use aura_registry::{
|
|
CampaignRunRecord, CellRealization, FamilyKind, Registry, StageRealization, StageSelection,
|
|
};
|
|
|
|
fn temp_dir(name: &str) -> PathBuf {
|
|
let dir = std::path::Path::new(env!("CARGO_TARGET_TMPDIR"))
|
|
.join(format!("aura-registry-campaign-e2e-{name}"));
|
|
let _ = fs::remove_dir_all(&dir);
|
|
fs::create_dir_all(&dir).expect("create temp dir");
|
|
dir.join("runs.jsonl")
|
|
}
|
|
|
|
fn tiny_report() -> RunReport {
|
|
RunReport {
|
|
manifest: RunManifest {
|
|
commit: "c".to_string(),
|
|
params: vec![],
|
|
defaults: vec![],
|
|
window: (Timestamp(1), Timestamp(2)),
|
|
seed: 0,
|
|
broker: "b".to_string(),
|
|
selection: None,
|
|
instrument: None,
|
|
topology_hash: None,
|
|
project: None,
|
|
},
|
|
metrics: RunMetrics { total_pips: 1.0, max_drawdown: 0.0, bias_sign_flips: 0, r: None },
|
|
}
|
|
}
|
|
|
|
fn campaign_run(campaign: &str) -> CampaignRunRecord {
|
|
CampaignRunRecord {
|
|
campaign: campaign.to_string(),
|
|
process: "proc".to_string(),
|
|
run: 0,
|
|
seed: 1,
|
|
cells: vec![CellRealization {
|
|
strategy: "strat-id".to_string(),
|
|
instrument: "EURUSD".to_string(),
|
|
window_ms: (0, 1000),
|
|
regime: None,
|
|
regime_ordinal: 0,
|
|
fault: None,
|
|
coverage: None,
|
|
stages: vec![StageRealization {
|
|
block: "std::sweep".to_string(),
|
|
family_id: Some(format!("{campaign}-0-EURUSD-w0-s0-0")),
|
|
survivor_ordinals: None,
|
|
selection: Some(StageSelection {
|
|
winner_ordinal: 0,
|
|
params: vec![("x".to_string(), Scalar::i64(1))],
|
|
selection: FamilySelection {
|
|
selection_metric: "sqn_normalized".to_string(),
|
|
n_trials: 1,
|
|
raw_winner_metric: 1.0,
|
|
mode: SelectionMode::Argmax,
|
|
deflated_score: None,
|
|
overfit_probability: None,
|
|
n_resamples: None,
|
|
block_len: None,
|
|
seed: None,
|
|
neighbourhood_score: None,
|
|
n_neighbours: None,
|
|
},
|
|
}),
|
|
bootstrap: None,
|
|
window_faults: vec![],
|
|
}],
|
|
}],
|
|
generalizations: vec![],
|
|
trace_name: None,
|
|
}
|
|
}
|
|
|
|
/// Property: the campaign-run store is a real on-disk file, not a cache tied to
|
|
/// one `Registry` instance — a record appended by one `Registry::open` handle
|
|
/// is visible to a SECOND, freshly-opened handle over the same path. This is
|
|
/// what makes the store usable across separate CLI invocations (author runs
|
|
/// `aura campaign run` twice, in two processes, against the same registry).
|
|
#[test]
|
|
fn campaign_run_persists_across_fresh_registry_handles() {
|
|
let path = temp_dir("persist_across_handles");
|
|
let writer = Registry::open(&path);
|
|
let run = writer.append_campaign_run(&campaign_run("camp-a")).expect("append");
|
|
assert_eq!(run, 0);
|
|
|
|
// a brand-new handle, as a second CLI invocation would construct
|
|
let reader = Registry::open(&path);
|
|
let loaded = reader.load_campaign_runs().expect("load from fresh handle");
|
|
assert_eq!(loaded.len(), 1);
|
|
assert_eq!(loaded[0].campaign, "camp-a");
|
|
assert_eq!(loaded[0].run, 0);
|
|
}
|
|
|
|
/// Property: `campaign_runs.jsonl` is a sibling store, wholly independent of
|
|
/// the existing `families.jsonl` flat-run store it sits beside — appending a
|
|
/// campaign run does not add, remove, or alter a single family member record,
|
|
/// and appending a family does not touch the campaign-run store. Two
|
|
/// independently-growing stores at the same registry root, per the module's
|
|
/// documented growth pattern.
|
|
#[test]
|
|
fn campaign_run_store_does_not_perturb_the_sibling_family_store() {
|
|
let path = temp_dir("sibling_isolation");
|
|
let reg = Registry::open(&path);
|
|
|
|
let before_families = reg.append_family("fam", FamilyKind::Sweep, &[tiny_report()]).unwrap();
|
|
let members_before = reg.load_family_members().expect("load families before");
|
|
assert_eq!(members_before.len(), 1);
|
|
|
|
reg.append_campaign_run(&campaign_run("camp-b")).expect("append campaign run");
|
|
|
|
// the family store is byte-for-byte unaffected by the campaign-run write
|
|
let members_after = reg.load_family_members().expect("load families after");
|
|
assert_eq!(members_after, members_before, "family store untouched by a campaign-run append");
|
|
assert_eq!(before_families, "fam-0");
|
|
|
|
// and the reverse: appending another family leaves the campaign-run store's
|
|
// single record alone
|
|
reg.append_family("fam2", FamilyKind::Sweep, &[tiny_report()]).unwrap();
|
|
let runs = reg.load_campaign_runs().expect("load campaign runs");
|
|
assert_eq!(runs.len(), 1, "campaign-run store untouched by a family append");
|
|
assert_eq!(runs[0].campaign, "camp-b");
|
|
}
|