Files
Aura/crates/aura-registry/tests/campaign_run_store_e2e.rs
T
claude d3b1a1aead feat(campaign,registry,cli): per-cell fault isolation — a failed cell is recorded, never a global abort
closes #272

A member fault (no-data, bind, run, or a caught panic) is now a recorded
per-cell outcome instead of aborting the whole campaign and discarding every
already-computed cell. The incident that motivated this (a 22-instrument
campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap)
now completes: the healthy cells persist, the gap cell is recorded as failed,
and the run exits 3.

Direction (owner decision 2026-07-14): run to completion and report
compromised results; no coverage preflight, no window synthesis.

Containment granularity:
- The CELL for a sweep-stage member fault (a grid hole structurally
  compromises winner selection, so the whole cell fails).
- The FOLD for a walk_forward member fault (independent time windows): the
  surviving folds pool into the family, failed folds are recorded as
  StageRealization.window_faults, and the summary names the ratio.

- aura-registry: additive CellFault / CellFaultKind (closed:
  no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus
  fault/coverage fields on CellRealization and window_faults on
  StageRealization — all serde-default-skipped, so pre-#272 campaign_runs
  lines parse and round-trip byte-identical.
- aura-campaign: run_cell returns a fault-annotated CellRealization instead of
  Err (execute's accumulate-then-append-once tail is unchanged and now
  persists every healthy cell + the one run record); a `contain` split keeps
  ExecFault::Registry and doc-shape preflight faults global while Member/Window
  become per-cell/per-fold. Member panics are caught with
  catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS)
  and recorded as MemberFault::Panic — a member panic no longer aborts the
  process. The wf stage partitions Registry faults (global) from Member/Window
  (per-fold) and filters faulted-fold placeholders (the
  "faulted-member-placeholder" broker sentinel) out of the persisted family.
- aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage
  derives effective bounds + interior gap months from the #264 archive
  primitives; present_campaign prints per-cell failure notes + a completion
  summary and threads the failed-cell count; a run with >=1 failed cell exits 3
  ("completed with failed cells") uniformly across `aura campaign run` and the
  dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result).
  Usage stays 2, refused-before-running stays 1, clean stays 0.

Tests: the global-abort pins flip to containment (execute + the two wf fault
tests → fold-containment + all-folds-fail-the-cell); new panic-containment
tests on both the sweep path (PanicRunner) and the wf path (this commit adds
the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered
cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3
register; a pre-#272-line byte-identical round-trip guard.

Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean.
Decision log: #272 comments (fork rationale, the fold Registry/Member split,
the placeholder sentinel, uniform exit-3).

Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry
though the additive fields also touch aura-campaign's exec.rs literals — the
loop absorbed it mechanically; a future plan for a cross-crate additive-field
change should scope every crate's construction sites in the first task.
2026-07-14 16:51:31 +02:00

139 lines
5.8 KiB
Rust

//! End-to-end coverage for the campaign-run store (cycle 0107 task 3): the
//! `campaign_runs.jsonl` sibling store's persistence and isolation properties,
//! driven through the PUBLIC `Registry` surface only (`append_family` /
//! `append_campaign_run` / `load_family_members` / `load_campaign_runs`) — the
//! in-crate unit tests in `lib.rs`/`lineage.rs` pin the counter and round-trip
//! shape on one live `Registry` instance; these tests pin the two properties a
//! real multi-invocation CLI depends on: the store survives a fresh `Registry`
//! handle (it is a real on-disk file, not an in-memory cache), and writing to
//! it never perturbs the existing family store it sits beside.
use std::fs;
use std::path::PathBuf;
use aura_core::{Scalar, Timestamp};
use aura_engine::{FamilySelection, RunManifest, RunMetrics, RunReport, SelectionMode};
use aura_registry::{
CampaignRunRecord, CellRealization, FamilyKind, Registry, StageRealization, StageSelection,
};
fn temp_dir(name: &str) -> PathBuf {
let dir = std::path::Path::new(env!("CARGO_TARGET_TMPDIR"))
.join(format!("aura-registry-campaign-e2e-{name}"));
let _ = fs::remove_dir_all(&dir);
fs::create_dir_all(&dir).expect("create temp dir");
dir.join("runs.jsonl")
}
fn tiny_report() -> RunReport {
RunReport {
manifest: RunManifest {
commit: "c".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(1), Timestamp(2)),
seed: 0,
broker: "b".to_string(),
selection: None,
instrument: None,
topology_hash: None,
project: None,
},
metrics: RunMetrics { total_pips: 1.0, max_drawdown: 0.0, bias_sign_flips: 0, r: None },
}
}
fn campaign_run(campaign: &str) -> CampaignRunRecord {
CampaignRunRecord {
campaign: campaign.to_string(),
process: "proc".to_string(),
run: 0,
seed: 1,
cells: vec![CellRealization {
strategy: "strat-id".to_string(),
instrument: "EURUSD".to_string(),
window_ms: (0, 1000),
regime: None,
regime_ordinal: 0,
fault: None,
coverage: None,
stages: vec![StageRealization {
block: "std::sweep".to_string(),
family_id: Some(format!("{campaign}-0-EURUSD-w0-s0-0")),
survivor_ordinals: None,
selection: Some(StageSelection {
winner_ordinal: 0,
params: vec![("x".to_string(), Scalar::i64(1))],
selection: FamilySelection {
selection_metric: "sqn_normalized".to_string(),
n_trials: 1,
raw_winner_metric: 1.0,
mode: SelectionMode::Argmax,
deflated_score: None,
overfit_probability: None,
n_resamples: None,
block_len: None,
seed: None,
neighbourhood_score: None,
n_neighbours: None,
},
}),
bootstrap: None,
window_faults: vec![],
}],
}],
generalizations: vec![],
trace_name: None,
}
}
/// Property: the campaign-run store is a real on-disk file, not a cache tied to
/// one `Registry` instance — a record appended by one `Registry::open` handle
/// is visible to a SECOND, freshly-opened handle over the same path. This is
/// what makes the store usable across separate CLI invocations (author runs
/// `aura campaign run` twice, in two processes, against the same registry).
#[test]
fn campaign_run_persists_across_fresh_registry_handles() {
let path = temp_dir("persist_across_handles");
let writer = Registry::open(&path);
let run = writer.append_campaign_run(&campaign_run("camp-a")).expect("append");
assert_eq!(run, 0);
// a brand-new handle, as a second CLI invocation would construct
let reader = Registry::open(&path);
let loaded = reader.load_campaign_runs().expect("load from fresh handle");
assert_eq!(loaded.len(), 1);
assert_eq!(loaded[0].campaign, "camp-a");
assert_eq!(loaded[0].run, 0);
}
/// Property: `campaign_runs.jsonl` is a sibling store, wholly independent of
/// the existing `families.jsonl` flat-run store it sits beside — appending a
/// campaign run does not add, remove, or alter a single family member record,
/// and appending a family does not touch the campaign-run store. Two
/// independently-growing stores at the same registry root, per the module's
/// documented growth pattern.
#[test]
fn campaign_run_store_does_not_perturb_the_sibling_family_store() {
let path = temp_dir("sibling_isolation");
let reg = Registry::open(&path);
let before_families = reg.append_family("fam", FamilyKind::Sweep, &[tiny_report()]).unwrap();
let members_before = reg.load_family_members().expect("load families before");
assert_eq!(members_before.len(), 1);
reg.append_campaign_run(&campaign_run("camp-b")).expect("append campaign run");
// the family store is byte-for-byte unaffected by the campaign-run write
let members_after = reg.load_family_members().expect("load families after");
assert_eq!(members_after, members_before, "family store untouched by a campaign-run append");
assert_eq!(before_families, "fam-0");
// and the reverse: appending another family leaves the campaign-run store's
// single record alone
reg.append_family("fam2", FamilyKind::Sweep, &[tiny_report()]).unwrap();
let runs = reg.load_campaign_runs().expect("load campaign runs");
assert_eq!(runs.len(), 1, "campaign-run store untouched by a family append");
assert_eq!(runs[0].campaign, "camp-b");
}