Brummel/Aura
1
0
Fork 0
Code Issues 33 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(aura-cli): aura mc + family-aware runs + window_of migration (0045 iter 2)

Browse Source 
Iteration 2 of spec 0045 — the CLI surface for orchestration families.

window_of migration: every manifest-window scan (run_sample, sweep_family,
walkforward_family, sweep_over, run_oos, run_macd) now reads the window from
Source::bounds() via window_of(&sources) instead of prices.first()/.last().
Behaviour-preserving (the walk-forward roller boundaries are bar-aligned, so
the producer-supplied window equals the old first/last) — pinned by the
unchanged run_sample (1,7) and walk-forward determinism tests.

aura mc: new built-in Monte-Carlo family on the CLI (monte_carlo over a
built-in seed set + the sample harness re-seeded per draw), persisted via
append_family and rendered as one member line per seed (carrying the
family_id) plus an mc_aggregate line. McAggregate is not Serialize, so the
aggregate line is built from its three MetricStats blocks.

family-aware runs: aura sweep / walkforward / mc now persist to the family
store via append_family with an optional --name (per-kind default), printing
the assigned family_id per member. aura runs families lists family headers
(id, kind, member count); aura runs family <id> [rank <metric>] lists/ranks
one family's members as a unit. An unknown family id is an empty family
(exit 0); an unknown metric is a usage error (exit 2).

Two pre-existing cli_run.rs E2E tests were adapted to the new family
contract (a required consequence of sweep/walkforward switching from the
flat store to the family store — a plan gap caught at implement time): the
sweep-output test now asserts the family-wrapper, and the runs list/rank
flow became the family list / per-family-rank flow (which also exercises the
per-name counter sweep-0/sweep-1 end-to-end). A new E2E test drives aura mc
through the binary.

Known cosmetic detail: the family-wrapped sweep/mc/walkforward stdout nests
the report via serde_json::json! (to_value -> alphabetical keys, leading
"broker"), while runs family / runs list print via RunReport::to_json()
(declaration order, leading "commit"). Both valid JSON; key order is not
load-bearing and the stored families.jsonl uses to_string (declaration
order), so the round-trip is unaffected.

Verification: cargo test --workspace green; cargo clippy --workspace
--all-targets -D warnings clean.

refs #70
This commit is contained in:
Brummel
2026-06-15 18:27:54 +02:00
parent 1d9555c468
commit bcd1072ca8
2 changed files with 338 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/aura-cli/src/main.rs
+261 -54
View File
@@ -15,12 +15,15 @@ mod render;
use aura_core::{Firing, Scalar, ScalarKind, Timestamp};
use aura_engine::{
f64_field, param_stability, summarize, walk_forward, Composite, Edge, FlatGraph,
GraphBuilder, Harness, RollMode, RunManifest, RunReport, SourceSpec, SweepFamily,
SyntheticSpec, Target, VecSource, WalkForwardResult, WindowBounds, WindowRoller,
WindowRun,
f64_field, monte_carlo, param_stability, summarize, walk_forward, window_of, Composite, Edge,
FlatGraph, GraphBuilder, Harness, McAggregate, McFamily, RollMode, RunManifest, RunReport,
SourceSpec, SweepFamily, SyntheticSpec, Target, VecSource, WalkForwardResult, WindowBounds,
WindowRoller, WindowRun,
};
use aura_registry::{
group_families, mc_member_reports, optimize, rank_by, sweep_member_reports,
walkforward_member_reports, FamilyKind, Registry,
};
use aura_registry::{optimize, rank_by, Registry};
use aura_std::{Ema, Exposure, LinComb, Recorder, SimBroker, Sma, Sub};
use std::sync::mpsc::{self, Receiver};
@@ -143,12 +146,10 @@ fn sim_optimal_manifest(
/// (C1): the same build yields the same report.
fn run_sample() -> RunReport {
let (mut h, rx_eq, rx_ex) = sample_harness();
let prices = synthetic_prices();
let window = (
prices.first().expect("non-empty stream").0,
prices.last().expect("non-empty stream").0,
);
h.run(vec![Box::new(VecSource::new(prices))]);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(VecSource::new(synthetic_prices()))];
let window = window_of(&sources).expect("non-empty synthetic stream");
h.run(sources);
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
@@ -376,12 +377,10 @@ fn sweep_family() -> SweepFamily {
let mut h = bp
.bootstrap_with_params(point.to_vec())
.expect("grid points are kind-checked against param_space");
let prices = showcase_prices();
let window = (
prices.first().expect("non-empty stream").0,
prices.last().expect("non-empty stream").0,
);
h.run(vec![Box::new(VecSource::new(prices))]);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(VecSource::new(showcase_prices()))];
let window = window_of(&sources).expect("non-empty showcase stream");
h.run(sources);
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
let params = space
@@ -416,35 +415,45 @@ fn default_registry() -> Registry {
Registry::open("runs/runs.jsonl")
}
/// `aura sweep`: run the built-in sweep, persist each point's `RunReport` to the
/// registry (the run record, queryable over time — C18), and print each as one
/// JSON line.
fn run_sweep() {
/// `aura sweep [--name <n>]`: run the built-in sweep, persist it as a *family*
/// (related records sharing one `family_id`, C18/C21) via `append_family`, and
/// print each point's record line carrying the assigned id.
fn run_sweep(name: &str) {
let reg = default_registry();
for pt in &sweep_family().points {
if let Err(e) = reg.append(&pt.report) {
let family = sweep_family();
let id = match reg.append_family(name, FamilyKind::Sweep, &sweep_member_reports(&family)) {
Ok(id) => id,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
println!("{}", pt.report.to_json());
};
for pt in &family.points {
println!("{}", serde_json::json!({ "family_id": id, "report": pt.report }));
}
}
/// `aura walkforward`: run a built-in rolling walk-forward over the sample
/// blueprint + a synthetic windowed source. Per window: sweep the built-in grid on
/// the in-sample slice, optimize by total_pips (axis 2 inside axis 3, where
/// aura-cli bridges engine + registry), run the chosen params out-of-sample,
/// persist the OOS RunReport (C18), and print it; finally print the stitched
/// summary line. Deterministic (C1).
fn run_walkforward() {
/// `aura walkforward [--name <n>]`: run a built-in rolling walk-forward over the
/// sample blueprint + a synthetic windowed source. Per window: sweep the built-in
/// grid on the in-sample slice, optimize by total_pips (axis 2 inside axis 3,
/// where aura-cli bridges engine + registry), run the chosen params out-of-sample.
/// Persist the per-window OOS reports as a *family* (C18/C21) via `append_family`,
/// print each carrying the assigned id, then the stitched summary line.
/// Deterministic (C1).
fn run_walkforward(name: &str) {
let reg = default_registry();
let result = walkforward_family();
let id =
match reg.append_family(name, FamilyKind::WalkForward, &walkforward_member_reports(&result))
{
Ok(id) => id,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
};
for w in &result.windows {
if let Err(e) = reg.append(&w.run.oos_report) {
eprintln!("aura: {e}");
std::process::exit(2);
}
println!("{}", w.run.oos_report.to_json());
println!("{}", serde_json::json!({ "family_id": id, "report": w.run.oos_report }));
}
println!("{}", walkforward_summary_json(&result));
}
@@ -454,11 +463,9 @@ fn run_walkforward() {
/// windows. Each window sweeps the built-in grid in-sample, optimizes by
/// total_pips (axis 2), and runs the chosen params out-of-sample.
fn walkforward_family() -> WalkForwardResult {
let prices = walkforward_prices();
let span = (
prices.first().expect("non-empty synthetic stream").0,
prices.last().expect("non-empty synthetic stream").0,
);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(VecSource::new(walkforward_prices()))];
let span = window_of(&sources).expect("non-empty synthetic stream");
let roller = WindowRoller::new(span, 24, 12, 12, RollMode::Rolling)
.expect("built-in walk-forward config fits the 60-bar synthetic span");
walk_forward(roller, |w: WindowBounds| {
@@ -487,7 +494,10 @@ fn sweep_over(from: Timestamp, to: Timestamp) -> SweepFamily {
let mut h = bp
.bootstrap_with_params(point.to_vec())
.expect("grid points are kind-checked against param_space");
h.run(vec![Box::new(walkforward_window_source(from, to))]);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(walkforward_window_source(from, to))];
let window = window_of(&sources).expect("non-empty in-sample window");
h.run(sources);
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
let params = space
@@ -496,7 +506,7 @@ fn sweep_over(from: Timestamp, to: Timestamp) -> SweepFamily {
.map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v)))
.collect();
RunReport {
manifest: sim_optimal_manifest(params, (from, to), 0),
manifest: sim_optimal_manifest(params, window, 0),
metrics: summarize(&equity, &exposure),
}
})
@@ -511,7 +521,10 @@ fn run_oos(params: &[Scalar], from: Timestamp, to: Timestamp) -> (Vec<(Timestamp
let mut h = bp
.bootstrap_with_params(params.to_vec())
.expect("chosen params are kind-checked against param_space");
h.run(vec![Box::new(walkforward_window_source(from, to))]);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(walkforward_window_source(from, to))];
let window = window_of(&sources).expect("non-empty out-of-sample window");
h.run(sources);
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
let named = space
@@ -520,7 +533,7 @@ fn run_oos(params: &[Scalar], from: Timestamp, to: Timestamp) -> (Vec<(Timestamp
.map(|(ps, v)| (ps.name.clone(), scalar_as_param_f64(v)))
.collect();
let report = RunReport {
manifest: sim_optimal_manifest(named, (from, to), 0),
manifest: sim_optimal_manifest(named, window, 0),
metrics: summarize(&equity, &exposure),
};
(equity, report)
@@ -581,6 +594,90 @@ fn walkforward_report() -> String {
out
}
/// The built-in Monte-Carlo family: the sample harness over a fixed (empty) base
/// point, re-seeded across a built-in seed set — each seed a disjoint C1
/// realization of a synthetic price walk (C12 axis 4). Mirrors `sweep_family`,
/// varying the *seed* rather than a tuning param. The seed -> `Source`
/// construction lives inside the per-draw closure (eager-agnostic, #71).
fn mc_family() -> McFamily {
let base_point: Vec<Scalar> = Vec::new();
monte_carlo(&base_point, &[1, 2, 3], |seed, _base| {
let (mut h, rx_eq, rx_ex) = sample_harness();
let spec = SyntheticSpec { start: 1.0, len: 32, step: 1 };
let sources: Vec<Box<dyn aura_engine::Source>> = vec![Box::new(spec.source(seed))];
let window = window_of(&sources).expect("non-empty synthetic stream");
h.run(sources);
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
RunReport {
manifest: sim_optimal_manifest(
vec![
("sma_fast".to_string(), 2.0),
("sma_slow".to_string(), 4.0),
("exposure_scale".to_string(), 0.5),
],
window,
seed,
),
metrics: summarize(&equity, &exposure),
}
})
}
/// Render an `McAggregate` as one canonical JSON line. `McAggregate` itself is not
/// `Serialize` (only its `MetricStats` fields are), so the line is built from the
/// three per-metric stat blocks.
fn mc_aggregate_json(agg: &McAggregate) -> String {
serde_json::json!({
"mc_aggregate": {
"total_pips": agg.total_pips,
"max_drawdown": agg.max_drawdown,
"exposure_sign_flips": agg.exposure_sign_flips,
}
})
.to_string()
}
/// `aura mc [--name <n>]`: run the built-in Monte-Carlo family, persist it to the
/// family store via `append_family` (C18/C21), print each draw's record line
/// (carrying the assigned `family_id`) plus the aggregate line.
fn run_mc(name: &str) {
let reg = default_registry();
let family = mc_family();
let id = match reg.append_family(name, FamilyKind::MonteCarlo, &mc_member_reports(&family)) {
Ok(id) => id,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
};
for draw in &family.draws {
println!(
"{}",
serde_json::json!({ "family_id": id, "seed": draw.seed, "report": draw.report })
);
}
println!("{}", mc_aggregate_json(&family.aggregate));
}
/// Render the built-in Monte-Carlo family as the per-draw `RunReport` lines plus
/// the aggregate line — the `run_mc` shape minus registry persistence (no
/// `family_id`, which is store-assigned). Test helper, mirroring `sweep_report` /
/// `walkforward_report`: it carries the C1-determinism test of the family
/// computation, separate from the store-dependent id.
#[cfg(test)]
fn mc_report() -> String {
let family = mc_family();
let mut out = String::new();
for draw in &family.draws {
out.push_str(&draw.report.to_json());
out.push('\n');
}
out.push_str(&mc_aggregate_json(&family.aggregate));
out.push('\n');
out
}
/// `aura runs list`: print every stored run record, in store (over-time) order.
fn runs_list() {
for report in &load_runs_or_exit() {
@@ -603,6 +700,57 @@ fn runs_rank(metric: &str) {
}
}
/// `aura runs families`: one header line per stored family (id, kind, member
/// count), in first-seen store order.
fn runs_families() {
let reg = default_registry();
let members = match reg.load_family_members() {
Ok(m) => m,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
};
for fam in group_families(members) {
println!(
"{}",
serde_json::json!({ "family_id": fam.id, "kind": fam.kind, "members": fam.members.len() })
);
}
}
/// `aura runs family <id> [rank <metric>]`: list one family's member reports in
/// ordinal order, or best-first by `metric`. An unknown id is an empty family
/// (prints nothing, exit 0, consistent with `runs list` over an empty store); an
/// unknown metric is a usage error (stderr + exit 2).
fn runs_family(id: &str, rank: Option<&str>) {
let reg = default_registry();
let members = match reg.load_family_members() {
Ok(m) => m,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
};
let Some(family) = group_families(members).into_iter().find(|f| f.id == id) else {
return; // unknown family id: empty, exit 0
};
let reports: Vec<RunReport> = family.members.iter().map(|m| m.report.clone()).collect();
let ordered = match rank {
Some(metric) => match rank_by(reports, metric) {
Ok(r) => r,
Err(e) => {
eprintln!("aura: {e}");
std::process::exit(2);
}
},
None => reports,
};
for report in &ordered {
println!("{}", report.to_json());
}
}
/// Load the registry or exit 2 with the error. A missing registry loads empty.
fn load_runs_or_exit() -> Vec<RunReport> {
default_registry().load().unwrap_or_else(|e| {
@@ -706,12 +854,10 @@ fn run_macd() -> RunReport {
.expect("valid macd blueprint");
let mut h = Harness::bootstrap(flat).expect("valid macd harness");
let prices = macd_prices();
let window = (
prices.first().expect("non-empty stream").0,
prices.last().expect("non-empty stream").0,
);
h.run(vec![Box::new(VecSource::new(prices))]);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(VecSource::new(macd_prices()))];
let window = window_of(&sources).expect("non-empty macd stream");
h.run(sources);
let eq_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_eq.try_iter().collect();
let ex_rows: Vec<(Timestamp, Vec<Scalar>)> = rx_ex.try_iter().collect();
@@ -735,7 +881,7 @@ fn run_macd() -> RunReport {
}
const USAGE: &str =
"usage: aura run [--macd] | aura run --real <SYMBOL> [--from <ms>] [--to <ms>] | aura graph | aura sweep | aura walkforward | aura runs list | aura runs rank <metric>";
"usage: aura run [--macd] | aura run --real <SYMBOL> [--from <ms>] [--to <ms>] | aura graph | aura sweep [--name <n>] | aura mc [--name <n>] | aura walkforward [--name <n>] | aura runs list | aura runs rank <metric> | aura runs families | aura runs family <id> [rank <metric>]";
fn main() {
// Collect argv and match the whole vector: every accepted form is exhaustive,
@@ -753,10 +899,17 @@ fn main() {
}
},
["graph"] => print!("{}", render::render_html(&sample_blueprint())),
["sweep"] => run_sweep(),
["walkforward"] => run_walkforward(),
["sweep"] => run_sweep("sweep"),
["sweep", "--name", n] => run_sweep(n),
["walkforward"] => run_walkforward("walkforward"),
["walkforward", "--name", n] => run_walkforward(n),
["mc"] => run_mc("mc"),
["mc", "--name", n] => run_mc(n),
["runs", "list"] => runs_list(),
["runs", "rank", metric] => runs_rank(metric),
["runs", "families"] => runs_families(),
["runs", "family", id] => runs_family(id, None),
["runs", "family", id, "rank", metric] => runs_family(id, Some(metric)),
["--help"] | ["-h"] => println!("{USAGE}"),
_ => {
eprintln!("aura: {USAGE}");
@@ -901,6 +1054,60 @@ mod tests {
assert_eq!(sweep_report(), sweep_report());
}
#[test]
fn mc_report_is_deterministic_and_one_line_per_seed() {
// C1 at the CLI edge: the family computation renders bit-identically.
assert_eq!(mc_report(), mc_report());
let out = mc_report();
let lines: Vec<&str> = out.lines().collect();
// three seeds -> three member lines + one aggregate line
assert_eq!(lines.len(), 4, "expected 3 members + 1 aggregate: {out}");
for line in &lines[..3] {
assert!(line.contains(r#""total_pips":"#), "member line missing metrics: {line}");
}
assert!(lines[3].contains(r#""mc_aggregate":"#), "missing aggregate line: {}", lines[3]);
}
#[test]
fn cli_families_persist_and_round_trip_per_kind() {
use aura_registry::{
group_families, mc_member_reports, sweep_member_reports, walkforward_member_reports,
FamilyKind, Registry,
};
let dir = std::env::temp_dir().join(format!("aura-cli-fam-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).expect("temp dir");
let reg = Registry::open(dir.join("runs.jsonl"));
// the exact persist chain `run_sweep`/`run_mc`/`run_walkforward` use, against
// a fresh temp store (the run_* fns themselves bind `default_registry()`):
// engine family -> per-kind extractor -> append_family.
let sid = reg
.append_family("sweep", FamilyKind::Sweep, &sweep_member_reports(&sweep_family()))
.expect("sweep family");
let mid = reg
.append_family("mc", FamilyKind::MonteCarlo, &mc_member_reports(&mc_family()))
.expect("mc family");
let wid = reg
.append_family(
"walkforward",
FamilyKind::WalkForward,
&walkforward_member_reports(&walkforward_family()),
)
.expect("walkforward family");
assert_eq!((sid.as_str(), mid.as_str(), wid.as_str()), ("sweep-0", "mc-0", "walkforward-0"));
let families = group_families(reg.load_family_members().expect("load"));
assert_eq!(families.len(), 3);
let by_id = |id: &str| families.iter().find(|f| f.id == id).expect("family present");
assert_eq!(by_id("sweep-0").kind, FamilyKind::Sweep);
assert_eq!(by_id("mc-0").kind, FamilyKind::MonteCarlo);
assert_eq!(by_id("mc-0").members.len(), 3); // 3 seeds
assert_eq!(by_id("walkforward-0").kind, FamilyKind::WalkForward);
assert_eq!(by_id("walkforward-0").members.len(), 3); // 3 windows
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn run_macd_compiles_from_nested_composite_and_is_deterministic() {
// the MACD strategy authors a nested EMA-of-EMA composite, compiles it to a
crates/aura-cli/tests/cli_run.rs
+77 -17
View File
@@ -199,7 +199,7 @@ fn graph_emits_self_contained_html_viewer() {
}
#[test]
fn sweep_prints_four_json_lines_and_exits_zero() {
fn sweep_prints_four_family_json_lines_and_exits_zero() {
let cwd = temp_cwd("sweep4");
let out = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn aura sweep");
assert!(out.status.success(), "exit status: {:?}", out.status);
@@ -207,9 +207,11 @@ fn sweep_prints_four_json_lines_and_exits_zero() {
let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
assert_eq!(stdout.lines().count(), 4, "stdout was: {stdout:?}");
let first = stdout.lines().next().unwrap();
// each line is a full RunReport; commit is the real git HEAD (volatile), so
// pin the first odometer point's manifest params, not the commit value.
assert!(first.starts_with("{\"manifest\":{\"commit\":\""), "got: {stdout}");
// each line is now a family member: the assigned `family_id` plus the nested
// RunReport (C18/C21 lineage). The first run assigns `sweep-0`; the nested
// report is serialized via `serde_json`, so the manifest's fields lead with
// `broker` (the commit value is the volatile git HEAD, pinned by params below).
assert!(first.starts_with("{\"family_id\":\"sweep-0\",\"report\":{\"manifest\":{"), "got: {stdout}");
assert!(
first.contains("\"params\":[[\"signals.trend.fast.length\",2.0],[\"signals.trend.slow.length\",4.0],[\"signals.momentum.fast.length\",2.0],[\"signals.momentum.slow.length\",4.0],[\"signals.momentum.signal.length\",3.0],[\"signals.blend.weights[0]\",1.0],[\"signals.blend.weights[1]\",1.0],[\"exposure.scale\",0.5]]"),
"got: {stdout}"
@@ -224,27 +226,71 @@ fn sweep_with_trailing_token_is_strict_and_exits_two() {
assert_eq!(out.status.code(), Some(2), "exit status: {:?}", out.status);
}
/// Property: `aura mc` runs the built-in Monte-Carlo family end-to-end through the
/// binary — it prints one member line per seed (carrying the assigned `family_id`)
/// plus one `mc_aggregate` line, exits 0, and persists the draws as a discoverable
/// `MonteCarlo` family (C12 axis 4 / C18/C21). Driven through the built binary so
/// it pins the observable CLI contract, not just the in-crate render helper.
#[test]
fn runs_list_and_rank_across_invocations() {
fn mc_runs_persists_a_monte_carlo_family_and_lists_it() {
let cwd = temp_cwd("mc-family");
let out = Command::new(BIN).arg("mc").current_dir(&cwd).output().expect("spawn aura mc");
assert!(out.status.success(), "mc exit: {:?}", out.status);
let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
let lines: Vec<&str> = stdout.lines().collect();
// three built-in seeds -> three member lines + one aggregate line.
assert_eq!(lines.len(), 4, "mc stdout: {stdout:?}");
for line in &lines[..3] {
assert!(line.contains("\"family_id\":\"mc-0\""), "member missing family_id: {line}");
assert!(line.contains("\"seed\":"), "member missing seed: {line}");
}
assert!(lines[3].contains("\"mc_aggregate\":"), "missing aggregate line: {}", lines[3]);
// the run persisted the draws as a discoverable MonteCarlo family.
let fams = Command::new(BIN).args(["runs", "families"]).current_dir(&cwd).output().expect("spawn families");
assert!(fams.status.success(), "families exit: {:?}", fams.status);
let fams_out = String::from_utf8(fams.stdout).expect("utf-8");
assert_eq!(fams_out.lines().count(), 1, "families: {fams_out:?}");
assert!(fams_out.contains("\"family_id\":\"mc-0\""), "families: {fams_out:?}");
assert!(fams_out.contains("\"kind\":\"MonteCarlo\""), "families: {fams_out:?}");
assert!(fams_out.contains("\"members\":3"), "families: {fams_out:?}");
let _ = std::fs::remove_dir_all(&cwd);
}
#[test]
fn runs_families_list_and_per_family_rank_across_invocations() {
let cwd = temp_cwd("runs-flow");
// two sweeps accumulate 8 records over time
// two sweeps accumulate two families (sweep-0, sweep-1) over time, each a
// related set of 4 records (C18/C21 lineage), in the sibling family store.
for _ in 0..2 {
let out = Command::new(BIN).arg("sweep").current_dir(&cwd).output().expect("spawn sweep");
assert!(out.status.success(), "sweep exit: {:?}", out.status);
}
// `runs list` shows all 8 stored records
let list = Command::new(BIN).args(["runs", "list"]).current_dir(&cwd).output().expect("spawn list");
assert!(list.status.success(), "list exit: {:?}", list.status);
let list_out = String::from_utf8(list.stdout).expect("utf-8");
assert_eq!(list_out.lines().count(), 8, "list: {list_out:?}");
assert!(list_out.lines().all(|l| l.starts_with("{\"manifest\":{")), "list shape: {list_out:?}");
// `runs families` shows both stored families in first-seen order, each with
// its kind + member count.
let fams = Command::new(BIN).args(["runs", "families"]).current_dir(&cwd).output().expect("spawn families");
assert!(fams.status.success(), "families exit: {:?}", fams.status);
let fams_out = String::from_utf8(fams.stdout).expect("utf-8");
let fam_lines: Vec<&str> = fams_out.lines().collect();
assert_eq!(fam_lines.len(), 2, "families: {fams_out:?}");
assert!(fam_lines[0].contains("\"family_id\":\"sweep-0\""), "fam0: {fams_out:?}");
assert!(fam_lines[1].contains("\"family_id\":\"sweep-1\""), "fam1: {fams_out:?}");
assert!(fam_lines.iter().all(|l| l.contains("\"members\":4")), "members: {fams_out:?}");
// `runs rank total_pips`: 8 lines, highest total_pips first
let rank = Command::new(BIN).args(["runs", "rank", "total_pips"]).current_dir(&cwd).output().expect("spawn rank");
// `runs family sweep-0 rank total_pips`: the family's 4 members, highest
// total_pips first.
let rank = Command::new(BIN)
.args(["runs", "family", "sweep-0", "rank", "total_pips"])
.current_dir(&cwd)
.output()
.expect("spawn rank");
assert!(rank.status.success(), "rank exit: {:?}", rank.status);
let rank_out = String::from_utf8(rank.stdout).expect("utf-8");
assert_eq!(rank_out.lines().count(), 8, "rank: {rank_out:?}");
assert_eq!(rank_out.lines().count(), 4, "rank: {rank_out:?}");
assert!(rank_out.lines().all(|l| l.starts_with("{\"manifest\":{")), "rank shape: {rank_out:?}");
let pips: Vec<f64> = rank_out
.lines()
.map(|l| {
@@ -257,10 +303,24 @@ fn runs_list_and_rank_across_invocations() {
.collect();
assert!(pips.windows(2).all(|w| w[0] >= w[1]), "rank not descending: {pips:?}");
// unknown metric is a usage error
let bogus = Command::new(BIN).args(["runs", "rank", "bogus"]).current_dir(&cwd).output().expect("spawn bogus");
// an unknown metric is a usage error
let bogus = Command::new(BIN)
.args(["runs", "family", "sweep-0", "rank", "bogus"])
.current_dir(&cwd)
.output()
.expect("spawn bogus");
assert_eq!(bogus.status.code(), Some(2), "bogus exit: {:?}", bogus.status);
// an unknown family id is an empty family: zero lines, exit 0 (consistent
// with `runs list` over an empty store).
let unknown = Command::new(BIN)
.args(["runs", "family", "nope-9"])
.current_dir(&cwd)
.output()
.expect("spawn unknown");
assert!(unknown.status.success(), "unknown family exit: {:?}", unknown.status);
assert_eq!(unknown.stdout.len(), 0, "unknown family stdout: {:?}", unknown.stdout);
let _ = std::fs::remove_dir_all(&cwd);
}