feat(aura-registry): optimize — argmax over a SweepFamily by a named metric
Ships C12 orchestration axis 2 (optimize): pick the single best point of a sweep family by a named RunMetrics field. The GREEN half of the RED-first iteration whose executable-spec landed in the previous commit. optimize(&SweepFamily, metric) -> Result<SweepPoint, RegistryError> returns the whole winning SweepPoint — its params coordinate AND its RunReport — because the caller needs the params that won, not just the metric. "Best" is fixed per metric (total_pips higher-is-better; max_drawdown / exposure_sign_flips lower-is-better); ties resolve to the earliest enumeration-order point (the family is in odometer order, and only a strictly-better later point displaces the incumbent). Single source of truth for "best", as #67 required ("reuse rank_by; do not fork"): rank_by's per-metric direction is extracted into a private `metric_cmp` helper that resolves the boundary metric *name* to a closed `enum Metric` once and returns one comparator closure. Both rank_by (sort by it) and optimize (argmax via `reduce`, strictly-better-displaces) now call it, so the per-metric direction lives in exactly one match. rank_by is behaviour-preserving — its existing tests pass unchanged. The sweep executor is untouched. No caller-supplied `direction`: one definition of best per metric, consistent with the shipped rank_by invariant (a caller direction would admit incoherent asks like the worst max_drawdown). An unknown metric is UnknownMetric. A SweepFamily is non-empty by construction (EmptyAxis is rejected upstream), so the argmax always yields a winner. CLI surface: the "pick the best" view is already served by `aura runs rank <metric>`, whose first line is the optimum over the persisted registry; optimize() is the new *programmatic* axis-2 entry, for World programs that hold a live SweepFamily and need the winning params back in-process — which the registry/RunReport path cannot give. No redundant CLI command added. closes #67
This commit is contained in:
@@ -5,12 +5,13 @@
|
||||
//! serde_json; display is the caller's concern (the CLI prints via
|
||||
//! `RunReport::to_json`).
|
||||
|
||||
use std::cmp::Ordering;
|
||||
use std::fmt;
|
||||
use std::fs;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use aura_engine::RunReport;
|
||||
use aura_engine::{RunReport, SweepFamily, SweepPoint};
|
||||
|
||||
/// An append-only run registry over a JSONL file: one serde_json line per
|
||||
/// `RunReport`.
|
||||
@@ -59,28 +60,75 @@ impl Registry {
|
||||
}
|
||||
}
|
||||
|
||||
/// Which metric a best-first comparison keys on. Resolves a metric *name* (a
|
||||
/// string at the API boundary) to a closed kind once, so the per-comparison hot
|
||||
/// path branches on this rather than re-parsing the name.
|
||||
#[derive(Clone, Copy)]
|
||||
enum Metric {
|
||||
TotalPips,
|
||||
MaxDrawdown,
|
||||
ExposureSignFlips,
|
||||
}
|
||||
|
||||
/// The per-metric **best-first ordering** of two reports: `Less` means `a` is
|
||||
/// better than `b`. "Best" is fixed by each metric's meaning: `total_pips`
|
||||
/// higher-is-better; `max_drawdown` and `exposure_sign_flips` lower-is-better.
|
||||
/// `f64` keys use `partial_cmp` (total, since metrics are finite by
|
||||
/// construction). An unknown metric name is a `RegistryError::UnknownMetric`.
|
||||
///
|
||||
/// The single source of truth for "best" — both [`rank_by`] (sort by it) and
|
||||
/// [`optimize`] (argmax by it) call this, so the per-metric direction lives in
|
||||
/// exactly one place. The metric name is matched once here; the returned closure
|
||||
/// carries the resolved `Metric`, so per-comparison work is just the key compare.
|
||||
fn metric_cmp(
|
||||
metric: &str,
|
||||
) -> Result<impl Fn(&RunReport, &RunReport) -> Ordering, RegistryError> {
|
||||
let metric = match metric {
|
||||
"total_pips" => Metric::TotalPips,
|
||||
"max_drawdown" => Metric::MaxDrawdown,
|
||||
"exposure_sign_flips" => Metric::ExposureSignFlips,
|
||||
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
||||
};
|
||||
Ok(move |a: &RunReport, b: &RunReport| match metric {
|
||||
// higher-is-better -> better when greater
|
||||
Metric::TotalPips => b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap(),
|
||||
// lower-is-better -> better when smaller
|
||||
Metric::MaxDrawdown => a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap(),
|
||||
Metric::ExposureSignFlips => {
|
||||
a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed
|
||||
/// by each metric's meaning: `total_pips` higher-is-better (descending);
|
||||
/// `max_drawdown` and `exposure_sign_flips` lower-is-better (ascending). A stable
|
||||
/// sort keeps file (insertion) order among ties; `f64` keys use `partial_cmp`
|
||||
/// (total, since metrics are finite by construction). An unknown metric name is a
|
||||
/// by each metric's meaning (see [`metric_cmp`]). A stable sort keeps file
|
||||
/// (insertion) order among ties. An unknown metric name is a
|
||||
/// `RegistryError::UnknownMetric`.
|
||||
pub fn rank_by(mut reports: Vec<RunReport>, metric: &str) -> Result<Vec<RunReport>, RegistryError> {
|
||||
match metric {
|
||||
"total_pips" => {
|
||||
reports.sort_by(|a, b| b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap())
|
||||
}
|
||||
"max_drawdown" => {
|
||||
reports.sort_by(|a, b| a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap())
|
||||
}
|
||||
"exposure_sign_flips" => {
|
||||
reports.sort_by(|a, b| a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips))
|
||||
}
|
||||
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
||||
}
|
||||
let cmp = metric_cmp(metric)?;
|
||||
reports.sort_by(|a, b| cmp(a, b));
|
||||
Ok(reports)
|
||||
}
|
||||
|
||||
/// `rank_by`'s argmax: return the single best `SweepPoint` of a `SweepFamily`
|
||||
/// under a named metric, by the same fixed per-metric sense of best (see
|
||||
/// [`metric_cmp`]). The whole point is carried — its `params` coordinate AND its
|
||||
/// `RunReport` — because the caller needs the params that won. Ties on the metric
|
||||
/// resolve to the **earliest** enumeration-order point (the family is already in
|
||||
/// odometer order; only a strictly-better later point displaces the incumbent).
|
||||
/// An unknown metric is a `RegistryError::UnknownMetric`. A `SweepFamily` is
|
||||
/// non-empty by construction (a sweep over a zero-point grid is rejected upstream
|
||||
/// by `EmptyAxis`), so `reduce` always yields a winner.
|
||||
pub fn optimize(family: &SweepFamily, metric: &str) -> Result<SweepPoint, RegistryError> {
|
||||
let cmp = metric_cmp(metric)?;
|
||||
let winner = family
|
||||
.points
|
||||
.iter()
|
||||
.reduce(|best, p| if cmp(&p.report, &best.report) == Ordering::Less { p } else { best })
|
||||
.expect("a SweepFamily is non-empty by construction (EmptyAxis is rejected upstream)");
|
||||
Ok(winner.clone())
|
||||
}
|
||||
|
||||
/// What can go wrong reading or ranking the registry.
|
||||
#[derive(Debug)]
|
||||
pub enum RegistryError {
|
||||
|
||||
Reference in New Issue
Block a user