feat(aura-registry): optimize — argmax over a SweepFamily by a named metric
Ships C12 orchestration axis 2 (optimize): pick the single best point of a sweep family by a named RunMetrics field. The GREEN half of the RED-first iteration whose executable-spec landed in the previous commit. optimize(&SweepFamily, metric) -> Result<SweepPoint, RegistryError> returns the whole winning SweepPoint — its params coordinate AND its RunReport — because the caller needs the params that won, not just the metric. "Best" is fixed per metric (total_pips higher-is-better; max_drawdown / exposure_sign_flips lower-is-better); ties resolve to the earliest enumeration-order point (the family is in odometer order, and only a strictly-better later point displaces the incumbent). Single source of truth for "best", as #67 required ("reuse rank_by; do not fork"): rank_by's per-metric direction is extracted into a private `metric_cmp` helper that resolves the boundary metric *name* to a closed `enum Metric` once and returns one comparator closure. Both rank_by (sort by it) and optimize (argmax via `reduce`, strictly-better-displaces) now call it, so the per-metric direction lives in exactly one match. rank_by is behaviour-preserving — its existing tests pass unchanged. The sweep executor is untouched. No caller-supplied `direction`: one definition of best per metric, consistent with the shipped rank_by invariant (a caller direction would admit incoherent asks like the worst max_drawdown). An unknown metric is UnknownMetric. A SweepFamily is non-empty by construction (EmptyAxis is rejected upstream), so the argmax always yields a winner. CLI surface: the "pick the best" view is already served by `aura runs rank <metric>`, whose first line is the optimum over the persisted registry; optimize() is the new *programmatic* axis-2 entry, for World programs that hold a live SweepFamily and need the winning params back in-process — which the registry/RunReport path cannot give. No redundant CLI command added. closes #67
This commit is contained in:
@@ -5,12 +5,13 @@
|
|||||||
//! serde_json; display is the caller's concern (the CLI prints via
|
//! serde_json; display is the caller's concern (the CLI prints via
|
||||||
//! `RunReport::to_json`).
|
//! `RunReport::to_json`).
|
||||||
|
|
||||||
|
use std::cmp::Ordering;
|
||||||
use std::fmt;
|
use std::fmt;
|
||||||
use std::fs;
|
use std::fs;
|
||||||
use std::io::Write;
|
use std::io::Write;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
use aura_engine::RunReport;
|
use aura_engine::{RunReport, SweepFamily, SweepPoint};
|
||||||
|
|
||||||
/// An append-only run registry over a JSONL file: one serde_json line per
|
/// An append-only run registry over a JSONL file: one serde_json line per
|
||||||
/// `RunReport`.
|
/// `RunReport`.
|
||||||
@@ -59,28 +60,75 @@ impl Registry {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Which metric a best-first comparison keys on. Resolves a metric *name* (a
|
||||||
|
/// string at the API boundary) to a closed kind once, so the per-comparison hot
|
||||||
|
/// path branches on this rather than re-parsing the name.
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
enum Metric {
|
||||||
|
TotalPips,
|
||||||
|
MaxDrawdown,
|
||||||
|
ExposureSignFlips,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The per-metric **best-first ordering** of two reports: `Less` means `a` is
|
||||||
|
/// better than `b`. "Best" is fixed by each metric's meaning: `total_pips`
|
||||||
|
/// higher-is-better; `max_drawdown` and `exposure_sign_flips` lower-is-better.
|
||||||
|
/// `f64` keys use `partial_cmp` (total, since metrics are finite by
|
||||||
|
/// construction). An unknown metric name is a `RegistryError::UnknownMetric`.
|
||||||
|
///
|
||||||
|
/// The single source of truth for "best" — both [`rank_by`] (sort by it) and
|
||||||
|
/// [`optimize`] (argmax by it) call this, so the per-metric direction lives in
|
||||||
|
/// exactly one place. The metric name is matched once here; the returned closure
|
||||||
|
/// carries the resolved `Metric`, so per-comparison work is just the key compare.
|
||||||
|
fn metric_cmp(
|
||||||
|
metric: &str,
|
||||||
|
) -> Result<impl Fn(&RunReport, &RunReport) -> Ordering, RegistryError> {
|
||||||
|
let metric = match metric {
|
||||||
|
"total_pips" => Metric::TotalPips,
|
||||||
|
"max_drawdown" => Metric::MaxDrawdown,
|
||||||
|
"exposure_sign_flips" => Metric::ExposureSignFlips,
|
||||||
|
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
||||||
|
};
|
||||||
|
Ok(move |a: &RunReport, b: &RunReport| match metric {
|
||||||
|
// higher-is-better -> better when greater
|
||||||
|
Metric::TotalPips => b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap(),
|
||||||
|
// lower-is-better -> better when smaller
|
||||||
|
Metric::MaxDrawdown => a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap(),
|
||||||
|
Metric::ExposureSignFlips => {
|
||||||
|
a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
/// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed
|
/// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed
|
||||||
/// by each metric's meaning: `total_pips` higher-is-better (descending);
|
/// by each metric's meaning (see [`metric_cmp`]). A stable sort keeps file
|
||||||
/// `max_drawdown` and `exposure_sign_flips` lower-is-better (ascending). A stable
|
/// (insertion) order among ties. An unknown metric name is a
|
||||||
/// sort keeps file (insertion) order among ties; `f64` keys use `partial_cmp`
|
|
||||||
/// (total, since metrics are finite by construction). An unknown metric name is a
|
|
||||||
/// `RegistryError::UnknownMetric`.
|
/// `RegistryError::UnknownMetric`.
|
||||||
pub fn rank_by(mut reports: Vec<RunReport>, metric: &str) -> Result<Vec<RunReport>, RegistryError> {
|
pub fn rank_by(mut reports: Vec<RunReport>, metric: &str) -> Result<Vec<RunReport>, RegistryError> {
|
||||||
match metric {
|
let cmp = metric_cmp(metric)?;
|
||||||
"total_pips" => {
|
reports.sort_by(|a, b| cmp(a, b));
|
||||||
reports.sort_by(|a, b| b.metrics.total_pips.partial_cmp(&a.metrics.total_pips).unwrap())
|
|
||||||
}
|
|
||||||
"max_drawdown" => {
|
|
||||||
reports.sort_by(|a, b| a.metrics.max_drawdown.partial_cmp(&b.metrics.max_drawdown).unwrap())
|
|
||||||
}
|
|
||||||
"exposure_sign_flips" => {
|
|
||||||
reports.sort_by(|a, b| a.metrics.exposure_sign_flips.cmp(&b.metrics.exposure_sign_flips))
|
|
||||||
}
|
|
||||||
other => return Err(RegistryError::UnknownMetric(other.to_string())),
|
|
||||||
}
|
|
||||||
Ok(reports)
|
Ok(reports)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// `rank_by`'s argmax: return the single best `SweepPoint` of a `SweepFamily`
|
||||||
|
/// under a named metric, by the same fixed per-metric sense of best (see
|
||||||
|
/// [`metric_cmp`]). The whole point is carried — its `params` coordinate AND its
|
||||||
|
/// `RunReport` — because the caller needs the params that won. Ties on the metric
|
||||||
|
/// resolve to the **earliest** enumeration-order point (the family is already in
|
||||||
|
/// odometer order; only a strictly-better later point displaces the incumbent).
|
||||||
|
/// An unknown metric is a `RegistryError::UnknownMetric`. A `SweepFamily` is
|
||||||
|
/// non-empty by construction (a sweep over a zero-point grid is rejected upstream
|
||||||
|
/// by `EmptyAxis`), so `reduce` always yields a winner.
|
||||||
|
pub fn optimize(family: &SweepFamily, metric: &str) -> Result<SweepPoint, RegistryError> {
|
||||||
|
let cmp = metric_cmp(metric)?;
|
||||||
|
let winner = family
|
||||||
|
.points
|
||||||
|
.iter()
|
||||||
|
.reduce(|best, p| if cmp(&p.report, &best.report) == Ordering::Less { p } else { best })
|
||||||
|
.expect("a SweepFamily is non-empty by construction (EmptyAxis is rejected upstream)");
|
||||||
|
Ok(winner.clone())
|
||||||
|
}
|
||||||
|
|
||||||
/// What can go wrong reading or ranking the registry.
|
/// What can go wrong reading or ranking the registry.
|
||||||
#[derive(Debug)]
|
#[derive(Debug)]
|
||||||
pub enum RegistryError {
|
pub enum RegistryError {
|
||||||
|
|||||||
Reference in New Issue
Block a user