Brummel/Aura
1
0
Fork 0
Code Issues 31 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: per-instrument pip metadata channel for the sim-optimal broker

Browse Source 
The data-server symbol stream carries only price bars, no instrument metadata.
So SimBroker was constructed with one global pip_size literal (0.0001, correct
only for 5-decimal FX), and `aura run --real <symbol>` emitted wrong-unit pip
equity for any non-FX instrument (AAPL.US, BTCUSD off by orders of magnitude).

Add the missing channel to *specify* per-instrument pip:

- aura-ingest: `InstrumentSpec { pip_size }` + `instrument_spec(symbol)` — a
  typed, Rust-authored vetted lookup (NOT Aura.toml; the later Aura.toml schema
  can subsume it). Seeded GER40/FRA40 = 1.0 (index points), EURUSD/GBPUSD/USDCAD
  = 0.0001 (5-dp FX majors); None for an un-specced symbol.
- aura-cli: `sample_harness` and `sim_optimal_manifest` gain a `pip_size`
  parameter; `run_sample_real` looks the pip up by symbol BEFORE any data access
  and refuses (exit 2) an un-specced instrument rather than guessing — honest by
  construction. The looked-up pip reaches both the broker divisor and the
  manifest broker label (`format!`). Every synthetic caller passes the unchanged
  0.0001, now named `SYNTHETIC_PIP_SIZE`. SimBroker (aura-std) is untouched.

Scope: narrowed to the CLI real path — the actual bug. The runnable GER40
examples already bake the correct 1.0 and are NOT buggy; centralizing them
through the lookup is deferred to #98.

Decisions (user-directed, recorded on #22): metadata channel = typed
InstrumentSpec at the source edge; un-specced real symbol refuses; seed =
GER40/FRA40 + FX majors; example refactor deferred. The spec went through the
grounding-check gate; the auto-sign panel surfaced the refusal-behaviour and
scope as genuine design decisions, escalated and resolved by the user.

Tests: instrument_spec unit; manifest pip rendering (1.0 -> "1", 0.0001
unchanged); non-gated CLI refusal (exit 2, no stdout leak, vetted symbol clears
the lookup); gated GER40 binary-boundary honesty (pip_size=1 in the emitted
manifest, deterministic). The pre-existing run_sample_real determinism test was
retargeted AAPL.US -> GER40 (AAPL.US is un-specced and would now refuse at the
lookup); the AAPL.US bounded-window streaming property still lives in
aura-ingest/tests/streaming_seam.rs (literal source, no spec lookup).

Verified: cargo build --workspace, clippy --all-targets -D warnings, and
cargo test --workspace all green; the gated GER40 path ran with local data.

closes #22
This commit is contained in:
Brummel
2026-06-18 19:36:43 +02:00
parent 47bcb29899
commit 26bec6d64a
3 changed files with 248 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/aura-cli/src/main.rs
+94 -30
View File
@@ -27,6 +27,14 @@ use aura_registry::{
use aura_std::{Ema, Exposure, LinComb, Recorder, SimBroker, Sma, Sub};
use std::sync::mpsc::{self, Receiver};
/// The pip size the built-in *synthetic* harnesses run at: a 5-decimal FX major
/// (`EURUSD`-shaped). The synthetic streams carry no instrument, so there is no
/// `instrument_spec` to thread; a single named source keeps the broker's divisor
/// (`sample_harness` / `SimBroker::builder`) and the recorded broker label
/// (`sim_optimal_manifest`) in lockstep, so they cannot silently drift apart.
/// The real path threads the looked-up `spec.pip_size` instead.
const SYNTHETIC_PIP_SIZE: f64 = 0.0001;
/// The built-in synthetic price stream: rises through t=4 then reverses, so the
/// demo trace carries one exposure sign flip and a real drawdown (C22 populated
/// trace). Deterministic and fixed (C1).
@@ -70,7 +78,7 @@ fn showcase_prices() -> Vec<(Timestamp, Scalar)> {
// The harness-plus-two-drained-sink-receivers tuple has exactly one call site
// (`run_sample`); a named type would be speculative abstraction this cycle.
#[allow(clippy::type_complexity)]
fn sample_harness() -> (
fn sample_harness(pip_size: f64) -> (
Harness,
Receiver<(Timestamp, Vec<Scalar>)>,
Receiver<(Timestamp, Vec<Scalar>)>,
@@ -88,7 +96,7 @@ fn sample_harness() -> (
Box::new(Sma::new(4)), // 1 slow SMA
Box::new(Sub::new()), // 2 spread
Box::new(Exposure::new(0.5)), // 3 exposure
Box::new(SimBroker::new(0.0001)), // 4 sim-optimal broker
Box::new(SimBroker::new(pip_size)), // 4 sim-optimal broker
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, tx_eq)), // 5 equity sink
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, tx_ex)), // 6 exposure sink
],
@@ -97,7 +105,7 @@ fn sample_harness() -> (
Sma::builder().schema().clone(),
Sub::builder().schema().clone(),
Exposure::builder().schema().clone(),
SimBroker::builder(0.0001).schema().clone(),
SimBroker::builder(pip_size).schema().clone(),
f64_recorder_sig(),
f64_recorder_sig(),
],
@@ -125,12 +133,13 @@ fn sample_harness() -> (
/// Build the sim-optimal `RunManifest`: the engine-external descriptor fields
/// (commit, seed-free synthetic run, broker label) are constant across the CLI's
/// built-in harnesses — only `params` and `window` vary. Centralizing the broker
/// label keeps it a single source (and a single edit point for #22's per-asset
/// pip work).
/// label keeps it a single source (and a single edit point for per-asset pip
/// work): the `pip_size` it renders is the one its caller ran the broker at.
fn sim_optimal_manifest(
params: Vec<(String, Scalar)>,
window: (Timestamp, Timestamp),
seed: u64,
pip_size: f64,
) -> RunManifest {
// Typed params pass straight through: the manifest carries self-describing
// Scalars, so a length stays `i64` and a scale `f64` in the record.
@@ -139,7 +148,7 @@ fn sim_optimal_manifest(
params,
window,
seed,
broker: "sim-optimal(pip_size=0.0001)".to_string(),
broker: format!("sim-optimal(pip_size={pip_size})"),
}
}
@@ -147,7 +156,7 @@ fn sim_optimal_manifest(
/// `f64_field` → `summarize` → pair with a `RunManifest`). Pure and deterministic
/// (C1): the same build yields the same report.
fn run_sample() -> RunReport {
let (mut h, rx_eq, rx_ex) = sample_harness();
let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE);
let sources: Vec<Box<dyn aura_engine::Source>> =
vec![Box::new(VecSource::new(synthetic_prices()))];
let window = window_of(&sources).expect("non-empty synthetic stream");
@@ -168,6 +177,7 @@ fn run_sample() -> RunReport {
],
window,
0,
SYNTHETIC_PIP_SIZE,
),
metrics,
}
@@ -181,7 +191,16 @@ fn run_sample() -> RunReport {
/// A no-local-data condition (unknown symbol, or a window overlapping no file / no
/// bars) is a usage error: stderr + exit(2), not a panic.
fn run_sample_real(symbol: &str, from_ms: Option<i64>, to_ms: Option<i64>) -> RunReport {
let (mut h, rx_eq, rx_ex) = sample_harness();
// Per-instrument pip: look up BEFORE any data access, so an un-specced symbol
// refuses without touching local data. Honest by construction.
let spec = match aura_ingest::instrument_spec(symbol) {
Some(s) => s,
None => {
eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' — refusing to run a real instrument with a guessed pip (add it to the instrument table)");
std::process::exit(2);
}
};
let (mut h, rx_eq, rx_ex) = sample_harness(spec.pip_size);
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
let no_data = || -> ! {
eprintln!(
@@ -231,6 +250,7 @@ fn run_sample_real(symbol: &str, from_ms: Option<i64>, to_ms: Option<i64>) -> Ru
],
window,
0,
spec.pip_size,
),
metrics,
}
@@ -323,7 +343,7 @@ fn sample_blueprint_with_sinks() -> (
let mut g = GraphBuilder::new("sample");
let sig = g.add(signals("signals"));
let exposure = g.add(Exposure::builder());
let broker = g.add(SimBroker::builder(0.0001));
let broker = g.add(SimBroker::builder(SYNTHETIC_PIP_SIZE));
let eq = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq));
let ex = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex));
let price = g.source_role("price", ScalarKind::F64);
@@ -376,7 +396,7 @@ fn sweep_family() -> SweepFamily {
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
RunReport {
manifest: sim_optimal_manifest(zip_params(&space, point), window, 0),
manifest: sim_optimal_manifest(zip_params(&space, point), window, 0, SYNTHETIC_PIP_SIZE),
metrics: summarize(&equity, &exposure),
}
})
@@ -495,7 +515,7 @@ fn sweep_over(from: Timestamp, to: Timestamp) -> SweepFamily {
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
RunReport {
manifest: sim_optimal_manifest(zip_params(&space, point), window, 0),
manifest: sim_optimal_manifest(zip_params(&space, point), window, 0, SYNTHETIC_PIP_SIZE),
metrics: summarize(&equity, &exposure),
}
})
@@ -517,7 +537,7 @@ fn run_oos(params: &[Cell], from: Timestamp, to: Timestamp) -> (Vec<(Timestamp,
let equity = f64_field(&rx_eq.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&rx_ex.try_iter().collect::<Vec<_>>(), 0);
let report = RunReport {
manifest: sim_optimal_manifest(zip_params(&space, params), window, 0),
manifest: sim_optimal_manifest(zip_params(&space, params), window, 0, SYNTHETIC_PIP_SIZE),
metrics: summarize(&equity, &exposure),
};
(equity, report)
@@ -586,7 +606,7 @@ fn walkforward_report() -> String {
fn mc_family() -> McFamily {
let base_point: Vec<Scalar> = Vec::new();
monte_carlo(&base_point, &[1, 2, 3], |seed, _base| {
let (mut h, rx_eq, rx_ex) = sample_harness();
let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE);
let spec = SyntheticSpec { start: 1.0, len: 32, step: 1 };
let sources: Vec<Box<dyn aura_engine::Source>> = vec![Box::new(spec.source(seed))];
let window = window_of(&sources).expect("non-empty synthetic stream");
@@ -602,6 +622,7 @@ fn mc_family() -> McFamily {
],
window,
seed,
SYNTHETIC_PIP_SIZE,
),
metrics: summarize(&equity, &exposure),
}
@@ -783,7 +804,7 @@ fn macd_strategy_blueprint(
let mut g = GraphBuilder::new("macd_strategy");
let macd_node = g.add(macd("macd"));
let exposure = g.add(Exposure::builder());
let broker = g.add(SimBroker::builder(0.0001));
let broker = g.add(SimBroker::builder(SYNTHETIC_PIP_SIZE));
let eq = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq));
let ex = g.add(Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_ex));
let price = g.source_role("price", ScalarKind::F64);
@@ -859,6 +880,7 @@ fn run_macd() -> RunReport {
],
window,
0,
SYNTHETIC_PIP_SIZE,
),
metrics,
}
@@ -906,6 +928,15 @@ fn main() {
mod tests {
use super::*;
// The vetted GER40 real-data window: the whole of September 2024 (UTC,
// inclusive), the same calendar month the gated ingest `ger40_breakout_real`
// test drives. Expressed in Unix-ms (`run_sample_real`'s window currency):
// `[2024-09-01T00:00:00Z, 2024-10-01T00:00:00Z - 1ms]`. Both gated GER40
// tests bound their runs to this window so the C1-determinism check stays
// fast — an unbounded `None, None` run drains the full archive every call.
const GER40_SEP2024_FROM_MS: i64 = 1_725_148_800_000;
const GER40_SEP2024_TO_MS: i64 = 1_727_740_799_999;
#[test]
fn walkforward_report_is_deterministic() {
// The built-in WFO render is byte-identical across two
@@ -943,7 +974,7 @@ mod tests {
/// drained sink trace is returned alongside the report. Every byte of both
/// is a function of `seed`.
fn run_sample_seeded(seed: u64) -> (RunReport, SeededTrace) {
let (mut h, rx_eq, rx_ex) = sample_harness();
let (mut h, rx_eq, rx_ex) = sample_harness(SYNTHETIC_PIP_SIZE);
let spec = SyntheticSpec { start: 1.0, len: 64, step: 1 };
let window = (Timestamp(1), Timestamp((spec.len as i64 - 1) * spec.step + 1));
h.run(vec![Box::new(spec.source(seed))]);
@@ -959,6 +990,7 @@ mod tests {
],
window,
seed,
SYNTHETIC_PIP_SIZE,
),
metrics,
};
@@ -1144,22 +1176,25 @@ mod tests {
/// `aura run --real <SYMBOL>` dogfoods the #71 streaming Source seam: the same
/// built-in sample signal-quality harness, but fed real M1 **close** bars
/// streamed lazily through `aura_ingest::M1FieldSource` (a `Box<dyn Source>`),
/// not synthetic `VecSource` prices. The property: over a bounded real window,
/// `run_sample_real` yields a `RunReport` whose `total_pips` is finite and is
/// C1-deterministic — two runs of the same window are bit-identical JSON.
/// not synthetic `VecSource` prices. The property: over the verified bounded
/// Sept-2024 GER40 window, `run_sample_real` yields a `RunReport` whose
/// `total_pips` is finite and is C1-deterministic — two runs of the same
/// window are bit-identical JSON. Bounding the window (vs the full unbounded
/// archive) keeps the determinism check fast.
///
/// Gated like the ingest `streaming_seam` test: skip (early return) when the
/// local Pepperstone archive is absent, so the test never fails on a machine
/// without the data. Uses the verified bounded 2006-08 `AAPL.US` window (the
/// same FROM_MS/TO_MS the ingest seam test drives) so the two-run determinism
/// check stays fast.
/// without the data. Uses `GER40` — a *vetted* symbol (pip 1.0): the
/// per-instrument-pip refusal makes `run_sample_real` reject an un-specced
/// symbol before any data access (`std::process::exit(2)`), so this CLI-level
/// test must drive a symbol in the instrument table. The bounded-window AAPL.US
/// streaming property still lives in the ingest `streaming_seam` test, which
/// builds its source literally without the spec lookup.
#[test]
fn run_sample_real_streams_real_close_bars_deterministically() {
// Same bounded 2006-08 inclusive-ms window the ingest streaming_seam test
// uses; AAPL.US M1 data is present there on a data-bearing machine.
const SYMBOL: &str = "AAPL.US";
const FROM_MS: i64 = 1_154_390_400_000;
const TO_MS: i64 = 1_157_068_799_999;
// GER40 is in the vetted instrument table (index pip 1.0); the un-specced
// AAPL.US would now refuse at the spec lookup before any data access.
const SYMBOL: &str = "GER40";
// Mirror skip_if_no_data: never fail where the local archive is absent.
let server = std::sync::Arc::new(data_server::DataServer::new(
@@ -1173,10 +1208,10 @@ mod tests {
return;
}
// The headline: a real-data run over the bounded window yields a finite,
// C1-deterministic RunReport.
let r1 = run_sample_real(SYMBOL, Some(FROM_MS), Some(TO_MS));
let r2 = run_sample_real(SYMBOL, Some(FROM_MS), Some(TO_MS));
// The headline: a real-data run over the bounded Sept-2024 window yields
// a finite, C1-deterministic RunReport.
let r1 = run_sample_real(SYMBOL, Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS));
let r2 = run_sample_real(SYMBOL, Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS));
assert!(
r1.metrics.total_pips.is_finite(),
@@ -1191,6 +1226,35 @@ mod tests {
);
}
#[test]
fn sim_optimal_manifest_renders_per_instrument_pip() {
let m = sim_optimal_manifest(vec![], (Timestamp(1), Timestamp(2)), 0, 1.0);
assert_eq!(m.broker, "sim-optimal(pip_size=1)");
let m2 = sim_optimal_manifest(vec![], (Timestamp(1), Timestamp(2)), 0, 0.0001);
assert_eq!(m2.broker, "sim-optimal(pip_size=0.0001)");
}
#[test]
fn run_real_ger40_uses_index_pip() {
// Gated: needs local GER40 data. Mirrors the existing real-path test's skip.
let server = data_server::DataServer::new(data_server::DEFAULT_DATA_PATH);
if !server.has_symbol("GER40") {
eprintln!("skip: no local GER40 data at {}", data_server::DEFAULT_DATA_PATH);
return;
}
// Bounded to the vetted Sept-2024 window so the pip-label + determinism
// checks run fast (not the full unbounded archive).
let report =
run_sample_real("GER40", Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS));
// The looked-up index pip (1.0) reaches the manifest — not the FX 0.0001.
assert_eq!(report.manifest.broker, "sim-optimal(pip_size=1)");
// Deterministic (C1): a second run yields the same report.
let again =
run_sample_real("GER40", Some(GER40_SEP2024_FROM_MS), Some(GER40_SEP2024_TO_MS));
assert_eq!(report.manifest.broker, again.manifest.broker);
assert_eq!(report.metrics.total_pips, again.metrics.total_pips);
}
/// `parse_real_args` accepts a bare symbol (no window) and a full
/// symbol + `--from`/`--to` pair in any order, and rejects a flag missing its
/// value — the pure arg grammar `main` relies on for `run --real`.