feat(0074): source real-path pip from the geometry sidecar; drop the authored floor

The data-server now ships per-symbol geometry sidecars, so aura no longer
hard-codes instrument geometry. A consumption grep showed production reads
exactly one `InstrumentSpec` field — `pip_size` — at the real-data path; the
other eight (incl. `instrument_id`) were constructed but never read.

- aura-cli resolves the real-path pip from the recorded sidecar
  (`instrument_geometry` -> `InstrumentGeometry.pip_size`), refusing (exit 2)
  on absent geometry. The authored table no longer gates: any symbol with a
  sidecar + bars now runs (new test: USDJPY, never vetted, resolves its JPY
  pip 0.01 from the sidecar).
- Removed the redundant authored floor: `InstrumentSpec`, `instrument_spec`,
  `VETTED_SYMBOLS`, and the table-only tests. The `InstrumentGeometry` seam
  (the surviving pip source) and its nameability guard are kept.
- `instrument_id` (the "find out"): no production consumer. The position-event
  table's `instrument_id` is a decoupled caller parameter; the engine never
  imports an instrument spec. Dropped with the struct.
- The example `pip_size_of` is re-sourced to the sidecar; the pure-structural
  blueprint tests use a literal pip to stay archive-independent.

Speculative deploy-grade fields and the #124 override/floor tiers are deferred
to the Stage-2 broker that will consume them, read from the sidecar geometry
then. Ledger (C8/C15/#22/#106) updated; #124 collapses to "recorded sidecar
geometry -> refuse" in the meantime.

Verified: cargo test --workspace (all green), clippy -D warnings clean, cargo
doc clean, with the local archive present so the sidecar paths run.

refs #124
This commit is contained in:
2026-06-25 19:06:12 +02:00
parent 4fd047adf3
commit 7e4b91ab5f
11 changed files with 198 additions and 464 deletions
+28 -22
View File
@@ -38,10 +38,10 @@ use std::collections::HashSet;
/// The pip size the built-in *synthetic* harnesses run at: a 5-decimal FX major
/// (`EURUSD`-shaped). The synthetic streams carry no instrument, so there is no
/// `instrument_spec` to thread; a single named source keeps the broker's divisor
/// (`sample_harness` / `SimBroker::builder`) and the recorded broker label
/// recorded geometry sidecar to thread; a single named source keeps the broker's
/// divisor (`sample_harness` / `SimBroker::builder`) and the recorded broker label
/// (`sim_optimal_manifest`) in lockstep, so they cannot silently drift apart.
/// The real path threads the looked-up `spec.pip_size` instead.
/// The real path threads the sidecar's looked-up `pip_size` instead.
const SYNTHETIC_PIP_SIZE: f64 = 0.0001;
/// Real walk-forward roller sizes (Fork D/F). `WindowRoller` takes sizes in the
@@ -723,15 +723,21 @@ fn no_real_data(symbol: &str) -> ! {
std::process::exit(2)
}
/// Look up the vetted per-instrument pip/spec, or refuse (stderr + exit 2) on an
/// un-specced symbol — the single home of the guessed-pip refusal message, shared
/// by `run_sample_real` and `DataSource::from_choice`. Refusing BEFORE any data
/// access keeps the pip honest by construction (never guessed).
fn instrument_spec_or_refuse(symbol: &str) -> aura_ingest::InstrumentSpec {
match aura_ingest::instrument_spec(symbol) {
Some(s) => s,
/// Resolve the per-instrument pip from the recorded geometry sidecar, or refuse
/// (stderr + exit 2) when the symbol has no recorded geometry — the single home of
/// the guessed-pip refusal, shared by `open_real_source` and `from_choice`. Reads
/// the symbol's geometry metadata (not bar data) before the run source opens, so an
/// instrument with no recorded geometry refuses without a guessed pip; the pip is
/// the provider's recorded value, honest by construction.
fn pip_or_refuse(server: &std::sync::Arc<data_server::DataServer>, symbol: &str) -> f64 {
match aura_ingest::instrument_geometry(server, symbol) {
Some(geo) => geo.pip_size,
None => {
eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' — refusing to run a real instrument with a guessed pip (add it to the instrument table)");
eprintln!(
"aura: no recorded geometry for symbol '{symbol}' at {}\
refusing to run a real instrument with a guessed pip",
data_server::DEFAULT_DATA_PATH
);
std::process::exit(2);
}
}
@@ -758,10 +764,10 @@ fn probe_window(
(first, last)
}
/// Open a real M1-close source for a vetted symbol over an optional window, returning
/// Open a real M1-close source for a recorded symbol over an optional window, returning
/// the run source paired with its manifest `window` and per-instrument `pip_size`.
/// Single home of the real-source construction the single-run handlers share — the
/// vetted-pip lookup, the `DataServer` `has_symbol` refusal, the probe-window pass, and
/// sidecar-pip lookup, the `DataServer` `has_symbol` refusal, the probe-window pass, and
/// the run-source `open` (each refusal an stderr + exit 2). Pre-data refusals keep the
/// pip honest by construction. Shared by `run_sample_real` and `run_stage1_r`.
fn open_real_source(
@@ -769,10 +775,10 @@ fn open_real_source(
from_ms: Option<i64>,
to_ms: Option<i64>,
) -> (Box<dyn aura_engine::Source>, (Timestamp, Timestamp), f64) {
// Per-instrument pip: look up BEFORE any data access, so an un-specced symbol
// refuses without touching local data. Honest by construction.
let spec = instrument_spec_or_refuse(symbol);
// Per-instrument pip from the recorded sidecar; resolved BEFORE bar-data access
// so an instrument with no geometry refuses without touching the archive.
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
let pip = pip_or_refuse(&server, symbol);
if !server.has_symbol(symbol) {
no_real_data(symbol);
}
@@ -783,24 +789,24 @@ fn open_real_source(
Some(s) => Box::new(s),
None => no_real_data(symbol),
};
(source, window, spec.pip_size)
(source, window, pip)
}
impl DataSource {
/// Build a provider from a parsed choice, or refuse (stderr + exit 2) on an
/// un-vetted symbol / absent data — both BEFORE any member runs (Fork C/G),
/// via the same `instrument_spec_or_refuse` / `no_real_data` helpers as
/// Build a provider from a parsed choice, or refuse (stderr + exit 2) on a symbol
/// with no recorded geometry / absent data — both BEFORE any member runs (Fork C/G),
/// via the same `pip_or_refuse` / `no_real_data` helpers as
/// `run_sample_real`.
fn from_choice(choice: DataChoice) -> DataSource {
match choice {
DataChoice::Synthetic => DataSource::Synthetic,
DataChoice::Real { symbol, from_ms, to_ms } => {
let spec = instrument_spec_or_refuse(&symbol);
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
let pip = pip_or_refuse(&server, &symbol);
if !server.has_symbol(&symbol) {
no_real_data(&symbol);
}
DataSource::Real { server, symbol, from_ms, to_ms, pip: spec.pip_size }
DataSource::Real { server, symbol, from_ms, to_ms, pip }
}
}
}
+109 -40
View File
@@ -131,10 +131,10 @@ fn run_with_trailing_token_is_strict_and_exits_two() {
}
#[test]
fn run_real_unspecced_symbol_refuses_with_exit_2() {
// The instrument_spec lookup precedes any data access, so an un-specced symbol
// refuses with NO local data required (CI-safe). "NONEXISTENT" is not in the
// vetted table.
fn run_real_no_geometry_symbol_refuses_with_exit_2() {
// The geometry-sidecar lookup precedes any bar-data access, so a symbol with no
// recorded geometry refuses with NO local data required (CI-safe). "NONEXISTENT"
// has no recorded geometry on any host.
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["run", "--real", "NONEXISTENT"])
.output()
@@ -142,11 +142,43 @@ fn run_real_unspecced_symbol_refuses_with_exit_2() {
assert_eq!(out.status.code(), Some(2), "expected exit 2");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
stderr.contains("no vetted pip/instrument spec for symbol 'NONEXISTENT'"),
stderr.contains("no recorded geometry for symbol 'NONEXISTENT'"),
"expected the per-instrument-pip refusal message, got: {stderr}"
);
}
/// Property: the pip-refusal is sourced ONLY from recorded provider geometry — the
/// removed authored instrument floor (the hand-authored "vetted" table) no
/// longer participates in the refusal path. The cycle-0074 headline is the floor's
/// REMOVAL, not a rename; the observable signature is that the refusal message
/// never points the user at a hand-authored table to extend. A regression that
/// re-introduced an authored fallback (or restored the old "add it to the
/// instrument table" wording) would re-leak this vocabulary. Asserts the OLD
/// authored-floor phrasing is ABSENT — the negative complement to the positive
/// "no recorded geometry" assertion above. Archive-independent: "NONEXISTENT" has
/// no sidecar on any host, so the refusal fires without local data.
#[test]
fn run_real_refusal_names_no_authored_floor() {
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["run", "--real", "NONEXISTENT"])
.output()
.expect("spawn aura");
assert_eq!(out.status.code(), Some(2), "expected exit 2");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
!stderr.contains("instrument table"),
"refusal must not point at a removed authored table, got: {stderr}"
);
assert!(
!stderr.contains("vetted"),
"refusal must not use the removed vetted-floor vocabulary, got: {stderr}"
);
assert!(
!stderr.contains("instrument spec"),
"refusal must not reference the removed authored instrument floor, got: {stderr}"
);
}
/// Property: the un-specced-symbol refusal is a CLEAN refusal — it emits NOTHING
/// on stdout, never a partial `RunReport`. The #22 acceptance is "refuse INSTEAD
/// of emitting nonsense": the pip lookup precedes harness construction and data
@@ -155,7 +187,7 @@ fn run_real_unspecced_symbol_refuses_with_exit_2() {
/// leak a `{"manifest":...}` line here while still exiting non-zero; this pins
/// that it cannot. Archive-independent (the lookup never touches local data).
#[test]
fn run_real_unspecced_symbol_emits_no_stdout_report() {
fn run_real_no_geometry_symbol_emits_no_stdout_report() {
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["run", "--real", "NONEXISTENT"])
.output()
@@ -168,25 +200,25 @@ fn run_real_unspecced_symbol_emits_no_stdout_report() {
);
}
/// Property: the pip-honesty refusal keys on the SYMBOL being un-specced, not on
/// `--real` being taken at all. A *vetted* symbol (`EURUSD`, in the instrument
/// table) gets PAST the `instrument_spec` lookup, so it never produces the
/// pip-refusal message — it either runs to a report (local data present) or hits
/// the DISTINCT no-local-data usage path ("no local data for symbol"). Either
/// way the "no vetted pip/instrument spec" string is absent. This distinguishes
/// the per-instrument-pip honesty lever from the unrelated arg-grammar / no-data
/// error paths, and is archive-independent (asserts only on the absence of the
/// pip-refusal string, true whether or not EURUSD data is on the machine).
/// Property: the pip-honesty refusal keys on the SYMBOL having no recorded geometry,
/// not on `--real` being taken at all. A symbol *with* a recorded sidecar (`EURUSD`)
/// gets PAST the geometry lookup, so it never produces the pip-refusal message — it
/// either runs to a report (local data present) or hits the DISTINCT no-local-data
/// usage path ("no local data for symbol"). Either way the "no recorded geometry"
/// string is absent. This distinguishes the per-instrument-pip honesty lever from
/// the unrelated arg-grammar / no-data error paths, and is archive-independent
/// (asserts only on the absence of the pip-refusal string, true whether or not
/// EURUSD data is on the machine).
#[test]
fn run_real_vetted_symbol_never_hits_the_pip_refusal() {
fn run_real_sidecar_symbol_never_hits_the_pip_refusal() {
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["run", "--real", "EURUSD"])
.output()
.expect("spawn aura");
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
!stderr.contains("no vetted pip/instrument spec"),
"a vetted symbol must clear the pip lookup, never the pip-refusal; stderr: {stderr}"
!stderr.contains("no recorded geometry"),
"a symbol with a recorded sidecar must clear the pip lookup, never the pip-refusal; stderr: {stderr}"
);
// It resolves to exactly one of the two legitimate outcomes: a clean report
// (exit 0, data present) or the distinct no-local-data refusal (exit 2).
@@ -200,24 +232,24 @@ fn run_real_vetted_symbol_never_hits_the_pip_refusal() {
),
Some(2) => assert!(
stderr.contains("no local data for symbol 'EURUSD'"),
"exit 2 for a vetted symbol must be the no-data path, got: {stderr}"
"exit 2 for a symbol with a recorded sidecar must be the no-data path, got: {stderr}"
),
other => panic!("unexpected exit for a vetted real run: {other:?}; stderr: {stderr}"),
other => panic!("unexpected exit for a real run with a recorded sidecar: {other:?}; stderr: {stderr}"),
}
}
/// Property: a vetted symbol threads its looked-up pip all the way to the binary's
/// emitted manifest — `aura run --real GER40` renders the INDEX pip
/// Property: a recorded-sidecar symbol threads its looked-up pip all the way to the
/// binary's emitted manifest — `aura run --real GER40` renders the INDEX pip
/// `sim-optimal(pip_size=1)`, not the FX `0.0001` literal the synthetic path uses.
/// This is the #22 headline at the built-binary boundary (criterion 2: GER40=1.0
/// vs EURUSD=0.0001 vs un-specced→refuse): the metadata channel reaches stdout, so
/// vs EURUSD=0.0001 vs no-geometry→refuse): the metadata channel reaches stdout, so
/// equity is honestly scaled per instrument. Gated on local GER40 data — skip
/// cleanly when the archive is absent (the project's skip-on-no-data convention),
/// so it never fails on a data-less machine; the no-data path is the distinct
/// "no local data" refusal (exit 2), not the pip-refusal, asserted above.
#[test]
fn run_real_vetted_index_pip_reaches_the_emitted_manifest() {
// The vetted GER40 Sept-2024 window (inclusive Unix-ms), the same calendar
fn run_real_sidecar_index_pip_reaches_the_emitted_manifest() {
// The GER40 Sept-2024 window (inclusive Unix-ms), the same calendar
// month the gated ingest path drives; bounding it keeps the run fast.
const FROM_MS: &str = "1725148800000";
const TO_MS: &str = "1727740799999";
@@ -226,13 +258,13 @@ fn run_real_vetted_index_pip_reaches_the_emitted_manifest() {
.output()
.expect("spawn aura");
// Skip on a data-less machine: a vetted symbol with no local data takes the
// distinct no-data path (exit 2, "no local data"), never the pip-refusal.
// Skip on a data-less machine: a recorded-sidecar symbol with no local data takes
// the distinct no-data path (exit 2, "no local data"), never the pip-refusal.
if out.status.code() == Some(2) {
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
stderr.contains("no local data for symbol 'GER40'"),
"exit 2 for vetted GER40 must be the no-data path, got: {stderr}"
"exit 2 for GER40 must be the no-data path, got: {stderr}"
);
eprintln!("skip: no local GER40 data");
return;
@@ -248,6 +280,43 @@ fn run_real_vetted_index_pip_reaches_the_emitted_manifest() {
);
}
/// Property: the authored instrument table no longer gates real runs — a symbol
/// absent from the (removed) vetted floor but carrying a recorded geometry sidecar
/// resolves its pip from the sidecar and runs. USDJPY (never vetted; JPY pip 0.01)
/// renders the JPY pip in the manifest, proving the pip is sourced from the provider
/// geometry, not a hand-authored table entry. Gated: skips cleanly when USDJPY has
/// no local geometry/data (no sidecar → "no recorded geometry"; sidecar but no bars
/// → "no local data").
#[test]
fn run_real_nonvetted_symbol_resolves_pip_from_sidecar() {
// FX trades continuously; the GER40 Sept-2024 window also has USDJPY bars.
const FROM_MS: &str = "1725148800000";
const TO_MS: &str = "1727740799999";
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
.args(["run", "--real", "USDJPY", "--from", FROM_MS, "--to", TO_MS])
.output()
.expect("spawn aura");
// Skip on a host without USDJPY geometry/data — either refusal is a clean skip,
// sourced from recorded geometry, never an authored-table miss.
if out.status.code() == Some(2) {
let stderr = String::from_utf8_lossy(&out.stderr);
if stderr.contains("no recorded geometry") || stderr.contains("no local data") {
eprintln!("skip: no local USDJPY geometry/data");
return;
}
}
assert_eq!(out.status.code(), Some(0), "USDJPY should resolve and run; exit: {:?}, stderr: {}",
out.status, String::from_utf8_lossy(&out.stderr));
let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
assert!(
stdout.contains("pip_size=0.01"),
"USDJPY must render the JPY sidecar pip (0.01) in the manifest, got: {}",
stdout.trim_end()
);
}
#[test]
fn run_trace_persists_taps_and_plain_run_writes_no_traces() {
let cwd = temp_cwd("run-trace");
@@ -372,13 +441,13 @@ fn run_real_with_trace_persists_taps_and_keeps_stdout_unchanged() {
.output()
.expect("spawn aura run --real --trace");
// Skip on a data-less machine: vetted GER40 with no local data takes the
// Skip on a data-less machine: GER40 with no local data takes the
// distinct no-data path (exit 2), never the pip-refusal, and writes no traces.
if out.status.code() == Some(2) {
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
stderr.contains("no local data for symbol 'GER40'"),
"exit 2 for vetted GER40 --trace must be the no-data path, got: {stderr}"
"exit 2 for GER40 --trace must be the no-data path, got: {stderr}"
);
eprintln!("skip: no local GER40 data");
let _ = std::fs::remove_dir_all(&cwd);
@@ -1098,10 +1167,10 @@ fn momentum_longonly_true_clamps_exposure_nonnegative_false_keeps_negatives() {
// These drive `aura sweep|walkforward --real <SYMBOL>` end to end over the local
// data-server archive. The gated tests need the Pepperstone M1 archive present;
// they skip (print + pass) on a data-less machine, the project's
// skip-on-no-data convention. The pip-refusal test is NOT gated: the un-specced
// symbol is rejected before any data access.
// skip-on-no-data convention. The pip-refusal test is NOT gated: a symbol with no
// recorded geometry is rejected before any data access.
// EURUSD 2024-06 (UTC inclusive ms): a bounded vetted real window.
// EURUSD 2024-06 (UTC inclusive ms): a bounded recorded-sidecar real window.
const EURUSD_JUN2024_FROM_MS: i64 = 1_717_200_000_000;
const EURUSD_JUN2024_TO_MS: i64 = 1_719_791_999_999;
@@ -1181,18 +1250,18 @@ fn walkforward_real_persists_one_oos_member_per_window() {
}
/// Property: the per-instrument pip refusal fires for `aura sweep --real
/// <unspecced>` exactly as it does for `aura run --real` — an un-vetted symbol
/// (`AAPL.US`, absent from the instrument table) is rejected with exit 2 and the
/// "no vetted pip" message BEFORE any data access. NOT gated: the refusal
/// precedes the archive entirely, so it is CI-safe and runs on every machine.
/// <no-geometry symbol>` exactly as it does for `aura run --real` — a symbol with no
/// recorded geometry (`NONEXISTENT`, no sidecar on any host) is rejected with exit 2
/// and the "no recorded geometry" message BEFORE any bar-data access. NOT gated: the
/// refusal precedes the archive entirely, so it is CI-safe and runs on every machine.
#[test]
fn sweep_real_unspecced_symbol_refuses_with_exit_2() {
fn sweep_real_no_geometry_symbol_refuses_with_exit_2() {
// not gated: the pip refusal happens before any data access.
let dir = temp_cwd("sweep_real_refuse");
let out = std::process::Command::new(BIN)
.args(["sweep", "--real", "AAPL.US"]).current_dir(&dir).output().unwrap();
.args(["sweep", "--real", "NONEXISTENT"]).current_dir(&dir).output().unwrap();
assert_eq!(out.status.code(), Some(2));
assert!(String::from_utf8_lossy(&out.stderr).contains("no vetted pip"));
assert!(String::from_utf8_lossy(&out.stderr).contains("no recorded geometry"));
let _ = std::fs::remove_dir_all(&dir);
}