diff --git a/docs/specs/0073-instrument-geometry-crosscheck.md b/docs/specs/0073-instrument-geometry-crosscheck.md new file mode 100644 index 0000000..107fe9a --- /dev/null +++ b/docs/specs/0073-instrument-geometry-crosscheck.md @@ -0,0 +1,278 @@ +# Consume DataServer instrument geometry: cross-check the vetted floor + two deploy fields — Design Spec + +**Date:** 2026-06-25 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +> **Reference issue:** Brummel/Aura #143 (consumer side) ⊕ data-server #3 (upstream +> reader, CLOSED). Scope fork resolved in the #143 reconciliation comment (cycle +> 0073): this cycle delivers geometry *consumption* + *cross-check* + two *additive* +> fields; it builds **no** runtime resolver and does **not** change +> `InstrumentSpec.quote_currency`'s type — both remain #124's. + +## Goal + +The data path now writes a per-symbol geometry sidecar (`.meta.json`) +beside the bars, and data-server #3 (CLOSED) shipped a reader that parses it into +neutral, broker-agnostic `InstrumentGeometry`. This cycle makes that geometry a +first-class, broker-agnostic input at aura's ingestion edge and uses it to +**cross-check the hand-authored vetted `InstrumentSpec` floor (#113) against +provider truth**, catching a typo in the authored table that would otherwise +silently mis-size every position. It also adds the two deploy-grade price-rounding +fields (`tick_size`, `digits`) the floor deferred, now that a provider source +supplies them. + +It deliberately does **not** build the four-tier runtime resolver or widen the +`quote_currency` type — those are #124's, and no broker/resolver consumer exists +yet (#116 / #120–123 open). "quote_currency beyond FX" is satisfied at the +geometry tier (`InstrumentGeometry.quote: String`), not by mutating the authored +floor's `&'static str` field. + +## Architecture + +Three changes, all at the `aura-ingest` ingestion edge (`crates/aura-ingest/src/lib.rs`), +plus a new integration test and a `Cargo.lock` bump: + +1. **Tier-2 source access.** Re-export `data_server::meta::InstrumentGeometry` and + add a thin, documented accessor `instrument_geometry(server, symbol)` over the + reader's `DataServer::symbol_meta`. This mirrors the crate's existing + `pub use data_server::{DataServer, DEFAULT_DATA_PATH}` discipline: a consumer + builds from `aura-ingest` alone and never names `data_server`, and the accessor + is the single vetted home where the C15 / untrusted-input / refuse-don't-guess + semantics of the recorded-metadata tier are stated. It surfaces the tier-2 + *source*; the tier-composing, authored-override-wins *resolver* stays #124. + +2. **Two additive `InstrumentSpec` fields.** `tick_size: f64` and `digits: u32`, + appended to the struct (it stays `Copy` — both are `Copy` scalars), authored for + the 5 vetted symbols from the verified sidecars. Additive per C15 + ("extensible without a signature break"). The construction sites the compiler + enumerates — the `instrument_spec()` table and the `instrument_spec_lookup_by_symbol` + unit test's 5 full literals — are updated in lock-step (a compile gate, not a + behaviour change). + +3. **Cross-check test.** A new integration test asserts, for every vetted symbol, + that the authored floor agrees with the provider geometry on `pip_size`, + `contract_size == lot_size`, `pip_value_per_lot == lot_size * pip_size` (the C10 + pip-value invariant against broker truth), `tick_size`, `digits`, and + `quote_currency`. It iterates the vetted set, so when #140 registers + NAS100 / USDJPY / USDCHF / NZDUSD the same test validates them for free. It + skips cleanly where the local Pepperstone archive is absent (hermetic + elsewhere), exactly as the existing real-data tests do. + +The `Cargo.lock` bump (`data-server` `0f5e665` → `694f96f2`) pulls the commit +carrying the reader; it is part of this cycle. + +## Concrete code shapes + +### Worked consumer example (the acceptance-criterion evidence) + +The audience is a downstream broker / deploy-step author building on `aura-ingest`. +This is the code they write — and what the cycle must make compile and run: + +```rust +use aura_ingest::{default_data_server, instrument_geometry, instrument_spec}; + +let server = default_data_server(); +let symbol = "EURUSD"; + +// The authored floor a broker sizes against — now with deploy-grade rounding inputs. +let spec = instrument_spec(symbol).expect("vetted symbol, or refuse-don't-guess"); +let price_decimals: u32 = spec.digits; // NEW: round a price for the order ticket +let tick: f64 = spec.tick_size; // NEW: snap a stop/limit to the tradable grid + +// The recorded provider geometry (tier 2 of #124), broker-agnostic, never raw JSON. +if let Some(geo) = instrument_geometry(&server, symbol) { + // Authoring/cross-check aid: provider truth for the same symbol. + debug_assert!((geo.lot_size * geo.pip_size - spec.pip_value_per_lot).abs() < 1e-9); + // Non-FX quote currency is available here as an owned String (geometry tier), + // without widening the &'static str authored floor (that type-fork is #124). + let _quote_ccy: &str = &geo.quote; +} +``` + +### The cross-check test the cycle delivers (`crates/aura-ingest/tests/instrument_geometry_crosscheck.rs`) + +```rust +//! Cross-check the hand-authored vetted InstrumentSpec floor (#113) against the +//! provider-recorded geometry sidecars (#143 — tier 2 of #124's resolution +//! hierarchy). Skips with a note where the local Pepperstone archive is absent, +//! so it is hermetic elsewhere and exercises broker truth where the files exist. + +use aura_ingest::{default_data_server, instrument_geometry, instrument_spec, DEFAULT_DATA_PATH}; + +/// The vetted symbols. Kept in lock-step with the authored table; #140 extends it. +const VETTED: [&str; 5] = ["GER40", "FRA40", "EURUSD", "GBPUSD", "USDCAD"]; + +#[test] +fn authored_floor_agrees_with_provider_geometry() { + let server = default_data_server(); + // Gate on a sidecar's presence (symbol_meta reads the file directly, not the + // bar index) — the same skip-if-absent discipline as the other real-data tests. + if instrument_geometry(&server, "EURUSD").is_none() { + eprintln!("skip: no local geometry sidecars at {DEFAULT_DATA_PATH} (EURUSD.meta.json absent)"); + return; + } + for sym in VETTED { + let spec = instrument_spec(sym).expect("vetted symbol"); + let geo = instrument_geometry(&server, sym) + .unwrap_or_else(|| panic!("vetted symbol {sym} must carry a geometry sidecar")); + + assert!((spec.pip_size - geo.pip_size).abs() < 1e-12, + "{sym}: authored pip_size {} != provider {}", spec.pip_size, geo.pip_size); + assert!((spec.contract_size - geo.lot_size).abs() < 1e-6, + "{sym}: authored contract_size {} != provider lot_size {}", spec.contract_size, geo.lot_size); + // the C10 pip-value invariant, now against broker truth + let provider_pip_value = geo.lot_size * geo.pip_size; + assert!((spec.pip_value_per_lot - provider_pip_value).abs() < 1e-9, + "{sym}: authored pip_value_per_lot {} != provider lot_size*pip_size {}", + spec.pip_value_per_lot, provider_pip_value); + assert!((spec.tick_size - geo.tick_size).abs() < 1e-12, + "{sym}: authored tick_size {} != provider {}", spec.tick_size, geo.tick_size); + assert_eq!(spec.digits, geo.digits, + "{sym}: authored digits {} != provider {}", spec.digits, geo.digits); + assert_eq!(spec.quote_currency, geo.quote, + "{sym}: authored quote_currency {} != provider {}", spec.quote_currency, geo.quote); + } +} +``` + +### `InstrumentSpec` — before → after (additive, stays `Copy`) + +```rust +// BEFORE (7 fields) +pub struct InstrumentSpec { + pub instrument_id: i64, + pub pip_size: f64, + pub contract_size: f64, + pub pip_value_per_lot: f64, + pub min_lot: f64, + pub lot_step: f64, + pub quote_currency: &'static str, +} + +// AFTER (9 fields — two appended; quote_currency type unchanged) +pub struct InstrumentSpec { + pub instrument_id: i64, + pub pip_size: f64, + pub contract_size: f64, + pub pip_value_per_lot: f64, + pub min_lot: f64, + pub lot_step: f64, + pub quote_currency: &'static str, + /// Smallest representable price increment (provider geometry; deploy-grade + /// price rounding). 1e-5 for a 5-decimal FX major, 0.1 for the indices. + pub tick_size: f64, + /// Decimal places in a quoted price (provider geometry; deploy-grade + /// price rounding). 5 for FX majors, 1 for the indices. + pub digits: u32, +} +``` + +### `instrument_spec()` table — authored values (from the verified sidecars) + +| symbol | tick_size | digits | +|--------|-----------|--------| +| GER40 | 0.1 | 1 | +| FRA40 | 0.1 | 1 | +| EURUSD | 1e-5 | 5 | +| GBPUSD | 1e-5 | 5 | +| USDCAD | 1e-5 | 5 | + +Each existing struct literal gains `tick_size: , digits: ,`. Example +(EURUSD): `… lot_step: 0.01, quote_currency: "USD", tick_size: 1e-5, digits: 5,`. + +### Re-export + accessor (`crates/aura-ingest/src/lib.rs`) + +```rust +// near the existing `pub use data_server::{DataServer, DEFAULT_DATA_PATH};` +/// Re-export of data-server's neutral instrument geometry (#143): a consumer reads +/// provider geometry through `aura-ingest` alone, never naming `data_server`. +pub use data_server::meta::InstrumentGeometry; + +// near `default_data_server` +/// Load the provider-recorded neutral geometry for `symbol` — the recorded +/// dataset-metadata tier (tier 2) of #124's resolution hierarchy — or `None` when +/// the dataset carries no usable sidecar (missing / unparseable / unsupported +/// version / a non-finite field; the reader absorbs every provider quirk). +/// +/// This surfaces the tier-2 *source*; the tier-composing, authored-override-wins +/// *resolver* is #124. The geometry is an **untrusted input** the authored override +/// supersedes — never a silent runtime fallback for an unvetted symbol +/// (refuse-don't-guess stands: `instrument_spec` still returns `None`). Geometry is +/// non-scalar reference metadata held beside the hot path (C15); it is **not** a +/// `Source` and never enters the engine's `(Timestamp, Scalar)` seam. +pub fn instrument_geometry(server: &Arc, symbol: &str) -> Option { + server.symbol_meta(symbol) +} +``` + +## Components + +- **`crates/aura-ingest/src/lib.rs`** — the re-export, the `instrument_geometry` + accessor, the two new `InstrumentSpec` fields, the five updated table literals, + and the updated `instrument_spec_lookup_by_symbol` unit-test literals (compile + gate). The `instrument_spec_pip_value_matches_contract_times_pip` and + `instrument_spec_ids_are_distinct` unit tests are unaffected. +- **`crates/aura-ingest/tests/instrument_geometry_crosscheck.rs`** — new; the + cross-check above. +- **`crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs`** — extend the + `every_vetted_symbol_resolves_to_a_complete_spec` property to also assert the two + new fields are sane (`tick_size > 0.0`, `digits > 0`), keeping the public-boundary + "no field left at a placeholder" guarantee complete. +- **`Cargo.lock`** — `data-server` bumped to `694f96f2` (carries the reader). + +## Data flow + +`.meta.json` on disk → (data-server, broker-facing) `symbol_meta` → +neutral `InstrumentGeometry` → (aura-ingest edge) `instrument_geometry` accessor → +consumer / cross-check test. The raw cTrader JSON never crosses into this repo; the +parser stays `pub(crate)` upstream. Geometry is keyed by symbol and held beside the +hot path — it never becomes a streamed scalar, and the engine's Source seam is +untouched (C15, invariant #4). + +## Error handling + +- **Refuse-don't-guess preserved.** `instrument_spec(unknown)` still returns `None`; + the geometry accessor is a *source for authoring/cross-check*, never a runtime + fallback that fabricates a spec for an unvetted symbol. +- **Untrusted input.** `instrument_geometry` returns `None` for a missing / + malformed / unsupported-version / non-finite sidecar (the reader's contract). The + cross-check test treats a *present archive with a missing vetted sidecar* as a + real failure (panic), but treats a *wholly absent archive* as skip — matching the + existing real-data tests. +- **No silent currency widening.** `quote_currency` stays `&'static str`; the + String-typed provider currency lives only in `InstrumentGeometry`. The cross-check + compares them for the vetted set (they agree), but does not fold the String into + the authored floor — that type transition is #124's. + +## Testing strategy + +- **Cross-check (new, real-data, skip-if-absent):** `authored_floor_agrees_with_provider_geometry` + — the headline. Verified to pass against the 30 live sidecars (all five vetted + symbols agree; the invariant `lot_size * pip_size == pip_value_per_lot` holds). +- **Compile gate:** the two new fields force the `instrument_spec()` table and the + unit-test literals to be updated together; `cargo build -p aura-ingest` is the + gate. +- **Extended public-boundary property:** `every_vetted_symbol_resolves_to_a_complete_spec` + gains `tick_size`/`digits` sanity asserts. +- **Regression:** all existing aura-ingest unit + integration tests stay green; no + golden metric anywhere else changes (this cycle touches no engine/CLI hot path). +- **Final gates:** `cargo test --workspace`, `cargo clippy --workspace --all-targets -- -D warnings`. + +## Acceptance criteria + +Against aura's default criterion ("solves a real user-facing problem and +contradicts no stated design commitment"): + +- **Real problem.** A downstream broker/deploy author reaches for `instrument_geometry` + to read broker-agnostic geometry and for `spec.tick_size` / `spec.digits` to round + prices to the tradable grid (the worked example above). The cross-check converts a + silent table typo — which would mis-size every position — into a build/test-time + failure against broker truth. +- **No commitment contradicted.** Determinism (C1), SoA/owned state (C7), one + output per node (C8), reference-metadata-beside-the-hot-path + additive-without- + signature-break (C15), refuse-don't-guess, untrusted-input, and engine-domain- + freedom all hold; the Source seam still carries only `(Timestamp, Scalar)`. The + resolver and the `quote_currency` type-fork are explicitly left to #124. +- **Measurable.** The headline test passes against live geometry; an injected + one-digit typo in the authored table makes it fail (the value it adds).