From 2844ace2ce3334baa495c3577f79608c6ed6e08c Mon Sep 17 00:00:00 2001 From: Brummel Date: Thu, 18 Jun 2026 18:58:56 +0200 Subject: [PATCH] spec: 0054 per-instrument pip metadata channel refs #22 --- docs/specs/0054-per-instrument-pip.md | 235 ++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 docs/specs/0054-per-instrument-pip.md diff --git a/docs/specs/0054-per-instrument-pip.md b/docs/specs/0054-per-instrument-pip.md new file mode 100644 index 0000000..5af4f2b --- /dev/null +++ b/docs/specs/0054-per-instrument-pip.md @@ -0,0 +1,235 @@ +# Per-Instrument Pip Metadata — Design Spec + +**Date:** 2026-06-18 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +The data-server symbol stream carries only price bars — it brings **no instrument +metadata**, in particular no pip/point size. That absence is the bug behind #22: +the sim-optimal broker is therefore constructed with a single global `pip_size` +literal (`0.0001`, correct only for 5-decimal FX), so on the CLI real path +`aura run --real ` produces wrong-unit pip equity for any non-FX instrument +(issue-#22 table: `AAPL.US` → `-8,693,884`, `BTCUSD` → `-3.2e9`). + +This cycle adds the missing **channel to specify per-instrument pip metadata**, and +threads it through the one observably-broken path — the CLI `aura run --real`. + +**Scope (narrowed, user-directed 2026-06-18, recorded on #22):** the metadata +channel + the CLI real path + the unknown-symbol refusal. The runnable real-data +examples (`ger40_breakout_*`) already bake the **correct** pip (`1.0` for the GER40/ +FRA40 indices) — they are *not* buggy; centralizing them through the lookup is a +consistency cleanup **deferred to a follow-up `idea` issue**. `SimBroker` is +unchanged. Portfolio/multi-instrument machinery is out of scope (`SimBroker` is and +stays single-instrument); **not** via `Aura.toml`. + +## Architecture + +Two new pieces + one threaded path, placed by the crate dependency graph +(`aura-cli → aura-ingest → {aura-std, aura-engine, aura-core}`): + +1. **`InstrumentSpec` — a typed per-instrument reference-metadata carrier**, home + **`aura-ingest`** (the ingestion/source edge, where `symbol` exists). Minimal + today (`pip_size` only); extensible (tick size, digits, quote currency) without a + signature break. It is metadata beside the hot path (C4/C7/C15): never streamed, + never on `Ctx`; the engine stays domain-free. + +2. **`instrument_spec(symbol) -> Option` — the lookup**, home + **`aura-ingest`**. A static, Rust-authored vetted table (NOT `Aura.toml` — the + later `Aura.toml` schema can subsume it). `None` for an unknown symbol: the + honesty lever — a real run for an instrument with no vetted pip cannot proceed + with a guessed value. + +3. **CLI real path threads the looked-up pip.** `SimBroker` keeps its `pip_size: f64` + divisor constructor (synthetic/seam/test fixtures legitimately build it with an + explicit value). The honesty is realized at the real-data call site, which obtains + pip **only** through `instrument_spec(symbol)` and never a literal. + +## Concrete code shapes + +### Worked user-facing behaviour (the acceptance evidence) + +The same command over differently-quoted instruments yields honestly-scaled pip +equity, with no pip literal in the invocation; an un-specced instrument refuses: + +```text +$ aura run --real GER40 --from … --to … +{"manifest":{…,"broker":"sim-optimal(pip_size=1)"}, "metrics":{…}} # index → pip 1.0 +$ aura run --real EURUSD --from … --to … +{"manifest":{…,"broker":"sim-optimal(pip_size=0.0001)"}, "metrics":{…}} # 5-dp FX → pip 0.0001 +$ aura run --real BTCUSD +aura: no vetted pip/instrument spec for symbol 'BTCUSD' — refusing to run a real +instrument with a guessed pip (add it to the instrument table). [exit 2] +``` + +GER40 equity is now scaled by `1.0` (was `0.0001`), so it is comparable to a real FX +run — this is the cross-asset mismatch the feature exists to fix, shown end-to-end. + +### New: `InstrumentSpec` + `instrument_spec` (aura-ingest) + +```rust +// crates/aura-ingest/src/lib.rs (new) + +/// Per-instrument reference metadata held beside the hot path (C7/C15): non-scalar, +/// never streamed. Minimal today — extend with tick size / digits / quote currency +/// as later cycles need, without breaking this signature. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct InstrumentSpec { + /// Price units per pip / point (> 0). The sim-optimal broker's divisor. + pub pip_size: f64, +} + +/// Look up per-instrument reference metadata by `symbol` at the ingestion edge. +/// Rust-authored vetted table (NOT `Aura.toml`); `None` for an instrument with no +/// vetted spec — the caller must refuse rather than guess a pip. +pub fn instrument_spec(symbol: &str) -> Option { + // SHAPE: explicit per-symbol entries, None for unknown. Indices quote in points + // (pip = 1.0); 5-decimal FX = 0.0001. NB: JPY pairs are 0.01, NOT 0.0001 — seed + // only vetted values. Planner pins the full set against the data-server symbols. + let pip = match symbol { + "GER40" | "FRA40" => 1.0, // index points + "EURUSD" | "GBPUSD" | "USDCAD" => 0.0001, // 5-dp FX majors (vetted subset) + _ => return None, + }; + Some(InstrumentSpec { pip_size: pip }) +} +``` + +### Changed: CLI real path (aura-cli) + +```rust +// crates/aura-cli/src/main.rs + +// BEFORE: sample_harness hard-codes the broker pip; the manifest label is a literal +fn sample_harness() -> (Harness, …) { … SimBroker::new(0.0001) … } // :91 +fn sim_optimal_manifest(params, window, seed) -> RunManifest { + RunManifest { …, broker: "sim-optimal(pip_size=0.0001)".to_string() } // :142 +} + +// AFTER: pip is a parameter, threaded from the per-instrument lookup +fn sample_harness(pip_size: f64) -> (Harness, …) { … SimBroker::new(pip_size) … } +fn sim_optimal_manifest(params, window, seed, pip_size: f64) -> RunManifest { + RunManifest { …, broker: format!("sim-optimal(pip_size={pip_size})") } // 1.0→"1", 0.0001→"0.0001" +} + +fn run_sample() -> RunReport { // synthetic — UNCHANGED behaviour + let (mut h, …) = sample_harness(0.0001); // sample instrument; cli_run.rs:35 pin holds + … sim_optimal_manifest(…, 0.0001) … +} + +fn run_sample_real(symbol: &str, from_ms, to_ms) -> RunReport { + // Look up FIRST — before any data access — so an un-specced symbol refuses + // without touching local data (CI-testable, no fixture). + let spec = match aura_ingest::instrument_spec(symbol) { + Some(s) => s, + None => { eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' …"); std::process::exit(2); } + }; + let (mut h, …) = sample_harness(spec.pip_size); + … // unchanged streaming over M1FieldSource::open(symbol, …); has_symbol/window guards follow + … sim_optimal_manifest(…, spec.pip_size) … +} +``` + +`SimBroker::label()` already renders `SimBroker({pip_size})` (sim_broker.rs:99), so +the node label tracks the looked-up pip; only the manifest *string* (a hand-built +literal today) needs parameterizing. + +### Call-site propagation (workspace-wide, mechanical) + +Two signatures gain a `pip_size` parameter; the type checker enumerates every +caller, and every caller that is NOT the real path passes its existing `0.0001` +unchanged — behaviour-preserving, no emitted-pip change, so existing drift pins stay +green: + +- `sample_harness(pip_size)` — **4** sites (main.rs:150,184,589,946). `run_sample_real` + passes the lookup; the synthetic `run_sample` + the two enriched/seeded sample + callers pass `0.0001`. +- `sim_optimal_manifest(…, pip_size)` — **8** sites (main.rs:163,226,379,498,520,597,853,954). + `run_sample_real` passes the lookup; the synthetic run / sweep / walkforward + (IS+OOS) / MC / macd / seeded callers pass `0.0001`. + +The exact enumeration is `rustc`'s: the signature change is the complete forcing +function; the rule is "the real path passes the lookup, every other caller passes its +existing `0.0001`." + +## Components + +| Component | Crate | Change | +|---|---|---| +| `InstrumentSpec` struct | aura-ingest | **new** — `{ pip_size: f64 }`, derives Clone/Copy/Debug/PartialEq | +| `instrument_spec(symbol)` | aura-ingest | **new** — static vetted table, `Option` | +| `sample_harness` | aura-cli | takes `pip_size: f64` (4 call sites propagate) | +| `sim_optimal_manifest` | aura-cli | takes `pip_size: f64`, formats it into the broker string (8 call sites propagate) | +| `run_sample_real` | aura-cli | look up by symbol FIRST; refuse (exit 2) on `None`; thread pip | +| `run_sample` + synthetic/sweep/mc/wf/macd callers | aura-cli | pass the unchanged `0.0001` | +| `SimBroker` | aura-std | **unchanged** (keeps `new(f64)` / `builder(f64)`) | + +## Data flow + +Real path (changed): `symbol` → `instrument_spec(symbol)` (before any data access) → +`InstrumentSpec { pip_size }` → `sample_harness(pip_size)` builds `SimBroker::new(pip_size)` +→ equity integrates `exposure·(Δprice)/pip_size` (sim_broker.rs:90) → +`sim_optimal_manifest(…, pip_size)` records the real pip in the manifest broker +string. No pip literal on the path. + +Synthetic / seam / test paths (unchanged): construct `SimBroker::new()` +directly; no lookup. + +## Error handling + +- **Unknown / un-specced instrument on a real run (new guard, checked FIRST):** + `run_sample_real` calls `instrument_spec(symbol)` before any data access. `None` → + a clear stderr refusal + `exit(2)`, without touching local data (so it is + observable and CI-testable with no fixture). Deliberate, observable **behaviour + change**: a real-data run for a non-seeded symbol previously produced silent + wrong-unit pips and now refuses (user-directed, recorded on #22). +- **Downstream guards unchanged and not depended on.** A specced symbol flows through + the existing `run_sample_real` body untouched — the `has_symbol`/empty-window usage + error and the streaming run are not modified; the spec adds a guard *ahead* of them + and rests on no current-behaviour claim about that pre-existing path. +- `pip_size > 0` is already asserted in `SimBroker::new` (sim_broker.rs:48); the table + holds only positive values. + +## Testing strategy + +- **Lookup unit (aura-ingest):** `instrument_spec("GER40") == Some(pip_size:1.0)`; + `instrument_spec("EURUSD") == Some(0.0001)`; `instrument_spec("NONEXISTENT") == None`. +- **Manifest pip (aura-cli):** `sim_optimal_manifest(…, 1.0)` → `broker == + "sim-optimal(pip_size=1)"`; with `0.0001` → the existing string. Pins the exact + `format!("{pip_size}")` rendering (`1.0_f64` → `1`). +- **Refusal (NON-gated, CI-covered):** the lookup precedes data access, so + `aura run --real ` exits `2` with the refusal message and **no + local data** — a plain `cli_run.rs` integration test, closing the current + zero-coverage of the `--real` path's refusal branch. +- **Real-path honesty (gated, skip if no local data — same gating as `real_bars.rs`):** + `aura run --real GER40 ` emits `sim-optimal(pip_size=1)` and equity scaled + by `1.0`; deterministic across two runs (C1). +- **Regression (unchanged behaviour):** the synthetic `aura run` still emits + `sim-optimal(pip_size=0.0001)` — `cli_run.rs:35` untouched; the seam tests + `real_bars.rs` / `streaming_seam.rs` (AAPL.US at 0.0001, a mechanism fixture) stay + green. + +## Acceptance criteria + +aura's feature-acceptance criterion (CLAUDE.md): the audience reaches for it, it +measurably improves correctness, and it reintroduces no failure class a core +invariant exists to prevent. + +1. **Reached for naturally.** A researcher runs `aura run --real ` with no + pip literal; the engine supplies the right pip from the metadata channel. +2. **Measurably more correct.** `--real GER40` equity is scaled by `1.0`, not + `0.0001`; an un-specced instrument refuses instead of emitting nonsense — the + issue-#22 wrong-unit class is removed on the CLI real path (the worked example + spans GER40=1.0 vs EURUSD=0.0001 vs BTCUSD→refuse). +3. **No invariant regressed.** Pip stays non-scalar reference metadata beside the hot + path (C4/C7/C15) — never streamed, never on `Ctx`; the engine stays domain-free. + Determinism (C1) preserved (pip fixed per symbol). `SimBroker` single-instrument + unchanged (C7). Not via `Aura.toml` (C16/C17). + +## Deferred (follow-up) + +The runnable-example pip centralization (give `ger40_breakout_blueprint` a `pip_size` +param; source `build_harness`/compare pip from `instrument_spec`) is **out of this +cycle's scope** — the examples already bake the correct `1.0`, so it is a consistency +cleanup, not a bug fix. Filed as a follow-up `idea` issue.