@@ -0,0 +1,235 @@
|
||||
# Per-Instrument Pip Metadata — Design Spec
|
||||
|
||||
**Date:** 2026-06-18
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
|
||||
## Goal
|
||||
|
||||
The data-server symbol stream carries only price bars — it brings **no instrument
|
||||
metadata**, in particular no pip/point size. That absence is the bug behind #22:
|
||||
the sim-optimal broker is therefore constructed with a single global `pip_size`
|
||||
literal (`0.0001`, correct only for 5-decimal FX), so on the CLI real path
|
||||
`aura run --real <symbol>` produces wrong-unit pip equity for any non-FX instrument
|
||||
(issue-#22 table: `AAPL.US` → `-8,693,884`, `BTCUSD` → `-3.2e9`).
|
||||
|
||||
This cycle adds the missing **channel to specify per-instrument pip metadata**, and
|
||||
threads it through the one observably-broken path — the CLI `aura run --real`.
|
||||
|
||||
**Scope (narrowed, user-directed 2026-06-18, recorded on #22):** the metadata
|
||||
channel + the CLI real path + the unknown-symbol refusal. The runnable real-data
|
||||
examples (`ger40_breakout_*`) already bake the **correct** pip (`1.0` for the GER40/
|
||||
FRA40 indices) — they are *not* buggy; centralizing them through the lookup is a
|
||||
consistency cleanup **deferred to a follow-up `idea` issue**. `SimBroker` is
|
||||
unchanged. Portfolio/multi-instrument machinery is out of scope (`SimBroker` is and
|
||||
stays single-instrument); **not** via `Aura.toml`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two new pieces + one threaded path, placed by the crate dependency graph
|
||||
(`aura-cli → aura-ingest → {aura-std, aura-engine, aura-core}`):
|
||||
|
||||
1. **`InstrumentSpec` — a typed per-instrument reference-metadata carrier**, home
|
||||
**`aura-ingest`** (the ingestion/source edge, where `symbol` exists). Minimal
|
||||
today (`pip_size` only); extensible (tick size, digits, quote currency) without a
|
||||
signature break. It is metadata beside the hot path (C4/C7/C15): never streamed,
|
||||
never on `Ctx`; the engine stays domain-free.
|
||||
|
||||
2. **`instrument_spec(symbol) -> Option<InstrumentSpec>` — the lookup**, home
|
||||
**`aura-ingest`**. A static, Rust-authored vetted table (NOT `Aura.toml` — the
|
||||
later `Aura.toml` schema can subsume it). `None` for an unknown symbol: the
|
||||
honesty lever — a real run for an instrument with no vetted pip cannot proceed
|
||||
with a guessed value.
|
||||
|
||||
3. **CLI real path threads the looked-up pip.** `SimBroker` keeps its `pip_size: f64`
|
||||
divisor constructor (synthetic/seam/test fixtures legitimately build it with an
|
||||
explicit value). The honesty is realized at the real-data call site, which obtains
|
||||
pip **only** through `instrument_spec(symbol)` and never a literal.
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### Worked user-facing behaviour (the acceptance evidence)
|
||||
|
||||
The same command over differently-quoted instruments yields honestly-scaled pip
|
||||
equity, with no pip literal in the invocation; an un-specced instrument refuses:
|
||||
|
||||
```text
|
||||
$ aura run --real GER40 --from … --to …
|
||||
{"manifest":{…,"broker":"sim-optimal(pip_size=1)"}, "metrics":{…}} # index → pip 1.0
|
||||
$ aura run --real EURUSD --from … --to …
|
||||
{"manifest":{…,"broker":"sim-optimal(pip_size=0.0001)"}, "metrics":{…}} # 5-dp FX → pip 0.0001
|
||||
$ aura run --real BTCUSD
|
||||
aura: no vetted pip/instrument spec for symbol 'BTCUSD' — refusing to run a real
|
||||
instrument with a guessed pip (add it to the instrument table). [exit 2]
|
||||
```
|
||||
|
||||
GER40 equity is now scaled by `1.0` (was `0.0001`), so it is comparable to a real FX
|
||||
run — this is the cross-asset mismatch the feature exists to fix, shown end-to-end.
|
||||
|
||||
### New: `InstrumentSpec` + `instrument_spec` (aura-ingest)
|
||||
|
||||
```rust
|
||||
// crates/aura-ingest/src/lib.rs (new)
|
||||
|
||||
/// Per-instrument reference metadata held beside the hot path (C7/C15): non-scalar,
|
||||
/// never streamed. Minimal today — extend with tick size / digits / quote currency
|
||||
/// as later cycles need, without breaking this signature.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub struct InstrumentSpec {
|
||||
/// Price units per pip / point (> 0). The sim-optimal broker's divisor.
|
||||
pub pip_size: f64,
|
||||
}
|
||||
|
||||
/// Look up per-instrument reference metadata by `symbol` at the ingestion edge.
|
||||
/// Rust-authored vetted table (NOT `Aura.toml`); `None` for an instrument with no
|
||||
/// vetted spec — the caller must refuse rather than guess a pip.
|
||||
pub fn instrument_spec(symbol: &str) -> Option<InstrumentSpec> {
|
||||
// SHAPE: explicit per-symbol entries, None for unknown. Indices quote in points
|
||||
// (pip = 1.0); 5-decimal FX = 0.0001. NB: JPY pairs are 0.01, NOT 0.0001 — seed
|
||||
// only vetted values. Planner pins the full set against the data-server symbols.
|
||||
let pip = match symbol {
|
||||
"GER40" | "FRA40" => 1.0, // index points
|
||||
"EURUSD" | "GBPUSD" | "USDCAD" => 0.0001, // 5-dp FX majors (vetted subset)
|
||||
_ => return None,
|
||||
};
|
||||
Some(InstrumentSpec { pip_size: pip })
|
||||
}
|
||||
```
|
||||
|
||||
### Changed: CLI real path (aura-cli)
|
||||
|
||||
```rust
|
||||
// crates/aura-cli/src/main.rs
|
||||
|
||||
// BEFORE: sample_harness hard-codes the broker pip; the manifest label is a literal
|
||||
fn sample_harness() -> (Harness, …) { … SimBroker::new(0.0001) … } // :91
|
||||
fn sim_optimal_manifest(params, window, seed) -> RunManifest {
|
||||
RunManifest { …, broker: "sim-optimal(pip_size=0.0001)".to_string() } // :142
|
||||
}
|
||||
|
||||
// AFTER: pip is a parameter, threaded from the per-instrument lookup
|
||||
fn sample_harness(pip_size: f64) -> (Harness, …) { … SimBroker::new(pip_size) … }
|
||||
fn sim_optimal_manifest(params, window, seed, pip_size: f64) -> RunManifest {
|
||||
RunManifest { …, broker: format!("sim-optimal(pip_size={pip_size})") } // 1.0→"1", 0.0001→"0.0001"
|
||||
}
|
||||
|
||||
fn run_sample() -> RunReport { // synthetic — UNCHANGED behaviour
|
||||
let (mut h, …) = sample_harness(0.0001); // sample instrument; cli_run.rs:35 pin holds
|
||||
… sim_optimal_manifest(…, 0.0001) …
|
||||
}
|
||||
|
||||
fn run_sample_real(symbol: &str, from_ms, to_ms) -> RunReport {
|
||||
// Look up FIRST — before any data access — so an un-specced symbol refuses
|
||||
// without touching local data (CI-testable, no fixture).
|
||||
let spec = match aura_ingest::instrument_spec(symbol) {
|
||||
Some(s) => s,
|
||||
None => { eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' …"); std::process::exit(2); }
|
||||
};
|
||||
let (mut h, …) = sample_harness(spec.pip_size);
|
||||
… // unchanged streaming over M1FieldSource::open(symbol, …); has_symbol/window guards follow
|
||||
… sim_optimal_manifest(…, spec.pip_size) …
|
||||
}
|
||||
```
|
||||
|
||||
`SimBroker::label()` already renders `SimBroker({pip_size})` (sim_broker.rs:99), so
|
||||
the node label tracks the looked-up pip; only the manifest *string* (a hand-built
|
||||
literal today) needs parameterizing.
|
||||
|
||||
### Call-site propagation (workspace-wide, mechanical)
|
||||
|
||||
Two signatures gain a `pip_size` parameter; the type checker enumerates every
|
||||
caller, and every caller that is NOT the real path passes its existing `0.0001`
|
||||
unchanged — behaviour-preserving, no emitted-pip change, so existing drift pins stay
|
||||
green:
|
||||
|
||||
- `sample_harness(pip_size)` — **4** sites (main.rs:150,184,589,946). `run_sample_real`
|
||||
passes the lookup; the synthetic `run_sample` + the two enriched/seeded sample
|
||||
callers pass `0.0001`.
|
||||
- `sim_optimal_manifest(…, pip_size)` — **8** sites (main.rs:163,226,379,498,520,597,853,954).
|
||||
`run_sample_real` passes the lookup; the synthetic run / sweep / walkforward
|
||||
(IS+OOS) / MC / macd / seeded callers pass `0.0001`.
|
||||
|
||||
The exact enumeration is `rustc`'s: the signature change is the complete forcing
|
||||
function; the rule is "the real path passes the lookup, every other caller passes its
|
||||
existing `0.0001`."
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Crate | Change |
|
||||
|---|---|---|
|
||||
| `InstrumentSpec` struct | aura-ingest | **new** — `{ pip_size: f64 }`, derives Clone/Copy/Debug/PartialEq |
|
||||
| `instrument_spec(symbol)` | aura-ingest | **new** — static vetted table, `Option<InstrumentSpec>` |
|
||||
| `sample_harness` | aura-cli | takes `pip_size: f64` (4 call sites propagate) |
|
||||
| `sim_optimal_manifest` | aura-cli | takes `pip_size: f64`, formats it into the broker string (8 call sites propagate) |
|
||||
| `run_sample_real` | aura-cli | look up by symbol FIRST; refuse (exit 2) on `None`; thread pip |
|
||||
| `run_sample` + synthetic/sweep/mc/wf/macd callers | aura-cli | pass the unchanged `0.0001` |
|
||||
| `SimBroker` | aura-std | **unchanged** (keeps `new(f64)` / `builder(f64)`) |
|
||||
|
||||
## Data flow
|
||||
|
||||
Real path (changed): `symbol` → `instrument_spec(symbol)` (before any data access) →
|
||||
`InstrumentSpec { pip_size }` → `sample_harness(pip_size)` builds `SimBroker::new(pip_size)`
|
||||
→ equity integrates `exposure·(Δprice)/pip_size` (sim_broker.rs:90) →
|
||||
`sim_optimal_manifest(…, pip_size)` records the real pip in the manifest broker
|
||||
string. No pip literal on the path.
|
||||
|
||||
Synthetic / seam / test paths (unchanged): construct `SimBroker::new(<literal>)`
|
||||
directly; no lookup.
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Unknown / un-specced instrument on a real run (new guard, checked FIRST):**
|
||||
`run_sample_real` calls `instrument_spec(symbol)` before any data access. `None` →
|
||||
a clear stderr refusal + `exit(2)`, without touching local data (so it is
|
||||
observable and CI-testable with no fixture). Deliberate, observable **behaviour
|
||||
change**: a real-data run for a non-seeded symbol previously produced silent
|
||||
wrong-unit pips and now refuses (user-directed, recorded on #22).
|
||||
- **Downstream guards unchanged and not depended on.** A specced symbol flows through
|
||||
the existing `run_sample_real` body untouched — the `has_symbol`/empty-window usage
|
||||
error and the streaming run are not modified; the spec adds a guard *ahead* of them
|
||||
and rests on no current-behaviour claim about that pre-existing path.
|
||||
- `pip_size > 0` is already asserted in `SimBroker::new` (sim_broker.rs:48); the table
|
||||
holds only positive values.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Lookup unit (aura-ingest):** `instrument_spec("GER40") == Some(pip_size:1.0)`;
|
||||
`instrument_spec("EURUSD") == Some(0.0001)`; `instrument_spec("NONEXISTENT") == None`.
|
||||
- **Manifest pip (aura-cli):** `sim_optimal_manifest(…, 1.0)` → `broker ==
|
||||
"sim-optimal(pip_size=1)"`; with `0.0001` → the existing string. Pins the exact
|
||||
`format!("{pip_size}")` rendering (`1.0_f64` → `1`).
|
||||
- **Refusal (NON-gated, CI-covered):** the lookup precedes data access, so
|
||||
`aura run --real <un-specced-symbol>` exits `2` with the refusal message and **no
|
||||
local data** — a plain `cli_run.rs` integration test, closing the current
|
||||
zero-coverage of the `--real` path's refusal branch.
|
||||
- **Real-path honesty (gated, skip if no local data — same gating as `real_bars.rs`):**
|
||||
`aura run --real GER40 <window>` emits `sim-optimal(pip_size=1)` and equity scaled
|
||||
by `1.0`; deterministic across two runs (C1).
|
||||
- **Regression (unchanged behaviour):** the synthetic `aura run` still emits
|
||||
`sim-optimal(pip_size=0.0001)` — `cli_run.rs:35` untouched; the seam tests
|
||||
`real_bars.rs` / `streaming_seam.rs` (AAPL.US at 0.0001, a mechanism fixture) stay
|
||||
green.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
aura's feature-acceptance criterion (CLAUDE.md): the audience reaches for it, it
|
||||
measurably improves correctness, and it reintroduces no failure class a core
|
||||
invariant exists to prevent.
|
||||
|
||||
1. **Reached for naturally.** A researcher runs `aura run --real <symbol>` with no
|
||||
pip literal; the engine supplies the right pip from the metadata channel.
|
||||
2. **Measurably more correct.** `--real GER40` equity is scaled by `1.0`, not
|
||||
`0.0001`; an un-specced instrument refuses instead of emitting nonsense — the
|
||||
issue-#22 wrong-unit class is removed on the CLI real path (the worked example
|
||||
spans GER40=1.0 vs EURUSD=0.0001 vs BTCUSD→refuse).
|
||||
3. **No invariant regressed.** Pip stays non-scalar reference metadata beside the hot
|
||||
path (C4/C7/C15) — never streamed, never on `Ctx`; the engine stays domain-free.
|
||||
Determinism (C1) preserved (pip fixed per symbol). `SimBroker` single-instrument
|
||||
unchanged (C7). Not via `Aura.toml` (C16/C17).
|
||||
|
||||
## Deferred (follow-up)
|
||||
|
||||
The runnable-example pip centralization (give `ger40_breakout_blueprint` a `pip_size`
|
||||
param; source `build_harness`/compare pip from `instrument_spec`) is **out of this
|
||||
cycle's scope** — the examples already bake the correct `1.0`, so it is a consistency
|
||||
cleanup, not a bug fix. Filed as a follow-up `idea` issue.
|
||||
Reference in New Issue
Block a user