diff --git a/docs/specs/0063-broker-foundation.md b/docs/specs/0063-broker-foundation.md new file mode 100644 index 0000000..12f00bd --- /dev/null +++ b/docs/specs/0063-broker-foundation.md @@ -0,0 +1,280 @@ +# 0063 broker-foundation — InstrumentSpec deploy metadata + PositionEvent schema — Design Spec + +**Date:** 2026-06-22 +**Status:** Draft — awaiting user spec review +**Authors:** orchestrator + Claude + +## Goal + +Lay the two data foundations the realistic-broker milestone (C10 A-side) stands +on, with **no broker and no derivation** yet — pure, fully-testable value types: + +1. **#113** — extend `InstrumentSpec` (aura-ingest) with the deploy-grade fields a + realistic broker needs to turn pips/volume into account currency + (`instrument_id`, `contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`, + `quote_currency`), and populate the vetted table for the existing + GER40/FRA40/EURUSD/GBPUSD/USDCAD set. The authored table is the **permanent + vetted floor** of the milestone's authored-override-wins resolution hierarchy + (the recorded-metadata tier is forward work, #124); the refuse-don't-guess + `None` arm stays the absolute floor. +2. **#114** — pin the canonical Rust representation of C10's derived + position-event table: `PositionAction { Buy, Sell, Close }` + `PositionEvent`, + living in `aura-engine` alongside `RunMetrics`/`ColumnarTrace` (a post-run + reduction, the same home as `summarize`). Pure schema — no friction, no + derivation logic. + +This is foundation only. `derive_position_events` (#115), the `RealisticBroker` +node (#116), and frictions/metrics/persist (#120–122) consume these shapes in +later cycles. + +## Architecture + +Both deliverables are **value types beside the hot path**, faithful to existing +contracts — they add no behaviour to the engine's run loop: + +- `InstrumentSpec` is non-scalar reference metadata held beside the hot path + (C7/C15) at the ingestion edge, keyed by symbol. C15's realization note already + promises this exact extension ("extensible to tick size / digits / quote + currency without a signature break"). The struct stays `Copy`. +- `PositionEvent` is the broker-independent audit row C10 already specifies down + to its columns (INDEX.md:471–501): `event_ts`, `action: i64` (buy/sell/close — + *direction IS the action*), `position_id: i64`, `instrument_id: i64`, + `volume: f64` unsigned; **no `open_ts`**; the signed-volume trick is forbidden. + It is a post-run value type (sibling of `RunMetrics`), never a per-`eval` node + output (C8) — which is why it lives as data in `aura-engine`, not as a node. + The engine stays headless and domain-free (C14): these are plain serializable + structs, no I/O, no instrument identity on the hot path. + +The one in-memory refinement over the ledger's columnar `action: i64`: the +canonical Rust type is a **closed enum** `PositionAction`, serde-encoded **as +i64** (ordinal `Buy=0, Sell=1, Close=2`). The enum makes an invalid action +unrepresentable in memory; the i64 encoding keeps the persisted/columnar form +ledger- and C7-faithful and makes #122's columnar projection trivial. An +out-of-range i64 fails `TryFrom` (the honesty lever, mirroring +`instrument_spec`'s `None`). + +## Concrete code shapes + +### Delivered code — behaviour this cycle ships (the empirical evidence) + +The extended vetted table (must-pass) and its refuse arm (must-fail-by-design): + +```rust +// aura-ingest/src/lib.rs — instrument_spec(), after extension +pub fn instrument_spec(symbol: &str) -> Option { + // (instrument_id, pip_size, contract_size, pip_value_per_lot, min_lot, lot_step, quote) + let s = match symbol { + // index CFDs: quote EUR, 1 contract = €1 per point, pip = 1.0 point + "GER40" => InstrumentSpec { instrument_id: 1, pip_size: 1.0, contract_size: 1.0, + pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01, + quote_currency: "EUR" }, + "FRA40" => InstrumentSpec { instrument_id: 2, pip_size: 1.0, contract_size: 1.0, + pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01, + quote_currency: "EUR" }, + // FX majors: standard lot = 100_000 units, pip = 0.0001, + // pip value = contract_size * pip_size in the quote currency (= 10) + "EURUSD" => InstrumentSpec { instrument_id: 3, pip_size: 0.0001, contract_size: 100_000.0, + pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01, + quote_currency: "USD" }, + "GBPUSD" => InstrumentSpec { instrument_id: 4, pip_size: 0.0001, contract_size: 100_000.0, + pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01, + quote_currency: "USD" }, + "USDCAD" => InstrumentSpec { instrument_id: 5, pip_size: 0.0001, contract_size: 100_000.0, + pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01, + quote_currency: "CAD" }, + _ => return None, // refuse-don't-guess — the absolute floor stays + }; + Some(s) +} +``` + +> **SUSPECT BYTES — vetted reference data, not validated by an existing test.** +> The contract numbers (contract sizes, pip values, min/lot steps) are +> industry-standard conventions, not broker-exact, and are **overridable by +> design** per the milestone's authored-override-wins contract. They are this +> spec's most fragile content; a later cycle (or an Aura.toml override) corrects +> any that prove wrong. The *invariants* over them are what the tests pin +> (below), not the literals' real-world exactness. + +The closed action mapping (must-pass round-trip + must-fail rejection): + +```rust +// aura-engine/src/report.rs +use std::convert::TryFrom; + +impl From for i64 { + fn from(a: PositionAction) -> i64 { + match a { PositionAction::Buy => 0, PositionAction::Sell => 1, PositionAction::Close => 2 } + } +} +impl TryFrom for PositionAction { + type Error = String; + fn try_from(v: i64) -> Result { + match v { + 0 => Ok(PositionAction::Buy), + 1 => Ok(PositionAction::Sell), + 2 => Ok(PositionAction::Close), + other => Err(format!("invalid PositionAction i64: {other}")), + } + } +} +// try_from(99) -> Err — an out-of-range action is rejected, never silently mapped. +``` + +### North star (cycle #115 — NOT built this cycle) + +How the foundation will be consumed, shown minimally for honesty (no code lands +here): + +```rust +// derive_position_events folds the recorded exposure first-difference into rows. +// A long open, then a stop-and-reverse (Close + Sell at the SAME event_ts): +PositionEvent { event_ts: t0, action: PositionAction::Buy, position_id: 1, instrument_id: 3, volume: lots }; +PositionEvent { event_ts: t9, action: PositionAction::Close, position_id: 1, instrument_id: 3, volume: lots }; +PositionEvent { event_ts: t9, action: PositionAction::Sell, position_id: 2, instrument_id: 3, volume: lots }; +// sizing lots = round_to(|exposure| * account / contract_value, lot_step); sub-min_lot drops (#115). +``` + +### Before → after struct shapes (secondary) + +`InstrumentSpec` — add six fields, keep `Copy`: + +```rust +// BEFORE +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct InstrumentSpec { + pub pip_size: f64, +} + +// AFTER +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct InstrumentSpec { + /// Stable per-instrument id — the position-event table's `instrument_id` + /// column. Assigned once, never renumbered. + pub instrument_id: i64, + /// Price units per pip / point (> 0). The sim-optimal broker's divisor. + pub pip_size: f64, + /// Units of the base instrument per 1.0 lot (FX standard lot = 100_000; + /// an index CFD = 1). + pub contract_size: f64, + /// Value of one pip per 1.0 lot, in the quote currency. Equals + /// `contract_size * pip_size` for every vetted instrument here; carried + /// explicitly because a real broker may not make it exactly so. + pub pip_value_per_lot: f64, + /// Smallest tradable volume and the volume granularity, both in lots. + pub min_lot: f64, + pub lot_step: f64, + /// Quote currency (ISO label only this cycle — no FX conversion yet). + pub quote_currency: &'static str, +} +``` + +`PositionAction` + `PositionEvent` — new in `aura-engine/src/report.rs`: + +```rust +/// The three position-event actions (C10). Direction IS the action; volume is +/// unsigned. Serde-encoded as its i64 mapping so the persisted/columnar form +/// stays C7-scalar and ledger-faithful (`action: i64`). +#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(into = "i64", try_from = "i64")] +pub enum PositionAction { Buy, Sell, Close } + +/// One row of C10's derived position-event table — the broker-independent audit +/// view (the first difference of the exposure state). A post-run value type +/// (sibling of `RunMetrics`), NOT a per-`eval` node output (C8). Multiple events +/// may share one `event_ts` (a reversal: Close then open, close-before-open). No +/// `open_ts` — a position's open time is its opening event's `event_ts`. +#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)] +pub struct PositionEvent { + pub event_ts: Timestamp, + pub action: PositionAction, + /// Monotonic, assigned at open. A Close references an existing `position_id`. + pub position_id: i64, + pub instrument_id: i64, + /// Lots, unsigned. A partial close carries its own (smaller) volume. + pub volume: f64, +} +``` + +## Components + +- **aura-ingest `InstrumentSpec` (extend)** — `crates/aura-ingest/src/lib.rs:346`. + Add the six fields; keep `#[derive(Clone, Copy, Debug, PartialEq)]`. Update the + doc comment to name the new fields. `Copy` is preserved because + `quote_currency` is `&'static str`, not `String`. +- **aura-ingest `instrument_spec()` table (populate)** — `:358`. Fill all six + fields for the five vetted symbols; keep the `_ => None` refuse arm. +- **aura-engine `PositionAction` (new)** — `crates/aura-engine/src/report.rs`. + Closed enum + `From for i64` + `TryFrom` + serde + `into`/`try_from` attributes. Re-export from the crate root if `RunMetrics` is + (match the existing re-export pattern). +- **aura-engine `PositionEvent` (new)** — same file. The row struct above. + +## Data flow + +No runtime/streaming flow changes this cycle — these are value types. The +relationships they encode (consumed later): + +- `instrument_spec(symbol)` → `InstrumentSpec` is resolved at the ingestion edge + at bootstrap; the broker (#116) and `derive_position_events` (#115) read the + **resolved injected spec**, never call `instrument_spec()` directly, so a + resolver (the override hierarchy, #124) can sit in front later without touching + them. (Wiring the injection is #115/#116, not this cycle.) +- `PositionEvent` rows are produced post-run by `derive_position_events` (#115) + from the recorded exposure stream; persisted dup-ts-safe (#122). This cycle + only fixes the shape every downstream piece wires to. + +## Error handling + +- **Un-specced symbol** → `instrument_spec()` returns `None`; the existing + `instrument_spec_or_refuse` (aura-cli `main.rs:766`) keeps refusing. No guessed + metadata enters a run. +- **Out-of-range action i64** → `PositionAction::try_from` returns `Err` (a + message naming the bad value); serde deserialization of a bad `action` fails + rather than silently coercing. +- **Compile fan-out (contained):** adding fields breaks every `InstrumentSpec { + pip_size }` literal construction — the table constructor and the three test + literals in `aura-ingest/src/lib.rs` (greps confirm: no other crate constructs + the struct; `instrument_spec_or_refuse` and the examples only *access* + `.pip_size`, which is unaffected). The plan threads all construction sites in + one task so the crate compiles before its build gate. +- No new panics; the engine stays headless (C14). + +## Testing strategy + +- **aura-ingest** (extend `instrument_spec_lookup_by_symbol`, add cases): + - Each of the five symbols returns the full expected `InstrumentSpec` (all six + fields), asserted by value. + - `instrument_spec("NONEXISTENT") == None` (refuse arm intact). + - Invariant: `pip_value_per_lot == contract_size * pip_size` for every vetted + instrument (pins the cross-field relationship, not the literals' real-world + exactness). + - Invariant: the five `instrument_id`s are distinct (uniqueness). +- **aura-engine** (new tests in `report.rs`): + - `PositionAction` round-trips: each variant → `i64` → `try_from` → same + variant. + - `PositionAction::try_from(3)` (and a negative) → `Err`. + - A `PositionEvent` serde JSON round-trip equals the original, and the + serialized `action` is a **bare integer** (e.g. the JSON contains `"action":1` + for `Sell`), confirming the i64 encoding. + - A reversal pair (Close then Sell) sharing one `event_ts` constructs and is + representable (documents close-before-open ordering; the ordering *logic* is + #115). +- **Workspace:** `cargo test --workspace` green; `cargo clippy --workspace + --all-targets -- -D warnings` clean. + +## Acceptance criteria + +1. `InstrumentSpec` carries the six new deploy-grade fields and stays `Copy`; the + vetted table populates all five instruments; the `_ => None` refuse arm + remains. +2. `PositionAction` is a closed three-value enum with a **total, round-tripping** + i64 mapping (`Buy=0, Sell=1, Close=2`); an out-of-range i64 is rejected. +3. `PositionEvent` matches the C10 column spec exactly — `event_ts`, enum + `action`, `position_id`, `instrument_id`, unsigned `volume`; **no `open_ts`**; + no signed-volume trick. Its serde shape is pinned (action encodes as i64). +4. Both types live where the ledger/issues place them (`InstrumentSpec` in + aura-ingest at the ingestion edge; `PositionEvent` in aura-engine beside + `RunMetrics`), and the engine stays headless (C14) — no broker, no derivation, + no I/O this cycle. +5. `cargo test --workspace` passes and `clippy -D warnings` is clean.