spec: 0063 broker-foundation — InstrumentSpec deploy metadata + PositionEvent schema (boss-signed)

The broker milestone's (C10 A-side) foundation: two pure value-type deliverables, no broker and no derivation yet. - #113: extend InstrumentSpec with deploy-grade fields (instrument_id, contract_size, pip_value_per_lot, min_lot, lot_step, quote_currency) + populate the vetted table for the five instruments; refuse-don't-guess arm stays. Permanent authored floor of the override hierarchy (#124). - #114: PositionAction { Buy, Sell, Close } (closed enum, serde-encoded as i64) + PositionEvent row in aura-engine, faithful to the C10 column spec (no open_ts, unsigned volume, direction is the action). Signed under /boss on a grounding-check PASS (all current-state assumptions ratified by green tests; the two new types are greenfield, expected). Derived fork decisions (quote_currency &'static str; action enum encoded as i64 0/1/2; stable instrument_id + overridable vetted values) recorded on #113. refs #113 #114
2026-06-22 19:14:07 +02:00
parent 3638d4822e
commit cc0dce19a3
1 changed files with 280 additions and 0 deletions
@@ -0,0 +1,280 @@
+# 0063 broker-foundation — InstrumentSpec deploy metadata + PositionEvent schema — Design Spec
+
+**Date:** 2026-06-22
+**Status:** Draft — awaiting user spec review
+**Authors:** orchestrator + Claude
+
+## Goal
+
+Lay the two data foundations the realistic-broker milestone (C10 A-side) stands
+on, with **no broker and no derivation** yet — pure, fully-testable value types:
+
+1. **#113** — extend `InstrumentSpec` (aura-ingest) with the deploy-grade fields a
+   realistic broker needs to turn pips/volume into account currency
+   (`instrument_id`, `contract_size`, `pip_value_per_lot`, `min_lot`, `lot_step`,
+   `quote_currency`), and populate the vetted table for the existing
+   GER40/FRA40/EURUSD/GBPUSD/USDCAD set. The authored table is the **permanent
+   vetted floor** of the milestone's authored-override-wins resolution hierarchy
+   (the recorded-metadata tier is forward work, #124); the refuse-don't-guess
+   `None` arm stays the absolute floor.
+2. **#114** — pin the canonical Rust representation of C10's derived
+   position-event table: `PositionAction { Buy, Sell, Close }` + `PositionEvent`,
+   living in `aura-engine` alongside `RunMetrics`/`ColumnarTrace` (a post-run
+   reduction, the same home as `summarize`). Pure schema — no friction, no
+   derivation logic.
+
+This is foundation only. `derive_position_events` (#115), the `RealisticBroker`
+node (#116), and frictions/metrics/persist (#120–122) consume these shapes in
+later cycles.
+
+## Architecture
+
+Both deliverables are **value types beside the hot path**, faithful to existing
+contracts — they add no behaviour to the engine's run loop:
+
+- `InstrumentSpec` is non-scalar reference metadata held beside the hot path
+  (C7/C15) at the ingestion edge, keyed by symbol. C15's realization note already
+  promises this exact extension ("extensible to tick size / digits / quote
+  currency without a signature break"). The struct stays `Copy`.
+- `PositionEvent` is the broker-independent audit row C10 already specifies down
+  to its columns (INDEX.md:471–501): `event_ts`, `action: i64` (buy/sell/close —
+  *direction IS the action*), `position_id: i64`, `instrument_id: i64`,
+  `volume: f64` unsigned; **no `open_ts`**; the signed-volume trick is forbidden.
+  It is a post-run value type (sibling of `RunMetrics`), never a per-`eval` node
+  output (C8) — which is why it lives as data in `aura-engine`, not as a node.
+  The engine stays headless and domain-free (C14): these are plain serializable
+  structs, no I/O, no instrument identity on the hot path.
+
+The one in-memory refinement over the ledger's columnar `action: i64`: the
+canonical Rust type is a **closed enum** `PositionAction`, serde-encoded **as
+i64** (ordinal `Buy=0, Sell=1, Close=2`). The enum makes an invalid action
+unrepresentable in memory; the i64 encoding keeps the persisted/columnar form
+ledger- and C7-faithful and makes #122's columnar projection trivial. An
+out-of-range i64 fails `TryFrom` (the honesty lever, mirroring
+`instrument_spec`'s `None`).
+
+## Concrete code shapes
+
+### Delivered code — behaviour this cycle ships (the empirical evidence)
+
+The extended vetted table (must-pass) and its refuse arm (must-fail-by-design):
+
+```rust
+// aura-ingest/src/lib.rs — instrument_spec(), after extension
+pub fn instrument_spec(symbol: &str) -> Option<InstrumentSpec> {
+    // (instrument_id, pip_size, contract_size, pip_value_per_lot, min_lot, lot_step, quote)
+    let s = match symbol {
+        // index CFDs: quote EUR, 1 contract = €1 per point, pip = 1.0 point
+        "GER40" => InstrumentSpec { instrument_id: 1, pip_size: 1.0, contract_size: 1.0,
+                                    pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01,
+                                    quote_currency: "EUR" },
+        "FRA40" => InstrumentSpec { instrument_id: 2, pip_size: 1.0, contract_size: 1.0,
+                                    pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01,
+                                    quote_currency: "EUR" },
+        // FX majors: standard lot = 100_000 units, pip = 0.0001,
+        // pip value = contract_size * pip_size in the quote currency (= 10)
+        "EURUSD" => InstrumentSpec { instrument_id: 3, pip_size: 0.0001, contract_size: 100_000.0,
+                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
+                                     quote_currency: "USD" },
+        "GBPUSD" => InstrumentSpec { instrument_id: 4, pip_size: 0.0001, contract_size: 100_000.0,
+                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
+                                     quote_currency: "USD" },
+        "USDCAD" => InstrumentSpec { instrument_id: 5, pip_size: 0.0001, contract_size: 100_000.0,
+                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
+                                     quote_currency: "CAD" },
+        _ => return None, // refuse-don't-guess — the absolute floor stays
+    };
+    Some(s)
+}
+```
+
+> **SUSPECT BYTES — vetted reference data, not validated by an existing test.**
+> The contract numbers (contract sizes, pip values, min/lot steps) are
+> industry-standard conventions, not broker-exact, and are **overridable by
+> design** per the milestone's authored-override-wins contract. They are this
+> spec's most fragile content; a later cycle (or an Aura.toml override) corrects
+> any that prove wrong. The *invariants* over them are what the tests pin
+> (below), not the literals' real-world exactness.
+
+The closed action mapping (must-pass round-trip + must-fail rejection):
+
+```rust
+// aura-engine/src/report.rs
+use std::convert::TryFrom;
+
+impl From<PositionAction> for i64 {
+    fn from(a: PositionAction) -> i64 {
+        match a { PositionAction::Buy => 0, PositionAction::Sell => 1, PositionAction::Close => 2 }
+    }
+}
+impl TryFrom<i64> for PositionAction {
+    type Error = String;
+    fn try_from(v: i64) -> Result<Self, String> {
+        match v {
+            0 => Ok(PositionAction::Buy),
+            1 => Ok(PositionAction::Sell),
+            2 => Ok(PositionAction::Close),
+            other => Err(format!("invalid PositionAction i64: {other}")),
+        }
+    }
+}
+// try_from(99) -> Err — an out-of-range action is rejected, never silently mapped.
+```
+
+### North star (cycle #115 — NOT built this cycle)
+
+How the foundation will be consumed, shown minimally for honesty (no code lands
+here):
+
+```rust
+// derive_position_events folds the recorded exposure first-difference into rows.
+// A long open, then a stop-and-reverse (Close + Sell at the SAME event_ts):
+PositionEvent { event_ts: t0, action: PositionAction::Buy,   position_id: 1, instrument_id: 3, volume: lots };
+PositionEvent { event_ts: t9, action: PositionAction::Close, position_id: 1, instrument_id: 3, volume: lots };
+PositionEvent { event_ts: t9, action: PositionAction::Sell,  position_id: 2, instrument_id: 3, volume: lots };
+// sizing lots = round_to(|exposure| * account / contract_value, lot_step); sub-min_lot drops (#115).
+```
+
+### Before → after struct shapes (secondary)
+
+`InstrumentSpec` — add six fields, keep `Copy`:
+
+```rust
+// BEFORE
+#[derive(Clone, Copy, Debug, PartialEq)]
+pub struct InstrumentSpec {
+    pub pip_size: f64,
+}
+
+// AFTER
+#[derive(Clone, Copy, Debug, PartialEq)]
+pub struct InstrumentSpec {
+    /// Stable per-instrument id — the position-event table's `instrument_id`
+    /// column. Assigned once, never renumbered.
+    pub instrument_id: i64,
+    /// Price units per pip / point (> 0). The sim-optimal broker's divisor.
+    pub pip_size: f64,
+    /// Units of the base instrument per 1.0 lot (FX standard lot = 100_000;
+    /// an index CFD = 1).
+    pub contract_size: f64,
+    /// Value of one pip per 1.0 lot, in the quote currency. Equals
+    /// `contract_size * pip_size` for every vetted instrument here; carried
+    /// explicitly because a real broker may not make it exactly so.
+    pub pip_value_per_lot: f64,
+    /// Smallest tradable volume and the volume granularity, both in lots.
+    pub min_lot: f64,
+    pub lot_step: f64,
+    /// Quote currency (ISO label only this cycle — no FX conversion yet).
+    pub quote_currency: &'static str,
+}
+```
+
+`PositionAction` + `PositionEvent` — new in `aura-engine/src/report.rs`:
+
+```rust
+/// The three position-event actions (C10). Direction IS the action; volume is
+/// unsigned. Serde-encoded as its i64 mapping so the persisted/columnar form
+/// stays C7-scalar and ledger-faithful (`action: i64`).
+#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
+#[serde(into = "i64", try_from = "i64")]
+pub enum PositionAction { Buy, Sell, Close }
+
+/// One row of C10's derived position-event table — the broker-independent audit
+/// view (the first difference of the exposure state). A post-run value type
+/// (sibling of `RunMetrics`), NOT a per-`eval` node output (C8). Multiple events
+/// may share one `event_ts` (a reversal: Close then open, close-before-open). No
+/// `open_ts` — a position's open time is its opening event's `event_ts`.
+#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
+pub struct PositionEvent {
+    pub event_ts: Timestamp,
+    pub action: PositionAction,
+    /// Monotonic, assigned at open. A Close references an existing `position_id`.
+    pub position_id: i64,
+    pub instrument_id: i64,
+    /// Lots, unsigned. A partial close carries its own (smaller) volume.
+    pub volume: f64,
+}
+```
+
+## Components
+
+- **aura-ingest `InstrumentSpec` (extend)** — `crates/aura-ingest/src/lib.rs:346`.
+  Add the six fields; keep `#[derive(Clone, Copy, Debug, PartialEq)]`. Update the
+  doc comment to name the new fields. `Copy` is preserved because
+  `quote_currency` is `&'static str`, not `String`.
+- **aura-ingest `instrument_spec()` table (populate)** — `:358`. Fill all six
+  fields for the five vetted symbols; keep the `_ => None` refuse arm.
+- **aura-engine `PositionAction` (new)** — `crates/aura-engine/src/report.rs`.
+  Closed enum + `From<PositionAction> for i64` + `TryFrom<i64>` + serde
+  `into`/`try_from` attributes. Re-export from the crate root if `RunMetrics` is
+  (match the existing re-export pattern).
+- **aura-engine `PositionEvent` (new)** — same file. The row struct above.
+
+## Data flow
+
+No runtime/streaming flow changes this cycle — these are value types. The
+relationships they encode (consumed later):
+
+- `instrument_spec(symbol)` → `InstrumentSpec` is resolved at the ingestion edge
+  at bootstrap; the broker (#116) and `derive_position_events` (#115) read the
+  **resolved injected spec**, never call `instrument_spec()` directly, so a
+  resolver (the override hierarchy, #124) can sit in front later without touching
+  them. (Wiring the injection is #115/#116, not this cycle.)
+- `PositionEvent` rows are produced post-run by `derive_position_events` (#115)
+  from the recorded exposure stream; persisted dup-ts-safe (#122). This cycle
+  only fixes the shape every downstream piece wires to.
+
+## Error handling
+
+- **Un-specced symbol** → `instrument_spec()` returns `None`; the existing
+  `instrument_spec_or_refuse` (aura-cli `main.rs:766`) keeps refusing. No guessed
+  metadata enters a run.
+- **Out-of-range action i64** → `PositionAction::try_from` returns `Err` (a
+  message naming the bad value); serde deserialization of a bad `action` fails
+  rather than silently coercing.
+- **Compile fan-out (contained):** adding fields breaks every `InstrumentSpec {
+  pip_size }` literal construction — the table constructor and the three test
+  literals in `aura-ingest/src/lib.rs` (greps confirm: no other crate constructs
+  the struct; `instrument_spec_or_refuse` and the examples only *access*
+  `.pip_size`, which is unaffected). The plan threads all construction sites in
+  one task so the crate compiles before its build gate.
+- No new panics; the engine stays headless (C14).
+
+## Testing strategy
+
+- **aura-ingest** (extend `instrument_spec_lookup_by_symbol`, add cases):
+  - Each of the five symbols returns the full expected `InstrumentSpec` (all six
+    fields), asserted by value.
+  - `instrument_spec("NONEXISTENT") == None` (refuse arm intact).
+  - Invariant: `pip_value_per_lot == contract_size * pip_size` for every vetted
+    instrument (pins the cross-field relationship, not the literals' real-world
+    exactness).
+  - Invariant: the five `instrument_id`s are distinct (uniqueness).
+- **aura-engine** (new tests in `report.rs`):
+  - `PositionAction` round-trips: each variant → `i64` → `try_from` → same
+    variant.
+  - `PositionAction::try_from(3)` (and a negative) → `Err`.
+  - A `PositionEvent` serde JSON round-trip equals the original, and the
+    serialized `action` is a **bare integer** (e.g. the JSON contains `"action":1`
+    for `Sell`), confirming the i64 encoding.
+  - A reversal pair (Close then Sell) sharing one `event_ts` constructs and is
+    representable (documents close-before-open ordering; the ordering *logic* is
+    #115).
+- **Workspace:** `cargo test --workspace` green; `cargo clippy --workspace
+  --all-targets -- -D warnings` clean.
+
+## Acceptance criteria
+
+1. `InstrumentSpec` carries the six new deploy-grade fields and stays `Copy`; the
+   vetted table populates all five instruments; the `_ => None` refuse arm
+   remains.
+2. `PositionAction` is a closed three-value enum with a **total, round-tripping**
+   i64 mapping (`Buy=0, Sell=1, Close=2`); an out-of-range i64 is rejected.
+3. `PositionEvent` matches the C10 column spec exactly — `event_ts`, enum
+   `action`, `position_id`, `instrument_id`, unsigned `volume`; **no `open_ts`**;
+   no signed-volume trick. Its serde shape is pinned (action encodes as i64).
+4. Both types live where the ledger/issues place them (`InstrumentSpec` in
+   aura-ingest at the ingestion edge; `PositionEvent` in aura-engine beside
+   `RunMetrics`), and the engine stays headless (C14) — no broker, no derivation,
+   no I/O this cycle.
+5. `cargo test --workspace` passes and `clippy -D warnings` is clean.