Aura/docs/specs/0063-broker-foundation.md

0063 broker-foundation — InstrumentSpec deploy metadata + PositionEvent schema — Design Spec

Date: 2026-06-22 Status: Draft — awaiting user spec review Authors: orchestrator + Claude

Goal

Lay the two data foundations the realistic-broker milestone (C10 A-side) stands on, with no broker and no derivation yet — pure, fully-testable value types:

1. #113 — extend InstrumentSpec (aura-ingest) with the deploy-grade fields a realistic broker needs to turn pips/volume into account currency (instrument_id, contract_size, pip_value_per_lot, min_lot, lot_step, quote_currency), and populate the vetted table for the existing GER40/FRA40/EURUSD/GBPUSD/USDCAD set. The authored table is the permanent vetted floor of the milestone's authored-override-wins resolution hierarchy (the recorded-metadata tier is forward work, #124); the refuse-don't-guess None arm stays the absolute floor.
2. #114 — pin the canonical Rust representation of C10's derived position-event table: PositionAction { Buy, Sell, Close } + PositionEvent, living in aura-engine alongside RunMetrics/ColumnarTrace (a post-run reduction, the same home as summarize). Pure schema — no friction, no derivation logic.

This is foundation only. derive_position_events (#115), the RealisticBroker node (#116), and frictions/metrics/persist (#120–122) consume these shapes in later cycles.

Architecture

Both deliverables are value types beside the hot path, faithful to existing contracts — they add no behaviour to the engine's run loop:

• InstrumentSpec is non-scalar reference metadata held beside the hot path (C7/C15) at the ingestion edge, keyed by symbol. C15's realization note already promises this exact extension ("extensible to tick size / digits / quote currency without a signature break"). The struct stays Copy.
• PositionEvent is the broker-independent audit row C10 already specifies down to its columns (INDEX.md:471–501): event_ts, action: i64 (buy/sell/close — direction IS the action), position_id: i64, instrument_id: i64, volume: f64 unsigned; no open_ts; the signed-volume trick is forbidden. It is a post-run value type (sibling of RunMetrics), never a per-eval node output (C8) — which is why it lives as data in aura-engine, not as a node. The engine stays headless and domain-free (C14): these are plain serializable structs, no I/O, no instrument identity on the hot path.

The one in-memory refinement over the ledger's columnar action: i64: the canonical Rust type is a closed enum PositionAction, serde-encoded as i64 (ordinal Buy=0, Sell=1, Close=2). The enum makes an invalid action unrepresentable in memory; the i64 encoding keeps the persisted/columnar form ledger- and C7-faithful and makes #122's columnar projection trivial. An out-of-range i64 fails TryFrom (the honesty lever, mirroring instrument_spec's None).

Concrete code shapes

Delivered code — behaviour this cycle ships (the empirical evidence)

The extended vetted table (must-pass) and its refuse arm (must-fail-by-design):

// aura-ingest/src/lib.rs — instrument_spec(), after extension
pub fn instrument_spec(symbol: &str) -> Option<InstrumentSpec> {
    // (instrument_id, pip_size, contract_size, pip_value_per_lot, min_lot, lot_step, quote)
    let s = match symbol {
        // index CFDs: quote EUR, 1 contract = €1 per point, pip = 1.0 point
        "GER40" => InstrumentSpec { instrument_id: 1, pip_size: 1.0, contract_size: 1.0,
                                    pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01,
                                    quote_currency: "EUR" },
        "FRA40" => InstrumentSpec { instrument_id: 2, pip_size: 1.0, contract_size: 1.0,
                                    pip_value_per_lot: 1.0, min_lot: 0.1, lot_step: 0.01,
                                    quote_currency: "EUR" },
        // FX majors: standard lot = 100_000 units, pip = 0.0001,
        // pip value = contract_size * pip_size in the quote currency (= 10)
        "EURUSD" => InstrumentSpec { instrument_id: 3, pip_size: 0.0001, contract_size: 100_000.0,
                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
                                     quote_currency: "USD" },
        "GBPUSD" => InstrumentSpec { instrument_id: 4, pip_size: 0.0001, contract_size: 100_000.0,
                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
                                     quote_currency: "USD" },
        "USDCAD" => InstrumentSpec { instrument_id: 5, pip_size: 0.0001, contract_size: 100_000.0,
                                     pip_value_per_lot: 10.0, min_lot: 0.01, lot_step: 0.01,
                                     quote_currency: "CAD" },
        _ => return None, // refuse-don't-guess — the absolute floor stays
    };
    Some(s)
}

SUSPECT BYTES — vetted reference data, not validated by an existing test. The contract numbers (contract sizes, pip values, min/lot steps) are industry-standard conventions, not broker-exact, and are overridable by design per the milestone's authored-override-wins contract. They are this spec's most fragile content; a later cycle (or an Aura.toml override) corrects any that prove wrong. The invariants over them are what the tests pin (below), not the literals' real-world exactness.

The closed action mapping (must-pass round-trip + must-fail rejection):

// aura-engine/src/report.rs
use std::convert::TryFrom;

impl From<PositionAction> for i64 {
    fn from(a: PositionAction) -> i64 {
        match a { PositionAction::Buy => 0, PositionAction::Sell => 1, PositionAction::Close => 2 }
    }
}
impl TryFrom<i64> for PositionAction {
    type Error = String;
    fn try_from(v: i64) -> Result<Self, String> {
        match v {
            0 => Ok(PositionAction::Buy),
            1 => Ok(PositionAction::Sell),
            2 => Ok(PositionAction::Close),
            other => Err(format!("invalid PositionAction i64: {other}")),
        }
    }
}
// try_from(99) -> Err — an out-of-range action is rejected, never silently mapped.

North star (cycle #115 — NOT built this cycle)

How the foundation will be consumed, shown minimally for honesty (no code lands here):

// derive_position_events folds the recorded exposure first-difference into rows.
// A long open, then a stop-and-reverse (Close + Sell at the SAME event_ts):
PositionEvent { event_ts: t0, action: PositionAction::Buy,   position_id: 1, instrument_id: 3, volume: lots };
PositionEvent { event_ts: t9, action: PositionAction::Close, position_id: 1, instrument_id: 3, volume: lots };
PositionEvent { event_ts: t9, action: PositionAction::Sell,  position_id: 2, instrument_id: 3, volume: lots };
// sizing lots = round_to(|exposure| * account / contract_value, lot_step); sub-min_lot drops (#115).

Before → after struct shapes (secondary)

InstrumentSpec — add six fields, keep Copy:

// BEFORE
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct InstrumentSpec {
    pub pip_size: f64,
}

// AFTER
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct InstrumentSpec {
    /// Stable per-instrument id — the position-event table's `instrument_id`
    /// column. Assigned once, never renumbered.
    pub instrument_id: i64,
    /// Price units per pip / point (> 0). The sim-optimal broker's divisor.
    pub pip_size: f64,
    /// Units of the base instrument per 1.0 lot (FX standard lot = 100_000;
    /// an index CFD = 1).
    pub contract_size: f64,
    /// Value of one pip per 1.0 lot, in the quote currency. Equals
    /// `contract_size * pip_size` for every vetted instrument here; carried
    /// explicitly because a real broker may not make it exactly so.
    pub pip_value_per_lot: f64,
    /// Smallest tradable volume and the volume granularity, both in lots.
    pub min_lot: f64,
    pub lot_step: f64,
    /// Quote currency (ISO label only this cycle — no FX conversion yet).
    pub quote_currency: &'static str,
}

PositionAction + PositionEvent — new in aura-engine/src/report.rs:

/// The three position-event actions (C10). Direction IS the action; volume is
/// unsigned. Serde-encoded as its i64 mapping so the persisted/columnar form
/// stays C7-scalar and ledger-faithful (`action: i64`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(into = "i64", try_from = "i64")]
pub enum PositionAction { Buy, Sell, Close }

/// One row of C10's derived position-event table — the broker-independent audit
/// view (the first difference of the exposure state). A post-run value type
/// (sibling of `RunMetrics`), NOT a per-`eval` node output (C8). Multiple events
/// may share one `event_ts` (a reversal: Close then open, close-before-open). No
/// `open_ts` — a position's open time is its opening event's `event_ts`.
#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct PositionEvent {
    pub event_ts: Timestamp,
    pub action: PositionAction,
    /// Monotonic, assigned at open. A Close references an existing `position_id`.
    pub position_id: i64,
    pub instrument_id: i64,
    /// Lots, unsigned. A partial close carries its own (smaller) volume.
    pub volume: f64,
}

Components

• aura-ingest InstrumentSpec (extend) — crates/aura-ingest/src/lib.rs:346. Add the six fields; keep #[derive(Clone, Copy, Debug, PartialEq)]. Update the doc comment to name the new fields. Copy is preserved because quote_currency is &'static str, not String.
• aura-ingest instrument_spec() table (populate) — :358. Fill all six fields for the five vetted symbols; keep the _ => None refuse arm.
• aura-engine PositionAction (new) — crates/aura-engine/src/report.rs. Closed enum + From<PositionAction> for i64 + TryFrom<i64> + serde into/try_from attributes. Re-export from the crate root if RunMetrics is (match the existing re-export pattern).
• aura-engine PositionEvent (new) — same file. The row struct above.

Data flow

No runtime/streaming flow changes this cycle — these are value types. The relationships they encode (consumed later):

• instrument_spec(symbol) → InstrumentSpec is resolved at the ingestion edge at bootstrap; the broker (#116) and derive_position_events (#115) read the resolved injected spec, never call instrument_spec() directly, so a resolver (the override hierarchy, #124) can sit in front later without touching them. (Wiring the injection is #115/#116, not this cycle.)
• PositionEvent rows are produced post-run by derive_position_events (#115) from the recorded exposure stream; persisted dup-ts-safe (#122). This cycle only fixes the shape every downstream piece wires to.

Error handling

• Un-specced symbol → instrument_spec() returns None; the existing instrument_spec_or_refuse (aura-cli main.rs:766) keeps refusing. No guessed metadata enters a run.
• Out-of-range action i64 → PositionAction::try_from returns Err (a message naming the bad value); serde deserialization of a bad action fails rather than silently coercing.
• Compile fan-out (contained): adding fields breaks every InstrumentSpec { pip_size } literal construction — the table constructor and the three test literals in aura-ingest/src/lib.rs (greps confirm: no other crate constructs the struct; instrument_spec_or_refuse and the examples only access .pip_size, which is unaffected). The plan threads all construction sites in one task so the crate compiles before its build gate.
• No new panics; the engine stays headless (C14).

Testing strategy

• aura-ingest (extend instrument_spec_lookup_by_symbol, add cases):
  • Each of the five symbols returns the full expected InstrumentSpec (all six fields), asserted by value.
  • instrument_spec("NONEXISTENT") == None (refuse arm intact).
  • Invariant: pip_value_per_lot == contract_size * pip_size for every vetted instrument (pins the cross-field relationship, not the literals' real-world exactness).
  • Invariant: the five instrument_ids are distinct (uniqueness).
• aura-engine (new tests in report.rs):
  • PositionAction round-trips: each variant → i64 → try_from → same variant.
  • PositionAction::try_from(3) (and a negative) → Err.
  • A PositionEvent serde JSON round-trip equals the original, and the serialized action is a bare integer (e.g. the JSON contains "action":1 for Sell), confirming the i64 encoding.
  • A reversal pair (Close then Sell) sharing one event_ts constructs and is representable (documents close-before-open ordering; the ordering logic is #115).
• Workspace: cargo test --workspace green; cargo clippy --workspace --all-targets -- -D warnings clean.

Acceptance criteria

1. InstrumentSpec carries the six new deploy-grade fields and stays Copy; the vetted table populates all five instruments; the _ => None refuse arm remains.
2. PositionAction is a closed three-value enum with a total, round-tripping i64 mapping (Buy=0, Sell=1, Close=2); an out-of-range i64 is rejected.
3. PositionEvent matches the C10 column spec exactly — event_ts, enum action, position_id, instrument_id, unsigned volume; no open_ts; no signed-volume trick. Its serde shape is pinned (action encodes as i64).
4. Both types live where the ledger/issues place them (InstrumentSpec in aura-ingest at the ingestion edge; PositionEvent in aura-engine beside RunMetrics), and the engine stays headless (C14) — no broker, no derivation, no I/O this cycle.
5. cargo test --workspace passes and clippy -D warnings is clean.