From 0d7deb3a91d4f61b78ed249d13573d355d360b57 Mon Sep 17 00:00:00 2001 From: Brummel Date: Thu, 4 Jun 2026 17:02:33 +0200 Subject: [PATCH] design(C10): reframe strategy output to intent/exposure stream; position table becomes a derived layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The DAG expresses exactly one state at time t and a node emits at most one record per eval (C8), so a *sequence* of position events — where one decision instant (a stop-and-reverse) needs a close AND an open at the same event_ts — cannot be the DAG's per-cycle output without violating C8. The state the DAG can express faithfully is the desired exposure (one value per cycle); the buy/sell/close events are its first difference, a derived consequence. C10 is reframed accordingly: the strategy's primary, backtestable output is an intent/exposure stream (one signed, bounded f64 in [-1,+1] per cycle). Signal quality is measured by the sim-optimal broker integrating exposure*return into a synthetic pip-equity curve. The broker-independent position-event table survives as a decoupled, derivable, downstream position-management layer (computed table, not a per-eval output) feeding realistic broker nodes for viability/deploy — no longer the DAG output nor the signal-quality measure. Touches the ledger contract (INDEX.md C10 + provenance/milestone/C20 ancillary), the always-loaded summary (CLAUDE.md invariant #7), the glossary (broker, equity stream, position table, realistic broker, signal, sim-optimal broker, strategy reframed + new exposure-stream entry), and the north-star layout doc. Sealed specs/plans (0001-0006) left as historical record. refs #4 #5 Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 26 +++++---- docs/design/INDEX.md | 119 +++++++++++++++++++++++++---------------- docs/glossary.md | 18 ++++--- docs/project-layout.md | 15 +++--- 4 files changed, 107 insertions(+), 71 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7c760b7..d6dd807 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -72,17 +72,21 @@ design decision, not a refactor, and belongs in the ledger. makes a live external call mid-replay. (See `~/.claude/CLAUDE.md` for the IONOS consent rule: external LLM calls happen at the recording/live-source edge, with explicit per-session consent, never inside a backtest.) -7. **Strategy result is a broker-independent position table.** A strategy - outputs not an equity curve but a time-ordered table of position events - (`event_ts, action[buy/sell/close], position_id, instrument_id, volume`; a - position's open time is its opening event's `event_ts`); the set of open - positions at t is its state. Brokers are downstream **consumer nodes** (not part - of the strategy) that consume the position-event stream (plus prices) and emit - an equity stream; several can be attached to the same position table at once, - giving directly comparable curves. Neutral evaluation uses a deterministic, - frictionless sim-optimal broker producing synthetic equity in **pips**; - realistic broker nodes add real friction/constraints in currency for viability - and deploy. +7. **Strategy output is an intent/exposure stream; position management is a + decoupled derived layer.** The DAG expresses one state at t (C8: ≤1 record per + `eval`), so a strategy's primary, backtestable output is a signed, bounded + **exposure** `f64 ∈ [-1,+1]` per cycle, not an equity curve. **Signal quality** + is measured by the **sim-optimal broker** — a downstream consumer node that + integrates `exposure·return` into a deterministic, frictionless synthetic + equity stream in **pips**. The broker-independent **position-event table** + (`event_ts, action[buy/sell/close], position_id, instrument_id, volume`) is a + **decoupled, derivable** layer — the first difference of the exposure state, a + computed table (not a per-`eval` output, since one decision instant may yield + >1 event) — that feeds **realistic broker nodes** (currency equity, real + frictions) for viability and deploy. The two broker classes attach at two + points: sim-optimal on the exposure stream, realistic on the derived + position-event table; brokers are ordinary downstream nodes, never part of the + strategy. 8. **Deploy artifacts are frozen.** Hot-reload (cdylib) is an authoring-loop tool only. The live bot is a statically-linked, versioned, frozen artifact — never hot-swapped (audit trail: this bot = this commit). diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index c2f9f36..4147d8c 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -7,7 +7,9 @@ its new rationale), never a silent refactor. Provenance: contracts C1–C18 were settled in the initial rough-sketch design interview (2026-06-03), walking the design tree root-to-leaf (C16–C18 and the -C10 refinement to a broker-independent position table came in follow-up turns). +C10 refinement to a broker-independent position table came in follow-up turns; +C10 was later reframed in cycle 0007 — the intent/exposure stream is the primary +output, the position table a derived layer — see C10). The `CLAUDE.md` **Domain invariants** section is the compressed, always-loaded summary of the subset that agents must never violate; this file is the fuller form with @@ -15,8 +17,8 @@ rationale. Vocabulary: a *contract* is one ledger entry. A *cycle* is one pipeline round; a *milestone* is a tracker container spanning many cycles (the first milestone is -the **walking skeleton**: ingest → one signal → deterministic backtest → -position table → sim-optimal broker → synthetic pip-equity metric). +the **walking skeleton**: ingest → one signal → exposure/intent → sim-optimal +broker → synthetic pip-equity signal-quality metric). --- @@ -241,49 +243,74 @@ forcing feedback through a visible delay node keeps the per-cycle determinism intact and the one legitimate feedback path explicit. Graph-as-data enables visualization, freezing, and re-parameterization for sweeps. -### C10 — Strategy result is a broker-independent position table; brokers are downstream nodes -**Guarantee.** A strategy's result is **not** an equity curve but a -**broker-independent, time-ordered table of position events**. The chain is -`signals (scores) → decision/sizing node → position-event output`. An event is -pure scalar columns (C7): `event_ts: timestamp`, `action: i64` (buy / sell / -close), `position_id: i64`, `instrument_id: i64`, `volume: f64` (unsigned — -direction is the `action`). A position's open time is the `event_ts` of its -opening event (there is no separate `open_ts`); a `close` references a -`position_id` and may be partial via its own `volume`. The **set of open -positions at time t** (opens minus closes with `event_ts ≤ t`) is the strategy's -*state* at t; the ordered sequence of these states is the result. Position -sizing and risk live here (they set `volume`); the portfolio is multi-instrument. +### C10 — Strategy output is an intent/exposure stream; position management is a decoupled derived layer; brokers are downstream nodes +**Guarantee.** A strategy's primary, backtestable output is **not** an equity +curve, **nor a position-event table**, but an **intent / exposure stream**: the +DAG expresses exactly one *state* at time t (C8 — a node emits at most one record +per `eval`), so a strategy emits one **signed, bounded exposure** `f64 ∈ +[-1, +1]` per cycle (per instrument; the portfolio is multi-instrument). Exposure +is the desired fractional position; **position sizing and risk live in the +decision/sizing node** that shapes a raw signal score into exposure. The chain is +`signals (scores) → decision/sizing node → exposure stream`. -A **broker is a downstream consumer node** (C8 / C9): it consumes the strategy's -position-event stream — plus the relevant price streams, to mark open positions — -and emits an **equity stream** as its output. It is *not* part of the strategy. -Because it is an ordinary node, **several brokers can be attached to the same -position table at once**, each emitting its own equity stream, so the resulting -curves are directly comparable. Two classes: **(a) the sim-optimal broker** — -deterministic, frictionless, perfect-fill execution producing a **synthetic -equity stream in pips** (no real currency, no real-broker constraints); the -neutral yardstick for comparing and optimizing strategy *logic*. **(b) realistic -broker nodes** (Pepperstone, …) — apply real spread / commission / slippage / -lot / margin, may reject or modify positions, and produce a currency equity -stream for viability and deployment. Pip PnL uses per-instrument pip metadata -(reference data beside the hot path, C7). Live: a realistic broker node consumes -the position events in real time and routes orders as a side effect; -reconciliation with the real account is an external adapter. -**Forbids.** Treating an equity curve as the strategy's output; baking a broker -into the strategy; a special external broker subsystem (a broker is an ordinary -node); storing `open_ts` (derive it from the opening event); a signed-volume -direction trick (use `action`); broker-specific assumptions leaking into the -strategy logic. -**Why.** A strategy can be judged neutrally only if its result is independent of -any real broker's frictions. The position table is that broker-independent -invariant: one table feeds many broker nodes, each yielding its own equity — so -attaching a synthetic and a real broker side by side gives two comparable -curves, and "same strategy, different broker" and "same decisions sim vs live" -both fall out. The sim-optimal pip curve is a level, currency-free playing field -for comparison; realistic broker nodes then test real-world viability. Modelling -the broker as a node (not a bespoke subsystem) keeps it within the one Node/graph -abstraction (C9). This supersedes the earlier "broker is part of the strategy" -framing. +**Signal quality** is evaluated by the **(a) sim-optimal broker** — a downstream +consumer node (C8/C9) that consumes the **exposure stream** plus the relevant +price stream and integrates `exposure(t-1) · (price(t) − price(t-1))` into a +**synthetic equity stream in pips**: deterministic, frictionless, perfect-fill +(no currency, no real-broker constraints). The exposure held *into* a cycle +(decided at t-1) earns that cycle's return, so the integration is causal (C2 — no +look-ahead). Pip PnL uses per-instrument pip metadata (reference data beside the +hot path, C7/C15). This pip-equity curve measures the **signal's quality**, not +an execution-modelled P&L; it is the primary research loop — backtest one signal, +combine it with another, backtest the combination. + +**Position management** — turning the exposure stream into a broker-independent, +time-ordered **table of position events** (pure scalar columns, C7: `event_ts: +timestamp`, `action: i64` buy / sell / close, `position_id: i64`, +`instrument_id: i64`, `volume: f64` unsigned — direction is the `action`; a +position's open time is its opening event's `event_ts`, no separate `open_ts`; a +`close` references a `position_id` and may be partial via its own `volume`) — is a +**decoupled, derivable, downstream** layer. The events are the **first difference +of the exposure state**, materialized as a *computed table* (where multiple +events may share one `event_ts` — e.g. a reversal's close + open), **never as a +per-`eval` node output** (which C8 caps at one record per cycle). This layer feeds +**(b) realistic broker nodes** (Pepperstone, …): downstream consumer nodes that +consume the position-event table plus prices, apply real spread / commission / +slippage / lot / margin, may reject or modify positions, and produce a +**currency** equity stream for viability and deployment. The position-event table +stays broker-independent — one table feeds many realistic brokers, giving +directly comparable currency curves. Live: a realistic broker node consumes the +events in real time and routes orders as a side effect; reconciliation with the +real account is an external adapter. + +So **the two broker classes attach at two different points**: the sim-optimal +broker on the **exposure stream** (signal quality, pips, now), realistic brokers +on the **derived position-event table** (execution viability, currency, later). +Both are ordinary downstream nodes (C8/C9); several can attach at once for +directly comparable curves. +**Forbids.** Treating an equity curve as the strategy's output; making the +**position-event table the strategy's direct DAG output** (it is derived, not +emitted per `eval` — a decision instant may need >1 event, which C8 forbids) or +the measure of signal quality; baking a broker into the strategy; a special +external broker subsystem (a broker is an ordinary node); storing `open_ts` +(derive it from the opening event); a signed-volume direction trick in the event +table (use `action`); broker-specific assumptions leaking into the strategy logic. +**Why.** The DAG is a synchronous reactive graph: at time t it holds exactly one +state, and a node emits at most one record per `eval` (C8). A *sequence of +position events* — where one decision instant (a stop-and-reverse) needs a close +**and** an open at the same `event_ts` — cannot be the DAG's per-cycle output +without violating C8. The state the DAG *can* express faithfully is the **desired +exposure** (one value per cycle); the position events are its first difference, a +derived consequence. Decoupling them resolves the C8↔C10 impedance and matches +how signal research actually proceeds: you first measure a signal's *quality* +(does this exposure, held over time, make pips?) independently of any execution +model; only later do you model position management and real-broker frictions for +deployment. The sim-optimal pip curve is a level, currency-free playing field for +comparing and combining signals; the derived position-event table + realistic +broker nodes then test real-world viability. Modelling brokers as nodes (not a +bespoke subsystem) keeps them within the one Node/graph abstraction (C9). This +supersedes the earlier "the strategy's output is the position-event table" +framing (cycle 0007 reframe). ### C11 — Generalized sources; record-then-replay determinism boundary **Guarantee.** A source is anything that produces timestamped scalar streams — @@ -433,7 +460,7 @@ explicit. ### C20 — Strategy ↔ harness; the harness is the root sim graph **Guarantee.** A **strategy** is a reusable composite-node blueprint (C9): broker-, data-, and viz-independent, with inputs declared as named **roles** -(symbol-agnostic where possible) and the position-event stream (C10) as output. +(symbol-agnostic where possible) and the intent/exposure stream (C10) as output. A **harness** (the experimental setup) is the **root sim graph** — sources bound to the strategy's input roles + the strategy + attached broker node(s) + sinks — and is itself produced by the bootstrap (C19). A harness *instance* is C1's diff --git a/docs/glossary.md b/docs/glossary.md index 5719c60..0cd3acd 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -37,7 +37,7 @@ A frozen artifact that is a deployed strategy + broker: the live trading program ### broker **Avoid:** — -A downstream consumer node that consumes the position table (plus the relevant price streams) and emits an equity stream; several can attach to one position table at once for directly comparable curves. Two classes: the sim-optimal broker and realistic brokers. +A downstream consumer node that emits an equity stream, attaching at one of two points: the **sim-optimal broker** consumes the **exposure stream** (integrating `exposure·return` → synthetic pip equity = signal quality); **realistic brokers** consume the derived **position-event table** (real frictions → currency equity). Several can attach at once for directly comparable curves. Not part of the strategy. Two classes: the sim-optimal broker and realistic brokers. ### cdylib **Avoid:** — @@ -57,7 +57,7 @@ A directed wiring link in a harness that forwards **one field** (`from_field`) o ### equity stream **Avoid:** — -A broker node's output: the marked value of open positions over time — synthetic pips for the sim-optimal broker, currency for realistic brokers. An equity *curve* is explicitly not a strategy's output (the position table is). +A broker node's output over time — synthetic pips for the sim-optimal broker (its `exposure·return` integral), currency for realistic brokers. An equity *curve* is explicitly not a strategy's output (the intent/exposure stream is). ### experiment **Avoid:** — @@ -67,6 +67,10 @@ A Rust (builder-API) definition of anything beyond a single backtest — sweep, **Avoid:** structural matrix The set of harness instances produced by varying the structural axes (strategy × instrument × broker × window), expressed as plain Rust loops. The outer orchestration over the structural dimension; the tuning sweep is the inner loop. +### exposure stream +**Avoid:** intent stream +A strategy's primary, backtestable DAG output: one signed, bounded `f64 ∈ [-1,+1]` per cycle — the desired fractional position (intent). The DAG expresses one state at t (C8: ≤1 record per `eval`), so exposure, not a position-event sequence, is what it emits. The sim-optimal broker integrates `exposure·return` into pip equity to measure signal quality; the position table is its derived first difference. + ### firing policy **Avoid:** — A per-input-group declaration of one of two firing modes: A = fire-on-any-fresh + hold (as-of join), B = all-fresh barrier (synchronizing join). A single node may mix an A input and a B group. @@ -113,11 +117,11 @@ The egui-native visual face (`aura play`) that plays any harness — program str ### position table **Avoid:** — -A strategy's broker-independent, time-ordered table of position events (scalar columns: `event_ts, action, position_id, instrument_id, volume`). The set of open positions at time t is the strategy's state; brokers are the downstream consumers of this table. +A broker-independent, time-ordered table of position events (scalar columns: `event_ts, action, position_id, instrument_id, volume`) — the **derived** first difference of a strategy's exposure stream, a **decoupled position-management layer** feeding realistic brokers / deploy. Computed, not emitted per `eval` (one decision instant may yield >1 event); **not** the strategy's direct DAG output (that is the intent/exposure stream). ### realistic broker **Avoid:** — -A broker node that applies real spread / commission / slippage / lot / margin (and may reject or modify positions), emitting a currency equity stream for viability and deploy. Contrasted with the sim-optimal broker. +A broker node that consumes the derived position-event table and applies real spread / commission / slippage / lot / margin (and may reject or modify positions), emitting a currency equity stream for viability and deploy. Contrasted with the sim-optimal broker (which consumes the exposure stream directly). ### resampler **Avoid:** — @@ -145,7 +149,7 @@ A node that exposes session context as scalar streams (`bars_since_open`, `in_se ### signal **Avoid:** — -A node whose output is a score, feeding the `signals (scores) → decision/sizing node → position events` chain. A specific node role — distinct from a general node and from a strategy. +A node whose output is a score, feeding the `signals (scores) → decision/sizing node → exposure stream` chain. A specific node role — distinct from a general node and from a strategy. ### sim **Avoid:** — @@ -153,7 +157,7 @@ The disjoint, executable unit of one deterministic harness run; the unit of para ### sim-optimal broker **Avoid:** — -The deterministic, frictionless, perfect-fill broker that emits synthetic equity in pips — the neutral, currency-free yardstick for comparing and optimizing strategy logic. Distinct from the Aura.toml "default broker" (a config role) and from realistic brokers. +The deterministic, frictionless, perfect-fill broker that consumes the **exposure stream** + prices and integrates `exposure·return` into synthetic equity in pips — the neutral, currency-free yardstick that measures **signal quality**. Distinct from the Aura.toml "default broker" (a config role) and from realistic brokers (which consume the derived position-event table). ### sink **Avoid:** — @@ -169,7 +173,7 @@ Anything that produces timestamped scalar streams — market data and non-financ ### strategy **Avoid:** — -A reusable composite-node blueprint — broker-, data-, and viz-independent, inputs declared as named roles — whose output is the position table (not an equity curve). Frozen with a broker into a bot for deploy. +A reusable composite-node blueprint — broker-, data-, and viz-independent, inputs declared as named roles — whose output is the intent/exposure stream (not an equity curve; the position table is a derived downstream layer). Frozen with a broker into a bot for deploy. ### structural axes **Avoid:** — diff --git a/docs/project-layout.md b/docs/project-layout.md index 452529e..528a23e 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -77,13 +77,14 @@ per-session consent. implements `schema` + `eval` against `aura-core`. 3. **Backtest:** `aura backtest nodes/third-candle-long --symbol GER40 --from 2020 --to 2024` → the strategy produces a broker-independent - **position table** (open/close events); the default **sim-optimal broker** - projects it into a synthetic **pip**-equity, yielding a metrics table - (trades, hit-rate, pip-P&L, max-DD, Sharpe) + a run record (manifest + - metrics) under `runs/`. Brokers are consumer **nodes**, and several can be - attached at once: add `--broker pepperstone` to get a realistic currency - curve *alongside* the default sim-optimal pip curve — two comparable equity - curves from the same position table. (Contract C10.) + **exposure stream** (one bounded signed value per cycle = intent); the default + **sim-optimal broker** integrates `exposure·return` into a synthetic + **pip**-equity — the signal's *quality* — yielding a metrics table + (pip-P&L, max-DD, Sharpe) + a run record (manifest + metrics) under `runs/`. + Brokers are consumer **nodes**: the sim-optimal one reads the exposure stream + directly, while a realistic broker reads the *derived* position-event table — + add `--broker pepperstone` to get a realistic currency curve *alongside* the + sim-optimal pip curve, two comparable equity curves. (Contract C10.) 4. **Sweep / Monte-Carlo / matrix — a Rust experiment.** Anything beyond a single backtest is an *experiment* in `experiments/` (Rust, builder API): a parameter sweep, Monte-Carlo over seeds, or a structural matrix like "these 10