spec: 0074 provider-sourced pip, drop authored instrument floor (boss-signed)
Resolve the real-path pip from the data-server geometry sidecar (instrument_geometry) and remove the redundant authored floor (InstrumentSpec / instrument_spec / VETTED_SYMBOLS) plus its table-only tests. Consumption grep: only pip_size is read in production; InstrumentSpec.instrument_id has no consumer (the position-event table's id is a decoupled caller parameter). Speculative geometry fields are deferred to the Stage-2 broker, read from the sidecar when needed. Signed via grounding-check PASS (all assumptions tied to currently-green tests, executed with the archive present). Decisions recorded on #124; #140 closed as obsolete. refs #124
This commit is contained in:
@@ -0,0 +1,238 @@
|
||||
# Provider-sourced pip; drop the authored instrument floor — Design Spec
|
||||
|
||||
**Date:** 2026-06-25
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
|
||||
## Goal
|
||||
|
||||
aura must define and take over only the instrument reference data it currently
|
||||
consumes, and source it from the data-server geometry sidecars rather than a
|
||||
hand-maintained table. A grep of every `InstrumentSpec` field *read* across
|
||||
`crates/` shows production code consumes exactly one datum — `pip_size` — at the
|
||||
real-data path. Every other field (`contract_size`, `pip_value_per_lot`,
|
||||
`min_lot`, `lot_step`, `quote_currency`, `tick_size`, `digits`, **and
|
||||
`instrument_id`**) is constructed but never read outside the table's own tests.
|
||||
|
||||
This cycle resolves the pip from the per-symbol sidecar
|
||||
(`instrument_geometry(server, symbol).pip_size`) and removes the now-redundant
|
||||
authored floor (`InstrumentSpec`, `instrument_spec()`, `VETTED_SYMBOLS`) together
|
||||
with the tests that exist only to validate it. The decision and its grounding are
|
||||
recorded on the reference issue (#124, reconciliation comment).
|
||||
|
||||
Deferred, not deleted (per the user directive "build it back in when the broker
|
||||
needs it"): the speculative fields are revived from `InstrumentGeometry` — which
|
||||
already carries `digits`, `lotSize`, `tickSize`, `quoteAsset` — when the Stage-2
|
||||
realistic-broker milestone actually consumes them. No knowledge is lost; the
|
||||
sidecar remains the source.
|
||||
|
||||
## Architecture
|
||||
|
||||
The instrument-reference resolution hierarchy of #124 (authored override >
|
||||
recorded metadata > authored floor > refuse) collapses, for what is built now, to
|
||||
its two ends: **recorded sidecar geometry → refuse-don't-guess**. The middle
|
||||
authored floor (tier 3) and the authored override (tier 1) are future work, built
|
||||
when a consumer needs them. The `instrument_geometry` accessor and the
|
||||
`InstrumentGeometry` re-export added in #143 are the surviving seam and become the
|
||||
sole pip source.
|
||||
|
||||
The `instrument_id` question (the directive's "find out"): the position-event
|
||||
table needs an `instrument_id`, but it already takes one as a **caller-supplied
|
||||
`i64` parameter** — `derive_position_events(record, instrument_id)`
|
||||
(report.rs:231) — and the engine deliberately never imports `InstrumentSpec`
|
||||
(report.rs:226). `InstrumentSpec.instrument_id` has no production reader; it is
|
||||
dropped with the struct. No symbol→id registry is consumed anywhere today, so
|
||||
none is built; when one is needed it is added then.
|
||||
|
||||
Invariants preserved: **refuse-don't-guess** (a symbol with no sidecar → exit 2,
|
||||
unchanged); **honest-pip-by-construction** (now the provider's recorded value
|
||||
instead of a hand-authored one — strictly more honest); **determinism / C6** (the
|
||||
sidecar is recorded data read once at bootstrap, never a live mid-replay call);
|
||||
the engine's `(Timestamp, Scalar)` seam is untouched (geometry stays beside the
|
||||
hot path, C7/C15).
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### Delivered — CLI pip resolution (the production path)
|
||||
|
||||
`crates/aura-cli/src/main.rs`. Before — table lookup, server-free, refuses on an
|
||||
un-vetted symbol:
|
||||
|
||||
```rust
|
||||
// ~line 730
|
||||
fn instrument_spec_or_refuse(symbol: &str) -> aura_ingest::InstrumentSpec {
|
||||
match aura_ingest::instrument_spec(symbol) {
|
||||
Some(s) => s,
|
||||
None => {
|
||||
eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' — \
|
||||
refusing to run a real instrument with a guessed pip \
|
||||
(add it to the instrument table)");
|
||||
std::process::exit(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After — resolve the pip from the recorded sidecar; refuse when absent. The pip is
|
||||
the only datum any caller used, so the helper returns `f64`:
|
||||
|
||||
```rust
|
||||
// the single home of the guessed-pip refusal, now sidecar-sourced. Reads the
|
||||
// per-symbol geometry metadata (not bar data) BEFORE the run source opens, so an
|
||||
// instrument with no recorded geometry refuses without touching the archive —
|
||||
// the pip stays honest by construction (the provider's value, never guessed).
|
||||
fn pip_or_refuse(server: &std::sync::Arc<data_server::DataServer>, symbol: &str) -> f64 {
|
||||
match aura_ingest::instrument_geometry(server, symbol) {
|
||||
Some(geo) => geo.pip_size,
|
||||
None => {
|
||||
eprintln!("aura: no recorded geometry for symbol '{symbol}' at {} — \
|
||||
refusing to run a real instrument with a guessed pip",
|
||||
data_server::DEFAULT_DATA_PATH);
|
||||
std::process::exit(2);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Both call sites build the server first, then resolve the pip (the refusal stays
|
||||
ahead of bar-data access). `open_real_source` (~line 767):
|
||||
|
||||
```rust
|
||||
// before: spec lookup, THEN server
|
||||
// let spec = instrument_spec_or_refuse(symbol);
|
||||
// let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
|
||||
// after: server first, then pip from its sidecar
|
||||
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
|
||||
let pip = pip_or_refuse(&server, symbol);
|
||||
if !server.has_symbol(symbol) { no_real_data(symbol); }
|
||||
// ... probe_window + open as today; return (source, window, pip)
|
||||
```
|
||||
|
||||
`DataSource::from_choice` (~line 794) takes the same reorder: build `server`,
|
||||
`let pip = pip_or_refuse(&server, &symbol);`, then `has_symbol`, then
|
||||
`DataSource::Real { server, symbol, from_ms, to_ms, pip }`.
|
||||
|
||||
### Delivered — aura-ingest library surface
|
||||
|
||||
`crates/aura-ingest/src/lib.rs`. **Removed:** `pub struct InstrumentSpec` (all 9
|
||||
fields), `pub fn instrument_spec`, `pub const VETTED_SYMBOLS`, and the three unit
|
||||
tests that only exercise them (`instrument_spec_lookup_by_symbol`,
|
||||
`instrument_spec_pip_value_matches_contract_times_pip`,
|
||||
`instrument_spec_ids_are_distinct`). **Kept unchanged:** the pip source —
|
||||
|
||||
```rust
|
||||
pub use data_server::meta::InstrumentGeometry;
|
||||
|
||||
/// Recorded per-symbol geometry at the ingestion edge: the provider's structural
|
||||
/// truth (digits, pip/tick size, lot size, quote currency), read from the
|
||||
/// dataset sidecar. Beside the hot path (C7/C15); never enters the engine seam.
|
||||
pub fn instrument_geometry(server: &Arc<DataServer>, symbol: &str) -> Option<InstrumentGeometry> {
|
||||
server.symbol_meta(symbol)
|
||||
}
|
||||
```
|
||||
|
||||
and its unit test `instrument_geometry_none_for_unknown_symbol`.
|
||||
|
||||
### Delivered — example fixtures
|
||||
|
||||
`crates/aura-ingest/examples/shared/breakout_real.rs`. `pip_size_of` resolves
|
||||
from the sidecar (real-data examples require the archive anyway):
|
||||
|
||||
```rust
|
||||
/// The example's pip divisor, sourced from the recorded geometry sidecar — the
|
||||
/// same provider truth the CLI `--real` path uses (#98), now provider-sourced.
|
||||
/// Panics on a symbol with no sidecar (examples only run known instruments with
|
||||
/// local data).
|
||||
pub fn pip_size_of(symbol: &str) -> f64 {
|
||||
aura_ingest::instrument_geometry(&aura_ingest::default_data_server(), symbol)
|
||||
.unwrap_or_else(|| panic!("no recorded geometry sidecar for {symbol}"))
|
||||
.pip_size
|
||||
}
|
||||
```
|
||||
|
||||
The **pure-structural** blueprint tests assert on `param_space()` topology, not on
|
||||
the pip, and are contracted to "run everywhere" (no archive). They pass a literal
|
||||
in place of the sidecar read so they stay hermetic —
|
||||
`crates/aura-ingest/tests/ger40_breakout_blueprint.rs`:
|
||||
|
||||
```rust
|
||||
// param_space_is_exactly_entry_and_exit_bar_targets (#37: "No data needed")
|
||||
// and blueprint_param_space_is_construction_deterministic: the pip is a throwaway
|
||||
// sizing arg here, never asserted, so a literal keeps these archive-independent.
|
||||
let (bp, _taps) = ger40_breakout_blueprint(1.0 /* GER40 pip: index point */, 15, 9, 0, Berlin);
|
||||
```
|
||||
|
||||
Real-data / archive-gated consumers (`ger40_breakout_world.rs`, the example
|
||||
binaries, the gated blueprint fidelity test) keep calling `pip_size_of` — they
|
||||
skip cleanly when the archive is absent and run with it present, as today. The
|
||||
cross-instrument compare example (`ger40_breakout_compare.rs`, variable `symbol`)
|
||||
keeps `pip_size_of(symbol)` and is now per-symbol sidecar-sourced.
|
||||
|
||||
### Removed — table-only test files
|
||||
|
||||
`crates/aura-ingest/tests/instrument_geometry_crosscheck.rs` (cross-checks the
|
||||
removed floor against the provider — moot once the floor is gone) and
|
||||
`crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs` (asserts on the
|
||||
removed fields).
|
||||
|
||||
## Components
|
||||
|
||||
- **aura-cli** — `pip_or_refuse` replaces `instrument_spec_or_refuse`; two call
|
||||
sites reordered (server before pip).
|
||||
- **aura-ingest (lib)** — public surface shrinks to `instrument_geometry` +
|
||||
`InstrumentGeometry`; the authored floor and its tests are removed.
|
||||
- **aura-ingest (examples/tests)** — `pip_size_of` repointed to the sidecar;
|
||||
pure-structural blueprint tests use a literal pip; two table-only test files
|
||||
deleted.
|
||||
- **Design ledger** (`docs/design/INDEX.md`) — the C15 realization note added in
|
||||
0073 (authored floor + cross-check + tick_size/digits) is rewritten to record
|
||||
the floor's removal and provider-sourced pip; the #124 hierarchy note records
|
||||
the collapse to "sidecar → refuse" with the override/floor tiers deferred.
|
||||
|
||||
## Data flow
|
||||
|
||||
`aura --real <SYMBOL>` → build `DataServer` → `pip_or_refuse(server, symbol)`
|
||||
reads `<SYMBOL>.meta.json` via `symbol_meta` → `pip_size` (or exit 2) →
|
||||
threaded into the real `Source` / SimBroker exactly as before. The
|
||||
position-event table's `instrument_id` continues to arrive as a caller parameter,
|
||||
untouched.
|
||||
|
||||
## Error handling
|
||||
|
||||
A symbol with no readable/parseable sidecar → `instrument_geometry` returns
|
||||
`None` → `pip_or_refuse` prints the refusal and `exit(2)`, before any bar-data
|
||||
access. Same exit code and pre-data ordering as the removed helper; only the
|
||||
message and the source change. Any test pinning the old refusal substring is
|
||||
updated to the new message.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **RED → GREEN (behaviour):** a test that `pip_or_refuse` (or the real path) for
|
||||
a symbol with a sidecar yields the sidecar's `pip_size`, and that an unknown
|
||||
symbol refuses (exit 2). The existing real-path tests already ratify "pip is
|
||||
threaded into the real source"; this cycle re-sources it without changing that
|
||||
observable.
|
||||
- **Deletions:** removing `InstrumentSpec`/`instrument_spec`/`VETTED_SYMBOLS` and
|
||||
their tests is safe by the consumption grep (no production reader). The compile
|
||||
itself enumerates any missed consumer.
|
||||
- **Hermetic structural tests stay hermetic:** the two pure blueprint tests must
|
||||
still pass with the archive absent (literal pip, no sidecar read).
|
||||
- **Gates:** `cargo test --workspace` green; `cargo clippy --workspace
|
||||
--all-targets -- -D warnings` clean. On this host the archive is present, so the
|
||||
sidecar-sourced paths actually execute.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. **Removes redundancy / improves correctness (the project criterion):** the
|
||||
authored floor duplicated the provider geometry byte-for-byte (proven by the
|
||||
#143 cross-check); removing it deletes a drift surface and sources the pip from
|
||||
the broker's recorded truth.
|
||||
2. **Reintroduces no failure class the core constraints forbid:** refuse-don't-
|
||||
guess preserved (exit 2 on absent geometry); determinism preserved (sidecar is
|
||||
recorded data read at bootstrap, C6); the engine `(Timestamp, Scalar)` seam
|
||||
untouched (C4/C7); no look-ahead introduced (static metadata).
|
||||
3. **Defines only what is needed:** the only instrument datum any current path
|
||||
consumes — `pip_size` — is the only one defined/sourced; the rest is deferred
|
||||
to the Stage-2 broker that will consume it, read from the sidecar then.
|
||||
4. `cargo test --workspace` and the clippy gate pass; the pure-structural tests
|
||||
pass with and without the local archive.
|
||||
Reference in New Issue
Block a user