spec: 0074 provider-sourced pip, drop authored instrument floor (boss-signed)

Resolve the real-path pip from the data-server geometry sidecar
(instrument_geometry) and remove the redundant authored floor
(InstrumentSpec / instrument_spec / VETTED_SYMBOLS) plus its table-only
tests. Consumption grep: only pip_size is read in production;
InstrumentSpec.instrument_id has no consumer (the position-event table's
id is a decoupled caller parameter). Speculative geometry fields are
deferred to the Stage-2 broker, read from the sidecar when needed.

Signed via grounding-check PASS (all assumptions tied to currently-green
tests, executed with the archive present). Decisions recorded on #124;
#140 closed as obsolete.

refs #124

docs/specs/0074-pip-from-sidecar.md

@@ -0,0 +1,238 @@
+# Provider-sourced pip; drop the authored instrument floor — Design Spec
+
+**Date:** 2026-06-25
+**Status:** Draft — awaiting user spec review
+**Authors:** orchestrator + Claude
+
+## Goal
+
+aura must define and take over only the instrument reference data it currently
+consumes, and source it from the data-server geometry sidecars rather than a
+hand-maintained table. A grep of every `InstrumentSpec` field *read* across
+`crates/` shows production code consumes exactly one datum — `pip_size` — at the
+real-data path. Every other field (`contract_size`, `pip_value_per_lot`,
+`min_lot`, `lot_step`, `quote_currency`, `tick_size`, `digits`, **and
+`instrument_id`**) is constructed but never read outside the table's own tests.
+
+This cycle resolves the pip from the per-symbol sidecar
+(`instrument_geometry(server, symbol).pip_size`) and removes the now-redundant
+authored floor (`InstrumentSpec`, `instrument_spec()`, `VETTED_SYMBOLS`) together
+with the tests that exist only to validate it. The decision and its grounding are
+recorded on the reference issue (#124, reconciliation comment).
+
+Deferred, not deleted (per the user directive "build it back in when the broker
+needs it"): the speculative fields are revived from `InstrumentGeometry` — which
+already carries `digits`, `lotSize`, `tickSize`, `quoteAsset` — when the Stage-2
+realistic-broker milestone actually consumes them. No knowledge is lost; the
+sidecar remains the source.
+
+## Architecture
+
+The instrument-reference resolution hierarchy of #124 (authored override >
+recorded metadata > authored floor > refuse) collapses, for what is built now, to
+its two ends: **recorded sidecar geometry → refuse-don't-guess**. The middle
+authored floor (tier 3) and the authored override (tier 1) are future work, built
+when a consumer needs them. The `instrument_geometry` accessor and the
+`InstrumentGeometry` re-export added in #143 are the surviving seam and become the
+sole pip source.
+
+The `instrument_id` question (the directive's "find out"): the position-event
+table needs an `instrument_id`, but it already takes one as a **caller-supplied
+`i64` parameter** — `derive_position_events(record, instrument_id)`
+(report.rs:231) — and the engine deliberately never imports `InstrumentSpec`
+(report.rs:226). `InstrumentSpec.instrument_id` has no production reader; it is
+dropped with the struct. No symbol→id registry is consumed anywhere today, so
+none is built; when one is needed it is added then.
+
+Invariants preserved: **refuse-don't-guess** (a symbol with no sidecar → exit 2,
+unchanged); **honest-pip-by-construction** (now the provider's recorded value
+instead of a hand-authored one — strictly more honest); **determinism / C6** (the
+sidecar is recorded data read once at bootstrap, never a live mid-replay call);
+the engine's `(Timestamp, Scalar)` seam is untouched (geometry stays beside the
+hot path, C7/C15).
+
+## Concrete code shapes
+
+### Delivered — CLI pip resolution (the production path)
+
+`crates/aura-cli/src/main.rs`. Before — table lookup, server-free, refuses on an
+un-vetted symbol:
+
+```rust
+// ~line 730
+fn instrument_spec_or_refuse(symbol: &str) -> aura_ingest::InstrumentSpec {
+    match aura_ingest::instrument_spec(symbol) {
+        Some(s) => s,
+        None => {
+            eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' — \
+                       refusing to run a real instrument with a guessed pip \
+                       (add it to the instrument table)");
+            std::process::exit(2);
+        }
+    }
+}
+```
+
+After — resolve the pip from the recorded sidecar; refuse when absent. The pip is
+the only datum any caller used, so the helper returns `f64`:
+
+```rust
+// the single home of the guessed-pip refusal, now sidecar-sourced. Reads the
+// per-symbol geometry metadata (not bar data) BEFORE the run source opens, so an
+// instrument with no recorded geometry refuses without touching the archive —
+// the pip stays honest by construction (the provider's value, never guessed).
+fn pip_or_refuse(server: &std::sync::Arc<data_server::DataServer>, symbol: &str) -> f64 {
+    match aura_ingest::instrument_geometry(server, symbol) {
+        Some(geo) => geo.pip_size,
+        None => {
+            eprintln!("aura: no recorded geometry for symbol '{symbol}' at {} — \
+                       refusing to run a real instrument with a guessed pip",
+                      data_server::DEFAULT_DATA_PATH);
+            std::process::exit(2);
+        }
+    }
+}
+```
+
+Both call sites build the server first, then resolve the pip (the refusal stays
+ahead of bar-data access). `open_real_source` (~line 767):
+
+```rust
+// before: spec lookup, THEN server
+//   let spec = instrument_spec_or_refuse(symbol);
+//   let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
+// after: server first, then pip from its sidecar
+let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
+let pip = pip_or_refuse(&server, symbol);
+if !server.has_symbol(symbol) { no_real_data(symbol); }
+// ... probe_window + open as today; return (source, window, pip)
+```
+
+`DataSource::from_choice` (~line 794) takes the same reorder: build `server`,
+`let pip = pip_or_refuse(&server, &symbol);`, then `has_symbol`, then
+`DataSource::Real { server, symbol, from_ms, to_ms, pip }`.
+
+### Delivered — aura-ingest library surface
+
+`crates/aura-ingest/src/lib.rs`. **Removed:** `pub struct InstrumentSpec` (all 9
+fields), `pub fn instrument_spec`, `pub const VETTED_SYMBOLS`, and the three unit
+tests that only exercise them (`instrument_spec_lookup_by_symbol`,
+`instrument_spec_pip_value_matches_contract_times_pip`,
+`instrument_spec_ids_are_distinct`). **Kept unchanged:** the pip source —
+
+```rust
+pub use data_server::meta::InstrumentGeometry;
+
+/// Recorded per-symbol geometry at the ingestion edge: the provider's structural
+/// truth (digits, pip/tick size, lot size, quote currency), read from the
+/// dataset sidecar. Beside the hot path (C7/C15); never enters the engine seam.
+pub fn instrument_geometry(server: &Arc<DataServer>, symbol: &str) -> Option<InstrumentGeometry> {
+    server.symbol_meta(symbol)
+}
+```
+
+and its unit test `instrument_geometry_none_for_unknown_symbol`.
+
+### Delivered — example fixtures
+
+`crates/aura-ingest/examples/shared/breakout_real.rs`. `pip_size_of` resolves
+from the sidecar (real-data examples require the archive anyway):
+
+```rust
+/// The example's pip divisor, sourced from the recorded geometry sidecar — the
+/// same provider truth the CLI `--real` path uses (#98), now provider-sourced.
+/// Panics on a symbol with no sidecar (examples only run known instruments with
+/// local data).
+pub fn pip_size_of(symbol: &str) -> f64 {
+    aura_ingest::instrument_geometry(&aura_ingest::default_data_server(), symbol)
+        .unwrap_or_else(|| panic!("no recorded geometry sidecar for {symbol}"))
+        .pip_size
+}
+```
+
+The **pure-structural** blueprint tests assert on `param_space()` topology, not on
+the pip, and are contracted to "run everywhere" (no archive). They pass a literal
+in place of the sidecar read so they stay hermetic —
+`crates/aura-ingest/tests/ger40_breakout_blueprint.rs`:
+
+```rust
+// param_space_is_exactly_entry_and_exit_bar_targets (#37: "No data needed")
+// and blueprint_param_space_is_construction_deterministic: the pip is a throwaway
+// sizing arg here, never asserted, so a literal keeps these archive-independent.
+let (bp, _taps) = ger40_breakout_blueprint(1.0 /* GER40 pip: index point */, 15, 9, 0, Berlin);
+```
+
+Real-data / archive-gated consumers (`ger40_breakout_world.rs`, the example
+binaries, the gated blueprint fidelity test) keep calling `pip_size_of` — they
+skip cleanly when the archive is absent and run with it present, as today. The
+cross-instrument compare example (`ger40_breakout_compare.rs`, variable `symbol`)
+keeps `pip_size_of(symbol)` and is now per-symbol sidecar-sourced.
+
+### Removed — table-only test files
+
+`crates/aura-ingest/tests/instrument_geometry_crosscheck.rs` (cross-checks the
+removed floor against the provider — moot once the floor is gone) and
+`crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs` (asserts on the
+removed fields).
+
+## Components
+
+- **aura-cli** — `pip_or_refuse` replaces `instrument_spec_or_refuse`; two call
+  sites reordered (server before pip).
+- **aura-ingest (lib)** — public surface shrinks to `instrument_geometry` +
+  `InstrumentGeometry`; the authored floor and its tests are removed.
+- **aura-ingest (examples/tests)** — `pip_size_of` repointed to the sidecar;
+  pure-structural blueprint tests use a literal pip; two table-only test files
+  deleted.
+- **Design ledger** (`docs/design/INDEX.md`) — the C15 realization note added in
+  0073 (authored floor + cross-check + tick_size/digits) is rewritten to record
+  the floor's removal and provider-sourced pip; the #124 hierarchy note records
+  the collapse to "sidecar → refuse" with the override/floor tiers deferred.
+
+## Data flow
+
+`aura --real <SYMBOL>` → build `DataServer` → `pip_or_refuse(server, symbol)`
+reads `<SYMBOL>.meta.json` via `symbol_meta` → `pip_size` (or exit 2) →
+threaded into the real `Source` / SimBroker exactly as before. The
+position-event table's `instrument_id` continues to arrive as a caller parameter,
+untouched.
+
+## Error handling
+
+A symbol with no readable/parseable sidecar → `instrument_geometry` returns
+`None` → `pip_or_refuse` prints the refusal and `exit(2)`, before any bar-data
+access. Same exit code and pre-data ordering as the removed helper; only the
+message and the source change. Any test pinning the old refusal substring is
+updated to the new message.
+
+## Testing strategy
+
+- **RED → GREEN (behaviour):** a test that `pip_or_refuse` (or the real path) for
+  a symbol with a sidecar yields the sidecar's `pip_size`, and that an unknown
+  symbol refuses (exit 2). The existing real-path tests already ratify "pip is
+  threaded into the real source"; this cycle re-sources it without changing that
+  observable.
+- **Deletions:** removing `InstrumentSpec`/`instrument_spec`/`VETTED_SYMBOLS` and
+  their tests is safe by the consumption grep (no production reader). The compile
+  itself enumerates any missed consumer.
+- **Hermetic structural tests stay hermetic:** the two pure blueprint tests must
+  still pass with the archive absent (literal pip, no sidecar read).
+- **Gates:** `cargo test --workspace` green; `cargo clippy --workspace
+  --all-targets -- -D warnings` clean. On this host the archive is present, so the
+  sidecar-sourced paths actually execute.
+
+## Acceptance criteria
+
+1. **Removes redundancy / improves correctness (the project criterion):** the
+   authored floor duplicated the provider geometry byte-for-byte (proven by the
+   #143 cross-check); removing it deletes a drift surface and sources the pip from
+   the broker's recorded truth.
+2. **Reintroduces no failure class the core constraints forbid:** refuse-don't-
+   guess preserved (exit 2 on absent geometry); determinism preserved (sidecar is
+   recorded data read at bootstrap, C6); the engine `(Timestamp, Scalar)` seam
+   untouched (C4/C7); no look-ahead introduced (static metadata).
+3. **Defines only what is needed:** the only instrument datum any current path
+   consumes — `pip_size` — is the only one defined/sourced; the rest is deferred
+   to the Stage-2 broker that will consume it, read from the sidecar then.
+4. `cargo test --workspace` and the clippy gate pass; the pure-structural tests
+   pass with and without the local archive.