audit(0074): cycle close — provider-sourced pip; authored floor removed; drift-clean
Architect audit: drift-clean. The ledger and source now point at the recorded geometry sidecar; no live reference to the removed authored floor survives in code or any live contract. Refuse-don't-guess preserved. Recorded debt (non-blocking, accepted): - fieldtests/milestone-runway references the removed instrument_spec; a frozen non-workspace snapshot, left untouched per the tracked-fieldtests convention. - cli_run.rs test fn names retain unspecced/vetted vocabulary (plan prescribed substring-only edits); cosmetic. #124 stays open for the deferred override/floor tiers the Stage-2 broker needs. Drops the ephemeral cycle spec + plan (rationale lifted to the ledger).
This commit is contained in:
@@ -1,385 +0,0 @@
|
|||||||
# Provider-sourced pip; drop the authored instrument floor — Implementation Plan
|
|
||||||
|
|
||||||
> **Parent spec:** `docs/specs/0074-pip-from-sidecar.md`
|
|
||||||
>
|
|
||||||
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` skill to run
|
|
||||||
> this plan. Steps use `- [ ]` checkboxes for tracking.
|
|
||||||
|
|
||||||
**Goal:** Resolve the real-path pip from the data-server geometry sidecar and
|
|
||||||
remove the now-redundant authored instrument floor (`InstrumentSpec`,
|
|
||||||
`instrument_spec`, `VETTED_SYMBOLS`) and its table-only tests, keeping only the
|
|
||||||
one datum production consumes — `pip_size`.
|
|
||||||
|
|
||||||
**Architecture:** The resolution collapses to `recorded sidecar geometry →
|
|
||||||
refuse-don't-guess`. `aura_ingest::instrument_geometry` (the #143 accessor) is the
|
|
||||||
sole pip source. The authored floor and its 9 fields (incl. the unconsumed
|
|
||||||
`instrument_id`) are deleted; speculative fields revive from `InstrumentGeometry`
|
|
||||||
when the Stage-2 broker consumes them. Tasks are dependency-ordered so every
|
|
||||||
intermediate `cargo test` gate is satisfiable: consumers are repointed (Tasks
|
|
||||||
1–2) *before* the definitions are removed (Task 3). Each task is self-contained
|
|
||||||
green (RED→GREEN within the task per TDD).
|
|
||||||
|
|
||||||
**Tech Stack:** aura-cli (real-data pip resolution), aura-ingest (lib surface +
|
|
||||||
example fixtures), the design ledger.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Files this plan creates or modifies:**
|
|
||||||
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs:726–806` — `instrument_spec_or_refuse`→`pip_or_refuse` (sidecar-sourced `f64`); reorder both call sites (server before pip)
|
|
||||||
- Modify: `crates/aura-cli/tests/cli_run.rs` — new RED test (non-vetted symbol resolves from sidecar) + refusal-message-pin updates + AAPL.US→NONEXISTENT premise fix
|
|
||||||
- Modify: `crates/aura-ingest/examples/shared/breakout_real.rs:71–80` — `pip_size_of` repointed to the sidecar
|
|
||||||
- Modify: `crates/aura-ingest/tests/ger40_breakout_blueprint.rs:40,67,68` — literal pip in the two pure-structural tests
|
|
||||||
- Modify: `crates/aura-ingest/src/lib.rs` — remove `InstrumentSpec` (361–390), `VETTED_SYMBOLS` (392–397), `instrument_spec` (399–436), 3 unit tests (453–557); fix the `:353` intra-doc link
|
|
||||||
- Delete: `crates/aura-ingest/tests/instrument_geometry_crosscheck.rs` (floor cross-check, moot)
|
|
||||||
- Delete: `crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs` (asserts removed fields)
|
|
||||||
- Create: `crates/aura-ingest/tests/instrument_geometry.rs` — relocated seam test guarding the surviving `InstrumentGeometry` re-export
|
|
||||||
- Modify: `docs/design/INDEX.md` — C15 realization note (~825–834), #124 hierarchy note, #22 realization (~604–609), #106 amendment (~1236)
|
|
||||||
- Modify: `crates/aura-engine/src/report.rs:226–227` — comment naming the removed `InstrumentSpec`
|
|
||||||
- Modify: `crates/aura-ingest/examples/ger40_breakout_compare.rs:8,43,50` — doc-prose naming `instrument_spec`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 1: aura-cli — resolve pip from the sidecar (RED → GREEN)
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `crates/aura-cli/tests/cli_run.rs` (new test after `run_real_vetted_index_pip_reaches_the_emitted_manifest` ~line 249; message pins ~145, ~188, ~1189–1195)
|
|
||||||
- Modify: `crates/aura-cli/src/main.rs:726–806`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Write the failing test (RED)**
|
|
||||||
|
|
||||||
Add this test — it captures the directive's whole point: the authored table no
|
|
||||||
longer gates; any symbol with recorded data resolves. USDJPY was never in the
|
|
||||||
vetted 5-symbol table but has a sidecar (`pip 0.01`) and bars on this host.
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// Property: the authored instrument table no longer gates real runs — a symbol
|
|
||||||
/// absent from the (removed) vetted floor but carrying a recorded geometry sidecar
|
|
||||||
/// resolves its pip from the sidecar and runs. USDJPY (never vetted; JPY pip 0.01)
|
|
||||||
/// renders the JPY pip in the manifest, proving the pip is sourced from the provider
|
|
||||||
/// geometry, not a hand-authored table entry. Gated: skips cleanly when USDJPY has
|
|
||||||
/// no local geometry/data (no sidecar → "no recorded geometry"; sidecar but no bars
|
|
||||||
/// → "no local data").
|
|
||||||
#[test]
|
|
||||||
fn run_real_nonvetted_symbol_resolves_pip_from_sidecar() {
|
|
||||||
// FX trades continuously; the GER40 Sept-2024 window also has USDJPY bars.
|
|
||||||
const FROM_MS: &str = "1725148800000";
|
|
||||||
const TO_MS: &str = "1727740799999";
|
|
||||||
let out = std::process::Command::new(env!("CARGO_BIN_EXE_aura"))
|
|
||||||
.args(["run", "--real", "USDJPY", "--from", FROM_MS, "--to", TO_MS])
|
|
||||||
.output()
|
|
||||||
.expect("spawn aura");
|
|
||||||
|
|
||||||
// Skip on a host without USDJPY geometry/data — either refusal is a clean skip,
|
|
||||||
// never the old table-miss "no vetted pip".
|
|
||||||
if out.status.code() == Some(2) {
|
|
||||||
let stderr = String::from_utf8_lossy(&out.stderr);
|
|
||||||
if stderr.contains("no recorded geometry") || stderr.contains("no local data") {
|
|
||||||
eprintln!("skip: no local USDJPY geometry/data");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
assert_eq!(out.status.code(), Some(0), "USDJPY should resolve and run; exit: {:?}, stderr: {}",
|
|
||||||
out.status, String::from_utf8_lossy(&out.stderr));
|
|
||||||
let stdout = String::from_utf8(out.stdout).expect("utf-8 stdout");
|
|
||||||
assert!(
|
|
||||||
stdout.contains("pip_size=0.01"),
|
|
||||||
"USDJPY must render the JPY sidecar pip (0.01) in the manifest, got: {}",
|
|
||||||
stdout.trim_end()
|
|
||||||
);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Run the test to verify it fails RED**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test cli_run run_real_nonvetted_symbol_resolves_pip_from_sidecar`
|
|
||||||
Expected: FAIL — current code refuses USDJPY (`instrument_spec("USDJPY")` is `None`
|
|
||||||
→ exit 2 with "no vetted pip/instrument spec"), which matches neither skip
|
|
||||||
substring, so `assert_eq!(.., Some(0))` fails. (Archive present on this host, so it
|
|
||||||
does not skip.)
|
|
||||||
|
|
||||||
- [ ] **Step 3: Replace the helper with a sidecar-sourced resolver**
|
|
||||||
|
|
||||||
Replace `instrument_spec_or_refuse` (main.rs:726–738):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// Resolve the per-instrument pip from the recorded geometry sidecar, or refuse
|
|
||||||
/// (stderr + exit 2) when the symbol has no recorded geometry — the single home of
|
|
||||||
/// the guessed-pip refusal, shared by `open_real_source` and `from_choice`. Reads
|
|
||||||
/// the symbol's geometry metadata (not bar data) before the run source opens, so an
|
|
||||||
/// instrument with no recorded geometry refuses without a guessed pip; the pip is
|
|
||||||
/// the provider's recorded value, honest by construction.
|
|
||||||
fn pip_or_refuse(server: &std::sync::Arc<data_server::DataServer>, symbol: &str) -> f64 {
|
|
||||||
match aura_ingest::instrument_geometry(server, symbol) {
|
|
||||||
Some(geo) => geo.pip_size,
|
|
||||||
None => {
|
|
||||||
eprintln!(
|
|
||||||
"aura: no recorded geometry for symbol '{symbol}' at {} — \
|
|
||||||
refusing to run a real instrument with a guessed pip",
|
|
||||||
data_server::DEFAULT_DATA_PATH
|
|
||||||
);
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 4: Reorder `open_real_source` (server before pip)**
|
|
||||||
|
|
||||||
In `open_real_source` (main.rs:767–787): build the server first, resolve the pip,
|
|
||||||
return `pip` in the tuple (was `spec.pip_size`).
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Per-instrument pip from the recorded sidecar; resolved BEFORE bar-data access
|
|
||||||
// so an instrument with no geometry refuses without touching the archive.
|
|
||||||
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
|
|
||||||
let pip = pip_or_refuse(&server, symbol);
|
|
||||||
if !server.has_symbol(symbol) {
|
|
||||||
no_real_data(symbol);
|
|
||||||
}
|
|
||||||
// ... probe_window + M1FieldSource::open unchanged ...
|
|
||||||
(source, window, pip)
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 5: Reorder `DataSource::from_choice`**
|
|
||||||
|
|
||||||
In `from_choice` (main.rs:794–805): build the server first, resolve the pip, set
|
|
||||||
`pip` in `DataSource::Real`. Update the doc-comment at :792
|
|
||||||
(`instrument_spec_or_refuse` → `pip_or_refuse`).
|
|
||||||
|
|
||||||
```rust
|
|
||||||
DataChoice::Real { symbol, from_ms, to_ms } => {
|
|
||||||
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
|
|
||||||
let pip = pip_or_refuse(&server, &symbol);
|
|
||||||
if !server.has_symbol(&symbol) {
|
|
||||||
no_real_data(&symbol);
|
|
||||||
}
|
|
||||||
DataSource::Real { server, symbol, from_ms, to_ms, pip }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 6: Update the refusal-message pins + the AAPL.US premise**
|
|
||||||
|
|
||||||
In `cli_run.rs`:
|
|
||||||
- `run_real_unspecced_symbol_refuses_with_exit_2` (~145): substring
|
|
||||||
`"no vetted pip/instrument spec for symbol 'NONEXISTENT'"` →
|
|
||||||
`"no recorded geometry for symbol 'NONEXISTENT'"`; update the comment ~135.
|
|
||||||
- `run_real_vetted_symbol_never_hits_the_pip_refusal` (~188):
|
|
||||||
`!stderr.contains("no vetted pip/instrument spec")` →
|
|
||||||
`!stderr.contains("no recorded geometry")`; update doc-prose ~171–179.
|
|
||||||
- `sweep_real_unspecced_symbol_refuses_with_exit_2` (~1189–1195): change the symbol
|
|
||||||
from `"AAPL.US"` to `"NONEXISTENT"` (AAPL.US now HAS a sidecar and would resolve,
|
|
||||||
not refuse), substring `"no vetted pip"` → `"no recorded geometry"`; update the
|
|
||||||
doc-comment ~1183–1187.
|
|
||||||
|
|
||||||
- [ ] **Step 7: Run the gates for this task**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-cli --test cli_run`
|
|
||||||
Expected: PASS — `run_real_nonvetted_symbol_resolves_pip_from_sidecar` is GREEN
|
|
||||||
(USDJPY resolves), `run_real_vetted_index_pip_reaches_the_emitted_manifest` still
|
|
||||||
GREEN (GER40 sidecar pip == 1.0, byte-identical manifest), and all three updated
|
|
||||||
refusal tests pass.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 2: aura-ingest examples — repoint `pip_size_of`; literal pip in pure tests
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `crates/aura-ingest/examples/shared/breakout_real.rs:71–80`
|
|
||||||
- Modify: `crates/aura-ingest/tests/ger40_breakout_blueprint.rs:40,67,68`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Repoint `pip_size_of` to the sidecar**
|
|
||||||
|
|
||||||
Replace `pip_size_of` (breakout_real.rs:71–80):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// The example's pip divisor, sourced from the recorded geometry sidecar — the same
|
|
||||||
/// provider truth the CLI `--real` path uses, now provider-sourced. Panics on a
|
|
||||||
/// symbol with no sidecar (the examples only run known instruments with local data).
|
|
||||||
pub fn pip_size_of(symbol: &str) -> f64 {
|
|
||||||
aura_ingest::instrument_geometry(&aura_ingest::default_data_server(), symbol)
|
|
||||||
.unwrap_or_else(|| panic!("no recorded geometry sidecar for {symbol}"))
|
|
||||||
.pip_size
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- [ ] **Step 2: Literal pip in the two pure-structural blueprint tests**
|
|
||||||
|
|
||||||
In `ger40_breakout_blueprint.rs`, the two tests asserting `param_space()` topology
|
|
||||||
are contracted to run with no archive and must not read the sidecar. Replace
|
|
||||||
`pip_size_of(SYMBOL)` with a literal at `:40` (in
|
|
||||||
`param_space_is_exactly_entry_and_exit_bar_targets`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// pure-structural: the pip is a throwaway sizing arg, never asserted here, so a
|
|
||||||
// literal keeps this archive-independent (the topology assertion is the point).
|
|
||||||
let (bp, _taps) = ger40_breakout_blueprint(1.0, 15, 9, 0, Berlin);
|
|
||||||
```
|
|
||||||
|
|
||||||
and at `:67`/`:68` (in `blueprint_param_space_is_construction_deterministic`):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let a = ger40_breakout_blueprint(1.0, 15, 9, 0, Berlin).0.param_space();
|
|
||||||
let b = ger40_breakout_blueprint(1.0, 15, 9, 0, Berlin).0.param_space();
|
|
||||||
```
|
|
||||||
|
|
||||||
(The gated `run_blueprint` helper at `:78` keeps `pip_size_of(SYMBOL)` — it runs
|
|
||||||
real data and skips when absent.)
|
|
||||||
|
|
||||||
- [ ] **Step 3: Run the gates for this task**
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-ingest --test ger40_breakout_blueprint`
|
|
||||||
Expected: PASS — the two pure tests pass; the gated fidelity test runs (archive
|
|
||||||
present) or skips (absent).
|
|
||||||
|
|
||||||
Run: `cargo test -p aura-ingest`
|
|
||||||
Expected: PASS — `instrument_spec` is now referenced only by its own unit tests and
|
|
||||||
the two doomed table-only test files; everything compiles and is green.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 3: aura-ingest lib — remove the authored floor; relocate the seam test
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Create: `crates/aura-ingest/tests/instrument_geometry.rs`
|
|
||||||
- Delete: `crates/aura-ingest/tests/instrument_geometry_crosscheck.rs`
|
|
||||||
- Delete: `crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs`
|
|
||||||
- Modify: `crates/aura-ingest/src/lib.rs` (delete 361–390, 392–397, 399–436, 453–557; fix `:353`)
|
|
||||||
|
|
||||||
- [ ] **Step 1: Relocate the surviving seam test into a new file**
|
|
||||||
|
|
||||||
Create `crates/aura-ingest/tests/instrument_geometry.rs` holding only the seam test
|
|
||||||
from `instrument_geometry_crosscheck.rs` (which guards the *surviving*
|
|
||||||
`aura_ingest::InstrumentGeometry` re-export from silent removal), dropping the
|
|
||||||
`instrument_spec` / `VETTED_SYMBOLS` imports:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! Guards the surviving provider-geometry seam: `aura_ingest::instrument_geometry`
|
|
||||||
//! and the `InstrumentGeometry` re-export must stay nameable from outside the crate
|
|
||||||
//! (a downstream broker consumes them). Dropping the re-export would break this.
|
|
||||||
use aura_ingest::{default_data_server, instrument_geometry, InstrumentGeometry};
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn provider_geometry_type_is_nameable_through_the_ingest_seam() {
|
|
||||||
// Name the re-exported type explicitly (not just `_`), so removing the
|
|
||||||
// `pub use data_server::meta::InstrumentGeometry` re-export fails to compile here.
|
|
||||||
let server = default_data_server();
|
|
||||||
let _maybe: Option<InstrumentGeometry> = instrument_geometry(&server, "EURUSD");
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
(Match the original test's exact body if it differs; the load-bearing property is
|
|
||||||
that it names `aura_ingest::InstrumentGeometry`.)
|
|
||||||
|
|
||||||
- [ ] **Step 2: Delete the two table-only test files**
|
|
||||||
|
|
||||||
Run: `git rm crates/aura-ingest/tests/instrument_geometry_crosscheck.rs crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs`
|
|
||||||
Expected: both files removed from the working tree.
|
|
||||||
|
|
||||||
- [ ] **Step 3: Remove the authored floor from the lib**
|
|
||||||
|
|
||||||
In `crates/aura-ingest/src/lib.rs` delete: `pub struct InstrumentSpec` with its doc
|
|
||||||
(361–390), `pub const VETTED_SYMBOLS` (392–397), `pub fn instrument_spec`
|
|
||||||
(399–436), and the three unit tests `instrument_spec_lookup_by_symbol`,
|
|
||||||
`instrument_spec_pip_value_matches_contract_times_pip`,
|
|
||||||
`instrument_spec_ids_are_distinct` (453–557). Keep `instrument_geometry`, the
|
|
||||||
`pub use data_server::meta::InstrumentGeometry` re-export, `default_data_server`,
|
|
||||||
and `instrument_geometry_none_for_unknown_symbol` untouched.
|
|
||||||
|
|
||||||
- [ ] **Step 4: Fix the broken intra-doc link**
|
|
||||||
|
|
||||||
In the surviving `instrument_geometry` doc comment (~lib.rs:353), the reference
|
|
||||||
`[instrument_spec]` now points at a removed item and breaks `cargo doc`. Reword to
|
|
||||||
drop the link, e.g. "Returns `None` for a symbol with no recorded sidecar — the
|
|
||||||
caller refuses rather than guess a pip."
|
|
||||||
|
|
||||||
- [ ] **Step 5: Run the gates for this task**
|
|
||||||
|
|
||||||
Run: `cargo test --workspace`
|
|
||||||
Expected: PASS — no remaining reference to `InstrumentSpec` / `instrument_spec` /
|
|
||||||
`VETTED_SYMBOLS`; the new `instrument_geometry` test runs.
|
|
||||||
|
|
||||||
Run: `cargo doc --workspace --no-deps 2>&1`
|
|
||||||
Expected: no warnings/errors (the `:353` intra-doc link is fixed).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Task 4: Ledger and prose hygiene
|
|
||||||
|
|
||||||
**Files:**
|
|
||||||
- Modify: `docs/design/INDEX.md` (~604–609, ~709, ~817–834, ~1236, and the #124 note)
|
|
||||||
- Modify: `crates/aura-engine/src/report.rs:226–227`
|
|
||||||
- Modify: `crates/aura-ingest/examples/ger40_breakout_compare.rs:8,43,50`
|
|
||||||
|
|
||||||
- [ ] **Step 1: Rewrite the ledger passages that assert the removed mechanism**
|
|
||||||
|
|
||||||
In `docs/design/INDEX.md`:
|
|
||||||
- The C15 realization note added in cycle 0073 (~825–834: authored floor +
|
|
||||||
cross-check + tick_size/digits): rewrite to record that the authored floor was
|
|
||||||
removed in cycle 0074, the real-path pip is sourced from the data-server geometry
|
|
||||||
sidecar (`instrument_geometry`), and the override/floor tiers of #124 are deferred
|
|
||||||
until a consumer (the Stage-2 broker) needs them, read from the sidecar then.
|
|
||||||
- The #124 hierarchy passage: record the collapse to `recorded sidecar geometry →
|
|
||||||
refuse`, tiers 1 (authored override) and 3 (authored floor) deferred.
|
|
||||||
- The #22 realization passage (~604–609) and the #106 amendment (~1236) that name
|
|
||||||
`InstrumentSpec { pip_size } + instrument_spec(symbol)`: update to "pip resolved
|
|
||||||
from the recorded geometry sidecar (`instrument_geometry`); refuse-don't-guess on
|
|
||||||
absent geometry". Reword the C8 derive note (~709, "never imports InstrumentSpec")
|
|
||||||
to not name the removed type while keeping its (still-true) point.
|
|
||||||
|
|
||||||
- [ ] **Step 2: Fix the engine comment**
|
|
||||||
|
|
||||||
In `crates/aura-engine/src/report.rs:226–227`, the comment "`instrument_id` is
|
|
||||||
supplied by the caller (the engine never imports `InstrumentSpec`)" names a removed
|
|
||||||
type. Reword: "the engine never imports an instrument spec from `aura-ingest`".
|
|
||||||
|
|
||||||
- [ ] **Step 3: Fix the example doc-prose**
|
|
||||||
|
|
||||||
In `crates/aura-ingest/examples/ger40_breakout_compare.rs` (~8, ~43, ~50),
|
|
||||||
doc-comments naming `instrument_spec`: reword to "the recorded geometry sidecar".
|
|
||||||
(The live `pip_size_of(symbol)` call ~60 already works post-Task-2.)
|
|
||||||
|
|
||||||
- [ ] **Step 4: Final gates**
|
|
||||||
|
|
||||||
Run: `git grep -nE '\b(InstrumentSpec|instrument_spec|VETTED_SYMBOLS)\b' -- crates/`
|
|
||||||
Expected: NO matches under `crates/` (the symbols are gone; only `fieldtests/` — a
|
|
||||||
frozen, non-workspace snapshot left untouched by design — may still name them).
|
|
||||||
|
|
||||||
Run: `cargo test --workspace`
|
|
||||||
Expected: PASS.
|
|
||||||
|
|
||||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
|
||||||
Expected: clean.
|
|
||||||
|
|
||||||
Run: `cargo doc --workspace --no-deps 2>&1`
|
|
||||||
Expected: clean.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Self-review
|
|
||||||
|
|
||||||
1. **Spec coverage:** every spec section is implemented — CLI pip resolution +
|
|
||||||
new-behaviour RED test (Task 1), example repoint + hermetic structural tests
|
|
||||||
(Task 2), lib surface shrink + deletions + seam-test relocation (Task 3),
|
|
||||||
ledger/prose updates (Task 4). ✓
|
|
||||||
2. **Placeholder scan:** no TBD/TODO/"implement later"/"similar to Task". ✓
|
|
||||||
3. **Type/name consistency:** `pip_or_refuse`, `instrument_geometry`,
|
|
||||||
`InstrumentGeometry`, `pip_size_of`, `default_data_server` used identically
|
|
||||||
across tasks. ✓
|
|
||||||
4. **Step granularity:** each step is a single 2–5-min edit or one Run command;
|
|
||||||
Task 1 is RED→GREEN within the one task (canonical TDD shape). ✓
|
|
||||||
5. **No commit steps:** none; the orchestrator commits the iter. The `git rm` in
|
|
||||||
Task 3 is a working-tree deletion, not a commit. ✓
|
|
||||||
6. **Pin/replacement contiguity:** the refusal substring `"no recorded geometry
|
|
||||||
for symbol '<sym>'"` appears contiguously in the `pip_or_refuse` `eprintln!`
|
|
||||||
(Task 1 Step 3) and in the assertions (Task 1 Step 6); the `eprintln!`
|
|
||||||
`\`-continuation joins to one line at runtime, so the substring is contiguous in
|
|
||||||
the emitted text. ✓
|
|
||||||
7. **Compile-gate vs deferred-caller ordering:** consumers are repointed (Tasks
|
|
||||||
1–2) BEFORE the definitions are removed (Task 3), so every task gate is
|
|
||||||
satisfiable; Task 3's removal threads no external caller (none remain). ✓
|
|
||||||
8. **Verification-filter strings resolve:** Task 1's filter
|
|
||||||
`run_real_nonvetted_symbol_resolves_pip_from_sidecar` is the exact new test
|
|
||||||
name; `--test cli_run` / `--test ger40_breakout_blueprint` are existing files;
|
|
||||||
Task 4's `git grep` is unfiltered-by-test. ✓
|
|
||||||
@@ -1,238 +0,0 @@
|
|||||||
# Provider-sourced pip; drop the authored instrument floor — Design Spec
|
|
||||||
|
|
||||||
**Date:** 2026-06-25
|
|
||||||
**Status:** Draft — awaiting user spec review
|
|
||||||
**Authors:** orchestrator + Claude
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
aura must define and take over only the instrument reference data it currently
|
|
||||||
consumes, and source it from the data-server geometry sidecars rather than a
|
|
||||||
hand-maintained table. A grep of every `InstrumentSpec` field *read* across
|
|
||||||
`crates/` shows production code consumes exactly one datum — `pip_size` — at the
|
|
||||||
real-data path. Every other field (`contract_size`, `pip_value_per_lot`,
|
|
||||||
`min_lot`, `lot_step`, `quote_currency`, `tick_size`, `digits`, **and
|
|
||||||
`instrument_id`**) is constructed but never read outside the table's own tests.
|
|
||||||
|
|
||||||
This cycle resolves the pip from the per-symbol sidecar
|
|
||||||
(`instrument_geometry(server, symbol).pip_size`) and removes the now-redundant
|
|
||||||
authored floor (`InstrumentSpec`, `instrument_spec()`, `VETTED_SYMBOLS`) together
|
|
||||||
with the tests that exist only to validate it. The decision and its grounding are
|
|
||||||
recorded on the reference issue (#124, reconciliation comment).
|
|
||||||
|
|
||||||
Deferred, not deleted (per the user directive "build it back in when the broker
|
|
||||||
needs it"): the speculative fields are revived from `InstrumentGeometry` — which
|
|
||||||
already carries `digits`, `lotSize`, `tickSize`, `quoteAsset` — when the Stage-2
|
|
||||||
realistic-broker milestone actually consumes them. No knowledge is lost; the
|
|
||||||
sidecar remains the source.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
The instrument-reference resolution hierarchy of #124 (authored override >
|
|
||||||
recorded metadata > authored floor > refuse) collapses, for what is built now, to
|
|
||||||
its two ends: **recorded sidecar geometry → refuse-don't-guess**. The middle
|
|
||||||
authored floor (tier 3) and the authored override (tier 1) are future work, built
|
|
||||||
when a consumer needs them. The `instrument_geometry` accessor and the
|
|
||||||
`InstrumentGeometry` re-export added in #143 are the surviving seam and become the
|
|
||||||
sole pip source.
|
|
||||||
|
|
||||||
The `instrument_id` question (the directive's "find out"): the position-event
|
|
||||||
table needs an `instrument_id`, but it already takes one as a **caller-supplied
|
|
||||||
`i64` parameter** — `derive_position_events(record, instrument_id)`
|
|
||||||
(report.rs:231) — and the engine deliberately never imports `InstrumentSpec`
|
|
||||||
(report.rs:226). `InstrumentSpec.instrument_id` has no production reader; it is
|
|
||||||
dropped with the struct. No symbol→id registry is consumed anywhere today, so
|
|
||||||
none is built; when one is needed it is added then.
|
|
||||||
|
|
||||||
Invariants preserved: **refuse-don't-guess** (a symbol with no sidecar → exit 2,
|
|
||||||
unchanged); **honest-pip-by-construction** (now the provider's recorded value
|
|
||||||
instead of a hand-authored one — strictly more honest); **determinism / C6** (the
|
|
||||||
sidecar is recorded data read once at bootstrap, never a live mid-replay call);
|
|
||||||
the engine's `(Timestamp, Scalar)` seam is untouched (geometry stays beside the
|
|
||||||
hot path, C7/C15).
|
|
||||||
|
|
||||||
## Concrete code shapes
|
|
||||||
|
|
||||||
### Delivered — CLI pip resolution (the production path)
|
|
||||||
|
|
||||||
`crates/aura-cli/src/main.rs`. Before — table lookup, server-free, refuses on an
|
|
||||||
un-vetted symbol:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// ~line 730
|
|
||||||
fn instrument_spec_or_refuse(symbol: &str) -> aura_ingest::InstrumentSpec {
|
|
||||||
match aura_ingest::instrument_spec(symbol) {
|
|
||||||
Some(s) => s,
|
|
||||||
None => {
|
|
||||||
eprintln!("aura: no vetted pip/instrument spec for symbol '{symbol}' — \
|
|
||||||
refusing to run a real instrument with a guessed pip \
|
|
||||||
(add it to the instrument table)");
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
After — resolve the pip from the recorded sidecar; refuse when absent. The pip is
|
|
||||||
the only datum any caller used, so the helper returns `f64`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// the single home of the guessed-pip refusal, now sidecar-sourced. Reads the
|
|
||||||
// per-symbol geometry metadata (not bar data) BEFORE the run source opens, so an
|
|
||||||
// instrument with no recorded geometry refuses without touching the archive —
|
|
||||||
// the pip stays honest by construction (the provider's value, never guessed).
|
|
||||||
fn pip_or_refuse(server: &std::sync::Arc<data_server::DataServer>, symbol: &str) -> f64 {
|
|
||||||
match aura_ingest::instrument_geometry(server, symbol) {
|
|
||||||
Some(geo) => geo.pip_size,
|
|
||||||
None => {
|
|
||||||
eprintln!("aura: no recorded geometry for symbol '{symbol}' at {} — \
|
|
||||||
refusing to run a real instrument with a guessed pip",
|
|
||||||
data_server::DEFAULT_DATA_PATH);
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Both call sites build the server first, then resolve the pip (the refusal stays
|
|
||||||
ahead of bar-data access). `open_real_source` (~line 767):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// before: spec lookup, THEN server
|
|
||||||
// let spec = instrument_spec_or_refuse(symbol);
|
|
||||||
// let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
|
|
||||||
// after: server first, then pip from its sidecar
|
|
||||||
let server = std::sync::Arc::new(data_server::DataServer::new(data_server::DEFAULT_DATA_PATH));
|
|
||||||
let pip = pip_or_refuse(&server, symbol);
|
|
||||||
if !server.has_symbol(symbol) { no_real_data(symbol); }
|
|
||||||
// ... probe_window + open as today; return (source, window, pip)
|
|
||||||
```
|
|
||||||
|
|
||||||
`DataSource::from_choice` (~line 794) takes the same reorder: build `server`,
|
|
||||||
`let pip = pip_or_refuse(&server, &symbol);`, then `has_symbol`, then
|
|
||||||
`DataSource::Real { server, symbol, from_ms, to_ms, pip }`.
|
|
||||||
|
|
||||||
### Delivered — aura-ingest library surface
|
|
||||||
|
|
||||||
`crates/aura-ingest/src/lib.rs`. **Removed:** `pub struct InstrumentSpec` (all 9
|
|
||||||
fields), `pub fn instrument_spec`, `pub const VETTED_SYMBOLS`, and the three unit
|
|
||||||
tests that only exercise them (`instrument_spec_lookup_by_symbol`,
|
|
||||||
`instrument_spec_pip_value_matches_contract_times_pip`,
|
|
||||||
`instrument_spec_ids_are_distinct`). **Kept unchanged:** the pip source —
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub use data_server::meta::InstrumentGeometry;
|
|
||||||
|
|
||||||
/// Recorded per-symbol geometry at the ingestion edge: the provider's structural
|
|
||||||
/// truth (digits, pip/tick size, lot size, quote currency), read from the
|
|
||||||
/// dataset sidecar. Beside the hot path (C7/C15); never enters the engine seam.
|
|
||||||
pub fn instrument_geometry(server: &Arc<DataServer>, symbol: &str) -> Option<InstrumentGeometry> {
|
|
||||||
server.symbol_meta(symbol)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
and its unit test `instrument_geometry_none_for_unknown_symbol`.
|
|
||||||
|
|
||||||
### Delivered — example fixtures
|
|
||||||
|
|
||||||
`crates/aura-ingest/examples/shared/breakout_real.rs`. `pip_size_of` resolves
|
|
||||||
from the sidecar (real-data examples require the archive anyway):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
/// The example's pip divisor, sourced from the recorded geometry sidecar — the
|
|
||||||
/// same provider truth the CLI `--real` path uses (#98), now provider-sourced.
|
|
||||||
/// Panics on a symbol with no sidecar (examples only run known instruments with
|
|
||||||
/// local data).
|
|
||||||
pub fn pip_size_of(symbol: &str) -> f64 {
|
|
||||||
aura_ingest::instrument_geometry(&aura_ingest::default_data_server(), symbol)
|
|
||||||
.unwrap_or_else(|| panic!("no recorded geometry sidecar for {symbol}"))
|
|
||||||
.pip_size
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The **pure-structural** blueprint tests assert on `param_space()` topology, not on
|
|
||||||
the pip, and are contracted to "run everywhere" (no archive). They pass a literal
|
|
||||||
in place of the sidecar read so they stay hermetic —
|
|
||||||
`crates/aura-ingest/tests/ger40_breakout_blueprint.rs`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// param_space_is_exactly_entry_and_exit_bar_targets (#37: "No data needed")
|
|
||||||
// and blueprint_param_space_is_construction_deterministic: the pip is a throwaway
|
|
||||||
// sizing arg here, never asserted, so a literal keeps these archive-independent.
|
|
||||||
let (bp, _taps) = ger40_breakout_blueprint(1.0 /* GER40 pip: index point */, 15, 9, 0, Berlin);
|
|
||||||
```
|
|
||||||
|
|
||||||
Real-data / archive-gated consumers (`ger40_breakout_world.rs`, the example
|
|
||||||
binaries, the gated blueprint fidelity test) keep calling `pip_size_of` — they
|
|
||||||
skip cleanly when the archive is absent and run with it present, as today. The
|
|
||||||
cross-instrument compare example (`ger40_breakout_compare.rs`, variable `symbol`)
|
|
||||||
keeps `pip_size_of(symbol)` and is now per-symbol sidecar-sourced.
|
|
||||||
|
|
||||||
### Removed — table-only test files
|
|
||||||
|
|
||||||
`crates/aura-ingest/tests/instrument_geometry_crosscheck.rs` (cross-checks the
|
|
||||||
removed floor against the provider — moot once the floor is gone) and
|
|
||||||
`crates/aura-ingest/tests/instrument_spec_deploy_metadata.rs` (asserts on the
|
|
||||||
removed fields).
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
- **aura-cli** — `pip_or_refuse` replaces `instrument_spec_or_refuse`; two call
|
|
||||||
sites reordered (server before pip).
|
|
||||||
- **aura-ingest (lib)** — public surface shrinks to `instrument_geometry` +
|
|
||||||
`InstrumentGeometry`; the authored floor and its tests are removed.
|
|
||||||
- **aura-ingest (examples/tests)** — `pip_size_of` repointed to the sidecar;
|
|
||||||
pure-structural blueprint tests use a literal pip; two table-only test files
|
|
||||||
deleted.
|
|
||||||
- **Design ledger** (`docs/design/INDEX.md`) — the C15 realization note added in
|
|
||||||
0073 (authored floor + cross-check + tick_size/digits) is rewritten to record
|
|
||||||
the floor's removal and provider-sourced pip; the #124 hierarchy note records
|
|
||||||
the collapse to "sidecar → refuse" with the override/floor tiers deferred.
|
|
||||||
|
|
||||||
## Data flow
|
|
||||||
|
|
||||||
`aura --real <SYMBOL>` → build `DataServer` → `pip_or_refuse(server, symbol)`
|
|
||||||
reads `<SYMBOL>.meta.json` via `symbol_meta` → `pip_size` (or exit 2) →
|
|
||||||
threaded into the real `Source` / SimBroker exactly as before. The
|
|
||||||
position-event table's `instrument_id` continues to arrive as a caller parameter,
|
|
||||||
untouched.
|
|
||||||
|
|
||||||
## Error handling
|
|
||||||
|
|
||||||
A symbol with no readable/parseable sidecar → `instrument_geometry` returns
|
|
||||||
`None` → `pip_or_refuse` prints the refusal and `exit(2)`, before any bar-data
|
|
||||||
access. Same exit code and pre-data ordering as the removed helper; only the
|
|
||||||
message and the source change. Any test pinning the old refusal substring is
|
|
||||||
updated to the new message.
|
|
||||||
|
|
||||||
## Testing strategy
|
|
||||||
|
|
||||||
- **RED → GREEN (behaviour):** a test that `pip_or_refuse` (or the real path) for
|
|
||||||
a symbol with a sidecar yields the sidecar's `pip_size`, and that an unknown
|
|
||||||
symbol refuses (exit 2). The existing real-path tests already ratify "pip is
|
|
||||||
threaded into the real source"; this cycle re-sources it without changing that
|
|
||||||
observable.
|
|
||||||
- **Deletions:** removing `InstrumentSpec`/`instrument_spec`/`VETTED_SYMBOLS` and
|
|
||||||
their tests is safe by the consumption grep (no production reader). The compile
|
|
||||||
itself enumerates any missed consumer.
|
|
||||||
- **Hermetic structural tests stay hermetic:** the two pure blueprint tests must
|
|
||||||
still pass with the archive absent (literal pip, no sidecar read).
|
|
||||||
- **Gates:** `cargo test --workspace` green; `cargo clippy --workspace
|
|
||||||
--all-targets -- -D warnings` clean. On this host the archive is present, so the
|
|
||||||
sidecar-sourced paths actually execute.
|
|
||||||
|
|
||||||
## Acceptance criteria
|
|
||||||
|
|
||||||
1. **Removes redundancy / improves correctness (the project criterion):** the
|
|
||||||
authored floor duplicated the provider geometry byte-for-byte (proven by the
|
|
||||||
#143 cross-check); removing it deletes a drift surface and sources the pip from
|
|
||||||
the broker's recorded truth.
|
|
||||||
2. **Reintroduces no failure class the core constraints forbid:** refuse-don't-
|
|
||||||
guess preserved (exit 2 on absent geometry); determinism preserved (sidecar is
|
|
||||||
recorded data read at bootstrap, C6); the engine `(Timestamp, Scalar)` seam
|
|
||||||
untouched (C4/C7); no look-ahead introduced (static metadata).
|
|
||||||
3. **Defines only what is needed:** the only instrument datum any current path
|
|
||||||
consumes — `pip_size` — is the only one defined/sourced; the rest is deferred
|
|
||||||
to the Stage-2 broker that will consume it, read from the sidecar then.
|
|
||||||
4. `cargo test --workspace` and the clippy gate pass; the pure-structural tests
|
|
||||||
pass with and without the local archive.
|
|
||||||
Reference in New Issue
Block a user