diff --git a/README.md b/README.md index 2e70946..d92f9ef 100644 --- a/README.md +++ b/README.md @@ -62,6 +62,17 @@ no other consumer holds it, evicted. All timestamps are **Unix milliseconds** (converted from the Delphi `TDateTime` epoch on load). +### Instrument geometry (`data_server::meta`) + +- `InstrumentGeometry { digits: u32, pip_size: f64, tick_size: f64, lot_size: f64, base: String, quote: String }` + — neutral, broker-agnostic price geometry for one symbol, loaded by + `DataServer::symbol_meta` from the symbol's `.meta.json` sidecar. + `base` / `quote` are passed through verbatim (the consumer resolves provider + codes against its own reference table). A missing sidecar, an unsupported + schema version, or any `null` / non-finite field yields `None` — never a + partial struct. +- `SUPPORTED_SCHEMA_VERSION: u64` — the sidecar `schemaVersion` the reader accepts. + ### Lower-level modules - `data_server::loader` — `load_m1_file` / `load_tick_file` (ZIP → parsed diff --git a/docs/plans/0001-symbol-meta-geometry-reader.md b/docs/plans/0001-symbol-meta-geometry-reader.md deleted file mode 100644 index c240e53..0000000 --- a/docs/plans/0001-symbol-meta-geometry-reader.md +++ /dev/null @@ -1,387 +0,0 @@ -# Per-symbol geometry sidecar reader — Implementation Plan - -> **Parent spec:** `docs/specs/0001-symbol-meta-geometry-reader.md` -> -> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement` -> skill to run this plan. Steps use `- [ ]` checkboxes for tracking. - -**Goal:** Add `DataServer::symbol_meta(symbol) -> Option`, -a reader that loads neutral price geometry from a `.meta.json` sidecar. - -**Architecture:** A new leaf module `src/meta.rs` owns the neutral -`InstrumentGeometry` struct and a pure `from_sidecar_bytes` parser (serde_json -`Value` extraction, absorbing scientific notation, `null`-for-non-finite, and -an unknown schema version). A new one-shot `&self` method on `DataServer` does -the single filesystem read via the existing `SymbolIndex::base_path`. The -reader is independent of the bar `FileCache` and streaming path. - -**Tech Stack:** Rust, `serde_json` (new direct dep), `tempfile` (existing -dev-dep) for the integration tests. - -**Files this plan creates or modifies:** - -- Create: `src/meta.rs` — `InstrumentGeometry`, `SUPPORTED_SCHEMA_VERSION`, - `from_sidecar_bytes`, private `finite`, and `#[cfg(test)] mod tests`. -- Modify: `src/lib.rs:21-23` — add `pub mod meta;` to the module-declaration block. -- Modify: `src/lib.rs:196-201` — add `symbol_meta` method to `impl DataServer`, - after `file_count`, before `stream_m1`. -- Modify: `src/lib.rs` (`#[cfg(test)] mod tests`, near line 792) — add a - `write_meta_file` helper and two integration tests. -- Modify: `Cargo.toml:7-10` — add `serde_json = "1"` to `[dependencies]`. -- Modify: `README.md:8` — update the dependency statement to include `serde_json`. -- Test: `src/meta.rs` unit tests — pin each absorbed parser quirk. -- Test: `src/lib.rs` integration tests — `symbol_meta` over a tempdir. - -**Out-of-scope (not touched):** `tests/data_server.rs` (its `/mnt`-skip-guard -idiom is unsuited to the deterministic tempdir tests); `Cargo.lock` (git-ignored -per the library convention — do NOT `git add` it even though `cargo build` -regenerates it). - ---- - -### Task 1: `meta` module — neutral geometry struct + sidecar parser - -**Files:** -- Modify: `Cargo.toml:7-10` -- Modify: `README.md:8` -- Create: `src/meta.rs` -- Modify: `src/lib.rs:21-23` - -- [ ] **Step 1: Add the `serde_json` dependency and update the README** - -In `Cargo.toml`, change the `[dependencies]` block from: - -```toml -[dependencies] -chrono = "0.4" -regex = "1.10" -zip = "2" -``` - -to: - -```toml -[dependencies] -chrono = "0.4" -regex = "1.10" -serde_json = "1" -zip = "2" -``` - -In `README.md`, change line 8 from: - -```markdown -Standalone leaf crate — depends only on `chrono`, `regex`, `zip`. -``` - -to: - -```markdown -Leaf crate — depends only on `chrono`, `regex`, `serde_json`, `zip`. -``` - -- [ ] **Step 2: Write `src/meta.rs` with the type, a stub parser, and the unit tests** - -Create `src/meta.rs` with this exact content (the parser body is a stub at this -step so the "parses to `Some`" tests fail RED): - -```rust -//! Reader for per-symbol `.meta.json` geometry sidecars. -//! -//! Turns the provider's flat JSON sidecar into neutral, broker-agnostic -//! price-interpretation geometry. Every format quirk (scientific-notation -//! numbers, `null` for a non-finite value, an unknown schema version) is -//! absorbed here so nothing provider-specific reaches a consumer. - -use serde_json::Value; - -/// The only sidecar schema version this reader understands. -pub const SUPPORTED_SCHEMA_VERSION: u64 = 2; - -/// Neutral, broker-agnostic price-interpretation geometry for one symbol. -/// -/// Intrinsic to the historical price series and effectively constant over its -/// span. Time-varying / account-dependent conditions are deliberately excluded. -#[derive(Debug, Clone, PartialEq)] -pub struct InstrumentGeometry { - /// Number of decimal places in a quoted price (e.g. 5 for EURUSD). - pub digits: u32, - /// Price increment of one pip (e.g. 0.0001 for EURUSD). - pub pip_size: f64, - /// Smallest representable price increment (e.g. 1e-05 for EURUSD). - pub tick_size: f64, - /// Contract size of one lot in base units (e.g. 100000 for EURUSD). - pub lot_size: f64, - /// Base asset code, passed through verbatim from the sidecar. - pub base: String, - /// Quote (price) currency code, passed through verbatim. - pub quote: String, -} - -impl InstrumentGeometry { - /// Parse sidecar JSON bytes into neutral geometry. - /// - /// Returns `None` if the JSON is malformed, the schema version is - /// unsupported, or any core field is missing / `null` / non-finite. - pub(crate) fn from_sidecar_bytes(bytes: &[u8]) -> Option { - // STUB — replaced in Step 4. - let _ = bytes; - None - } -} - -/// A finite `f64` from a JSON number, or `None` for missing / null / non-finite. -fn finite(v: Option<&Value>) -> Option { - let n = v?.as_f64()?; - n.is_finite().then_some(n) -} - -#[cfg(test)] -mod tests { - use super::*; - - const EURUSD: &str = r#"{ - "schemaVersion": 2, - "symbol": "EURUSD", - "generatedAtUtc": "2026-06-25T11:26:37Z", - "generator": "MS_SimpleExport (cTrader.Automate)", - "description": "Euro vs US Dollar", - "digits": 5, - "pipSize": 0.0001, - "tickSize": 1E-05, - "lotSize": 100000, - "baseAsset": "EUR", - "quoteAsset": "USD" - }"#; - - #[test] - fn parses_canonical_sidecar_with_scientific_notation_and_integer_lot() { - let g = InstrumentGeometry::from_sidecar_bytes(EURUSD.as_bytes()).unwrap(); - assert_eq!( - g, - InstrumentGeometry { - digits: 5, - pip_size: 0.0001, - tick_size: 1e-05, - lot_size: 100_000.0, - base: "EUR".to_string(), - quote: "USD".to_string(), - } - ); - } - - #[test] - fn integer_valued_geometry_parses_as_float() { - // GER40-shaped: pipSize 1, digits 1, lotSize 1. - let json = r#"{"schemaVersion":2,"digits":1,"pipSize":1,"tickSize":0.1,"lotSize":1,"baseAsset":"DE40","quoteAsset":"EUR"}"#; - let g = InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).unwrap(); - assert_eq!(g.pip_size, 1.0); - assert_eq!(g.digits, 1); - assert_eq!(g.lot_size, 1.0); - } - - #[test] - fn provider_base_codes_pass_through_verbatim() { - let nas = r#"{"schemaVersion":2,"digits":1,"pipSize":1,"tickSize":0.1,"lotSize":1,"baseAsset":"USTEC","quoteAsset":"USD"}"#; - assert_eq!( - InstrumentGeometry::from_sidecar_bytes(nas.as_bytes()).unwrap().base, - "USTEC" - ); - let aapl = r#"{"schemaVersion":2,"digits":2,"pipSize":0.01,"tickSize":0.01,"lotSize":1,"baseAsset":"AAPL.US","quoteAsset":"USD"}"#; - assert_eq!( - InstrumentGeometry::from_sidecar_bytes(aapl.as_bytes()).unwrap().base, - "AAPL.US" - ); - } - - #[test] - fn unsupported_schema_version_yields_none() { - let json = r#"{"schemaVersion":99,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"lotSize":100000,"baseAsset":"EUR","quoteAsset":"USD"}"#; - assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none()); - } - - #[test] - fn null_core_field_yields_none() { - let json = r#"{"schemaVersion":2,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"lotSize":null,"baseAsset":"EUR","quoteAsset":"USD"}"#; - assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none()); - } - - #[test] - fn missing_core_field_yields_none() { - let json = r#"{"schemaVersion":2,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"baseAsset":"EUR","quoteAsset":"USD"}"#; - assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none()); - } - - #[test] - fn malformed_json_yields_none() { - assert!(InstrumentGeometry::from_sidecar_bytes(b"not json").is_none()); - } -} -``` - -Then add the module declaration in `src/lib.rs`, changing the block at lines 21-23 -from: - -```rust -pub mod cache; -pub mod loader; -pub mod records; -``` - -to: - -```rust -pub mod cache; -pub mod loader; -pub mod meta; -pub mod records; -``` - -- [ ] **Step 3: Run the meta unit tests — verify RED** - -Run: `cargo test -p data-server --lib meta::` -Expected: the suite COMPILES and the 7 `meta::tests::*` tests run; the three -that expect `Some` FAIL (`parses_canonical_sidecar_with_scientific_notation_and_integer_lot`, -`integer_valued_geometry_parses_as_float`, `provider_base_codes_pass_through_verbatim`) -with `called Option::unwrap() on a None value` (the stub returns `None`). The -four `*_yields_none` / `malformed_*` tests pass against the stub. - -- [ ] **Step 4: Implement the real parser** - -In `src/meta.rs`, replace the stub body of `from_sidecar_bytes`: - -```rust - // STUB — replaced in Step 4. - let _ = bytes; - None -``` - -with: - -```rust - let v: Value = serde_json::from_slice(bytes).ok()?; - if v.get("schemaVersion").and_then(Value::as_u64)? != SUPPORTED_SCHEMA_VERSION { - return None; - } - Some(Self { - digits: v.get("digits").and_then(Value::as_u64)? as u32, - pip_size: finite(v.get("pipSize"))?, - tick_size: finite(v.get("tickSize"))?, - lot_size: finite(v.get("lotSize"))?, - base: v.get("baseAsset").and_then(Value::as_str)?.to_string(), - quote: v.get("quoteAsset").and_then(Value::as_str)?.to_string(), - }) -``` - -- [ ] **Step 5: Run the meta unit tests — verify GREEN** - -Run: `cargo test -p data-server --lib meta::` -Expected: PASS — all 7 `meta::tests::*` tests green. - ---- - -### Task 2: `DataServer::symbol_meta` method + integration tests - -**Files:** -- Modify: `src/lib.rs:196-201` (add method to `impl DataServer`) -- Modify: `src/lib.rs` (`#[cfg(test)] mod tests`, near line 792) - -- [ ] **Step 1: Write the integration tests (RED)** - -In `src/lib.rs`, inside `#[cfg(test)] mod tests` (after the existing -`write_m1_file` helper, before the first `#[test]`), add a helper: - -```rust - /// Writes a `.meta.json` sidecar into `dir` so the reader can - /// load it. The body is written verbatim. - fn write_meta_file(dir: &Path, symbol: &str, body: &str) { - std::fs::write(dir.join(format!("{symbol}.meta.json")), body).unwrap(); - } -``` - -Then add two tests at the end of the same `mod tests` (before its closing `}`): - -```rust - #[test] - fn test_symbol_meta_reads_sidecar() { - let dir = tempfile::tempdir().unwrap(); - write_meta_file( - dir.path(), - "EURUSD", - r#"{ - "schemaVersion": 2, "digits": 5, "pipSize": 0.0001, - "tickSize": 1E-05, "lotSize": 100000, - "baseAsset": "EUR", "quoteAsset": "USD" - }"#, - ); - let server = DataServer::new(dir.path()); - let g = server.symbol_meta("EURUSD").unwrap(); - assert_eq!(g.digits, 5); - assert_eq!(g.pip_size, 0.0001); - assert_eq!(g.tick_size, 1e-05); - assert_eq!(g.lot_size, 100_000.0); - assert_eq!(g.base, "EUR"); - assert_eq!(g.quote, "USD"); - // Issue's worked derivation: pip_value_per_lot = lot_size * pip_size. - assert!((g.lot_size * g.pip_size - 10.0).abs() < 1e-9); - } - - #[test] - fn test_symbol_meta_missing_sidecar_is_none() { - let dir = tempfile::tempdir().unwrap(); - // A symbol with a bar file but no sidecar -> None. - write_m1_file(dir.path(), "GBPUSD", 2020, 6); - let server = DataServer::new(dir.path()); - assert!(server.symbol_meta("GBPUSD").is_none()); - // An entirely unknown symbol -> None. - assert!(server.symbol_meta("NOPE").is_none()); - } -``` - -- [ ] **Step 2: Run the integration tests — verify RED** - -Run: `cargo test -p data-server --lib test_symbol_meta` -Expected: COMPILE ERROR — `no method named symbol_meta found for ... DataServer`. -(The method does not exist yet; this is the RED state. The two -`test_symbol_meta_*` tests are the ones gated.) - -- [ ] **Step 3: Add the `symbol_meta` method** - -In `src/lib.rs`, inside `impl DataServer`, immediately after the `file_count` -method (which ends at line 200, before `stream_m1` at line 202), insert: - -```rust - /// Loads the neutral price geometry for a symbol from its - /// `.meta.json` sidecar, or `None` if no usable sidecar exists. - /// - /// A missing sidecar is normal (a symbol may have bars but no sidecar) and - /// returns `None` silently. A present-but-unusable sidecar (malformed, - /// unsupported schema version, or an incomplete / `null` / non-finite core - /// field) returns `None` after a one-line stderr warning. The returned - /// geometry is always complete; the consumer never sees a partial struct. - pub fn symbol_meta(&self, symbol: &str) -> Option { - let path = self.index.base_path.join(format!("{symbol}.meta.json")); - let bytes = std::fs::read(&path).ok()?; // missing / unreadable -> silent None - let geo = meta::InstrumentGeometry::from_sidecar_bytes(&bytes); - if geo.is_none() { - eprintln!( - "Warning: {} present but unusable (bad/incomplete geometry)", - path.display() - ); - } - geo - } -``` - -- [ ] **Step 4: Run the integration tests — verify GREEN** - -Run: `cargo test -p data-server --lib test_symbol_meta` -Expected: PASS — `test_symbol_meta_reads_sidecar` and -`test_symbol_meta_missing_sidecar_is_none` both green. - -- [ ] **Step 5: Run the full suite — verify no regression** - -Run: `cargo test -p data-server` -Expected: PASS — all tests green (existing suite + 7 meta unit tests + 2 -`symbol_meta` integration tests). No test is filtered out; the result line -reports the full count. diff --git a/docs/specs/0001-symbol-meta-geometry-reader.md b/docs/specs/0001-symbol-meta-geometry-reader.md deleted file mode 100644 index 0c14e3b..0000000 --- a/docs/specs/0001-symbol-meta-geometry-reader.md +++ /dev/null @@ -1,284 +0,0 @@ -# Per-symbol geometry sidecar reader — Design Spec - -**Date:** 2026-06-25 -**Status:** Draft — awaiting user spec review -**Authors:** orchestrator + Claude - -Tracker: Brummel/data-server#3 (reference issue; fork decisions logged there). - -## Goal - -Add a reader that loads a per-symbol `.meta.json` sidecar from the data -directory this server already owns and returns **neutral** price-interpretation -geometry — `digits`, `pip_size`, `tick_size`, `lot_size`, `base`, `quote` — so a -downstream consumer can ask one server for a symbol's bars *and* its geometry -without parsing provider JSON itself. - -The sidecar carries the price-interpretation facts the bars lack (how many -decimals a price has, what a pip is, what one lot is worth, which currency the -price is quoted in). Keeping the sidecar's filename, schema version, and format -quirks (scientific-notation numbers, `null` for a non-finite value, an unknown -schema version) inside this broker-facing library is what lets the consumer stay -broker-agnostic: it receives a plain Rust struct or nothing, never raw JSON. - -Non-goals: time-varying / account-dependent conditions (commission, swap, -deposit-currency pip value, volume limits, trading status) — a separate, -time-indexed concern, not this reader's job. No currency-code normalization (see -§Error handling, base/quote). No caching (see §Components). - -## Architecture - -A new leaf module `src/meta.rs` owns the neutral type and the pure parser; a new -one-shot method on `DataServer` owns the filesystem read. - -```text - DataServer - ┌───────────────────────────────────────┐ - │ index: SymbolIndex { base_path, … } │ - │ cache: Arc (bars only) │ - │ │ - │ symbol_meta(&self, symbol) ────────────┼──► fs::read(base_path/.meta.json) - └───────────────────────────────────────┘ │ - ▼ - meta::InstrumentGeometry::from_sidecar_bytes(&[u8]) -> Option<…> -``` - -The reader is independent of the bar `FileCache` and the streaming path: it does -not touch the cache, spawns no thread, and needs no `Arc`. It reaches the -data directory through the existing `SymbolIndex::base_path` — the same field -the bar file paths are built from (ratified by `test_file_path_construction` in -`src/lib.rs`, which pins `base_path` as the data directory). Membership in the -bar index is **not** required: a symbol with a sidecar but no bar files, or bar -files but no sidecar, is handled by the file existing or not. - -## Concrete code shapes - -### Worked consumer program (the acceptance evidence) - -What an Aura-side consumer (the counterpart, Brummel/Aura#143) writes — bars and -geometry from one server, no provider JSON in sight: - -```rust -use std::sync::Arc; -use data_server::DataServer; - -let server = Arc::new(DataServer::new("/mnt/tickdata/Pepperstone")); - -if let Some(geo) = server.symbol_meta("EURUSD") { - // geo: InstrumentGeometry — neutral, plain fields - assert_eq!(geo.digits, 5); - assert_eq!(geo.pip_size, 0.0001); - assert_eq!(geo.tick_size, 1e-05); // sidecar wrote it as 1E-05 - assert_eq!(geo.lot_size, 100_000.0); - assert_eq!(geo.quote, "USD"); - - // The consumer derives the rest; nothing provider-specific reached it. - let pip_value_per_lot = geo.lot_size * geo.pip_size; // 100000 * 0.0001 = 10 USD -} - -// Pairs with the existing symbol surface. -for sym in server.symbols() { - if let Some(geo) = server.symbol_meta(&sym) { - // … cross-check geo.base / geo.quote against the consumer's vetted table. - } -} -``` - -`symbol_meta` returns `None` for: no sidecar file, malformed JSON, an -unsupported schema version, or any missing / `null` / non-finite core field -(see §Error handling). The consumer never sees a partial geometry. - -### Implementation shapes (secondary) - -New file `src/meta.rs` — before: does not exist. After: - -```rust -//! Reader for per-symbol `.meta.json` geometry sidecars. - -use serde_json::Value; - -/// The only sidecar schema version this reader understands. -pub const SUPPORTED_SCHEMA_VERSION: u64 = 2; - -/// Neutral, broker-agnostic price-interpretation geometry for one symbol. -/// -/// Intrinsic to the historical price series and effectively constant over its -/// span. Derived/account-dependent conditions are deliberately excluded. -#[derive(Debug, Clone, PartialEq)] -pub struct InstrumentGeometry { - /// Number of decimal places in a quoted price (e.g. 5 for EURUSD). - pub digits: u32, - /// Price increment of one pip (e.g. 0.0001 for EURUSD). - pub pip_size: f64, - /// Smallest representable price increment (e.g. 1e-05 for EURUSD). - pub tick_size: f64, - /// Contract size of one lot in base units (e.g. 100000 for EURUSD). - pub lot_size: f64, - /// Base asset code, passed through verbatim from the sidecar. - pub base: String, - /// Quote (price) currency code, passed through verbatim. - pub quote: String, -} - -impl InstrumentGeometry { - /// Parse sidecar JSON bytes into neutral geometry. - /// - /// Returns `None` if the JSON is malformed, the schema version is - /// unsupported, or any core field is missing / `null` / non-finite. - pub(crate) fn from_sidecar_bytes(bytes: &[u8]) -> Option { - let v: Value = serde_json::from_slice(bytes).ok()?; - if v.get("schemaVersion").and_then(Value::as_u64)? != SUPPORTED_SCHEMA_VERSION { - return None; - } - Some(Self { - digits: v.get("digits").and_then(Value::as_u64)? as u32, - pip_size: finite(v.get("pipSize"))?, - tick_size: finite(v.get("tickSize"))?, - lot_size: finite(v.get("lotSize"))?, - base: v.get("baseAsset").and_then(Value::as_str)?.to_string(), - quote: v.get("quoteAsset").and_then(Value::as_str)?.to_string(), - }) - } -} - -/// A finite `f64` from a JSON number, or `None` for missing / null / non-finite. -/// `Value::as_f64` already rejects `null` and non-numbers; this also rejects -/// `NaN`/`Inf` defensively. -fn finite(v: Option<&Value>) -> Option { - let n = v?.as_f64()?; - n.is_finite().then_some(n) -} -``` - -New method on `DataServer` in `src/lib.rs` — before: absent. After: - -```rust -/// Loads the neutral price geometry for a symbol from its -/// `.meta.json` sidecar, or `None` if no usable sidecar exists. -/// -/// A missing sidecar is normal (`None`, silent). A present-but-unusable -/// sidecar (malformed, unsupported version, incomplete) returns `None` -/// after a one-line stderr warning. -pub fn symbol_meta(&self, symbol: &str) -> Option { - let path = self.index.base_path.join(format!("{symbol}.meta.json")); - let bytes = std::fs::read(&path).ok()?; // missing file -> silent None - let geo = meta::InstrumentGeometry::from_sidecar_bytes(&bytes); - if geo.is_none() { - eprintln!("Warning: {} present but unusable (bad/incomplete geometry)", path.display()); - } - geo -} -``` - -Module wiring in `src/lib.rs` — add `pub mod meta;` alongside the existing -`pub mod cache; pub mod loader; pub mod records;`. `base_path` is already a field -of the private `SymbolIndex`; `symbol_meta` is a `DataServer` method in the same -crate, so it reads `self.index.base_path` directly (no visibility change). - -`Cargo.toml` — add one direct dependency: - -```toml -serde_json = "1" -``` - -## Components - -- **`meta::InstrumentGeometry`** — the public neutral struct (six fields, - `Debug + Clone + PartialEq`). Excludes `symbol`, `description`, `generator`, - `generatedAtUtc` (the most provider-specific fields; `symbol` is already known - to the caller). -- **`meta::InstrumentGeometry::from_sidecar_bytes`** — `pub(crate)` pure parser, - bytes → `Option`. No filesystem, fully unit-testable. Absorbs every - format quirk here. -- **`meta::SUPPORTED_SCHEMA_VERSION`** — the `u64` gate (currently `2`). -- **`DataServer::symbol_meta`** — `&self` one-shot read; the only filesystem - touch. No caching: geometry files are tiny and read rarely (typically once per - symbol); a cache map + invalidation is complexity for no stated need, and the - bar `FileCache` exists for a different problem (large, repeatedly-streamed - files). - -## Data flow - -1. Consumer calls `server.symbol_meta("EURUSD")`. -2. Path built: `/EURUSD.meta.json` (symbols may contain dots, e.g. - `AAPL.US.meta.json`; the literal join handles them). -3. `fs::read` → on error (missing/unreadable) return `None` silently. -4. Bytes → `from_sidecar_bytes`: parse JSON; gate `schemaVersion == 2`; extract - the six fields with the null/non-finite guard. -5. `Some(geometry)` on success; `None` + one stderr warning on a present but - unusable file. - -## Error handling - -The reader absorbs every documented provider quirk so none reaches a consumer: - -- **Scientific notation** (`1E-05`, in 7 of 30 live files) — valid JSON, parses - as a number; `Value::as_f64` handles it. No special code. -- **`null` for a non-finite broker value** — `Value::as_f64`/`as_u64` return - `None` for `null`; that propagates to `symbol_meta` returning `None`. The - consumer never sees a hole. (No `null` exists in the current 30 samples; this - is defensive.) `finite()` additionally rejects `NaN`/`Inf`. -- **Unknown `schemaVersion`** — a reader-side decision, not a consumer's: an - unsupported version returns `None` (+ the call-site warning), never an error - type the consumer must handle. -- **base / quote provider codes** — passed through **verbatim** as `String`. - `baseAsset` is wildly heterogeneous across the 30 samples (ISO `EUR`/`USD`, - index codes `DE40`/`USTEC`/`JP225`, descriptive `Bitcoin`/`Copper`, stock - tickers `AAPL.US`); there is no well-defined "neutral" target and no mapping - table, so normalization is not even definable here. The consumer (Aura#143) - cross-checks the value against its own hand-authored vetted table and never - reads the raw JSON — resolution is the consumer's job by design. "Neutral" - means the struct's shape and semantics are broker-agnostic (a plain Rust type, - parsed numbers, no `serde_json::Value`, no raw JSON, no provider filename - leaking), not that every string is canonicalized. -- **Incomplete object** (a missing core field) — same channel as `null`: - `None`. - -A *missing* sidecar (a symbol with bars but no `.meta.json`) is an expected, -non-error condition → silent `None`. Only a *present* file that cannot yield a -complete geometry warns. - -## Testing strategy - -Unit tests in `src/meta.rs` against the pure `from_sidecar_bytes` (no -filesystem) — each pins one absorbed quirk: - -- Canonical `EURUSD` sidecar → `Some` with exact fields - (`digits 5, pip_size 0.0001, tick_size 1e-05, lot_size 100000.0, base "EUR", - quote "USD"`). Pins scientific-notation `tickSize` and integer `lotSize`. -- Integer-valued geometry (`GER40`: `pipSize 1`, `digits 1`) → `pip_size 1.0`. -- Provider base code passes through unchanged (`NAS100` → `base == "USTEC"`; - dotted `AAPL.US` → `base == "AAPL.US"`). -- Unsupported `schemaVersion` (e.g. `99`) → `None`. -- `null` core field (e.g. `"lotSize": null`) → `None`. -- Missing core field → `None`. -- Malformed JSON → `None`. - -Integration test in `tests/data_server.rs` (or `src/lib.rs` tests) using a -`tempfile::tempdir`: - -- Write `EURUSD.meta.json` into the dir → `server.symbol_meta("EURUSD")` is - `Some` with the expected geometry. -- A symbol with a bar file but no sidecar → `symbol_meta` is `None`. -- An unknown symbol → `symbol_meta` is `None`. - -All tests are deterministic and self-contained (no `/mnt` dependency), matching -the existing test suite's tempdir pattern. - -## Acceptance criteria - -- A consumer obtains a symbol's bars (existing `stream_*`) and its neutral - geometry (`symbol_meta`) from one `DataServer`, pairing with - `has_symbol`/`symbols`. -- The public API exposes only the plain `InstrumentGeometry` struct and `Option` - — no `serde_json` type, no raw JSON, no provider filename, no error type. -- Every documented quirk (scientific notation, `null`-for-non-finite, unknown - schema version, heterogeneous base codes) is absorbed: the consumer sees a - complete neutral geometry or `None`, never a partial struct or a provider - artefact. -- `pip_value_per_lot = lot_size * pip_size` reproduces the issue's verified - values (EURUSD 10 USD; USDJPY 1000 JPY; XAUUSD 10 USD) from the returned - fields. -- The new `serde_json` dependency is the only addition to the dependency set; - the README's dependency statement is updated to reflect it. -- The full suite stays green; new tests pin each absorbed quirk.