Brummel/data-server
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

audit: cycle #3 tidy (clean)

Browse Source 
Cycle-close for the per-symbol geometry sidecar reader (#3).

Architect drift review (0f5e665..be2489a): code matches the spec verbatim;
the README "## Interface" contract was already updated in lockstep (symbol_meta
row + serde_json dependency line). Two drift items, both resolved here:
- [medium] Retire the ephemeral active-cycle artefacts: git-rm the spec and
  plan (recoverable from history via `git show <rev>:<path>`), per the
  docs/specs + docs/plans lifecycle convention.
- [low] Document meta::InstrumentGeometry to parity with the other public
  record types: add a "data_server::meta" subsection enumerating its six
  fields and SUPPORTED_SCHEMA_VERSION.

Regression gate (all green):
- cargo build   -> exit 0, finished clean
- cargo clippy  -> exit 0, 0 warnings
- cargo test    -> exit 0; 36 lib + 4 integration + 0 doc tests passed

No baseline to update (the project has no metric baseline; cargo is the gate).

refs #3
This commit is contained in:
Brummel
2026-06-25 14:44:37 +02:00
parent be2489ac95
commit 694f96f2b2
3 changed files with 11 additions and 671 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+11
View File
@@ -62,6 +62,17 @@ no other consumer holds it, evicted.
All timestamps are **Unix milliseconds** (converted from the Delphi All timestamps are **Unix milliseconds** (converted from the Delphi
`TDateTime` epoch on load). `TDateTime` epoch on load).
### Instrument geometry (`data_server::meta`)
- `InstrumentGeometry { digits: u32, pip_size: f64, tick_size: f64, lot_size: f64, base: String, quote: String }`
— neutral, broker-agnostic price geometry for one symbol, loaded by
`DataServer::symbol_meta` from the symbol's `<SYMBOL>.meta.json` sidecar.
`base` / `quote` are passed through verbatim (the consumer resolves provider
codes against its own reference table). A missing sidecar, an unsupported
schema version, or any `null` / non-finite field yields `None` — never a
partial struct.
- `SUPPORTED_SCHEMA_VERSION: u64` — the sidecar `schemaVersion` the reader accepts.
### Lower-level modules ### Lower-level modules
- `data_server::loader` — `load_m1_file` / `load_tick_file` (ZIP → parsed - `data_server::loader` — `load_m1_file` / `load_tick_file` (ZIP → parsed
docs/plans/0001-symbol-meta-geometry-reader.md
-387
View File
@@ -1,387 +0,0 @@
# Per-symbol geometry sidecar reader — Implementation Plan
> **Parent spec:** `docs/specs/0001-symbol-meta-geometry-reader.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use the `implement`
> skill to run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Add `DataServer::symbol_meta(symbol) -> Option<InstrumentGeometry>`,
a reader that loads neutral price geometry from a `<SYMBOL>.meta.json` sidecar.
**Architecture:** A new leaf module `src/meta.rs` owns the neutral
`InstrumentGeometry` struct and a pure `from_sidecar_bytes` parser (serde_json
`Value` extraction, absorbing scientific notation, `null`-for-non-finite, and
an unknown schema version). A new one-shot `&self` method on `DataServer` does
the single filesystem read via the existing `SymbolIndex::base_path`. The
reader is independent of the bar `FileCache` and streaming path.
**Tech Stack:** Rust, `serde_json` (new direct dep), `tempfile` (existing
dev-dep) for the integration tests.
**Files this plan creates or modifies:**
- Create: `src/meta.rs` — `InstrumentGeometry`, `SUPPORTED_SCHEMA_VERSION`,
`from_sidecar_bytes`, private `finite`, and `#[cfg(test)] mod tests`.
- Modify: `src/lib.rs:21-23` — add `pub mod meta;` to the module-declaration block.
- Modify: `src/lib.rs:196-201` — add `symbol_meta` method to `impl DataServer`,
after `file_count`, before `stream_m1`.
- Modify: `src/lib.rs` (`#[cfg(test)] mod tests`, near line 792) — add a
`write_meta_file` helper and two integration tests.
- Modify: `Cargo.toml:7-10` — add `serde_json = "1"` to `[dependencies]`.
- Modify: `README.md:8` — update the dependency statement to include `serde_json`.
- Test: `src/meta.rs` unit tests — pin each absorbed parser quirk.
- Test: `src/lib.rs` integration tests — `symbol_meta` over a tempdir.
**Out-of-scope (not touched):** `tests/data_server.rs` (its `/mnt`-skip-guard
idiom is unsuited to the deterministic tempdir tests); `Cargo.lock` (git-ignored
per the library convention — do NOT `git add` it even though `cargo build`
regenerates it).
---
### Task 1: `meta` module — neutral geometry struct + sidecar parser
**Files:**
- Modify: `Cargo.toml:7-10`
- Modify: `README.md:8`
- Create: `src/meta.rs`
- Modify: `src/lib.rs:21-23`
- [ ] **Step 1: Add the `serde_json` dependency and update the README**
In `Cargo.toml`, change the `[dependencies]` block from:
```toml
[dependencies]
chrono = "0.4"
regex = "1.10"
zip = "2"
```
to:
```toml
[dependencies]
chrono = "0.4"
regex = "1.10"
serde_json = "1"
zip = "2"
```
In `README.md`, change line 8 from:
```markdown
Standalone leaf crate — depends only on `chrono`, `regex`, `zip`.
```
to:
```markdown
Leaf crate — depends only on `chrono`, `regex`, `serde_json`, `zip`.
```
- [ ] **Step 2: Write `src/meta.rs` with the type, a stub parser, and the unit tests**
Create `src/meta.rs` with this exact content (the parser body is a stub at this
step so the "parses to `Some`" tests fail RED):
```rust
//! Reader for per-symbol `<SYMBOL>.meta.json` geometry sidecars.
//!
//! Turns the provider's flat JSON sidecar into neutral, broker-agnostic
//! price-interpretation geometry. Every format quirk (scientific-notation
//! numbers, `null` for a non-finite value, an unknown schema version) is
//! absorbed here so nothing provider-specific reaches a consumer.
use serde_json::Value;
/// The only sidecar schema version this reader understands.
pub const SUPPORTED_SCHEMA_VERSION: u64 = 2;
/// Neutral, broker-agnostic price-interpretation geometry for one symbol.
///
/// Intrinsic to the historical price series and effectively constant over its
/// span. Time-varying / account-dependent conditions are deliberately excluded.
#[derive(Debug, Clone, PartialEq)]
pub struct InstrumentGeometry {
/// Number of decimal places in a quoted price (e.g. 5 for EURUSD).
pub digits: u32,
/// Price increment of one pip (e.g. 0.0001 for EURUSD).
pub pip_size: f64,
/// Smallest representable price increment (e.g. 1e-05 for EURUSD).
pub tick_size: f64,
/// Contract size of one lot in base units (e.g. 100000 for EURUSD).
pub lot_size: f64,
/// Base asset code, passed through verbatim from the sidecar.
pub base: String,
/// Quote (price) currency code, passed through verbatim.
pub quote: String,
}
impl InstrumentGeometry {
/// Parse sidecar JSON bytes into neutral geometry.
///
/// Returns `None` if the JSON is malformed, the schema version is
/// unsupported, or any core field is missing / `null` / non-finite.
pub(crate) fn from_sidecar_bytes(bytes: &[u8]) -> Option<Self> {
// STUB — replaced in Step 4.
let _ = bytes;
None
}
}
/// A finite `f64` from a JSON number, or `None` for missing / null / non-finite.
fn finite(v: Option<&Value>) -> Option<f64> {
let n = v?.as_f64()?;
n.is_finite().then_some(n)
}
#[cfg(test)]
mod tests {
use super::*;
const EURUSD: &str = r#"{
"schemaVersion": 2,
"symbol": "EURUSD",
"generatedAtUtc": "2026-06-25T11:26:37Z",
"generator": "MS_SimpleExport (cTrader.Automate)",
"description": "Euro vs US Dollar",
"digits": 5,
"pipSize": 0.0001,
"tickSize": 1E-05,
"lotSize": 100000,
"baseAsset": "EUR",
"quoteAsset": "USD"
}"#;
#[test]
fn parses_canonical_sidecar_with_scientific_notation_and_integer_lot() {
let g = InstrumentGeometry::from_sidecar_bytes(EURUSD.as_bytes()).unwrap();
assert_eq!(
g,
InstrumentGeometry {
digits: 5,
pip_size: 0.0001,
tick_size: 1e-05,
lot_size: 100_000.0,
base: "EUR".to_string(),
quote: "USD".to_string(),
}
);
}
#[test]
fn integer_valued_geometry_parses_as_float() {
// GER40-shaped: pipSize 1, digits 1, lotSize 1.
let json = r#"{"schemaVersion":2,"digits":1,"pipSize":1,"tickSize":0.1,"lotSize":1,"baseAsset":"DE40","quoteAsset":"EUR"}"#;
let g = InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).unwrap();
assert_eq!(g.pip_size, 1.0);
assert_eq!(g.digits, 1);
assert_eq!(g.lot_size, 1.0);
}
#[test]
fn provider_base_codes_pass_through_verbatim() {
let nas = r#"{"schemaVersion":2,"digits":1,"pipSize":1,"tickSize":0.1,"lotSize":1,"baseAsset":"USTEC","quoteAsset":"USD"}"#;
assert_eq!(
InstrumentGeometry::from_sidecar_bytes(nas.as_bytes()).unwrap().base,
"USTEC"
);
let aapl = r#"{"schemaVersion":2,"digits":2,"pipSize":0.01,"tickSize":0.01,"lotSize":1,"baseAsset":"AAPL.US","quoteAsset":"USD"}"#;
assert_eq!(
InstrumentGeometry::from_sidecar_bytes(aapl.as_bytes()).unwrap().base,
"AAPL.US"
);
}
#[test]
fn unsupported_schema_version_yields_none() {
let json = r#"{"schemaVersion":99,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"lotSize":100000,"baseAsset":"EUR","quoteAsset":"USD"}"#;
assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none());
}
#[test]
fn null_core_field_yields_none() {
let json = r#"{"schemaVersion":2,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"lotSize":null,"baseAsset":"EUR","quoteAsset":"USD"}"#;
assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none());
}
#[test]
fn missing_core_field_yields_none() {
let json = r#"{"schemaVersion":2,"digits":5,"pipSize":0.0001,"tickSize":1E-05,"baseAsset":"EUR","quoteAsset":"USD"}"#;
assert!(InstrumentGeometry::from_sidecar_bytes(json.as_bytes()).is_none());
}
#[test]
fn malformed_json_yields_none() {
assert!(InstrumentGeometry::from_sidecar_bytes(b"not json").is_none());
}
}
```
Then add the module declaration in `src/lib.rs`, changing the block at lines 21-23
from:
```rust
pub mod cache;
pub mod loader;
pub mod records;
```
to:
```rust
pub mod cache;
pub mod loader;
pub mod meta;
pub mod records;
```
- [ ] **Step 3: Run the meta unit tests — verify RED**
Run: `cargo test -p data-server --lib meta::`
Expected: the suite COMPILES and the 7 `meta::tests::*` tests run; the three
that expect `Some` FAIL (`parses_canonical_sidecar_with_scientific_notation_and_integer_lot`,
`integer_valued_geometry_parses_as_float`, `provider_base_codes_pass_through_verbatim`)
with `called Option::unwrap() on a None value` (the stub returns `None`). The
four `*_yields_none` / `malformed_*` tests pass against the stub.
- [ ] **Step 4: Implement the real parser**
In `src/meta.rs`, replace the stub body of `from_sidecar_bytes`:
```rust
// STUB — replaced in Step 4.
let _ = bytes;
None
```
with:
```rust
let v: Value = serde_json::from_slice(bytes).ok()?;
if v.get("schemaVersion").and_then(Value::as_u64)? != SUPPORTED_SCHEMA_VERSION {
return None;
}
Some(Self {
digits: v.get("digits").and_then(Value::as_u64)? as u32,
pip_size: finite(v.get("pipSize"))?,
tick_size: finite(v.get("tickSize"))?,
lot_size: finite(v.get("lotSize"))?,
base: v.get("baseAsset").and_then(Value::as_str)?.to_string(),
quote: v.get("quoteAsset").and_then(Value::as_str)?.to_string(),
})
```
- [ ] **Step 5: Run the meta unit tests — verify GREEN**
Run: `cargo test -p data-server --lib meta::`
Expected: PASS — all 7 `meta::tests::*` tests green.
---
### Task 2: `DataServer::symbol_meta` method + integration tests
**Files:**
- Modify: `src/lib.rs:196-201` (add method to `impl DataServer`)
- Modify: `src/lib.rs` (`#[cfg(test)] mod tests`, near line 792)
- [ ] **Step 1: Write the integration tests (RED)**
In `src/lib.rs`, inside `#[cfg(test)] mod tests` (after the existing
`write_m1_file` helper, before the first `#[test]`), add a helper:
```rust
/// Writes a `<SYMBOL>.meta.json` sidecar into `dir` so the reader can
/// load it. The body is written verbatim.
fn write_meta_file(dir: &Path, symbol: &str, body: &str) {
std::fs::write(dir.join(format!("{symbol}.meta.json")), body).unwrap();
}
```
Then add two tests at the end of the same `mod tests` (before its closing `}`):
```rust
#[test]
fn test_symbol_meta_reads_sidecar() {
let dir = tempfile::tempdir().unwrap();
write_meta_file(
dir.path(),
"EURUSD",
r#"{
"schemaVersion": 2, "digits": 5, "pipSize": 0.0001,
"tickSize": 1E-05, "lotSize": 100000,
"baseAsset": "EUR", "quoteAsset": "USD"
}"#,
);
let server = DataServer::new(dir.path());
let g = server.symbol_meta("EURUSD").unwrap();
assert_eq!(g.digits, 5);
assert_eq!(g.pip_size, 0.0001);
assert_eq!(g.tick_size, 1e-05);
assert_eq!(g.lot_size, 100_000.0);
assert_eq!(g.base, "EUR");
assert_eq!(g.quote, "USD");
// Issue's worked derivation: pip_value_per_lot = lot_size * pip_size.
assert!((g.lot_size * g.pip_size - 10.0).abs() < 1e-9);
}
#[test]
fn test_symbol_meta_missing_sidecar_is_none() {
let dir = tempfile::tempdir().unwrap();
// A symbol with a bar file but no sidecar -> None.
write_m1_file(dir.path(), "GBPUSD", 2020, 6);
let server = DataServer::new(dir.path());
assert!(server.symbol_meta("GBPUSD").is_none());
// An entirely unknown symbol -> None.
assert!(server.symbol_meta("NOPE").is_none());
}
```
- [ ] **Step 2: Run the integration tests — verify RED**
Run: `cargo test -p data-server --lib test_symbol_meta`
Expected: COMPILE ERROR — `no method named symbol_meta found for ... DataServer`.
(The method does not exist yet; this is the RED state. The two
`test_symbol_meta_*` tests are the ones gated.)
- [ ] **Step 3: Add the `symbol_meta` method**
In `src/lib.rs`, inside `impl DataServer`, immediately after the `file_count`
method (which ends at line 200, before `stream_m1` at line 202), insert:
```rust
/// Loads the neutral price geometry for a symbol from its
/// `<SYMBOL>.meta.json` sidecar, or `None` if no usable sidecar exists.
///
/// A missing sidecar is normal (a symbol may have bars but no sidecar) and
/// returns `None` silently. A present-but-unusable sidecar (malformed,
/// unsupported schema version, or an incomplete / `null` / non-finite core
/// field) returns `None` after a one-line stderr warning. The returned
/// geometry is always complete; the consumer never sees a partial struct.
pub fn symbol_meta(&self, symbol: &str) -> Option<meta::InstrumentGeometry> {
let path = self.index.base_path.join(format!("{symbol}.meta.json"));
let bytes = std::fs::read(&path).ok()?; // missing / unreadable -> silent None
let geo = meta::InstrumentGeometry::from_sidecar_bytes(&bytes);
if geo.is_none() {
eprintln!(
"Warning: {} present but unusable (bad/incomplete geometry)",
path.display()
);
}
geo
}
```
- [ ] **Step 4: Run the integration tests — verify GREEN**
Run: `cargo test -p data-server --lib test_symbol_meta`
Expected: PASS — `test_symbol_meta_reads_sidecar` and
`test_symbol_meta_missing_sidecar_is_none` both green.
- [ ] **Step 5: Run the full suite — verify no regression**
Run: `cargo test -p data-server`
Expected: PASS — all tests green (existing suite + 7 meta unit tests + 2
`symbol_meta` integration tests). No test is filtered out; the result line
reports the full count.
docs/specs/0001-symbol-meta-geometry-reader.md
-284
View File
@@ -1,284 +0,0 @@
# Per-symbol geometry sidecar reader — Design Spec
**Date:** 2026-06-25
**Status:** Draft — awaiting user spec review
**Authors:** orchestrator + Claude
Tracker: Brummel/data-server#3 (reference issue; fork decisions logged there).
## Goal
Add a reader that loads a per-symbol `<SYMBOL>.meta.json` sidecar from the data
directory this server already owns and returns **neutral** price-interpretation
geometry — `digits`, `pip_size`, `tick_size`, `lot_size`, `base`, `quote` — so a
downstream consumer can ask one server for a symbol's bars *and* its geometry
without parsing provider JSON itself.
The sidecar carries the price-interpretation facts the bars lack (how many
decimals a price has, what a pip is, what one lot is worth, which currency the
price is quoted in). Keeping the sidecar's filename, schema version, and format
quirks (scientific-notation numbers, `null` for a non-finite value, an unknown
schema version) inside this broker-facing library is what lets the consumer stay
broker-agnostic: it receives a plain Rust struct or nothing, never raw JSON.
Non-goals: time-varying / account-dependent conditions (commission, swap,
deposit-currency pip value, volume limits, trading status) — a separate,
time-indexed concern, not this reader's job. No currency-code normalization (see
§Error handling, base/quote). No caching (see §Components).
## Architecture
A new leaf module `src/meta.rs` owns the neutral type and the pure parser; a new
one-shot method on `DataServer` owns the filesystem read.
```text
DataServer
┌───────────────────────────────────────┐
│ index: SymbolIndex { base_path, … } │
│ cache: Arc<FileCache> (bars only) │
│ │
│ symbol_meta(&self, symbol) ────────────┼──► fs::read(base_path/<symbol>.meta.json)
└───────────────────────────────────────┘ │
▼
meta::InstrumentGeometry::from_sidecar_bytes(&[u8]) -> Option<…>
```
The reader is independent of the bar `FileCache` and the streaming path: it does
not touch the cache, spawns no thread, and needs no `Arc<Self>`. It reaches the
data directory through the existing `SymbolIndex::base_path` — the same field
the bar file paths are built from (ratified by `test_file_path_construction` in
`src/lib.rs`, which pins `base_path` as the data directory). Membership in the
bar index is **not** required: a symbol with a sidecar but no bar files, or bar
files but no sidecar, is handled by the file existing or not.
## Concrete code shapes
### Worked consumer program (the acceptance evidence)
What an Aura-side consumer (the counterpart, Brummel/Aura#143) writes — bars and
geometry from one server, no provider JSON in sight:
```rust
use std::sync::Arc;
use data_server::DataServer;
let server = Arc::new(DataServer::new("/mnt/tickdata/Pepperstone"));
if let Some(geo) = server.symbol_meta("EURUSD") {
// geo: InstrumentGeometry — neutral, plain fields
assert_eq!(geo.digits, 5);
assert_eq!(geo.pip_size, 0.0001);
assert_eq!(geo.tick_size, 1e-05); // sidecar wrote it as 1E-05
assert_eq!(geo.lot_size, 100_000.0);
assert_eq!(geo.quote, "USD");
// The consumer derives the rest; nothing provider-specific reached it.
let pip_value_per_lot = geo.lot_size * geo.pip_size; // 100000 * 0.0001 = 10 USD
}
// Pairs with the existing symbol surface.
for sym in server.symbols() {
if let Some(geo) = server.symbol_meta(&sym) {
// … cross-check geo.base / geo.quote against the consumer's vetted table.
}
}
```
`symbol_meta` returns `None` for: no sidecar file, malformed JSON, an
unsupported schema version, or any missing / `null` / non-finite core field
(see §Error handling). The consumer never sees a partial geometry.
### Implementation shapes (secondary)
New file `src/meta.rs` — before: does not exist. After:
```rust
//! Reader for per-symbol `<SYMBOL>.meta.json` geometry sidecars.
use serde_json::Value;
/// The only sidecar schema version this reader understands.
pub const SUPPORTED_SCHEMA_VERSION: u64 = 2;
/// Neutral, broker-agnostic price-interpretation geometry for one symbol.
///
/// Intrinsic to the historical price series and effectively constant over its
/// span. Derived/account-dependent conditions are deliberately excluded.
#[derive(Debug, Clone, PartialEq)]
pub struct InstrumentGeometry {
/// Number of decimal places in a quoted price (e.g. 5 for EURUSD).
pub digits: u32,
/// Price increment of one pip (e.g. 0.0001 for EURUSD).
pub pip_size: f64,
/// Smallest representable price increment (e.g. 1e-05 for EURUSD).
pub tick_size: f64,
/// Contract size of one lot in base units (e.g. 100000 for EURUSD).
pub lot_size: f64,
/// Base asset code, passed through verbatim from the sidecar.
pub base: String,
/// Quote (price) currency code, passed through verbatim.
pub quote: String,
}
impl InstrumentGeometry {
/// Parse sidecar JSON bytes into neutral geometry.
///
/// Returns `None` if the JSON is malformed, the schema version is
/// unsupported, or any core field is missing / `null` / non-finite.
pub(crate) fn from_sidecar_bytes(bytes: &[u8]) -> Option<Self> {
let v: Value = serde_json::from_slice(bytes).ok()?;
if v.get("schemaVersion").and_then(Value::as_u64)? != SUPPORTED_SCHEMA_VERSION {
return None;
}
Some(Self {
digits: v.get("digits").and_then(Value::as_u64)? as u32,
pip_size: finite(v.get("pipSize"))?,
tick_size: finite(v.get("tickSize"))?,
lot_size: finite(v.get("lotSize"))?,
base: v.get("baseAsset").and_then(Value::as_str)?.to_string(),
quote: v.get("quoteAsset").and_then(Value::as_str)?.to_string(),
})
}
}
/// A finite `f64` from a JSON number, or `None` for missing / null / non-finite.
/// `Value::as_f64` already rejects `null` and non-numbers; this also rejects
/// `NaN`/`Inf` defensively.
fn finite(v: Option<&Value>) -> Option<f64> {
let n = v?.as_f64()?;
n.is_finite().then_some(n)
}
```
New method on `DataServer` in `src/lib.rs` — before: absent. After:
```rust
/// Loads the neutral price geometry for a symbol from its
/// `<SYMBOL>.meta.json` sidecar, or `None` if no usable sidecar exists.
///
/// A missing sidecar is normal (`None`, silent). A present-but-unusable
/// sidecar (malformed, unsupported version, incomplete) returns `None`
/// after a one-line stderr warning.
pub fn symbol_meta(&self, symbol: &str) -> Option<meta::InstrumentGeometry> {
let path = self.index.base_path.join(format!("{symbol}.meta.json"));
let bytes = std::fs::read(&path).ok()?; // missing file -> silent None
let geo = meta::InstrumentGeometry::from_sidecar_bytes(&bytes);
if geo.is_none() {
eprintln!("Warning: {} present but unusable (bad/incomplete geometry)", path.display());
}
geo
}
```
Module wiring in `src/lib.rs` — add `pub mod meta;` alongside the existing
`pub mod cache; pub mod loader; pub mod records;`. `base_path` is already a field
of the private `SymbolIndex`; `symbol_meta` is a `DataServer` method in the same
crate, so it reads `self.index.base_path` directly (no visibility change).
`Cargo.toml` — add one direct dependency:
```toml
serde_json = "1"
```
## Components
- **`meta::InstrumentGeometry`** — the public neutral struct (six fields,
`Debug + Clone + PartialEq`). Excludes `symbol`, `description`, `generator`,
`generatedAtUtc` (the most provider-specific fields; `symbol` is already known
to the caller).
- **`meta::InstrumentGeometry::from_sidecar_bytes`** — `pub(crate)` pure parser,
bytes → `Option<Self>`. No filesystem, fully unit-testable. Absorbs every
format quirk here.
- **`meta::SUPPORTED_SCHEMA_VERSION`** — the `u64` gate (currently `2`).
- **`DataServer::symbol_meta`** — `&self` one-shot read; the only filesystem
touch. No caching: geometry files are tiny and read rarely (typically once per
symbol); a cache map + invalidation is complexity for no stated need, and the
bar `FileCache` exists for a different problem (large, repeatedly-streamed
files).
## Data flow
1. Consumer calls `server.symbol_meta("EURUSD")`.
2. Path built: `<base_path>/EURUSD.meta.json` (symbols may contain dots, e.g.
`AAPL.US.meta.json`; the literal join handles them).
3. `fs::read` → on error (missing/unreadable) return `None` silently.
4. Bytes → `from_sidecar_bytes`: parse JSON; gate `schemaVersion == 2`; extract
the six fields with the null/non-finite guard.
5. `Some(geometry)` on success; `None` + one stderr warning on a present but
unusable file.
## Error handling
The reader absorbs every documented provider quirk so none reaches a consumer:
- **Scientific notation** (`1E-05`, in 7 of 30 live files) — valid JSON, parses
as a number; `Value::as_f64` handles it. No special code.
- **`null` for a non-finite broker value** — `Value::as_f64`/`as_u64` return
`None` for `null`; that propagates to `symbol_meta` returning `None`. The
consumer never sees a hole. (No `null` exists in the current 30 samples; this
is defensive.) `finite()` additionally rejects `NaN`/`Inf`.
- **Unknown `schemaVersion`** — a reader-side decision, not a consumer's: an
unsupported version returns `None` (+ the call-site warning), never an error
type the consumer must handle.
- **base / quote provider codes** — passed through **verbatim** as `String`.
`baseAsset` is wildly heterogeneous across the 30 samples (ISO `EUR`/`USD`,
index codes `DE40`/`USTEC`/`JP225`, descriptive `Bitcoin`/`Copper`, stock
tickers `AAPL.US`); there is no well-defined "neutral" target and no mapping
table, so normalization is not even definable here. The consumer (Aura#143)
cross-checks the value against its own hand-authored vetted table and never
reads the raw JSON — resolution is the consumer's job by design. "Neutral"
means the struct's shape and semantics are broker-agnostic (a plain Rust type,
parsed numbers, no `serde_json::Value`, no raw JSON, no provider filename
leaking), not that every string is canonicalized.
- **Incomplete object** (a missing core field) — same channel as `null`:
`None`.
A *missing* sidecar (a symbol with bars but no `.meta.json`) is an expected,
non-error condition → silent `None`. Only a *present* file that cannot yield a
complete geometry warns.
## Testing strategy
Unit tests in `src/meta.rs` against the pure `from_sidecar_bytes` (no
filesystem) — each pins one absorbed quirk:
- Canonical `EURUSD` sidecar → `Some` with exact fields
(`digits 5, pip_size 0.0001, tick_size 1e-05, lot_size 100000.0, base "EUR",
quote "USD"`). Pins scientific-notation `tickSize` and integer `lotSize`.
- Integer-valued geometry (`GER40`: `pipSize 1`, `digits 1`) → `pip_size 1.0`.
- Provider base code passes through unchanged (`NAS100` → `base == "USTEC"`;
dotted `AAPL.US` → `base == "AAPL.US"`).
- Unsupported `schemaVersion` (e.g. `99`) → `None`.
- `null` core field (e.g. `"lotSize": null`) → `None`.
- Missing core field → `None`.
- Malformed JSON → `None`.
Integration test in `tests/data_server.rs` (or `src/lib.rs` tests) using a
`tempfile::tempdir`:
- Write `EURUSD.meta.json` into the dir → `server.symbol_meta("EURUSD")` is
`Some` with the expected geometry.
- A symbol with a bar file but no sidecar → `symbol_meta` is `None`.
- An unknown symbol → `symbol_meta` is `None`.
All tests are deterministic and self-contained (no `/mnt` dependency), matching
the existing test suite's tempdir pattern.
## Acceptance criteria
- A consumer obtains a symbol's bars (existing `stream_*`) and its neutral
geometry (`symbol_meta`) from one `DataServer`, pairing with
`has_symbol`/`symbols`.
- The public API exposes only the plain `InstrumentGeometry` struct and `Option`
— no `serde_json` type, no raw JSON, no provider filename, no error type.
- Every documented quirk (scientific notation, `null`-for-non-finite, unknown
schema version, heterogeneous base codes) is absorbed: the consumer sees a
complete neutral geometry or `None`, never a partial struct or a provider
artefact.
- `pip_value_per_lot = lot_size * pip_size` reproduces the issue's verified
values (EURUSD 10 USD; USDJPY 1000 JPY; XAUUSD 10 USD) from the returned
fields.
- The new `serde_json` dependency is the only addition to the dependency set;
the README's dependency statement is updated to reflect it.
- The full suite stays green; new tests pin each absorbed quirk.