Cycle-close for the per-symbol geometry sidecar reader (#3).
Architect drift review (0f5e665..be2489a): code matches the spec verbatim;
the README "## Interface" contract was already updated in lockstep (symbol_meta
row + serde_json dependency line). Two drift items, both resolved here:
- [medium] Retire the ephemeral active-cycle artefacts: git-rm the spec and
plan (recoverable from history via `git show <rev>:<path>`), per the
docs/specs + docs/plans lifecycle convention.
- [low] Document meta::InstrumentGeometry to parity with the other public
record types: add a "data_server::meta" subsection enumerating its six
fields and SUPPORTED_SCHEMA_VERSION.
Regression gate (all green):
- cargo build -> exit 0, finished clean
- cargo clippy -> exit 0, 0 warnings
- cargo test -> exit 0; 36 lib + 4 integration + 0 doc tests passed
No baseline to update (the project has no metric baseline; cargo is the gate).
refs #3
Implements #3: DataServer::symbol_meta(symbol) -> Option<InstrumentGeometry>
loads a <SYMBOL>.meta.json sidecar from the owned data directory and returns
neutral, broker-agnostic price geometry (digits, pip_size, tick_size,
lot_size, base, quote). A consumer can now obtain a symbol's bars and its
price-interpretation geometry from one server.
Design (forks logged on #3):
- Neutral output is the struct's shape/semantics, not currency-code
normalization: base/quote pass through verbatim. The live sidecars carry
wildly heterogeneous base codes (ISO, index codes, descriptive names, stock
tickers) with no definable neutral target; the downstream consumer
cross-checks them against its own vetted table.
- Format quirks are absorbed in a pure parser (meta::from_sidecar_bytes):
scientific notation parses natively; a null / non-finite / missing core
field or an unsupported schemaVersion yields None, never a partial struct.
- A missing sidecar is a normal None (silent); a present-but-unusable one
warns to stderr, matching the existing loader's degrade-and-warn style.
- serde_json (Value extraction, no derive) is the new direct dependency,
chosen over a hand-rolled parser for robust number/escape handling; the
README dependency statement is updated to match.
- No caching and no Arc<Self>: geometry files are tiny and read rarely, and
the read needs no prefetch. Lookup is not gated on bar-index membership.
Verified: cargo build and clippy clean (0 warnings); full suite green
(7 parser unit tests + 5 symbol_meta integration tests pin each absorbed
quirk and the index-independent lookup). Cargo.lock stays untracked per the
library convention.
closes#3
Settled prose design from #3: a reader returning neutral InstrumentGeometry
(digits, pip_size, tick_size, lot_size, base, quote) from <SYMBOL>.meta.json.
Signed autonomously under /boss on the grounding-check PASS; fork decisions
(base/quote pass-through, unknown-version -> None, serde_json, no caching)
are logged on the reference issue.
refs #3
test_real_concurrent_access depended on /mnt data (skipped otherwise)
and asserted equal counts over a partial 10-chunk read, which thread
scheduling could make uneven. The SymbolGuard ownership model closes
the underlying races, so concurrent consumers reading the same symbol
to completion read an identical, fixed record set regardless of
scheduling.
Replace it with a self-contained inline test: eight threads each fully
drain a stream over temp files and must read identical counts. Runs
everywhere with no /mnt dependency; verified deterministic across 30
consecutive runs.
stream_*_windowed loaded the first file and spawned a prefetch before
retaining the symbol, leaving loaded files at refcount 0 in a window
where a concurrent release could evict them (redundant reload), and a
prefetch thread could insert after an evict, leaving a refcount-0
orphan (leak).
Introduce a SymbolGuard RAII type: creating it retains the symbol,
dropping it releases. ensure_*_loaded now takes &SymbolGuard, making
"retain before load" a compile-time precondition that closes the
reload race structurally. prefetch_* clones a guard synchronously
before spawning and holds it across the insert, so the refcount
cannot reach zero between load and insert, closing the orphan leak.
Add an always-on load counter (DataServer::files_loaded) and a
deterministic regression test asserting each file is read from disk
exactly once across prefetch and a second consumer, locking the
no-redundant-I/O performance contract the fix must not regress.
closes#1
Real-tickdata integration test (skip-guarded on /mnt/tickdata) carved
out of the myc repo so the crate is self-testing on reuse. Paths
adjusted myc::ast::data_server:: -> data_server:: ; otherwise verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Concise API reference for the public surface (entry points, DataServer
query/stream methods, SymbolChunkIter, record types, lower-level modules)
plus a usage example, concurrency notes and the data-file naming convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
High-performance, thread-safe loader and cache for binary market data
files (M1/tick). No dependency on the rest of myc; only chrono, regex, zip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brummel