test_real_concurrent_access depended on /mnt data (skipped otherwise)
and asserted equal counts over a partial 10-chunk read, which thread
scheduling could make uneven. The SymbolGuard ownership model closes
the underlying races, so concurrent consumers reading the same symbol
to completion read an identical, fixed record set regardless of
scheduling.
Replace it with a self-contained inline test: eight threads each fully
drain a stream over temp files and must read identical counts. Runs
everywhere with no /mnt dependency; verified deterministic across 30
consecutive runs.
stream_*_windowed loaded the first file and spawned a prefetch before
retaining the symbol, leaving loaded files at refcount 0 in a window
where a concurrent release could evict them (redundant reload), and a
prefetch thread could insert after an evict, leaving a refcount-0
orphan (leak).
Introduce a SymbolGuard RAII type: creating it retains the symbol,
dropping it releases. ensure_*_loaded now takes &SymbolGuard, making
"retain before load" a compile-time precondition that closes the
reload race structurally. prefetch_* clones a guard synchronously
before spawning and holds it across the insert, so the refcount
cannot reach zero between load and insert, closing the orphan leak.
Add an always-on load counter (DataServer::files_loaded) and a
deterministic regression test asserting each file is read from disk
exactly once across prefetch and a second consumer, locking the
no-redundant-I/O performance contract the fix must not regress.
closes#1
Real-tickdata integration test (skip-guarded on /mnt/tickdata) carved
out of the myc repo so the crate is self-testing on reuse. Paths
adjusted myc::ast::data_server:: -> data_server:: ; otherwise verbatim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Concise API reference for the public surface (entry points, DataServer
query/stream methods, SymbolChunkIter, record types, lower-level modules)
plus a usage example, concurrency notes and the data-file naming convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
High-performance, thread-safe loader and cache for binary market data
files (M1/tick). No dependency on the rest of myc; only chrono, regex, zip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brummel