feat: Add data server for market data loading
This commit introduces a new data server module (`src/ast/data_server`) designed for high-performance, thread-safe loading and caching of market data. Key components: - `DataServer`: Manages symbol indexing and delegates loading/caching. - `SymbolIndex`: Scans the data directory and maintains a sorted index of data files per symbol. - `FileCache`: Implements a thread-safe cache using `RwLock` and a loading guard to prevent duplicate work. - `loader`: Handles ZIP decompression and binary record parsing from `.bin` files. - `records`: Defines raw and parsed data structures for M1 and tick data. - `SymbolChunkIter`: Provides an iterator over pre-parsed data chunks, prefetching subsequent files. This architecture allows multiple VM threads to access market data concurrently without redundant I/O, mirroring the strategy used in the original Delphi `TDataServer`.
This commit is contained in:
@@ -13,6 +13,7 @@ fastrand = "2.3"
|
||||
rmcp = { version = "1.2", features = ["server", "transport-io"] }
|
||||
tokio = { version = "1", features = ["rt", "io-std", "macros"] }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
zip = "2"
|
||||
|
||||
[dev-dependencies]
|
||||
insta = { version = "1.39", features = ["yaml"] }
|
||||
|
||||
Reference in New Issue
Block a user