The data_server module had no dependency on the rest of myc (only
chrono/regex/zip), so it was moved verbatim into a standalone leaf
crate at its own Gitea repo and is now consumed as a git dependency.
A 'pub use ::data_server;' re-export in src/ast/mod.rs keeps all
existing myc::ast::data_server::… paths valid; no call sites changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit introduces time windowing functionality to the M1 and tick
data streams.
Key changes:
- **`DataServer::stream_m1_windowed` and
`DataServer::stream_tick_windowed`**: New methods added to
`DataServer` to allow streaming data within a specified `from_ms` and
`to_ms` range.
- **`SymbolChunkIter` modifications**: The `SymbolChunkIter` struct now
includes `from_ms`, `to_ms`, and `exhausted` fields to manage time
window filtering.
- **`filter_chunk` helper function**: A new utility function to
efficiently filter individual data chunks based on the time window.
This function also handles early termination when `to_ms` is exceeded.
- **`HasTimestamp` trait**: Introduced to provide a common interface for
accessing timestamps across different record types (`M1Parsed`,
`TickParsed`), enabling generic filtering.
- **`filter_files` method**: Added to `DataServer` to pre-filter file
keys based on the time window before creating iterators. This
optimizes file loading by skipping files that fall entirely outside
the desired range.
- **`create-m1-stream` and `create-tick-stream`**: The native functions
in `data_streams.rs` are updated to accept optional `DateTime`
arguments for `from` and `to` timestamps. They now also handle unknown
symbols by returning `Value::Void` and return an empty stream if the
time window results in no data.
- **Documentation**: Updated doc comments for the stream creation
functions to reflect the new time windowing capabilities and to
clarify behavior for unknown symbols and empty time windows.
These changes significantly enhance the flexibility of data streaming,
allowing users to query specific time ranges of M1 and tick data more
efficiently.
The `FileCache` now uses `std::sync::Condvar` to signal when files have
finished loading. This replaces the previous spin-wait mechanism,
significantly improving efficiency by allowing waiting threads to sleep
until explicitly woken up.
The `wait_for_load` method now blocks on the `Condvar`, and the
`insert_m1` and `insert_tick` methods call `notify_all` after updating
the cache and removing the loading guard. This ensures that all waiting
threads are woken up and can re-check the cache.
Additionally, `ensure_m1_loaded` and `ensure_tick_loaded` in
`DataServer` have been simplified to directly call `wait_for_load` when
another thread is already loading the file, removing the redundant
synchronous loading logic.
Refactor FileCache to use Condvar
Brummel