d2d0d0f4fd6e1780d37dbffdd4c8a2f0c8f957de
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brummel
|
5b5f034be9 |
feat(aura-ingest): GER40 session-breakout on real M1 bars (examples/)
Run the Phase-1 session-breakout node vocabulary over REAL GER40 M1 bars
from the data-server archive — the first real-data exercise of the shipped
strategy nodes (Resample/Delay/Gt/Session/EqConst/And/Latch + SimBroker).
It is a pure composition of shipped aura-std nodes (no project-specific
signal code), so it lives in the engine repo's examples/ carveout (C9), not
a separate consumer crate. The breakout FlatGraph is reused VERBATIM from
aura-engine/tests/ger40_breakout.rs; only the sources change — four real
M1FieldSource (open/high/low/close, in the fixed C4 merge order) replace the
synthetic VecSources, with pip_size = 1.0 (GER40 is an index).
- examples/ger40_breakout_real.rs — runnable demo over a Sept-2024 window:
prints the RunReport metrics plus a per-session entry/exit trace.
- examples/shared/breakout_real.rs — the single 13-node wiring (the proven
11 + two trace taps), shared by the example and the test via #[path] so it
is never duplicated.
- tests/ger40_breakout_real.rs — gated determinism test (mirrors
real_bars.rs): skips where the archive is absent; where present, asserts
finite metrics, binary Latch exposure (held in {0.0, 1.0}), and
bit-identical C1 reruns of both the report and the recorded series.
chrono / chrono-tz added as aura-ingest dev-dependencies (the Session
timezone and the UTC window bounds), test-only — never on the normal path.
|
||
|
Brummel
|
8158e97dc6 |
feat(aura-ingest): streaming M1FieldSource — lazy data-server window through the seam
Completes the Source ingestion seam (plan Tasks 3-4): a real, lazily-streamed
data-server source proving the producer seam against the actual data source,
and closing the cycle-0011 deliberate eager-materialization gap.
- M1Field + a free decode(field, bar) — a pure projection of one M1 bar into
(epoch-ns ts, Scalar), reusing unix_ms_to_epoch_ns so ms->ns stays the one
C3 normalization point. Pure ⇒ hermetically unit-tested without an iterator.
- M1FieldSource: a streaming Source over a data-server M1 window. Holds one
Arc<[M1Parsed]> chunk (a zero-copy clone of the cache's chunk) + a cursor +
a pre-decoded head (so peek(&self) is cheap over a &mut next_chunk refill);
constructs each Scalar per-pull, refills when the chunk drains. Resident
footprint is O(one chunk), independent of window length.
- aura-engine moved [dev-dependencies] -> [dependencies] (M1FieldSource is a
library item implementing aura_engine::Source, not a test-only type).
- A gated streaming integration test (skips where local data is absent):
* end-to-end backtest through the seam to a RunReport, C1-deterministic
(two runs bit-identical);
* residency predicate — driving the source the way run() does (peek/next)
over the *unbounded* archive (~1.77M bars locally), peak resident_records
stays <= one data-server chunk (CHUNK_SIZE) while total >> CHUNK_SIZE:
O(one chunk), NOT O(window). A O(window) source would peak at `total`;
* two-field sharing (close + volume over one window, same ts axis).
The eager load_m1_window / close_stream path is kept (still valid for bounded
loads); the gap is closed by a streaming path now existing, not by deleting the
eager one. The residency test drives the full archive rather than the plan's
narrow 2006-08 window, which holds only ~21 bars locally — too few for the
> CHUNK_SIZE non-vacuity precondition; the unbounded archive is the stronger
"independent of window length" proof.
Verified: cargo build/test/clippy --workspace clean; the 3 gated tests ran
against real local data and passed; aura-ingest/aura-engine docs clean (two
pre-existing aura-registry doc-link warnings are from an earlier commit, not
this cycle).
closes #71
|
||
|
Brummel
|
eec876975c |
feat(engine): serde foundation + amend C16 dependency policy (cycle D / iter 1)
Iteration 1 of the run-registry cycle (#33): lay the dependency + serde foundation the registry's typed read-path needs, and amend the contract that forbade it. Contract change (load-bearing — C18/C16). C16's blanket "zero-external- dependency by commitment" + "aura-ingest is the sole external-dependency firewall" framing is struck and replaced by a considered, per-case dependency policy: dependencies are admitted by deliberate review (what a crate pulls in vs. what it buys), with particular scrutiny for anything entering the frozen deploy artifact (C13); standard vetted crates (serde, rayon, ...) pass that review and are used wherever they do the job, including in the bot; hand-rolling what a vetted crate does is the anti-pattern. C16's engine/project split + three-tier node reuse are unchanged. The five source comments that asserted the struck framing (aura-engine/aura-ingest Cargo.toml + aura-ingest/report.rs docs) are rewritten to match; no live source or ledger text still asserts it. Per-case justification for serde (the policy now requires one): it gives the run-report types a typed (de)serialization path — exactly what ranking/compare over stored runs needs, and what the writer-only hand-rolled JSON (C14) could never provide; closes the typed-read-path gap (#17). Its closure is tiny, ubiquitous, heavily audited, and its output deterministic (C1-safe). serde_json is test-only this iteration (dev-dependency); no production serde_json yet. Changes: [workspace.dependencies] serde (derive) + serde_json; serde derives on Timestamp (aura-core) and RunMetrics/RunManifest/RunReport (aura-engine), with serde_json round-trip tests (window renders as a [from,to] integer array via the transparent Timestamp newtype). The hand-rolled to_json writers are untouched (still drive stdout). No aura-registry crate, no SweepPoint change — those are iterations 2 and 3. Verified: cargo test --workspace green (incl. the two new round-trip tests); clippy --workspace --all-targets -D warnings clean; no surviving assertion of the struck zero-dep framing in live source or the ledger. refs #33 |
||
|
Brummel
|
a24729e97f |
feat(aura-ingest): data-server M1 ingestion at the one merge boundary
aura's first real data source (closes #7, Walking-skeleton milestone). A new aura-ingest workspace crate transposes data-server's AoS M1Parsed records into SoA base columns (C7), normalizes Unix-ms to canonical epoch-ns at the one ingestion boundary (C3), and feeds the existing k-way merge a real close-price stream so a backtest runs over real bars, deterministically (C1). Surface: - unix_ms_to_epoch_ns(time_ms) -> Timestamp (= ms * 1_000_000), the single C3 unit normalization. - M1Columns: the OHLCV bar as a bundle of base columns (open/high/low/close/ spread: f64, volume: i64, ts: epoch-ns) — SoA, C7. - transpose_m1(&[M1Parsed]) -> M1Columns: pure AoS->SoA (C1). - M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>: the price input the SMA-cross sample strategy consumes. - load_m1_window(server, symbol, from_ms, to_ms): drains data-server's chronological chunks, transposes once at the boundary. Design (per spec 0011): - Eager materialization, not a lazy/shared Source abstraction: C12's cross-sim Arc<[T]> sharing has no consumer until the multi-sim orchestration cycle; deferred, the transpose logic carries over. - aura-ingest is the external-dependency firewall. data-server transitively pulls chrono + regex + zip (+ their trees) into Cargo.lock; isolating it in this one crate keeps aura-core/std/engine zero-external-dependency and the frozen deploy artifact (C6 replays recorded streams, never re-ingests) clean. cargo test --workspace now needs a populated cargo cache (one Gitea fetch, done). - Scope: boundary + tests only, no aura run CLI arg surface (M1 first; tick and the CLI wiring are follow-ups). Tests: 6 hermetic unit tests on hand-built M1Parsed (normalization, field-wise transpose, purity, close_stream order, empty edges) + one gated integration test (tests/real_bars.rs) that runs the cycle-0007 sample harness over real AAPL.US 2006-08 close bars and asserts finite metrics + two-run bit-identical JSON (C1); it skips cleanly where /mnt/tickdata is absent. On this machine it ran the real path. Workspace gates green: test (86), clippy -D warnings, doc -D warnings. Minor: transient unused-import warnings in the new lib.rs across the additive build steps resolved once load_m1_window consumed the imports; final -D warnings gate clean. |