From 682e459554412abc6d8e1de378387dadb76b20c7 Mon Sep 17 00:00:00 2001 From: Brummel Date: Mon, 15 Jun 2026 10:38:15 +0200 Subject: [PATCH] =?UTF-8?q?audit:=20cycle=200041=20=E2=80=94=20drift-clean?= =?UTF-8?q?=20(Source=20ingestion=20seam)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Architect drift review over 8b330e3..HEAD (the Source-seam cycle #71 plus the interim #67 optimize work and refactors). No semantic or contract drift: the seam faithfully preserves C3 (ms→epoch-ns at the one ingestion boundary), C4 (tie-by-source-index, byte-for-byte), and C7 (field-per-source SoA); the full suite is byte-identical green and the gated streaming tests pass on real data. Regression gate: `cargo test --workspace` green (the project declares no dedicated regression script; the suite is the gate). Tidied stale prose the cycle left behind (no behaviour change): - docs/design/INDEX.md (C12): replaced the cycle-0011 "deliberate eager gap" status note with a cycle-0041 realization note — the producer `Source` seam + streaming `M1FieldSource` now deliver per-source O(one-chunk) streaming; precisely scopes what remains open (cross-*sim* Arc<[T]> window sharing, still unbuilt until the orchestration families #66/#68/#69 consume it). - aura-core/src/lib.rs: dropped the now-shipped "Source trait + data-server ingestion" from the module doc's "still to come" list (aura-engine's twin line was fixed in the cycle; aura-core's was missed). - aura-ingest/src/lib.rs: module header now documents both coexisting paths (eager load_m1_window/M1Columns vs lazy streaming M1FieldSource). - aura-registry/src/lib.rs: rank_by/optimize doc comments linked to the private `metric_cmp` (a public→private intra-doc-link warning from the #67 commit); demoted to plain code spans, so `cargo doc --workspace` is now warning-clean. --- crates/aura-core/src/lib.rs | 3 +-- crates/aura-ingest/src/lib.rs | 8 ++++++++ crates/aura-registry/src/lib.rs | 4 ++-- docs/design/INDEX.md | 18 ++++++++++++------ 4 files changed, 23 insertions(+), 10 deletions(-) diff --git a/crates/aura-core/src/lib.rs b/crates/aura-core/src/lib.rs index b061f34..5edfce7 100644 --- a/crates/aura-core/src/lib.rs +++ b/crates/aura-core/src/lib.rs @@ -24,8 +24,7 @@ //! access into each input (closing the cycle-0001 read-side gap on //! [`AnyColumn`]). //! -//! Still to come (subsequent cycles): the `Source` trait + data-server ingestion -//! and source-native time normalization (C3/C11), the broker-independent +//! Still to come (subsequent cycles): the broker-independent //! position-event output and downstream broker nodes (C10), and the run registry //! (C18/C22). diff --git a/crates/aura-ingest/src/lib.rs b/crates/aura-ingest/src/lib.rs index 9135e5b..a411226 100644 --- a/crates/aura-ingest/src/lib.rs +++ b/crates/aura-ingest/src/lib.rs @@ -7,6 +7,14 @@ //! as the engine source-stream shape (`Vec<(Timestamp, Scalar)>`) the SMA-cross //! sample strategy consumes. //! +//! Two ingestion shapes coexist: the **eager** [`load_m1_window`] → [`M1Columns`] +//! transpose (a bounded window materialized to owned SoA columns), and the **lazy +//! streaming** [`M1FieldSource`] — a `Source` that pulls one data-server +//! `Arc<[M1Parsed]>` chunk at a time and decodes each `Scalar` on demand +//! (resident O(one chunk), not O(window length)) for backtests over long +//! horizons. Both normalize ms→epoch-ns through the one [`unix_ms_to_epoch_ns`] +//! seam (C3). +//! //! This crate is the **data-source ingestion edge**: it links `data-server` //! (and its transitive `chrono`/`regex`/`zip`) and normalizes it into aura's //! columns here. Dependencies follow the amended C16 per-case policy (INDEX.md), diff --git a/crates/aura-registry/src/lib.rs b/crates/aura-registry/src/lib.rs index bbae347..fd9308d 100644 --- a/crates/aura-registry/src/lib.rs +++ b/crates/aura-registry/src/lib.rs @@ -101,7 +101,7 @@ fn metric_cmp( } /// Sort `reports` **best-first** by a named metric. "Best" is per-metric, fixed -/// by each metric's meaning (see [`metric_cmp`]). A stable sort keeps file +/// by each metric's meaning (see `metric_cmp`). A stable sort keeps file /// (insertion) order among ties. An unknown metric name is a /// `RegistryError::UnknownMetric`. pub fn rank_by(mut reports: Vec, metric: &str) -> Result, RegistryError> { @@ -112,7 +112,7 @@ pub fn rank_by(mut reports: Vec, metric: &str) -> Result` chunks zero-copy across sims — the cross-sim -`Arc<[T]>` sharing has no consumer until the orchestration axes above are built. -That sharing remains the target for the orchestration cycle that introduces -them; the transpose logic is unchanged by it. A known, deliberate gap, not drift. +**Realization (cycle 0041).** The eager ingestion of cycle 0011 is no longer the +only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trait +(`peek`/`next`, object-safe) the k-way merge drives — and a streaming +**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing +**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and +decoding each `Scalar` on demand: resident O(one chunk), not O(window length), +so a 20-year window streams in the same memory as a one-day one. The eager +`load_m1_window`/`close_stream` path is kept for bounded loads (the gap closes by +a streaming path *existing*, not by deleting the eager one). **Still open:** +cross-*sim* `Arc<[T]>` sharing — one window shared zero-copy across many disjoint +sweep sims — has no consumer until the orchestration families (axes 2–4 above: +#66/#68/#69) are built, and remains the target for those cycles. **Forbids.** Baking a specific search strategy (Bayesian/genetic) into the primitive — those are pluggable policies atop the atomic unit; recompiling on a param change.