docs: narrow streaming residency claim from whole-process RSS to the source ring

The C12 realization and the 0041 spec claimed streaming residency is O(one chunk) at the process level ("a 20-year window streams in the same memory as a one-day one"). That holds only for the aura source ring (M1FieldSource::resident_records(), bounded by one chunk and correctly tested); whole-process RSS grows O(records-touched) because data-server's FileCache retains each window's parsed chunks (~56 B/record) read-only for the pass. Reproduced: 6 MiB @ 1mo -> 173 MiB @ ~10y, linear.

That retention is the replay-many optimization (load-once; close+volume share one parse via the field-agnostic FileKey), not a leak, and no always-on eviction can bound a single forward pass without regressing it. This narrows the docs/comments to the true per-source-ring property and names the FileCache per-window retention as the real, replay-amortized process cost. The streaming_seam assert is unchanged (it was always the ring bound; only its labelling confused ring vs process). The deferred opt-in non-retaining streaming read path is parked as Brummel/data-server#2.

closes #95
This commit is contained in:
2026-06-17 22:09:32 +02:00
parent dee25afcd5
commit 0d7c2c16f4
4 changed files with 32 additions and 13 deletions
+8 -4
View File
@@ -561,10 +561,14 @@ only path. `Harness::run` is re-typed to a **producer seam** — a `Source` trai
(`peek`/`next`, object-safe) the k-way merge drives — and a streaming
**`M1FieldSource`** (`aura-ingest`) pulls a data-server window lazily, borrowing
**one** `Arc<[M1Parsed]>` chunk per pull (zero-copy *within* a source) and
decoding each `Scalar` on demand: resident O(one chunk), not O(window length),
so a 20-year window streams in the same memory as a one-day one. The eager
`load_m1_window`/`close_stream` path is kept for bounded loads (the gap closes by
a streaming path *existing*, not by deleting the eager one). **Still open:**
decoding each `Scalar` on demand: the **source ring** is resident O(one chunk),
not O(window length) — the measured `resident_records()` predicate, a *per-source*
bound, not whole-process RSS. (Data-server's `FileCache` retains each window's
parsed chunks read-only for the pass — ~56 B/record — so process residency is
O(records-touched); that is the **replay-many** sharing C12 wants — one window
parsed once across a sweep family — not a leak. The single-pass cost is tracked as
#95.) The eager `load_m1_window`/`close_stream` path is kept for bounded loads (the
gap closes by a streaming path *existing*, not by deleting the eager one). **Still open:**
cross-*sim* `Arc<[T]>` sharing — one window shared zero-copy across many disjoint
sweep sims — has no consumer until the orchestration families (axes 24 above:
#66/#68/#69) are built, and remains the target for those cycles.