From 96603cf9539ef3b5eb92f2011abbcba42a2ab299 Mon Sep 17 00:00:00 2001 From: Brummel Date: Wed, 3 Jun 2026 11:06:27 +0200 Subject: [PATCH] docs: record data-server and RustAst as external components The ledger named both sibling projects only conceptually; their concrete location lived only in session memory. Add an "External components" section so the repo is the single source of truth. - data-server: path + Gitea URL, its role as the first data source, and the concrete API shape (Arc<[T]> chunks, next_chunk, stream_*_windowed). Note its records are AoS (M1Parsed/TickParsed), so the ingestion boundary (C3) transposes them into SoA columns (C7) and normalizes time_ms -> epoch-ns. - RustAst: not just a conceptual reference but a working reference implementation of the streaming substrate. Map its src/ast/rtl/ layer (RingBuffer, ScalarValue, ScalarSeries, SoA RecordSeries, the register/ delay node, seeded generator) to the contracts it prefigures (C4/C5/C7/ C8/C9/C12/C19), and state how aura sharpens it. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/design/INDEX.md | 49 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/docs/design/INDEX.md b/docs/design/INDEX.md index c6ea9a0..214ffdb 100644 --- a/docs/design/INDEX.md +++ b/docs/design/INDEX.md @@ -43,6 +43,55 @@ sweeps, Monte-Carlo, comparison). The deterministic single-harness engine is the --- +## External components + +Two sibling projects live outside this repo and are named throughout the +contracts. Their concrete location is recorded here so the repo is the single +source of truth — not session memory. + +- **`data-server`** — `~/dev/libs/data-server`, Gitea `Brummel/data-server` + (`http://192.168.178.103:3000/Brummel/data-server.git`). aura's **first data + source** and the one reused component (Foundation; C3, C11, C12). A standalone + leaf crate (deps: `chrono`, `regex`, `zip`) that loads Pepperstone **M1/tick** + binary files and shares them lock-free as `Arc<[T]>` chunks + (`CHUNK_SIZE = 1024`) via `SymbolChunkIter::next_chunk` (an inherent method + driven by `while let`, not the `Iterator` trait); `stream_*_windowed(from_ms, + to_ms)` provides C12's data-window with inclusive Unix-ms bounds. Records are + **AoS** (`M1Parsed`/`TickParsed`, `time_ms: i64` Unix-ms). The ingestion + boundary (C3) therefore **transposes** these AoS records into aura's SoA + columns (C7) and normalizes `time_ms` → canonical epoch-ns at that one + boundary. Pulled in as a cargo git dependency when the ingestion task starts; + deliberately absent from the bare skeleton so the workspace compiles without + the network. +- **RustAst (`myc`)** — `~/dev/RustAst`, Gitea `Brummel/RustAst` + (`http://192.168.178.103:3000/Brummel/RustAst.git`). The predecessor DSL attempt + (Foundation): **a conceptual reference, never a dependency** — its DSL authoring + surface and `Value`/HM-inference machinery are exactly what aura rejects (C17). + But its **`src/ast/rtl/` layer is a working reference implementation of the very + streaming substrate aura rebuilds**, and is worth reading before authoring + `aura-core` — these are not just concepts, they exist as code: + - `rtl/series/data.rs` — `RingBuffer` with financial-style indexing (index 0 + = newest), `total_count` (the **run-count** of C5) and `lookback_limit` (C8's + pre-sized window); the `ScalarValue` marker trait ("flat scalars only, no + String/Record" = C7's closed scalar set); `ScalarSeries` and the + **SoA** `RecordSeries` for composites (C7's "OHLCV = a bundle of base columns"). + - `rtl/series/mod.rs` — `create_typed_series`, dispatching element type → storage + backend (the type-specialized **factory** of C19). + - `rtl/streams/mod.rs` — `Signal { cycle_id, value }` (C4's cycle clock) and the + `Stream` / `Observer` / `ObservableStream` push traits (the reactive model of + C4/C5). + - `rtl/streams/register.rs` — the RTL **"register" / delay node** (the one + explicit feedback path of C5/C9) plus a seeded, reproducible OHLC generator + (C12's seed-as-input). + + aura reimplements these natively and **sharpens** them: types are monomorphized + and edges type-erased to the four scalar kinds with direct dispatch (C7) instead + of carrying boxed `Value`s, and input history is shared zero-copy as `Arc<[T]>` + (C12) instead of a `VecDeque`. RustAst shows the *shape*; aura makes it + fast and deterministic. + +--- + ## Contracts ### C1 — Determinism and disjoint parallelism