Brummel
77ad977623
Per-cycle fieldtest of the aura-ingest ingestion boundary (issue #7), from the public interface only. 3 examples (transpose/normalize core; close_stream -> deterministic harness run with epoch-ns end-to-end; load_m1_window over real AAPL.US 2006-08 bars + None paths), all green. 0 bugs, 0 friction. Two spec_gaps named for follow-up (neither a code defect; both filed to the tracker): - load_m1_window returns Some(empty M1Columns) — not None — for an in-coverage but bar-empty window. The rustdoc's "None if no data in [from_ms,to_ms]" reads bar-level but the realized boundary is file-level (it inherits data-server's file-skip None). A consumer matching None == "no bars" is wrong for the in-coverage-empty case. -> tighten the load_m1_window rustdoc (one line). - AAPL.US (the carrier/integration-test symbol) has only ~21 M1 bars/month, so the SMA-cross never warms and the end-to-end run yields all-zero (degenerate but valid) metrics. The boundary is correct (C2 warm-up); the milestone's shipped demo data is too thin to show a non-degenerate trace. -> weigh against C22's "newcomer sees a populated trace" before the Walking-skeleton close (document expected density or name a denser demo symbol/window).
188 lines
12 KiB
Markdown
188 lines
12 KiB
Markdown
# Fieldtest — cycle-0011 — 2026-06-04
|
||
|
||
**Status:** Draft — awaiting orchestrator triage
|
||
**Author:** fieldtester (dispatched by fieldtest skill)
|
||
|
||
## Scope
|
||
|
||
Cycle 0011 (Walking-skeleton milestone, #7) shipped a NEW public crate
|
||
`aura-ingest` — aura's first real data-source ingestion boundary and the
|
||
workspace's external-dependency firewall (it alone links `data-server` and its
|
||
transitive `chrono`/`regex`/`zip`). Public surface field-tested:
|
||
|
||
1. `unix_ms_to_epoch_ns(time_ms: i64) -> Timestamp` — the single C3 unit
|
||
normalization (`= ms * 1_000_000`), at this one boundary and nowhere else.
|
||
2. `M1Columns` — the OHLCV bar transposed AoS → SoA (C7): public fields
|
||
`ts: Vec<Timestamp>`, `open/high/low/close/spread: Vec<f64>`,
|
||
`volume: Vec<i64>`; derives Clone + Debug + PartialEq.
|
||
3. `transpose_m1(&[M1Parsed]) -> M1Columns` — pure AoS → SoA, normalizing time
|
||
at the boundary.
|
||
4. `M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>` — the close column as
|
||
the engine source-stream shape the SMA-cross sample strategy consumes.
|
||
5. `load_m1_window(&Arc<DataServer>, symbol, from_ms, to_ms) -> Option<M1Columns>`
|
||
— drains a data-server M1 window and transposes once at the boundary.
|
||
|
||
Tested from the public interface only: `cargo doc -p aura-ingest` /
|
||
`cargo doc -p data-server` rustdoc, the design ledger (C1/C3/C7/C10/C12/C16),
|
||
the `data-server` README, and the observable behaviour of the functions —
|
||
never `crates/*/src`. The build exercised: `cargo build --workspace` (green from
|
||
HEAD), then each fixture via
|
||
`cargo run --manifest-path fieldtests/cycle-0011-ingest/Cargo.toml --bin <name>`
|
||
so HEAD source is always what runs. The fixture crate is a standalone non-member
|
||
crate path-depending on `aura-core/-engine/-std/-ingest` plus the `data-server`
|
||
git dep, exactly as a C16 research project that ingests recorded bars would.
|
||
|
||
## Examples
|
||
|
||
### fieldtests/cycle-0011-ingest/c0011_1_transpose_core.rs — the pure transpose/normalize core (axis a)
|
||
- Drives `unix_ms_to_epoch_ns` on representative values (0, 1 ms, a real 2006-era
|
||
Unix-ms), then `transpose_m1` on three hand-built `M1Parsed` bars with distinct
|
||
per-field values (so a swapped column would be caught), checks each SoA column
|
||
field-wise, index alignment (`ts[i]` ↔ `close[i]`), `volume` kept i64, purity
|
||
(identical bars → equal columns via PartialEq), `close_stream` shape, and the
|
||
empty-input edge.
|
||
- Why it fits: this is the C3/C7 boundary contract a project author transposing
|
||
recorded data exercises, verified from rustdoc alone.
|
||
- Outcome: built and ran first try; every assertion held. `ms*1e6` exact,
|
||
field-wise transpose correct, volume i64, pure, `close_stream` =
|
||
`(normalized ts, close as Scalar::F64)`, empty → empty. Matched expected.
|
||
|
||
### fieldtests/cycle-0011-ingest/c0011_2_close_stream_run.rs — close_stream → a real engine run (axis b)
|
||
- Transposes synthetic OHLCV bars (close rises then reverses), takes
|
||
`close_stream()`, feeds it straight into a Harness (SMA(2)/SMA(4) → Sub →
|
||
Exposure(4) → SimBroker(1.0) → two shipped `aura_std::Recorder` sinks), runs,
|
||
drains, `f64_field` + `summarize`, asserts finite metrics, ≥1 exposure flip,
|
||
that recorded equity timestamps equal the ingested normalized bar timestamps
|
||
(epoch-ns end to end), and that a disjoint second run is bit-identical (C1).
|
||
- Why it fits: this is the C3 → C1 hand-off — the `(Timestamp, Scalar)` shape
|
||
`close_stream` returns is exactly `Harness::run`'s source-stream input shape;
|
||
the ingestion boundary must connect to the engine with no hand-authored stream.
|
||
- Outcome: built and ran first try. equity `[0,0,0,0,1,0,-0.5]`, exposure flips
|
||
once at the reversal, `total_pips -0.5`, `max_drawdown 1.5`, recorded ts ==
|
||
ingested ts, two runs byte-identical. Matched expected.
|
||
|
||
### fieldtests/cycle-0011-ingest/c0011_3_load_window_real.rs — load_m1_window against real data (axis c)
|
||
- Opens the local `data-server` archive (`DEFAULT_DATA_PATH`), loads the real
|
||
AAPL.US 2006-08 M1 window, checks SoA column lengths aligned, every bar ts
|
||
normalized to epoch-ns and inside the requested window, `close_stream`
|
||
non-decreasing, then runs an end-to-end SMA-cross signal-quality backtest over
|
||
the real close bars (deterministic across two runs, C1). Then exercises the
|
||
documented `None` paths (far-future window, unknown symbol) and the
|
||
in-coverage-but-empty sub-window grey zone. Skips cleanly where
|
||
`/mnt/tickdata` is absent.
|
||
- Why it fits: the milestone's headline — "a backtest runs over real bars" —
|
||
driven exactly as a downstream researcher would, plus the documented `None`
|
||
path.
|
||
- Outcome: built and ran first try on the real archive. Loaded **21** bars for
|
||
2006-08; ts normalized and in-window; `close_stream` ascending; the end-to-end
|
||
run was deterministic; far-future window → `None`, unknown symbol → `None`;
|
||
an in-coverage empty sub-window → `Some(empty)`. Surfaced two spec_gaps (below).
|
||
|
||
## Findings
|
||
|
||
### [working] transpose_m1 / unix_ms_to_epoch_ns / close_stream match the C3/C7 contract from rustdoc alone
|
||
- Example: c0011_1.
|
||
- What happened: `unix_ms_to_epoch_ns` is exactly `ms * 1_000_000` (checked at 0,
|
||
1 ms, and a real 2006 ms = 1_154_390_400_000 → 1_154_390_400_000_000_000 ns);
|
||
`transpose_m1` produces field-wise-correct SoA columns with no column swap,
|
||
`volume` kept `i64` (C7's one i64 column), `ts[i]` aligned with `close[i]`;
|
||
it is pure (identical bars → PartialEq-equal columns, C1); `close_stream`
|
||
yields `(normalized ts, Scalar::F64(close))` per bar; empty input → empty
|
||
columns and empty stream.
|
||
- Why working: the entire pure boundary core is reachable, correct, and
|
||
unambiguous from the public rustdoc + ledger (C3/C7/C1) without reading any
|
||
implementation source. The public-fielded `M1Columns` + the `M1Parsed` public
|
||
fields make a hand-built fixture trivial to author.
|
||
- Recommended action: carry-on.
|
||
|
||
### [working] close_stream feeds Harness::run directly; epoch-ns flows end to end; deterministic
|
||
- Example: c0011_2.
|
||
- What happened: `close_stream()`'s `Vec<(Timestamp, Scalar)>` is exactly the
|
||
shape `Harness::run(vec![stream])` consumes — no adapter. Over a rise-then-
|
||
reverse close series the chain warmed, the recorded equity-curve timestamps
|
||
equalled the ingested (normalized epoch-ns) bar timestamps verbatim
|
||
(normalization survives the whole run loop intact), one exposure sign flip was
|
||
registered at the reversal, and a disjoint second run produced a bit-identical
|
||
`RunReport.to_json()` (C1).
|
||
- Why working: the C3 (ingestion) → C1 (deterministic run) seam — the milestone's
|
||
reason to exist — is reachable and correct from the public surface as a
|
||
downstream consumer, mirroring the cycle-0007/0009 patterns over real ingested
|
||
data rather than a hand-authored stream.
|
||
- Recommended action: carry-on.
|
||
|
||
### [working] load_m1_window over real bars: in-window normalized, ascending, deterministic; far-future & unknown-symbol None paths hold
|
||
- Example: c0011_3.
|
||
- What happened: against the real archive, `load_m1_window("AAPL.US", 2006-08
|
||
window)` returned `Some(M1Columns)` with 21 bars; all SoA columns equal length,
|
||
every ts normalized to epoch-ns and inside the requested (×1e6) window,
|
||
`close_stream` non-decreasing in ts; the end-to-end backtest over the real
|
||
closes ran and was bit-identical across two disjoint runs. A far-future window
|
||
for the present symbol → `None`; an unknown symbol → `None`, both as the
|
||
rustdoc documents.
|
||
- Why working: the real-data path — open archive, load a window, transpose once
|
||
at the boundary, run a deterministic backtest — works first try from the public
|
||
surface, and the two documented `None` conditions (no file overlaps the window;
|
||
unknown symbol) behave as stated.
|
||
- Recommended action: carry-on.
|
||
|
||
### [spec_gap] `Some(empty)` vs `None`: an in-coverage but bar-empty window returns `Some(empty M1Columns)`, not `None`
|
||
- Example: c0011_3 (grey-zone block).
|
||
- What happened: the `load_m1_window` rustdoc states "`None` if the symbol has no
|
||
data in `[from_ms, to_ms]`". Empirically, `None` is **file-level**: it occurs
|
||
when no data-server file overlaps the window (e.g. a far-future window — those
|
||
files are skipped without I/O) or the symbol is unknown. But a **narrow window
|
||
inside a loaded file's date coverage that happens to contain zero bars**
|
||
(a single 2006-08-01 day inside the loaded 2006-08 file) returns
|
||
`Some(M1Columns { ... empty vecs ... })`, **not** `None`. So a downstream
|
||
consumer pattern-matching `None` to mean "no bars here" is wrong for the
|
||
in-coverage-empty case; it must additionally check `cols.close.is_empty()`.
|
||
- Why spec_gap: "no data in `[from_ms, to_ms]`" reads bar-level on the public
|
||
surface, but the realized boundary is file-level (it inherits data-server's own
|
||
file-skip `None`). Two readings are equally plausible — "`None` ⇔ zero bars"
|
||
(what the wording suggests) vs "`None` ⇔ no overlapping file, else `Some`
|
||
(possibly empty)" (what ships). I asserted the **observed (file-level)** reading
|
||
in the fixture; a tightening to the bar-level reading would flip that assertion.
|
||
- Recommended action: ratify-or-tighten the ledger/rustdoc. Either document
|
||
precisely ("`None` only when no file overlaps the window or the symbol is
|
||
unknown; an in-coverage window with no bars returns `Some` with empty columns —
|
||
check `.close.is_empty()`"), or collapse the empty case into `None` so the
|
||
binary `Some`/`None` cleanly means "has bars" / "has none". One rustdoc line on
|
||
`load_m1_window` closes it.
|
||
|
||
### [spec_gap] the only documented real symbol is too sparse for a meaningful signal-quality run over a realistic window
|
||
- Example: c0011_3.
|
||
- What happened: AAPL.US — the symbol named in the carrier and the gated
|
||
integration test — has **21 M1 bars for all of August 2006** and **~100 for the
|
||
whole year 2006** (verified via `load_m1_window` over a year/month/day window).
|
||
The end-to-end backtest therefore ran but produced **all-zero metrics**
|
||
(`total_pips 0, max_drawdown 0, exposure_sign_flips 0`): with any realistic
|
||
SMA-cross (even SMA(10)/SMA(30)) the slow SMA never warms over ≤21 bars, so the
|
||
signal never fires (correct C2 warm-up / no-look-ahead behaviour — **not a
|
||
bug**). The boundary itself is faultless; the data behind the milestone's
|
||
shipped example is too thin to demonstrate a *non-degenerate* run.
|
||
- Why spec_gap: the milestone promise is "a backtest runs over real bars" — which
|
||
is technically met — but a downstream consumer following the documented surface
|
||
(this symbol, this archive) cannot reach a *meaningful* signal-quality result,
|
||
and nothing on the public surface signals that 21 bars/month is the expected
|
||
density (vs a window-bounds bug, a corrupt file, or a normalization error). The
|
||
consumer guesses "this is just sparse early data"; "the window is mis-bounded"
|
||
is an equally plausible reading from the surface alone. I picked the
|
||
sparse-data reading (the bounds were honoured, ts are in-window, the year window
|
||
also returns only ~100 bars).
|
||
- Recommended action: ratify (document expected per-symbol bar density, or name a
|
||
denser default symbol/window for the milestone demo) **or** plan a follow-up
|
||
that points the walking-skeleton sample at a real window dense enough to warm a
|
||
realistic signal — so "runs over real bars" demonstrably yields a non-degenerate
|
||
trace, consistent with C22's "newcomer sees a populated trace". No code change
|
||
to `aura-ingest` implied; this is about which data the milestone showcases.
|
||
|
||
## Recommendation summary
|
||
|
||
| Finding | Class | Action |
|
||
|---|---|---|
|
||
| transpose/normalize/close_stream match C3/C7 from rustdoc | working | carry-on |
|
||
| close_stream → deterministic engine run, epoch-ns end to end | working | carry-on |
|
||
| load_m1_window real bars + documented None paths | working | carry-on |
|
||
| `Some(empty)` vs `None` boundary not crisp | spec_gap | ratify / tighten the ledger + rustdoc |
|
||
| AAPL.US too sparse for a non-degenerate milestone run | spec_gap | ratify / plan a denser demo symbol+window |
|