fieldtest: cycle-0011 — 3 examples, 5 findings

Per-cycle fieldtest of the aura-ingest ingestion boundary (issue #7), from the
public interface only. 3 examples (transpose/normalize core; close_stream ->
deterministic harness run with epoch-ns end-to-end; load_m1_window over real
AAPL.US 2006-08 bars + None paths), all green. 0 bugs, 0 friction.

Two spec_gaps named for follow-up (neither a code defect; both filed to the
tracker):
- load_m1_window returns Some(empty M1Columns) — not None — for an in-coverage
  but bar-empty window. The rustdoc's "None if no data in [from_ms,to_ms]" reads
  bar-level but the realized boundary is file-level (it inherits data-server's
  file-skip None). A consumer matching None == "no bars" is wrong for the
  in-coverage-empty case. -> tighten the load_m1_window rustdoc (one line).
- AAPL.US (the carrier/integration-test symbol) has only ~21 M1 bars/month, so
  the SMA-cross never warms and the end-to-end run yields all-zero (degenerate
  but valid) metrics. The boundary is correct (C2 warm-up); the milestone's
  shipped demo data is too thin to show a non-degenerate trace. -> weigh against
  C22's "newcomer sees a populated trace" before the Walking-skeleton close
  (document expected density or name a denser demo symbol/window).
This commit is contained in:
2026-06-04 22:04:48 +02:00
parent 63ea7eb3b1
commit 77ad977623
6 changed files with 1614 additions and 0 deletions
+187
View File
@@ -0,0 +1,187 @@
# Fieldtest — cycle-0011 — 2026-06-04
**Status:** Draft — awaiting orchestrator triage
**Author:** fieldtester (dispatched by fieldtest skill)
## Scope
Cycle 0011 (Walking-skeleton milestone, #7) shipped a NEW public crate
`aura-ingest` — aura's first real data-source ingestion boundary and the
workspace's external-dependency firewall (it alone links `data-server` and its
transitive `chrono`/`regex`/`zip`). Public surface field-tested:
1. `unix_ms_to_epoch_ns(time_ms: i64) -> Timestamp` — the single C3 unit
normalization (`= ms * 1_000_000`), at this one boundary and nowhere else.
2. `M1Columns` — the OHLCV bar transposed AoS → SoA (C7): public fields
`ts: Vec<Timestamp>`, `open/high/low/close/spread: Vec<f64>`,
`volume: Vec<i64>`; derives Clone + Debug + PartialEq.
3. `transpose_m1(&[M1Parsed]) -> M1Columns` — pure AoS → SoA, normalizing time
at the boundary.
4. `M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>` — the close column as
the engine source-stream shape the SMA-cross sample strategy consumes.
5. `load_m1_window(&Arc<DataServer>, symbol, from_ms, to_ms) -> Option<M1Columns>`
— drains a data-server M1 window and transposes once at the boundary.
Tested from the public interface only: `cargo doc -p aura-ingest` /
`cargo doc -p data-server` rustdoc, the design ledger (C1/C3/C7/C10/C12/C16),
the `data-server` README, and the observable behaviour of the functions —
never `crates/*/src`. The build exercised: `cargo build --workspace` (green from
HEAD), then each fixture via
`cargo run --manifest-path fieldtests/cycle-0011-ingest/Cargo.toml --bin <name>`
so HEAD source is always what runs. The fixture crate is a standalone non-member
crate path-depending on `aura-core/-engine/-std/-ingest` plus the `data-server`
git dep, exactly as a C16 research project that ingests recorded bars would.
## Examples
### fieldtests/cycle-0011-ingest/c0011_1_transpose_core.rs — the pure transpose/normalize core (axis a)
- Drives `unix_ms_to_epoch_ns` on representative values (0, 1 ms, a real 2006-era
Unix-ms), then `transpose_m1` on three hand-built `M1Parsed` bars with distinct
per-field values (so a swapped column would be caught), checks each SoA column
field-wise, index alignment (`ts[i]``close[i]`), `volume` kept i64, purity
(identical bars → equal columns via PartialEq), `close_stream` shape, and the
empty-input edge.
- Why it fits: this is the C3/C7 boundary contract a project author transposing
recorded data exercises, verified from rustdoc alone.
- Outcome: built and ran first try; every assertion held. `ms*1e6` exact,
field-wise transpose correct, volume i64, pure, `close_stream` =
`(normalized ts, close as Scalar::F64)`, empty → empty. Matched expected.
### fieldtests/cycle-0011-ingest/c0011_2_close_stream_run.rs — close_stream → a real engine run (axis b)
- Transposes synthetic OHLCV bars (close rises then reverses), takes
`close_stream()`, feeds it straight into a Harness (SMA(2)/SMA(4) → Sub →
Exposure(4) → SimBroker(1.0) → two shipped `aura_std::Recorder` sinks), runs,
drains, `f64_field` + `summarize`, asserts finite metrics, ≥1 exposure flip,
that recorded equity timestamps equal the ingested normalized bar timestamps
(epoch-ns end to end), and that a disjoint second run is bit-identical (C1).
- Why it fits: this is the C3 → C1 hand-off — the `(Timestamp, Scalar)` shape
`close_stream` returns is exactly `Harness::run`'s source-stream input shape;
the ingestion boundary must connect to the engine with no hand-authored stream.
- Outcome: built and ran first try. equity `[0,0,0,0,1,0,-0.5]`, exposure flips
once at the reversal, `total_pips -0.5`, `max_drawdown 1.5`, recorded ts ==
ingested ts, two runs byte-identical. Matched expected.
### fieldtests/cycle-0011-ingest/c0011_3_load_window_real.rs — load_m1_window against real data (axis c)
- Opens the local `data-server` archive (`DEFAULT_DATA_PATH`), loads the real
AAPL.US 2006-08 M1 window, checks SoA column lengths aligned, every bar ts
normalized to epoch-ns and inside the requested window, `close_stream`
non-decreasing, then runs an end-to-end SMA-cross signal-quality backtest over
the real close bars (deterministic across two runs, C1). Then exercises the
documented `None` paths (far-future window, unknown symbol) and the
in-coverage-but-empty sub-window grey zone. Skips cleanly where
`/mnt/tickdata` is absent.
- Why it fits: the milestone's headline — "a backtest runs over real bars" —
driven exactly as a downstream researcher would, plus the documented `None`
path.
- Outcome: built and ran first try on the real archive. Loaded **21** bars for
2006-08; ts normalized and in-window; `close_stream` ascending; the end-to-end
run was deterministic; far-future window → `None`, unknown symbol → `None`;
an in-coverage empty sub-window → `Some(empty)`. Surfaced two spec_gaps (below).
## Findings
### [working] transpose_m1 / unix_ms_to_epoch_ns / close_stream match the C3/C7 contract from rustdoc alone
- Example: c0011_1.
- What happened: `unix_ms_to_epoch_ns` is exactly `ms * 1_000_000` (checked at 0,
1 ms, and a real 2006 ms = 1_154_390_400_000 → 1_154_390_400_000_000_000 ns);
`transpose_m1` produces field-wise-correct SoA columns with no column swap,
`volume` kept `i64` (C7's one i64 column), `ts[i]` aligned with `close[i]`;
it is pure (identical bars → PartialEq-equal columns, C1); `close_stream`
yields `(normalized ts, Scalar::F64(close))` per bar; empty input → empty
columns and empty stream.
- Why working: the entire pure boundary core is reachable, correct, and
unambiguous from the public rustdoc + ledger (C3/C7/C1) without reading any
implementation source. The public-fielded `M1Columns` + the `M1Parsed` public
fields make a hand-built fixture trivial to author.
- Recommended action: carry-on.
### [working] close_stream feeds Harness::run directly; epoch-ns flows end to end; deterministic
- Example: c0011_2.
- What happened: `close_stream()`'s `Vec<(Timestamp, Scalar)>` is exactly the
shape `Harness::run(vec![stream])` consumes — no adapter. Over a rise-then-
reverse close series the chain warmed, the recorded equity-curve timestamps
equalled the ingested (normalized epoch-ns) bar timestamps verbatim
(normalization survives the whole run loop intact), one exposure sign flip was
registered at the reversal, and a disjoint second run produced a bit-identical
`RunReport.to_json()` (C1).
- Why working: the C3 (ingestion) → C1 (deterministic run) seam — the milestone's
reason to exist — is reachable and correct from the public surface as a
downstream consumer, mirroring the cycle-0007/0009 patterns over real ingested
data rather than a hand-authored stream.
- Recommended action: carry-on.
### [working] load_m1_window over real bars: in-window normalized, ascending, deterministic; far-future & unknown-symbol None paths hold
- Example: c0011_3.
- What happened: against the real archive, `load_m1_window("AAPL.US", 2006-08
window)` returned `Some(M1Columns)` with 21 bars; all SoA columns equal length,
every ts normalized to epoch-ns and inside the requested (×1e6) window,
`close_stream` non-decreasing in ts; the end-to-end backtest over the real
closes ran and was bit-identical across two disjoint runs. A far-future window
for the present symbol → `None`; an unknown symbol → `None`, both as the
rustdoc documents.
- Why working: the real-data path — open archive, load a window, transpose once
at the boundary, run a deterministic backtest — works first try from the public
surface, and the two documented `None` conditions (no file overlaps the window;
unknown symbol) behave as stated.
- Recommended action: carry-on.
### [spec_gap] `Some(empty)` vs `None`: an in-coverage but bar-empty window returns `Some(empty M1Columns)`, not `None`
- Example: c0011_3 (grey-zone block).
- What happened: the `load_m1_window` rustdoc states "`None` if the symbol has no
data in `[from_ms, to_ms]`". Empirically, `None` is **file-level**: it occurs
when no data-server file overlaps the window (e.g. a far-future window — those
files are skipped without I/O) or the symbol is unknown. But a **narrow window
inside a loaded file's date coverage that happens to contain zero bars**
(a single 2006-08-01 day inside the loaded 2006-08 file) returns
`Some(M1Columns { ... empty vecs ... })`, **not** `None`. So a downstream
consumer pattern-matching `None` to mean "no bars here" is wrong for the
in-coverage-empty case; it must additionally check `cols.close.is_empty()`.
- Why spec_gap: "no data in `[from_ms, to_ms]`" reads bar-level on the public
surface, but the realized boundary is file-level (it inherits data-server's own
file-skip `None`). Two readings are equally plausible — "`None` ⇔ zero bars"
(what the wording suggests) vs "`None` ⇔ no overlapping file, else `Some`
(possibly empty)" (what ships). I asserted the **observed (file-level)** reading
in the fixture; a tightening to the bar-level reading would flip that assertion.
- Recommended action: ratify-or-tighten the ledger/rustdoc. Either document
precisely ("`None` only when no file overlaps the window or the symbol is
unknown; an in-coverage window with no bars returns `Some` with empty columns —
check `.close.is_empty()`"), or collapse the empty case into `None` so the
binary `Some`/`None` cleanly means "has bars" / "has none". One rustdoc line on
`load_m1_window` closes it.
### [spec_gap] the only documented real symbol is too sparse for a meaningful signal-quality run over a realistic window
- Example: c0011_3.
- What happened: AAPL.US — the symbol named in the carrier and the gated
integration test — has **21 M1 bars for all of August 2006** and **~100 for the
whole year 2006** (verified via `load_m1_window` over a year/month/day window).
The end-to-end backtest therefore ran but produced **all-zero metrics**
(`total_pips 0, max_drawdown 0, exposure_sign_flips 0`): with any realistic
SMA-cross (even SMA(10)/SMA(30)) the slow SMA never warms over ≤21 bars, so the
signal never fires (correct C2 warm-up / no-look-ahead behaviour — **not a
bug**). The boundary itself is faultless; the data behind the milestone's
shipped example is too thin to demonstrate a *non-degenerate* run.
- Why spec_gap: the milestone promise is "a backtest runs over real bars" — which
is technically met — but a downstream consumer following the documented surface
(this symbol, this archive) cannot reach a *meaningful* signal-quality result,
and nothing on the public surface signals that 21 bars/month is the expected
density (vs a window-bounds bug, a corrupt file, or a normalization error). The
consumer guesses "this is just sparse early data"; "the window is mis-bounded"
is an equally plausible reading from the surface alone. I picked the
sparse-data reading (the bounds were honoured, ts are in-window, the year window
also returns only ~100 bars).
- Recommended action: ratify (document expected per-symbol bar density, or name a
denser default symbol/window for the milestone demo) **or** plan a follow-up
that points the walking-skeleton sample at a real window dense enough to warm a
realistic signal — so "runs over real bars" demonstrably yields a non-degenerate
trace, consistent with C22's "newcomer sees a populated trace". No code change
to `aura-ingest` implied; this is about which data the milestone showcases.
## Recommendation summary
| Finding | Class | Action |
|---|---|---|
| transpose/normalize/close_stream match C3/C7 from rustdoc | working | carry-on |
| close_stream → deterministic engine run, epoch-ns end to end | working | carry-on |
| load_m1_window real bars + documented None paths | working | carry-on |
| `Some(empty)` vs `None` boundary not crisp | spec_gap | ratify / tighten the ledger + rustdoc |
| AAPL.US too sparse for a non-degenerate milestone run | spec_gap | ratify / plan a denser demo symbol+window |
+938
View File
@@ -0,0 +1,938 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aes"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "android_system_properties"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311"
dependencies = [
"libc",
]
[[package]]
name = "arbitrary"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
dependencies = [
"derive_arbitrary",
]
[[package]]
name = "aura-core"
version = "0.1.0"
[[package]]
name = "aura-engine"
version = "0.1.0"
dependencies = [
"aura-core",
]
[[package]]
name = "aura-ingest"
version = "0.1.0"
dependencies = [
"aura-core",
"data-server",
]
[[package]]
name = "aura-std"
version = "0.1.0"
dependencies = [
"aura-core",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "byteorder"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "bzip2"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49ecfb22d906f800d4fe833b6282cf4dc1c298f5057ca0b5445e5c209735ca47"
dependencies = [
"bzip2-sys",
]
[[package]]
name = "bzip2-sys"
version = "0.1.13+1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "225bff33b2141874fe80d71e07d6eec4f85c5c216453dd96388240f96e1acc14"
dependencies = [
"cc",
"pkg-config",
]
[[package]]
name = "c0011-fieldtest"
version = "0.0.0"
dependencies = [
"aura-core",
"aura-engine",
"aura-ingest",
"aura-std",
"data-server",
]
[[package]]
name = "cc"
version = "1.2.63"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "556e016178bb5662a08681bbe0f00f8e17631781a4dfc8c45e466e4b185ec27f"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "chrono"
version = "0.4.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327"
dependencies = [
"iana-time-zone",
"js-sys",
"num-traits",
"wasm-bindgen",
"windows-link",
]
[[package]]
name = "cipher"
version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [
"crypto-common",
"inout",
]
[[package]]
name = "constant_time_eq"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c74b8349d32d297c9134b8c88677813a227df8f779daa29bfc29c183fe3dca6"
[[package]]
name = "core-foundation-sys"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "crc"
version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5eb8a2a1cd12ab0d987a5d5e825195d372001a4094a0376319d5a0ad71c1ba0d"
dependencies = [
"crc-catalog",
]
[[package]]
name = "crc-catalog"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853"
[[package]]
name = "crc32fast"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "data-server"
version = "0.1.0"
source = "git+http://192.168.178.103:3000/Brummel/data-server.git?branch=main#0f5e6655b90eeab0f43a8b0f330b8d1632e80ed4"
dependencies = [
"chrono",
"regex",
"zip",
]
[[package]]
name = "deflate64"
version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac6b926516df9c60bfa16e107b21086399f8285a44ca9711344b9e553c5146e2"
[[package]]
name = "deranged"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
dependencies = [
"powerfmt",
]
[[package]]
name = "derive_arbitrary"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e567bd82dcff979e4b03460c307b3cdc9e96fde3d73bed1496d2bc75d9dd62a"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
"subtle",
]
[[package]]
name = "displaydoc"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ac70aa55017e108007fbaf5aa0f54b021c98f92ff8af59d42eda9da96e3dd4f"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "flate2"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
]
[[package]]
name = "futures-core"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-task"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "getrandom"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi",
"wasip2",
"wasm-bindgen",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "hmac"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e"
dependencies = [
"digest",
]
[[package]]
name = "iana-time-zone"
version = "0.1.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
dependencies = [
"android_system_properties",
"core-foundation-sys",
"iana-time-zone-haiku",
"js-sys",
"log",
"wasm-bindgen",
"windows-core",
]
[[package]]
name = "iana-time-zone-haiku"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
dependencies = [
"cc",
]
[[package]]
name = "indexmap"
version = "2.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "inout"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
dependencies = [
"generic-array",
]
[[package]]
name = "jobserver"
version = "0.1.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
dependencies = [
"getrandom",
"libc",
]
[[package]]
name = "js-sys"
version = "0.3.99"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "142bc4740e452c1e57ade0cbc129f139c9093e354346f0872ef985f4f5cf5f11"
dependencies = [
"cfg-if",
"futures-util",
"once_cell",
"wasm-bindgen",
]
[[package]]
name = "libc"
version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "log"
version = "0.4.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "953f07c43838f8e6f9758cab68bf5bed85465e7587ebe0b823f1bcd81978ad3a"
[[package]]
name = "lzma-rs"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "297e814c836ae64db86b36cf2a557ba54368d03f6afcd7d947c266692f71115e"
dependencies = [
"byteorder",
"crc",
]
[[package]]
name = "lzma-sys"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5fda04ab3764e6cde78b9974eec4f779acaba7c4e84b36eca3cf77c581b85d27"
dependencies = [
"cc",
"libc",
"pkg-config",
]
[[package]]
name = "memchr"
version = "2.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8"
[[package]]
name = "miniz_oxide"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "num-conv"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pbkdf2"
version = "0.12.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ed6a7761f76e3b9f92dfb0a60a6a6477c61024b775147ff0973a02653abaf2"
dependencies = [
"digest",
"hmac",
]
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pkg-config"
version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "powerfmt"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "regex"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sha1"
version = "0.10.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3bf829a2d51ab4a5ddf1352d8470c140cadc8301b2ae1789db023f01cedd6ba"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "thiserror"
version = "2.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4288b5bcbc7920c07a1149a35cf9590a2aa808e0bc1eafaade0b80947865fbc4"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "2.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "time"
version = "0.3.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "743bd48c283afc0388f9b8827b976905fb217ad9e647fae3a379a9283c4def2c"
dependencies = [
"deranged",
"num-conv",
"powerfmt",
"serde_core",
"time-core",
]
[[package]]
name = "time-core"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7694e1cfe791f8d31026952abf09c69ca6f6fa4e1a1229e18988f06a04a12dca"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wasip2"
version = "1.0.3+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ed04576f974d2b2fba0f38c51dbc5518011e38c36bf1143164be765528fd409"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "916151b09da36bd82f6615cbf3a419e2f0ba23a03c6160e8e92eb6bd4aa1dec6"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "299047362ccbfce148b67ab7e73349f77748e00c8296f9542adfad2ad82c5c5e"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a929b2c61f11ba3e9bc35b50c1f25cb38e0e892c0c231ae2b8cf78d5dad4437"
dependencies = [
"unicode-ident",
]
[[package]]
name = "windows-core"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
dependencies = [
"windows-link",
]
[[package]]
name = "wit-bindgen"
version = "0.57.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
[[package]]
name = "xz2"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "388c44dc09d76f1536602ead6d325eb532f5c122f17782bd57fb47baeeb767e2"
dependencies = [
"lzma-sys",
]
[[package]]
name = "zeroize"
version = "1.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
dependencies = [
"zeroize_derive",
]
[[package]]
name = "zeroize_derive"
version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85a5b4158499876c763cb03bc4e49185d3cccbabb15b33c627f7884f43db852e"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zip"
version = "2.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fabe6324e908f85a1c52063ce7aa26b68dcb7eb6dbc83a2d148403c9bc3eba50"
dependencies = [
"aes",
"arbitrary",
"bzip2",
"constant_time_eq",
"crc32fast",
"crossbeam-utils",
"deflate64",
"displaydoc",
"flate2",
"getrandom",
"hmac",
"indexmap",
"lzma-rs",
"memchr",
"pbkdf2",
"sha1",
"thiserror",
"time",
"xz2",
"zeroize",
"zopfli",
"zstd",
]
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zstd"
version = "0.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e91ee311a569c327171651566e07972200e76fcfe2242a4fa446149a3881c08a"
dependencies = [
"zstd-safe",
]
[[package]]
name = "zstd-safe"
version = "7.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f49c4d5f0abb602a93fb8736af2a4f4dd9512e36f7f570d66e65ff867ed3b9d"
dependencies = [
"zstd-sys",
]
[[package]]
name = "zstd-sys"
version = "2.0.16+zstd.1.5.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e19ebc2adc8f83e43039e79776e3fda8ca919132d68a1fed6a5faca2683748"
dependencies = [
"cc",
"pkg-config",
]
+37
View File
@@ -0,0 +1,37 @@
# Standalone downstream-consumer crate for the cycle-0011 fieldtest
# (the aura-ingest data-server M1 ingestion boundary).
#
# Like the cycle-0007..0010 fixtures, this is NOT a member of the aura
# workspace — it path-depends on the engine crates exactly as a real research
# project (C16) would. It additionally path-deps aura-ingest (this cycle's new
# crate) and the external data-server git dep, as a project that ingests real
# recorded bars would. Built/run via
# cargo run --manifest-path fieldtests/cycle-0011-ingest/Cargo.toml --bin <name>
# so HEAD source is always what runs.
# Empty [workspace] table: marks this fixture crate as its OWN workspace root.
[workspace]
[package]
name = "c0011-fieldtest"
version = "0.0.0"
edition = "2024"
publish = false
[dependencies]
aura-core = { path = "../../crates/aura-core" }
aura-engine = { path = "../../crates/aura-engine" }
aura-std = { path = "../../crates/aura-std" }
aura-ingest = { path = "../../crates/aura-ingest" }
data-server = { git = "http://192.168.178.103:3000/Brummel/data-server.git", branch = "main" }
[[bin]]
name = "c0011_1_transpose_core"
path = "c0011_1_transpose_core.rs"
[[bin]]
name = "c0011_2_close_stream_run"
path = "c0011_2_close_stream_run.rs"
[[bin]]
name = "c0011_3_load_window_real"
path = "c0011_3_load_window_real.rs"
@@ -0,0 +1,93 @@
//! Fieldtest c0011 #1 — the pure transpose/normalize core (axis a).
//!
//! Question under test: a project author who has recorded a handful of M1 bars
//! (as data-server `M1Parsed` AoS records) calls `transpose_m1`, inspects the
//! resulting SoA `M1Columns`, and the C3/C7 boundary contract — field-wise
//! transpose, Unix-ms -> epoch-ns normalization, volume kept as i64 — holds
//! exactly as the public rustdoc promises, WITHOUT reading crates/*/src.
//!
//! Public-surface facts used (aura-ingest rustdoc + design ledger C3/C7):
//! - `unix_ms_to_epoch_ns(time_ms) -> Timestamp` = "Normalize Unix-millisecond
//! to canonical epoch-ns" (the single C3 unit normalization). The feat-commit
//! surface line states `= ms * 1_000_000`.
//! - `transpose_m1(&[M1Parsed]) -> M1Columns` — "Pure: identical bars yield
//! identical columns (C1)". AoS -> SoA (C7).
//! - `M1Columns { ts: Vec<Timestamp>, open/high/low/close/spread: Vec<f64>,
//! volume: Vec<i64> }` — public fields; "All columns share an index: ts[i]
//! is the timestamp of close[i]"; derives Clone + Debug + PartialEq.
//! - `M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>` — close column
//! zipped with normalized ts as `Scalar::F64`.
//! - data-server rustdoc: `M1Parsed { time_ms: i64, open/high/low/close/spread:
//! f64, volume: i64 }`, all public fields, Copy.
use aura_core::{Scalar, Timestamp};
use aura_ingest::{M1Columns, transpose_m1, unix_ms_to_epoch_ns};
use data_server::records::M1Parsed;
fn main() {
// ---- Part 1: the unit-normalization primitive, standalone (C3). --------
// The surface claims ns = ms * 1_000_000. Check a few representative values
// a project author would care about: zero, a small ms, and a realistic
// 2006-era Pepperstone Unix-ms timestamp.
assert_eq!(unix_ms_to_epoch_ns(0), Timestamp(0), "zero maps to zero");
assert_eq!(unix_ms_to_epoch_ns(1), Timestamp(1_000_000), "1 ms = 1e6 ns");
// 2006-08-01T00:00:00Z is 1_154_390_400_000 ms; ns must be *1e6 of that.
let ms_2006 = 1_154_390_400_000_i64;
assert_eq!(
unix_ms_to_epoch_ns(ms_2006),
Timestamp(ms_2006 * 1_000_000),
"realistic 2006 ms scales by exactly 1e6"
);
println!("unix_ms_to_epoch_ns: 0->0, 1->1e6, 2006 ms -> {:?}", unix_ms_to_epoch_ns(ms_2006));
// ---- Part 2: field-wise AoS -> SoA transpose (C7). ---------------------
// Three hand-built bars, distinct values per field so a transposition that
// swapped two columns (e.g. high<->low) would be caught.
let bars = [
M1Parsed { time_ms: 1_000, open: 10.0, high: 11.0, low: 9.5, close: 10.5, spread: 0.2, volume: 100 },
M1Parsed { time_ms: 2_000, open: 10.5, high: 12.0, low: 10.1, close: 11.8, spread: 0.3, volume: 250 },
M1Parsed { time_ms: 3_000, open: 11.8, high: 12.4, low: 11.0, close: 11.2, spread: 0.25, volume: 75 },
];
let cols: M1Columns = transpose_m1(&bars);
println!("transposed M1Columns = {cols:?}");
// Each column must be exactly the corresponding field of each bar, in order.
assert_eq!(cols.ts, vec![Timestamp(1_000_000_000), Timestamp(2_000_000_000), Timestamp(3_000_000_000)],
"ts column = each bar's time_ms normalized to epoch-ns");
assert_eq!(cols.open, vec![10.0, 10.5, 11.8], "open column");
assert_eq!(cols.high, vec![11.0, 12.0, 12.4], "high column (not swapped with low)");
assert_eq!(cols.low, vec![9.5, 10.1, 11.0], "low column");
assert_eq!(cols.close, vec![10.5, 11.8, 11.2], "close column");
assert_eq!(cols.spread, vec![0.2, 0.3, 0.25], "spread column");
// volume stays i64 (C7: the one i64 column), never coerced to f64.
assert_eq!(cols.volume, vec![100_i64, 250, 75], "volume column kept as i64");
// Index alignment: ts[i] is the timestamp of close[i] (rustdoc claim).
for i in 0..bars.len() {
assert_eq!(cols.ts[i], unix_ms_to_epoch_ns(bars[i].time_ms), "ts[{i}] normalized");
assert_eq!(cols.close[i], bars[i].close, "close[{i}] aligned");
}
// ---- Part 3: purity (C1) — identical bars -> identical columns. --------
let cols_again = transpose_m1(&bars);
assert_eq!(cols, cols_again, "transpose is pure: same input -> same output (PartialEq)");
// ---- Part 4: close_stream — the engine source-stream shape (C7). -------
// The close column zipped with normalized ts as Scalar::F64.
let stream = cols.close_stream();
println!("close_stream = {stream:?}");
let expected: Vec<(Timestamp, Scalar)> = vec![
(Timestamp(1_000_000_000), Scalar::F64(10.5)),
(Timestamp(2_000_000_000), Scalar::F64(11.8)),
(Timestamp(3_000_000_000), Scalar::F64(11.2)),
];
assert_eq!(stream, expected, "close_stream = (normalized ts, close as Scalar::F64) per bar");
// ---- Part 5: empty edge — a window with no bars. -----------------------
let empty = transpose_m1(&[]);
assert!(empty.ts.is_empty() && empty.close.is_empty(), "empty input -> empty columns");
assert!(empty.close_stream().is_empty(), "empty columns -> empty close_stream");
println!("c0011_1 OK: transpose_m1 / unix_ms_to_epoch_ns / close_stream match the rustdoc C3/C7 contract");
}
@@ -0,0 +1,178 @@
//! Fieldtest c0011 #2 — close_stream -> a real engine run (axis b).
//!
//! Question under test: a project author transposes recorded M1 bars to
//! `M1Columns`, takes `close_stream()`, and feeds it straight into a Harness
//! (SMA-cross -> Exposure -> SimBroker -> Recorder, the cycle-0007/0009 pattern
//! over the shipped aura_std::Recorder), runs it, drains the recorded equity
//! curve, and the recorded stream is what a downstream consumer expects — the
//! ingestion boundary connects to the engine without a hand-authored stream.
//!
//! This is the C3 -> C1 hand-off: the (Timestamp, Scalar) shape `close_stream`
//! returns is exactly `Harness::run`'s source-stream input shape.
//!
//! Public-surface facts used (rustdoc + design ledger):
//! - aura_ingest::{M1Columns via transpose_m1, close_stream}.
//! - aura_std::{Sma, Sub, Exposure, SimBroker, Recorder}.
//! - aura_engine::{Harness, Edge, SourceSpec, Target, RunManifest, RunReport,
//! f64_field, summarize}.
//! - SimBroker slots: 0 = exposure, 1 = price (ledger C10 Realization 0007).
use std::sync::mpsc;
use aura_core::{Firing, Scalar, ScalarKind, Timestamp};
use aura_engine::{Edge, Harness, RunManifest, RunReport, SourceSpec, Target, f64_field, summarize};
use aura_ingest::transpose_m1;
use aura_std::{Exposure, Recorder, SimBroker, Sma, Sub};
use data_server::records::M1Parsed;
/// Build a deterministic synthetic set of M1 bars (close rises then reverses),
/// the way a project author would after recording a real window. We only use
/// the close column downstream, but transpose_m1 carries the full OHLCV bundle.
fn synthetic_bars() -> Vec<M1Parsed> {
// close: 100,102,104,106,108,106,104 — a rise then a reversal, so the
// SMA-cross flips sign once (exercises exposure_sign_flips like the CLI demo).
let closes = [100.0, 102.0, 104.0, 106.0, 108.0, 106.0, 104.0];
closes
.iter()
.enumerate()
.map(|(i, &c)| M1Parsed {
time_ms: (i as i64 + 1) * 60_000, // one M1 bar per minute, Unix-ms
open: c,
high: c + 0.5,
low: c - 0.5,
close: c,
spread: 0.1,
volume: 1_000,
})
.collect()
}
fn main() {
// --- Ingestion boundary: AoS bars -> SoA columns -> close source stream ---
let bars = synthetic_bars();
let cols = transpose_m1(&bars);
let price_stream: Vec<(Timestamp, Scalar)> = cols.close_stream();
println!("close source stream from ingestion = {price_stream:?}");
// The C3 precondition close_stream documents: ascending in timestamp.
for w in price_stream.windows(2) {
assert!(w[0].0 < w[1].0, "close_stream ascending in timestamp (C3 precondition)");
}
// --- Wire the signal-quality harness over the ingested close stream. ---
let (eq_tx, eq_rx) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
let (exp_tx, exp_rx) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
// nodes: 0=SMA(2) fast, 1=SMA(4) slow, 2=Sub, 3=Exposure(4), 4=SimBroker(1.0),
// 5=equity sink (taps broker), 6=exposure sink (taps Exposure).
let mut h = Harness::bootstrap(
vec![
Box::new(Sma::new(2)),
Box::new(Sma::new(4)),
Box::new(Sub::new()),
Box::new(Exposure::new(4.0)),
Box::new(SimBroker::new(1.0)),
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, eq_tx)),
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, exp_tx)),
],
vec![SourceSpec {
kind: ScalarKind::F64,
targets: vec![
Target { node: 0, slot: 0 }, // -> SMA fast
Target { node: 1, slot: 0 }, // -> SMA slow
Target { node: 4, slot: 1 }, // -> SimBroker price (slot 1)
],
}],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 }, // SMA2 -> Sub.in0
Edge { from: 1, to: 2, slot: 1, from_field: 0 }, // SMA4 -> Sub.in1
Edge { from: 2, to: 3, slot: 0, from_field: 0 }, // Sub -> Exposure
Edge { from: 3, to: 4, slot: 0, from_field: 0 }, // Exposure -> SimBroker.in0
Edge { from: 3, to: 6, slot: 0, from_field: 0 }, // Exposure -> exposure sink
Edge { from: 4, to: 5, slot: 0, from_field: 0 }, // SimBroker -> equity sink
],
)
.expect("valid signal-quality DAG over the ingested close stream");
// Run #1 over the ingested stream.
h.run(vec![price_stream.clone()]);
let equity_rows: Vec<(Timestamp, Vec<Scalar>)> = eq_rx.try_iter().collect();
let exposure_rows: Vec<(Timestamp, Vec<Scalar>)> = exp_rx.try_iter().collect();
println!("equity rows = {equity_rows:?}");
println!("exposure rows = {exposure_rows:?}");
let equity = f64_field(&equity_rows, 0);
let exposure = f64_field(&exposure_rows, 0);
let metrics = summarize(&equity, &exposure);
println!("metrics over ingested bars = {metrics:?}");
// The recorded equity curve must be non-empty (the source stream warmed the
// chain) and the metrics finite — the downstream "did the ingested data
// actually drive a sim?" check.
assert!(!equity_rows.is_empty(), "ingested close stream produced a recorded equity curve");
assert!(metrics.total_pips.is_finite(), "total_pips finite");
assert!(metrics.max_drawdown.is_finite() && metrics.max_drawdown >= 0.0, "drawdown finite, >= 0");
// The reversal in the close series must register at least one exposure flip.
assert!(metrics.exposure_sign_flips >= 1, "the rise-then-reverse close flips exposure at least once");
// The recorded equity timestamps are exactly the ingested (normalized) bar
// timestamps — confirming epoch-ns flowed end to end, unchanged, through the
// run loop (the broker fires on every price-fresh cycle, C10 Realization).
let recorded_ts: Vec<Timestamp> = equity_rows.iter().map(|(t, _)| *t).collect();
let ingested_ts: Vec<Timestamp> = price_stream.iter().map(|(t, _)| *t).collect();
assert_eq!(recorded_ts, ingested_ts, "recorded equity ts == ingested normalized bar ts (epoch-ns end to end)");
// --- Determinism (C1): a second disjoint run over the same ingested stream
// yields a bit-identical report. ---
let (eq_tx2, eq_rx2) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
let (exp_tx2, exp_rx2) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
let mut h2 = Harness::bootstrap(
vec![
Box::new(Sma::new(2)),
Box::new(Sma::new(4)),
Box::new(Sub::new()),
Box::new(Exposure::new(4.0)),
Box::new(SimBroker::new(1.0)),
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, eq_tx2)),
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, exp_tx2)),
],
vec![SourceSpec {
kind: ScalarKind::F64,
targets: vec![
Target { node: 0, slot: 0 },
Target { node: 1, slot: 0 },
Target { node: 4, slot: 1 },
],
}],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 },
Edge { from: 1, to: 2, slot: 1, from_field: 0 },
Edge { from: 2, to: 3, slot: 0, from_field: 0 },
Edge { from: 3, to: 4, slot: 0, from_field: 0 },
Edge { from: 3, to: 6, slot: 0, from_field: 0 },
Edge { from: 4, to: 5, slot: 0, from_field: 0 },
],
)
.expect("valid second DAG");
h2.run(vec![price_stream]);
let equity2 = f64_field(&eq_rx2.try_iter().collect::<Vec<_>>(), 0);
let exposure2 = f64_field(&exp_rx2.try_iter().collect::<Vec<_>>(), 0);
let metrics2 = summarize(&equity2, &exposure2);
let report1 = RunReport {
manifest: RunManifest {
commit: "c0011-ingested-synthetic".to_string(),
params: vec![("sma_fast".to_string(), 2.0), ("sma_slow".to_string(), 4.0)],
window: (ingested_ts[0], *ingested_ts.last().unwrap()),
seed: 0,
broker: "sim-optimal(pip_size=1)".to_string(),
},
metrics,
};
let report2 = RunReport {
manifest: report1.manifest.clone(),
metrics: metrics2,
};
assert_eq!(report1.to_json(), report2.to_json(), "two runs over the same ingested stream are bit-identical (C1)");
println!("c0011_2 OK: ingested close_stream drove a deterministic end-to-end signal-quality run");
}
@@ -0,0 +1,181 @@
//! Fieldtest c0011 #3 — load_m1_window against real data (axis c).
//!
//! Question under test: a downstream researcher opens the local data-server
//! archive, asks `load_m1_window` for a real symbol/window, gets back SoA
//! `M1Columns`, takes `close_stream()`, and runs an end-to-end signal-quality
//! backtest over REAL recorded AAPL.US 2006 M1 close bars — deterministically.
//! And: the documented `None` path (symbol/window miss) behaves as the rustdoc
//! says.
//!
//! Public-surface facts used (rustdoc + data-server README):
//! - aura_ingest::load_m1_window(&Arc<DataServer>, symbol, from_ms, to_ms)
//! -> Option<M1Columns>; "None if the symbol has no data in [from_ms,
//! to_ms]"; inclusive Unix-ms bounds (data-server's contract).
//! - data_server::{DataServer, DEFAULT_DATA_PATH}; DataServer::new(base_path);
//! has_symbol(symbol) -> bool; stream is over Arc<DataServer>.
//! - close_stream() ascending in timestamp -> Harness::run source shape.
//!
//! Skips cleanly (exit 0, prints SKIP) where the local archive is absent, so it
//! is hermetic on machines without /mnt/tickdata — mirroring the gated
//! integration test's documented behaviour.
use std::path::Path;
use std::sync::{Arc, mpsc};
use aura_core::{Firing, Scalar, ScalarKind, Timestamp};
use aura_engine::{Edge, Harness, SourceSpec, Target, f64_field, summarize};
use aura_ingest::load_m1_window;
use aura_std::{Exposure, Recorder, SimBroker, Sma, Sub};
use data_server::{DEFAULT_DATA_PATH, DataServer};
// AAPL.US 2006-08 is present per the carrier. Window the whole month, inclusive.
const SYMBOL: &str = "AAPL.US";
// 2006-08-01T00:00:00Z .. 2006-09-01T00:00:00Z in Unix-ms.
const FROM_MS: i64 = 1_154_390_400_000;
const TO_MS: i64 = 1_157_068_800_000;
fn main() {
if !Path::new(DEFAULT_DATA_PATH).exists() {
println!("c0011_3 SKIP: {DEFAULT_DATA_PATH} absent (no local archive)");
return;
}
let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
if !server.has_symbol(SYMBOL) {
println!("c0011_3 SKIP: symbol {SYMBOL} not in local archive");
return;
}
// ---- The happy path: load a real M1 window. ----------------------------
let cols = match load_m1_window(&server, SYMBOL, FROM_MS, TO_MS) {
Some(c) => c,
None => {
println!("c0011_3 SKIP: {SYMBOL} present but no bars in the 2006-08 window");
return;
}
};
let n = cols.close.len();
println!("loaded {n} real M1 bars for {SYMBOL} in 2006-08");
assert!(n > 0, "a present symbol+window yields at least one bar");
// SoA column lengths are all aligned (C7: one index across columns).
assert_eq!(cols.ts.len(), n, "ts column length matches close");
assert_eq!(cols.open.len(), n, "open length");
assert_eq!(cols.high.len(), n, "high length");
assert_eq!(cols.low.len(), n, "low length");
assert_eq!(cols.spread.len(), n, "spread length");
assert_eq!(cols.volume.len(), n, "volume (i64) length");
// Timestamps normalized to epoch-ns and inside the requested window
// (the boundary normalizes Unix-ms -> ns; the window bounds are Unix-ms).
let from_ns = Timestamp(FROM_MS * 1_000_000);
let to_ns = Timestamp(TO_MS * 1_000_000);
for (i, t) in cols.ts.iter().enumerate() {
assert!(*t >= from_ns && *t <= to_ns, "bar {i} ts {t:?} inside the normalized window");
}
// ---- close_stream is ascending (the C3 precondition Harness::run needs). -
let price_stream: Vec<(Timestamp, Scalar)> = cols.close_stream();
assert_eq!(price_stream.len(), n, "one close-stream entry per bar");
for w in price_stream.windows(2) {
assert!(w[0].0 <= w[1].0, "real close_stream non-decreasing in timestamp (C3)");
}
println!(
"first bar = (ts {:?}, close {:?}), last = (ts {:?}, close {:?})",
price_stream[0].0, price_stream[0].1,
price_stream[n - 1].0, price_stream[n - 1].1,
);
// ---- End-to-end signal-quality run over REAL close bars. ---------------
let report1 = run_signal_quality(&price_stream);
let report2 = run_signal_quality(&price_stream); // disjoint second run
println!("metrics over real AAPL.US 2006-08 close bars = {report1:?}");
assert!(report1.0.is_finite(), "total_pips finite over real bars");
assert!(report1.1.is_finite() && report1.1 >= 0.0, "max_drawdown finite, >= 0 over real bars");
// Determinism (C1): same recorded input -> bit-identical metrics.
assert_eq!(report1, report2, "two runs over the same real window are bit-identical (C1)");
// ---- The documented None path: a symbol with no data in the window. ----
// A far-future window for a present symbol -> data-server's own None.
let future_from = 30_000_000_000_000_i64; // ~year 2920 in Unix-ms
let future_to = 30_000_086_400_000_i64;
let miss = load_m1_window(&server, SYMBOL, future_from, future_to);
assert!(miss.is_none(), "present symbol, empty window -> None (rustdoc)");
println!("None-path: {SYMBOL} with a far-future window -> {miss:?}");
// An unknown symbol -> None as well.
let miss_sym = load_m1_window(&server, "NO.SUCH.SYMBOL_XYZ", FROM_MS, TO_MS);
assert!(miss_sym.is_none(), "unknown symbol -> None (rustdoc / data-server contract)");
println!("None-path: unknown symbol -> {miss_sym:?}");
// ---- The grey zone: a window INSIDE the file's date coverage but with no
// bars in it. The rustdoc says "None if the symbol has no data in
// [from_ms, to_ms]". A narrow single-day window inside 2006-08 (whose
// file IS loaded) but containing zero bars returns Some(EMPTY columns),
// NOT None — data-server's `None` is FILE-level (no file overlaps the
// window, skipped without I/O), while a loaded file that yields no bar
// in the exact sub-window transposes to Some(empty). Recorded as a
// spec_gap: the Some(empty) vs None boundary is not crisp on the public
// surface. We assert the OBSERVED reading.
let one_day_from = FROM_MS; // 2006-08-01T00:00Z
let one_day_to = FROM_MS + 86_400_000; // +1 day
let narrow = load_m1_window(&server, SYMBOL, one_day_from, one_day_to);
match &narrow {
Some(c) if c.close.is_empty() => {
println!("grey-zone: in-coverage empty sub-window -> Some(EMPTY), not None");
}
Some(c) => println!("grey-zone: in-coverage sub-window -> Some({} bars)", c.close.len()),
None => println!("grey-zone: in-coverage empty sub-window -> None"),
}
// Observed on this archive: Some(empty). Asserting the observed reading so
// the fixture is a faithful record (a future tightening would flip this).
assert!(
matches!(&narrow, Some(c) if c.close.is_empty()),
"in-coverage empty sub-window returns Some(empty), not None (observed reading)"
);
println!("c0011_3 OK: load_m1_window drove a deterministic end-to-end run over real bars; None paths hold");
}
/// SMA-cross -> Exposure -> SimBroker -> Recorder; returns (total_pips,
/// max_drawdown, exposure_sign_flips) as the summarized metrics tuple.
fn run_signal_quality(price_stream: &[(Timestamp, Scalar)]) -> (f64, f64, u64) {
let (eq_tx, eq_rx) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
let (exp_tx, exp_rx) = mpsc::channel::<(Timestamp, Vec<Scalar>)>();
let mut h = Harness::bootstrap(
vec![
Box::new(Sma::new(10)),
Box::new(Sma::new(30)),
Box::new(Sub::new()),
Box::new(Exposure::new(1.0)),
Box::new(SimBroker::new(0.0001)), // AAPL.US pip_size; reference metadata
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, eq_tx)),
Box::new(Recorder::new(&[ScalarKind::F64], Firing::Any, exp_tx)),
],
vec![SourceSpec {
kind: ScalarKind::F64,
targets: vec![
Target { node: 0, slot: 0 },
Target { node: 1, slot: 0 },
Target { node: 4, slot: 1 },
],
}],
vec![
Edge { from: 0, to: 2, slot: 0, from_field: 0 },
Edge { from: 1, to: 2, slot: 1, from_field: 0 },
Edge { from: 2, to: 3, slot: 0, from_field: 0 },
Edge { from: 3, to: 4, slot: 0, from_field: 0 },
Edge { from: 3, to: 6, slot: 0, from_field: 0 },
Edge { from: 4, to: 5, slot: 0, from_field: 0 },
],
)
.expect("valid real-bars signal-quality DAG");
h.run(vec![price_stream.to_vec()]);
let equity = f64_field(&eq_rx.try_iter().collect::<Vec<_>>(), 0);
let exposure = f64_field(&exp_rx.try_iter().collect::<Vec<_>>(), 0);
let m = summarize(&equity, &exposure);
(m.total_pips, m.max_drawdown, m.exposure_sign_flips)
}