Brummel
77ad977623
Per-cycle fieldtest of the aura-ingest ingestion boundary (issue #7), from the public interface only. 3 examples (transpose/normalize core; close_stream -> deterministic harness run with epoch-ns end-to-end; load_m1_window over real AAPL.US 2006-08 bars + None paths), all green. 0 bugs, 0 friction. Two spec_gaps named for follow-up (neither a code defect; both filed to the tracker): - load_m1_window returns Some(empty M1Columns) — not None — for an in-coverage but bar-empty window. The rustdoc's "None if no data in [from_ms,to_ms]" reads bar-level but the realized boundary is file-level (it inherits data-server's file-skip None). A consumer matching None == "no bars" is wrong for the in-coverage-empty case. -> tighten the load_m1_window rustdoc (one line). - AAPL.US (the carrier/integration-test symbol) has only ~21 M1 bars/month, so the SMA-cross never warms and the end-to-end run yields all-zero (degenerate but valid) metrics. The boundary is correct (C2 warm-up); the milestone's shipped demo data is too thin to show a non-degenerate trace. -> weigh against C22's "newcomer sees a populated trace" before the Walking-skeleton close (document expected density or name a denser demo symbol/window).
94 lines
5.1 KiB
Rust
94 lines
5.1 KiB
Rust
//! Fieldtest c0011 #1 — the pure transpose/normalize core (axis a).
|
|
//!
|
|
//! Question under test: a project author who has recorded a handful of M1 bars
|
|
//! (as data-server `M1Parsed` AoS records) calls `transpose_m1`, inspects the
|
|
//! resulting SoA `M1Columns`, and the C3/C7 boundary contract — field-wise
|
|
//! transpose, Unix-ms -> epoch-ns normalization, volume kept as i64 — holds
|
|
//! exactly as the public rustdoc promises, WITHOUT reading crates/*/src.
|
|
//!
|
|
//! Public-surface facts used (aura-ingest rustdoc + design ledger C3/C7):
|
|
//! - `unix_ms_to_epoch_ns(time_ms) -> Timestamp` = "Normalize Unix-millisecond
|
|
//! to canonical epoch-ns" (the single C3 unit normalization). The feat-commit
|
|
//! surface line states `= ms * 1_000_000`.
|
|
//! - `transpose_m1(&[M1Parsed]) -> M1Columns` — "Pure: identical bars yield
|
|
//! identical columns (C1)". AoS -> SoA (C7).
|
|
//! - `M1Columns { ts: Vec<Timestamp>, open/high/low/close/spread: Vec<f64>,
|
|
//! volume: Vec<i64> }` — public fields; "All columns share an index: ts[i]
|
|
//! is the timestamp of close[i]"; derives Clone + Debug + PartialEq.
|
|
//! - `M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>` — close column
|
|
//! zipped with normalized ts as `Scalar::F64`.
|
|
//! - data-server rustdoc: `M1Parsed { time_ms: i64, open/high/low/close/spread:
|
|
//! f64, volume: i64 }`, all public fields, Copy.
|
|
|
|
use aura_core::{Scalar, Timestamp};
|
|
use aura_ingest::{M1Columns, transpose_m1, unix_ms_to_epoch_ns};
|
|
use data_server::records::M1Parsed;
|
|
|
|
fn main() {
|
|
// ---- Part 1: the unit-normalization primitive, standalone (C3). --------
|
|
// The surface claims ns = ms * 1_000_000. Check a few representative values
|
|
// a project author would care about: zero, a small ms, and a realistic
|
|
// 2006-era Pepperstone Unix-ms timestamp.
|
|
assert_eq!(unix_ms_to_epoch_ns(0), Timestamp(0), "zero maps to zero");
|
|
assert_eq!(unix_ms_to_epoch_ns(1), Timestamp(1_000_000), "1 ms = 1e6 ns");
|
|
// 2006-08-01T00:00:00Z is 1_154_390_400_000 ms; ns must be *1e6 of that.
|
|
let ms_2006 = 1_154_390_400_000_i64;
|
|
assert_eq!(
|
|
unix_ms_to_epoch_ns(ms_2006),
|
|
Timestamp(ms_2006 * 1_000_000),
|
|
"realistic 2006 ms scales by exactly 1e6"
|
|
);
|
|
println!("unix_ms_to_epoch_ns: 0->0, 1->1e6, 2006 ms -> {:?}", unix_ms_to_epoch_ns(ms_2006));
|
|
|
|
// ---- Part 2: field-wise AoS -> SoA transpose (C7). ---------------------
|
|
// Three hand-built bars, distinct values per field so a transposition that
|
|
// swapped two columns (e.g. high<->low) would be caught.
|
|
let bars = [
|
|
M1Parsed { time_ms: 1_000, open: 10.0, high: 11.0, low: 9.5, close: 10.5, spread: 0.2, volume: 100 },
|
|
M1Parsed { time_ms: 2_000, open: 10.5, high: 12.0, low: 10.1, close: 11.8, spread: 0.3, volume: 250 },
|
|
M1Parsed { time_ms: 3_000, open: 11.8, high: 12.4, low: 11.0, close: 11.2, spread: 0.25, volume: 75 },
|
|
];
|
|
|
|
let cols: M1Columns = transpose_m1(&bars);
|
|
println!("transposed M1Columns = {cols:?}");
|
|
|
|
// Each column must be exactly the corresponding field of each bar, in order.
|
|
assert_eq!(cols.ts, vec![Timestamp(1_000_000_000), Timestamp(2_000_000_000), Timestamp(3_000_000_000)],
|
|
"ts column = each bar's time_ms normalized to epoch-ns");
|
|
assert_eq!(cols.open, vec![10.0, 10.5, 11.8], "open column");
|
|
assert_eq!(cols.high, vec![11.0, 12.0, 12.4], "high column (not swapped with low)");
|
|
assert_eq!(cols.low, vec![9.5, 10.1, 11.0], "low column");
|
|
assert_eq!(cols.close, vec![10.5, 11.8, 11.2], "close column");
|
|
assert_eq!(cols.spread, vec![0.2, 0.3, 0.25], "spread column");
|
|
// volume stays i64 (C7: the one i64 column), never coerced to f64.
|
|
assert_eq!(cols.volume, vec![100_i64, 250, 75], "volume column kept as i64");
|
|
|
|
// Index alignment: ts[i] is the timestamp of close[i] (rustdoc claim).
|
|
for i in 0..bars.len() {
|
|
assert_eq!(cols.ts[i], unix_ms_to_epoch_ns(bars[i].time_ms), "ts[{i}] normalized");
|
|
assert_eq!(cols.close[i], bars[i].close, "close[{i}] aligned");
|
|
}
|
|
|
|
// ---- Part 3: purity (C1) — identical bars -> identical columns. --------
|
|
let cols_again = transpose_m1(&bars);
|
|
assert_eq!(cols, cols_again, "transpose is pure: same input -> same output (PartialEq)");
|
|
|
|
// ---- Part 4: close_stream — the engine source-stream shape (C7). -------
|
|
// The close column zipped with normalized ts as Scalar::F64.
|
|
let stream = cols.close_stream();
|
|
println!("close_stream = {stream:?}");
|
|
let expected: Vec<(Timestamp, Scalar)> = vec![
|
|
(Timestamp(1_000_000_000), Scalar::F64(10.5)),
|
|
(Timestamp(2_000_000_000), Scalar::F64(11.8)),
|
|
(Timestamp(3_000_000_000), Scalar::F64(11.2)),
|
|
];
|
|
assert_eq!(stream, expected, "close_stream = (normalized ts, close as Scalar::F64) per bar");
|
|
|
|
// ---- Part 5: empty edge — a window with no bars. -----------------------
|
|
let empty = transpose_m1(&[]);
|
|
assert!(empty.ts.is_empty() && empty.close.is_empty(), "empty input -> empty columns");
|
|
assert!(empty.close_stream().is_empty(), "empty columns -> empty close_stream");
|
|
|
|
println!("c0011_1 OK: transpose_m1 / unix_ms_to_epoch_ns / close_stream match the rustdoc C3/C7 contract");
|
|
}
|