Files
AILang/docs/plans/embedding-abi-m5.3.md
T
Brummel 67027ab0f9 plan: embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest
Final functional M5 iteration (spec ae905de; builds on green
m5.1 204c171 + m5.2 b724cd1 whose leak-proof is resolved through
dbd76e5). 3 tasks: (1) additive adapter::fold_window (windowed
sibling of fold_symbol) + chrono dev-dep + a second
[[bin]] timeshard_runner declaration; (2) tests/timeshard.rs
written FIRST — RED is the deterministic `env!(CARGO_BIN_EXE_
timeshard_runner) not defined` compile failure (bin absent); (3)
the timeshard_runner bin → GREEN, with the spec §3 strength-ordered
assertions (a) per-shard bit-exact vs single-thread windowed host
fold (THE boundary-invisibility proof), (b) Σ bit-exact by
construction, (c) whole-window within REL_TOL=1e-6 (derived from
the recursive-summation error bound, never bit-exact — spec
explicit), plus the now-deterministic leak-Σ and a journal-only
friction-timing capture.

Boss decisions in-plan: new bin not an arg-mode (shipped symbol-fan
path byte-untouched, no regression surface); chrono dev-only (date
math in the test, bin date-math-free; Invariant 1 unaffected);
concrete pin EURUSD 2017_03/04/05 (recon-verified present, no
gaps; data-server has no month-listing API); friction is a journal
observation, no bench gate, no timing assertion (flake-free);
pre-existing DESIGN.md:2358-2360 M4-drift explicitly out of scope.
Milestone close + mandatory audit run after Boss-verification.
2026-05-19 02:30:46 +02:00

25 KiB
Raw Blame History

embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest — Implementation Plan

Parent spec: docs/specs/2026-05-19-embedding-abi-m5.md

For agentic workers: REQUIRED SUB-SKILL: use skills/implement to run this plan. Steps use - [ ] checkboxes for tracking.

Goal: Prove the chunk/window-boundary-invisibility claim the M4 retirement rests on — one symbol, N disjoint month-windows, each shard's (acc,n) bit-exact vs a single-thread host fold of that exact window — and harvest the per-tick-FFI cost at real volume for the P2 decision. The final functional M5 iteration before milestone close.

Architecture: Builds on shipped+green m5.1/m5.2 (adapter, Kernel, MidPriceStream, atomic runtime, build.rs runtime-tracking, deterministic leak-proof). Adds a windowed adapter sibling fold_window, a new single-responsibility [[bin]] timeshard_runner (one thread per disjoint month-window, each owning its KernelCtx: !Send makes one-ctx-per-thread a compile-time guarantee), and tests/timeshard.rs asserting the spec §3 strength-ordered (a)/(b)/(c) plus the (now-deterministic) leak-Σ, with a journal-only friction timing capture. The shipped symbol-fan path (swarm_runner.rs, swarm.rs) is NOT touched.

Tech Stack: Rust (edition 2021); data-server (stream_tick_windowed, filter_files month-granularity windowing); chrono (dev-dep, (y,m)→unix_ms bounds, data-server's own precedent); std::thread; std::time::Instant; the M3 staticlib via the m5.1 build.rs.


Files this plan creates or modifies:

  • Modify: ail-embed/Cargo.toml — add chrono = "0.4" to [dev-dependencies]; add a second [[bin]] timeshard_runner
  • Modify: ail-embed/src/adapter.rs — add fold_window (windowed sibling of fold_symbol)
  • Create: ail-embed/src/bin/timeshard_runner.rs — one thread per disjoint window; prints RESULT/TIMING
  • Test: ail-embed/tests/timeshard.rs — boundary-invisibility (a)/(b)/(c) + leak-Σ + friction capture

Cross-reference anchors (read-only, do not edit):

  • ail-embed/src/adapter.rs:62-69fold_symbol (the no-window sibling fold_window mirrors); MidPriceStream :24-60
  • ail-embed/src/lib.rs:81-100Kernel/Kernel::run<I:IntoIterator<Item=f64>> (unchanged; reused)
  • ail-embed/src/bin/swarm_runner.rs:1-73 — symbol-fan runner shape timeshard_runner parallels (NOT modified)
  • ail-embed/tests/swarm.rs:42-44,65-86skip_if_no_data / run_swarm_or_skip idioms timeshard.rs mirrors (NOT modified)
  • ../../libs/data-server/src/lib.rs:241-268stream_tick_windowed(self:&Arc<Self>,&str,Option<i64>,Option<i64>)->Option<SymbolChunkIter<TickParsed>>; filter_files:274-308 (month-granularity inclusive window)
  • ../../libs/data-server/src/lib.rs:696-699 — the chrono::NaiveDate::from_ymd_opt(..).and_hms_opt(..).and_utc().timestamp_millis() bound precedent
  • docs/DESIGN.md:2358-2360 — pre-existing "additive M4 concern" drift; spec §"Out of scope" — NOT m5.3's to fix, audit must not charge it here

Task 1: Windowed adapter + chrono dev-dep + bin declaration

Files:

  • Modify: ail-embed/Cargo.toml

  • Modify: ail-embed/src/adapter.rs

  • Step 1: Add fold_window to ail-embed/src/adapter.rs

Append to ail-embed/src/adapter.rs, immediately after the fold_symbol function (currently ends at :69), before the #[cfg(test)] mod tests block:

/// Fold the mid-price stream of `symbol` restricted to the inclusive
/// Unix-millisecond window `[from_ms, to_ms]` through the M3 kernel.
/// `None` if the symbol has no tick stream in that window. The window
/// is the bound (no tick cap): the result is `(Σ mid, count)` over
/// exactly the records data-server yields for that window, in stream
/// order — identical f64-add sequence to a single-thread host fold of
/// the same window, hence bit-exact against it. The windowed sibling
/// of `fold_symbol`; same `MidPriceStream` + `Kernel` path, only the
/// data-server entry differs (`stream_tick_windowed`).
pub fn fold_window(
    server: &Arc<DataServer>,
    symbol: &str,
    from_ms: i64,
    to_ms: i64,
) -> Option<(f64, i64)> {
    let it = server.stream_tick_windowed(symbol, Some(from_ms), Some(to_ms))?;
    Some(Kernel::new().run(MidPriceStream::new(it)))
}
  • Step 2: Add chrono dev-dep + the timeshard_runner bin to ail-embed/Cargo.toml

The exact current tail of ail-embed/Cargo.toml is:

[dev-dependencies]
zip = "2"
tempfile = "3"

[[bin]]
name = "swarm_runner"
path = "src/bin/swarm_runner.rs"

Replace it verbatim with:

[dev-dependencies]
zip = "2"
tempfile = "3"
# Test-only: the time-shard test derives (year,month) -> Unix-ms
# window bounds via chrono (data-server's own bound precedent,
# data-server/src/lib.rs:696-699) and passes raw ms to the bin, so
# the bin itself stays date-math-free. chrono is already transitively
# present via data-server; this is the test-direct declaration.
chrono = "0.4"

[[bin]]
name = "swarm_runner"
path = "src/bin/swarm_runner.rs"

[[bin]]
name = "timeshard_runner"
path = "src/bin/timeshard_runner.rs"
  • Step 3: Verify the crate builds and m5.1/m5.2 stay green (no regression)

Run: cargo test --manifest-path ail-embed/Cargo.toml --lib Expected: PASS — kernel_run_sums_prices + adapter::tests::tick_to_px_is_mid ok; 0 failed. (fold_window is additive; [[bin]] timeshard_runner references a not-yet-created file but --lib does not build bins, so this isolates the adapter/lib regression check. The bin + full-suite gate is Task 3 Step 4.)


Task 2: The time-shard E2E test (RED — bin does not exist yet)

Files:

  • Create: ail-embed/tests/timeshard.rs

  • Step 1: Write ail-embed/tests/timeshard.rs

//! Time-shard boundary-invisibility E2E (spec Architecture §3
//! second assertion / Testing strategy §3). One symbol, N disjoint
//! single-month windows, one thread per window (each owning its
//! `Kernel` — `Ctx: !Send`). Asserts, in spec strength order:
//!  (a) each shard's `(acc,n)` == a single-thread host fold of THAT
//!      EXACT window in stream order, BIT-EXACT — the
//!      boundary-invisibility proof (the kernel result for a window
//!      is independent of how data-server chunked it);
//!  (b) the host-summed `(Σacc,Σn)` == the identically-summed
//!      reference partials, bit-exact by construction;
//!  (c) the whole-window single-thread total within an
//!      f64-reassociation tolerance only (cross-shard re-association
//!      is host arithmetic, NOT a kernel property — asserting it
//!      bit-exact would be WRONG, per the spec).
//! Plus the global leak-Σ (deterministic since dbd76e5) and a
//! journal-only friction timing capture (NO bench gate).
//!
//! Concrete pin: EURUSD, months 2017-03 / 2017-04 / 2017-05 (all
//! present on /mnt, contiguous, no gaps — recon-verified). data-server
//! has no public month-listing API, so the months are hard-pinned.
//! Skips when /mnt absent — data-server's own precedent.

use std::process::Command;
use std::sync::Arc;

use chrono::{NaiveDate, TimeZone, Utc};
use data_server::{DataServer, DEFAULT_DATA_PATH};

const SYMBOL: &str = "EURUSD";
/// Three disjoint single-month windows. Contiguous so the whole-window
/// union is exactly 2017-03-01 .. 2017-05-31 23:59:59.999.
const SHARDS: [(i32, u32); 3] = [(2017, 3), (2017, 4), (2017, 5)];
/// Relative tolerance for the (c) whole-window cross-check. Derivation:
/// recursive f64 summation of n terms has error <= (n-1)*eps*Σ|x_i|.
/// One EURUSD month ~250k ticks => 3 shards ~750k adds of O(1) mids;
/// Σ|x| ~ 8.3e5, (n-1)*eps ~ 8.3e-11 => abs err ~ 6.8e-5, rel ~ 8e-11.
/// 1e-6 is ~4 orders of safety yet still catches any real bug
/// (a genuine miscount is off by >> 1 ppm).
const REL_TOL: f64 = 1e-6;

fn skip_if_no_data() -> bool {
    !std::path::Path::new(DEFAULT_DATA_PATH).exists()
}

/// Inclusive Unix-ms bounds for the whole of calendar month `(y, m)`.
/// `filter_files` windows at month granularity via
/// `unix_ms_to_year_month`, and `filter_chunk` is record-level
/// inclusive `[from, to]`; first-instant .. last-instant of the month
/// selects exactly that month's file and all its records.
fn month_window(y: i32, m: u32) -> (i64, i64) {
    let from = Utc
        .from_utc_datetime(
            &NaiveDate::from_ymd_opt(y, m, 1)
                .unwrap()
                .and_hms_opt(0, 0, 0)
                .unwrap(),
        )
        .timestamp_millis();
    let (ny, nm) = if m == 12 { (y + 1, 1) } else { (y, m + 1) };
    let next_from = Utc
        .from_utc_datetime(
            &NaiveDate::from_ymd_opt(ny, nm, 1)
                .unwrap()
                .and_hms_opt(0, 0, 0)
                .unwrap(),
        )
        .timestamp_millis();
    (from, next_from - 1)
}

/// Single-thread host reference: fold the mid stream over EXACTLY the
/// inclusive `[from_ms, to_ms]` window in data-server stream order
/// (`acc += mid; n += 1`). Bit-exact with `adapter::fold_window`
/// because the f64-add order is identical.
fn reference_window(server: &Arc<DataServer>, from_ms: i64, to_ms: i64) -> (f64, i64) {
    let mut it = server
        .stream_tick_windowed(SYMBOL, Some(from_ms), Some(to_ms))
        .expect("EURUSD has a tick stream in the pinned window");
    let (mut acc, mut n) = (0.0_f64, 0_i64);
    while let Some(chunk) = it.next_chunk() {
        for r in chunk.iter() {
            acc += (r.ask + r.bid) / 2.0;
            n += 1;
        }
    }
    (acc, n)
}

#[test]
fn timeshard_boundary_invisibility_and_leak_free() {
    if skip_if_no_data() {
        eprintln!("skipping: {DEFAULT_DATA_PATH} absent (mirrors data-server)");
        return;
    }

    let windows: Vec<(i64, i64)> =
        SHARDS.iter().map(|&(y, m)| month_window(y, m)).collect();

    // Spawn the time-shard runner: <data_dir> <symbol> then one
    // "from:to" arg per shard. It folds each window on its own thread
    // (own Kernel) and prints `RESULT <idx> <from> <to> <accbits> <n>`
    // + a final `TIMING total_ticks=<N> elapsed_ms=<M>` line.
    let mut cmd = Command::new(env!("CARGO_BIN_EXE_timeshard_runner"));
    cmd.arg(DEFAULT_DATA_PATH).arg(SYMBOL);
    for (f, t) in &windows {
        cmd.arg(format!("{f}:{t}"));
    }
    let out = cmd
        .env("AILANG_RC_STATS", "1")
        .output()
        .expect("spawn timeshard_runner");
    let stdout = String::from_utf8_lossy(&out.stdout);
    let stderr = String::from_utf8_lossy(&out.stderr);
    assert!(
        out.status.success(),
        "timeshard_runner exited {:?}\nstdout:\n{stdout}\nstderr:\n{stderr}",
        out.status.code()
    );

    let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));

    // Parse RESULT lines into (idx -> (from,to,acc,n)).
    let mut shards: Vec<(usize, i64, i64, f64, i64)> = Vec::new();
    for line in stdout.lines().filter(|l| l.starts_with("RESULT ")) {
        let mut t = line.split_whitespace();
        let _ = t.next(); // RESULT
        let idx: usize = t.next().unwrap().parse().unwrap();
        let from: i64 = t.next().unwrap().parse().unwrap();
        let to: i64 = t.next().unwrap().parse().unwrap();
        let acc = f64::from_bits(
            u64::from_str_radix(t.next().unwrap(), 16).unwrap(),
        );
        let n: i64 = t.next().unwrap().parse().unwrap();
        shards.push((idx, from, to, acc, n));
    }
    shards.sort_by_key(|s| s.0);
    assert_eq!(
        shards.len(),
        SHARDS.len(),
        "expected {} shard RESULT lines, got {}",
        SHARDS.len(),
        shards.len()
    );

    // (a) Per-shard BIT-EXACT vs a single-thread host fold of that
    //     exact window. THE boundary-invisibility proof.
    let mut sum_acc = 0.0_f64;
    let mut sum_n = 0_i64;
    let mut ref_sum_acc = 0.0_f64;
    let mut ref_sum_n = 0_i64;
    for (i, &(idx, from, to, acc, n)) in shards.iter().enumerate() {
        assert_eq!(idx, i, "shard indices contiguous 0..N");
        assert_eq!((from, to), windows[i], "shard {i} window matches pin");
        let (r_acc, r_n) = reference_window(&server, from, to);
        assert_eq!(
            acc.to_bits(),
            r_acc.to_bits(),
            "shard {i} ({from}..{to}): kernel acc BIT-EXACT vs \
             single-thread host fold of that exact window \
             (boundary-invisibility)"
        );
        assert_eq!(n, r_n, "shard {i}: tick count == reference");
        assert!(r_n > 0, "shard {i}: non-empty window");
        sum_acc += acc;
        sum_n += n;
        ref_sum_acc += r_acc;
        ref_sum_n += r_n;
    }

    // (b) Host-summed partials == identically-summed reference
    //     partials, BIT-EXACT by construction (same values, same
    //     host summation order).
    assert_eq!(
        sum_acc.to_bits(),
        ref_sum_acc.to_bits(),
        "Σ shard acc bit-exact vs Σ reference partials"
    );
    assert_eq!(sum_n, ref_sum_n, "Σ shard n == Σ reference n");

    // (c) Whole-window single-thread total within REL_TOL only —
    //     cross-shard re-association is host arithmetic, NOT a kernel
    //     property; asserting bit-exact here would be WRONG (spec §3).
    let whole_from = windows.first().unwrap().0;
    let whole_to = windows.last().unwrap().1;
    let (whole_acc, whole_n) = reference_window(&server, whole_from, whole_to);
    assert_eq!(
        whole_n, sum_n,
        "whole-window tick count == Σ shard n (windows are an exact \
         disjoint partition of the union)"
    );
    let rel = ((whole_acc - sum_acc) / whole_acc).abs();
    assert!(
        rel < REL_TOL,
        "whole-window total {whole_acc} vs Σ partials {sum_acc}: \
         relative diff {rel:e} must be < {REL_TOL:e} (f64 \
         reassociation only — NOT a kernel discrepancy)"
    );

    // Global leak-Σ across ALL ailang_rc_stats: lines (deterministic
    // since dbd76e5: atomic global counters + build.rs runtime-track).
    let mut allocs: u64 = 0;
    let mut frees: u64 = 0;
    let mut seen = 0usize;
    for line in stderr.lines().filter(|l| l.starts_with("ailang_rc_stats:")) {
        seen += 1;
        for tok in line.split_whitespace() {
            if let Some(v) = tok.strip_prefix("allocs=") {
                allocs += v.parse::<u64>().expect("allocs= u64");
            } else if let Some(v) = tok.strip_prefix("frees=") {
                frees += v.parse::<u64>().expect("frees= u64");
            }
        }
    }
    assert!(seen > 0, "no ailang_rc_stats line; stderr:\n{stderr}");
    assert_eq!(
        allocs, frees,
        "globally leak-free: Σallocs==Σfrees across {seen} stat line(s)"
    );

    // Friction harvest: journal-only observation, NO assertion (the
    // value feeds the P2 batch-crossing decision; it is not a gate).
    if let Some(tl) = stdout.lines().find(|l| l.starts_with("TIMING ")) {
        eprintln!("m5.3 friction-harvest: {tl} (Σn={sum_n})");
    }
}
  • Step 2: Run the test — verify it is RED (bin does not exist)

Run: cargo test --manifest-path ail-embed/Cargo.toml --test timeshard Expected: FAIL to compile — error: environment variable \CARGO_BIN_EXE_timeshard_runner` not defined at compile time(thebin timeshard_runnertarget's source file does not exist yet, so Cargo does not set the env varenv!` reads). This is the deterministic RED.


Task 3: The timeshard_runner bin (GREEN)

Files:

  • Create: ail-embed/src/bin/timeshard_runner.rs

  • Step 1: Write ail-embed/src/bin/timeshard_runner.rs

//! Time-shard swarm harness (spec §3). Run as a subprocess by
//! `tests/timeshard.rs` under `AILANG_RC_STATS=1`.
//!
//! argv: `<data_dir> <symbol> <from:to> [<from:to> ...]` — one
//! inclusive Unix-ms window per shard. Date math lives in the test
//! (chrono); this bin is date-math-free. Spawns one thread per
//! window, each building its OWN `Kernel` (`Ctx: !Send` forces
//! one-ctx-per-thread — a shared-ctx design is `error[E0277]`).
//! Prints, in shard order, `RESULT <idx> <from> <to> <accbits:016x>
//! <n>`, then one `TIMING total_ticks=<Σn> elapsed_ms=<wall>` line
//! (the friction-harvest observation; the test records it in the
//! close-out journal, it is not asserted).

use std::sync::Arc;
use std::time::Instant;

use ail_embed::adapter::fold_window;
use data_server::DataServer;

fn main() {
    let mut args = std::env::args().skip(1);
    let data_dir = args.next().expect("argv[1] = data dir");
    let symbol = args.next().expect("argv[2] = symbol");
    let windows: Vec<(i64, i64)> = args
        .map(|a| {
            let (f, t) = a.split_once(':').expect("window arg is from:to");
            (
                f.parse::<i64>().expect("from_ms i64"),
                t.parse::<i64>().expect("to_ms i64"),
            )
        })
        .collect();
    assert!(
        windows.len() >= 2,
        "need >= 2 disjoint windows for a time-shard, got {}",
        windows.len()
    );

    let server = Arc::new(DataServer::new(&data_dir));

    let started = Instant::now();
    let handles: Vec<_> = windows
        .iter()
        .copied()
        .enumerate()
        .map(|(idx, (from, to))| {
            let srv = Arc::clone(&server);
            let sym = symbol.clone();
            // Each thread builds its own Kernel inside the closure —
            // Kernel/Ctx are !Send, so this is the ONLY shape that
            // compiles (one ctx per thread, type-enforced).
            std::thread::spawn(move || {
                let (acc, n) = fold_window(&srv, &sym, from, to)
                    .expect("window has a tick stream");
                (idx, from, to, acc, n)
            })
        })
        .collect();

    let mut results: Vec<(usize, i64, i64, f64, i64)> = handles
        .into_iter()
        .map(|h| h.join().expect("shard thread panicked"))
        .collect();
    let elapsed_ms = started.elapsed().as_millis();
    results.sort_by_key(|r| r.0);

    let mut total_ticks: i64 = 0;
    for (idx, from, to, acc, n) in &results {
        total_ticks += *n;
        println!("RESULT {idx} {from} {to} {:016x} {n}", acc.to_bits());
    }
    println!("TIMING total_ticks={total_ticks} elapsed_ms={elapsed_ms}");
}
  • Step 2: Run the time-shard test — verify GREEN

Run: cargo test --manifest-path ail-embed/Cargo.toml --test timeshard Expected: PASS — test timeshard_boundary_invisibility_and_leak_free ... ok; 1 passed; 0 failed. The eprintln! shows the friction-harvest TIMING line (captured for the journal).

  • Step 3: Determinism — run the time-shard test ×5

Run: for i in 1 2 3 4 5; do cargo test --manifest-path ail-embed/Cargo.toml --test timeshard 2>&1 | grep 'test result:'; done Expected: test result: ok. 1 passed; 0 failed; 0 ignored; ... every run (5/5). The leak-Σ is deterministic post-dbd76e5; per-shard bit-exactness is deterministic by same-order f64 adds. Any FAIL here is a hard stop (the leak-proof saga's lesson: one green run is insufficient).

  • Step 4: Full ail-embed suite + the M5 acceptance gates

Run: cargo test --manifest-path ail-embed/Cargo.toml Expected: PASS — lib (kernel_run_sums_prices, adapter::tests::tick_to_px_is_mid), smoke (hermetic_smoke_data_server_roundtrip), swarm (symbol_fan_swarm_bit_exact, symbol_fan_swarm_leak_free), timeshard (timeshard_boundary_invisibility_and_leak_free), rt_archive_freshness all ok; 0 failed, 0 ignored across all test binaries (the m5.2 leak-proof stays green; nothing re-quarantined).

  • Step 5: Invariant 1 + no-compiler-surface-diff gate

Run: cargo metadata --format-version 1 --manifest-path Cargo.toml --no-deps 2>/dev/null | grep -c data-server | cat Expected: 0 — the AILang workspace graph has no data-server (adding chrono as an ail-embed dev-dep does not enter it; ail-embed is its own workspace root).

Run: git status --porcelain | grep -E '^\?\?|^ ?M' | grep -vE '^.. (ail-embed/|docs/|bench/orchestrator-stats/)' Expected: empty output — m5.3 touches only ail-embed/**, docs/, and the stats file. Zero diff to crates/ailang-*, crates/ail/, runtime/, examples/*.ail, root Cargo.toml.

  • Step 6: Isolated runtime RED still green (no regression)

Run: cargo test -p ail --test embed_rc_global_stats_race 2>&1 | grep 'test result:' Expected: test result: ok. 1 passed; 0 failed; ... — the runtime/build are untouched by m5.3; the audit-trail RED stays GREEN.


Boss decisions (recorded for the audit)

  • Friction-harvest is journal-only. timeshard_runner emits a TIMING total_ticks=N elapsed_ms=M line; the test eprintln!s it and the Boss records the derived per-tick-FFI cost (elapsed / total_ticks) in the m5.3 close-out journal as the P2 flat-array-decision input. There is NO timing assertion and NO bench/ baseline touched (recon confirmed no bench harness is invoked by ail-embed) — a wall-clock gate would flake; the spec (:302-305) is explicit it is an observation, not a perf gate.
  • chrono is a dev-dependency only. The date math lives in the test; the bin takes raw from:to ms args and is date-math-free. Invariant 1 is unaffected (compiler crates untouched; the AILang workspace graph stays data-server/chrono-free since ail-embed is its own workspace root — gated in Task 3 Step 5).
  • New bin, not an arg-mode on swarm_runner. Keeps the now-green m5.2 symbol-fan path byte-untouched (no regression surface) and each runner single-responsibility; its own CARGO_BIN_EXE_timeshard_runner is the established subprocess wiring.
  • (c) is a tolerance check, never bit-exact. Cross-shard re-association is host f64 arithmetic, not a kernel property; the spec and the m5-spec self-review are explicit. REL_TOL = 1e-6 is derived from the recursive-summation error bound with ~4 orders of safety margin (shown in the test).
  • Pre-existing DESIGN.md drift out of scope. docs/DESIGN.md:2358-2360 ("additive M4 concern") is pre-existing M4-retirement drift, NOT m5.3's to fix; the audit must not charge it to m5.3 (spec §"Out of scope"). m5.3 ships zero runtime//crates//DESIGN.md change.
  • Milestone close next. m5.3 is the last functional M5 iteration; after Boss-verification the milestone closes and audit (mandatory) runs separately.

Self-review (planner Step 5)

  1. Spec coverage. Architecture §3 time-shard second assertion → Task 2/3 (per-shard bit-exact (a), Σ bit-exact (b), whole-window tolerance (c)). Testing §3 → tests/timeshard.rs. Friction-harvest (:302-305, Acceptance :331-332) → TIMING line + journal record (Boss decision). Leak-Σ acceptance (:323-324) → the leak-Σ block. Invariant 1 / no-surface-diff (:309-314, :328-330) → Task 3 Step 5. skip-if-absent (:326-327) → skip_if_no_data. Symbol-fan / hermetic smoke already shipped (m5.2/m5.1) — not re-implemented; verified non-regressed in Task 3 Step 4. Audit runs separately (not an m5.3 task).
  2. Placeholder scan. No "TBD/TODO/implement later/similar to/add appropriate". SYMBOL/SHARDS/REL_TOL are concrete pinned constants with recon-verified data + a derived tolerance, not placeholders.
  3. Type/name consistency. fold_window, MidPriceStream, Kernel, timeshard_runner, CARGO_BIN_EXE_timeshard_runner, RESULT <idx> <from> <to> <accbits:016x> <n> (written in Task 3 / parsed in Task 2), TIMING total_ticks=<N> elapsed_ms=<M>, month_window, reference_window, SHARDS=[(2017,3),(2017,4),(2017,5)], SYMBOL="EURUSD", REL_TOL=1e-6, timeshard_boundary_invisibility_and_leak_free — identical across tasks and verification filters.
  4. Step granularity. Each step is one file write or one command; 2-5 min. Largest paste (Task 2 Step 1) is a single verbatim test.
  5. No commit steps. None. Work stays in the working tree.
  6. Pin/replacement substring contiguity. The one verbatim replacement (Task 1 Step 2, the [dev-dependencies]/[[bin]] block) is not paired with any contains(...)/grep over its own body. The greps (Task 3 Step 5) assert over cargo metadata / git status output, not over a verbatim text body this plan writes — no contiguity hazard.
  7. Compile-gate vs deferred-caller. No signature change to an existing function (only the additive fold_window; fold_symbol and all callers untouched). Task 1 Step 3 uses --lib precisely because the [[bin]] timeshard_runner path does not exist until Task 3 — a full cargo test in Task 1 would fail to build the declared bin. The full-suite/bin gate is correctly placed in Task 3 Step 4 (after the bin exists). This ordering is deliberate and internally consistent — --lib is satisfiable at Task 1 (bins are not built by --lib); the RED in Task 2 Step 2 is the intended env! compile failure, not a plan defect.
  8. Verification filter strings resolve. --lib, --test timeshard, --test embed_rc_global_stats_race, timeshard_boundary_invisibility_and_leak_free are all defined by this plan (or are the shipped isolated RED) in the same tasks that filter on them — resolve by construction. Task 3 Steps 4/5/6 use the unfiltered suite or explicit grep -c … == 0 / empty-output / 1 passed assertions, so "nothing ran" cannot masquerade as success. Task 2 Step 2's "RED" is a compile failure (deterministic, not a 0-filtered false-green).