Files
AILang/docs/plans/embedding-abi-m5.3.md
T
Brummel 67027ab0f9 plan: embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest
Final functional M5 iteration (spec ae905de; builds on green
m5.1 204c171 + m5.2 b724cd1 whose leak-proof is resolved through
dbd76e5). 3 tasks: (1) additive adapter::fold_window (windowed
sibling of fold_symbol) + chrono dev-dep + a second
[[bin]] timeshard_runner declaration; (2) tests/timeshard.rs
written FIRST — RED is the deterministic `env!(CARGO_BIN_EXE_
timeshard_runner) not defined` compile failure (bin absent); (3)
the timeshard_runner bin → GREEN, with the spec §3 strength-ordered
assertions (a) per-shard bit-exact vs single-thread windowed host
fold (THE boundary-invisibility proof), (b) Σ bit-exact by
construction, (c) whole-window within REL_TOL=1e-6 (derived from
the recursive-summation error bound, never bit-exact — spec
explicit), plus the now-deterministic leak-Σ and a journal-only
friction-timing capture.

Boss decisions in-plan: new bin not an arg-mode (shipped symbol-fan
path byte-untouched, no regression surface); chrono dev-only (date
math in the test, bin date-math-free; Invariant 1 unaffected);
concrete pin EURUSD 2017_03/04/05 (recon-verified present, no
gaps; data-server has no month-listing API); friction is a journal
observation, no bench gate, no timing assertion (flake-free);
pre-existing DESIGN.md:2358-2360 M4-drift explicitly out of scope.
Milestone close + mandatory audit run after Boss-verification.
2026-05-19 02:30:46 +02:00

589 lines
25 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# embedding-abi-m5.3 — time-shard boundary-invisibility proof + friction harvest — Implementation Plan
> **Parent spec:** `docs/specs/2026-05-19-embedding-abi-m5.md`
>
> **For agentic workers:** REQUIRED SUB-SKILL: use `skills/implement`
> to run this plan. Steps use `- [ ]` checkboxes for tracking.
**Goal:** Prove the chunk/window-boundary-invisibility claim the M4
retirement rests on — one symbol, N disjoint month-windows, each
shard's `(acc,n)` bit-exact vs a single-thread host fold of that exact
window — and harvest the per-tick-FFI cost at real volume for the P2
decision. The final functional M5 iteration before milestone close.
**Architecture:** Builds on shipped+green m5.1/m5.2 (adapter,
`Kernel`, `MidPriceStream`, atomic runtime, build.rs runtime-tracking,
deterministic leak-proof). Adds a windowed adapter sibling
`fold_window`, a new single-responsibility `[[bin]] timeshard_runner`
(one thread per disjoint month-window, each owning its `Kernel`
`Ctx: !Send` makes one-ctx-per-thread a compile-time guarantee), and
`tests/timeshard.rs` asserting the spec §3 strength-ordered (a)/(b)/(c)
plus the (now-deterministic) leak-Σ, with a journal-only friction
timing capture. The shipped symbol-fan path (`swarm_runner.rs`,
`swarm.rs`) is NOT touched.
**Tech Stack:** Rust (edition 2021); `data-server`
(`stream_tick_windowed`, `filter_files` month-granularity windowing);
`chrono` (dev-dep, `(y,m)→unix_ms` bounds, data-server's own
precedent); `std::thread`; `std::time::Instant`; the M3 staticlib via
the m5.1 `build.rs`.
---
**Files this plan creates or modifies:**
- Modify: `ail-embed/Cargo.toml` — add `chrono = "0.4"` to `[dev-dependencies]`; add a second `[[bin]] timeshard_runner`
- Modify: `ail-embed/src/adapter.rs` — add `fold_window` (windowed sibling of `fold_symbol`)
- Create: `ail-embed/src/bin/timeshard_runner.rs` — one thread per disjoint window; prints `RESULT`/`TIMING`
- Test: `ail-embed/tests/timeshard.rs` — boundary-invisibility (a)/(b)/(c) + leak-Σ + friction capture
Cross-reference anchors (read-only, do not edit):
- `ail-embed/src/adapter.rs:62-69``fold_symbol` (the no-window sibling `fold_window` mirrors); `MidPriceStream` `:24-60`
- `ail-embed/src/lib.rs:81-100``Kernel`/`Kernel::run<I:IntoIterator<Item=f64>>` (unchanged; reused)
- `ail-embed/src/bin/swarm_runner.rs:1-73` — symbol-fan runner shape `timeshard_runner` parallels (NOT modified)
- `ail-embed/tests/swarm.rs:42-44,65-86``skip_if_no_data` / `run_swarm_or_skip` idioms `timeshard.rs` mirrors (NOT modified)
- `../../libs/data-server/src/lib.rs:241-268``stream_tick_windowed(self:&Arc<Self>,&str,Option<i64>,Option<i64>)->Option<SymbolChunkIter<TickParsed>>`; `filter_files:274-308` (month-granularity inclusive window)
- `../../libs/data-server/src/lib.rs:696-699` — the `chrono::NaiveDate::from_ymd_opt(..).and_hms_opt(..).and_utc().timestamp_millis()` bound precedent
- `docs/DESIGN.md:2358-2360` — pre-existing "additive M4 concern" drift; spec §"Out of scope" — NOT m5.3's to fix, audit must not charge it here
---
### Task 1: Windowed adapter + chrono dev-dep + bin declaration
**Files:**
- Modify: `ail-embed/Cargo.toml`
- Modify: `ail-embed/src/adapter.rs`
- [ ] **Step 1: Add `fold_window` to `ail-embed/src/adapter.rs`**
Append to `ail-embed/src/adapter.rs`, immediately after the
`fold_symbol` function (currently ends at `:69`), *before* the
`#[cfg(test)] mod tests` block:
```rust
/// Fold the mid-price stream of `symbol` restricted to the inclusive
/// Unix-millisecond window `[from_ms, to_ms]` through the M3 kernel.
/// `None` if the symbol has no tick stream in that window. The window
/// is the bound (no tick cap): the result is `(Σ mid, count)` over
/// exactly the records data-server yields for that window, in stream
/// order — identical f64-add sequence to a single-thread host fold of
/// the same window, hence bit-exact against it. The windowed sibling
/// of `fold_symbol`; same `MidPriceStream` + `Kernel` path, only the
/// data-server entry differs (`stream_tick_windowed`).
pub fn fold_window(
server: &Arc<DataServer>,
symbol: &str,
from_ms: i64,
to_ms: i64,
) -> Option<(f64, i64)> {
let it = server.stream_tick_windowed(symbol, Some(from_ms), Some(to_ms))?;
Some(Kernel::new().run(MidPriceStream::new(it)))
}
```
- [ ] **Step 2: Add `chrono` dev-dep + the `timeshard_runner` bin to `ail-embed/Cargo.toml`**
The exact current tail of `ail-embed/Cargo.toml` is:
```toml
[dev-dependencies]
zip = "2"
tempfile = "3"
[[bin]]
name = "swarm_runner"
path = "src/bin/swarm_runner.rs"
```
Replace it verbatim with:
```toml
[dev-dependencies]
zip = "2"
tempfile = "3"
# Test-only: the time-shard test derives (year,month) -> Unix-ms
# window bounds via chrono (data-server's own bound precedent,
# data-server/src/lib.rs:696-699) and passes raw ms to the bin, so
# the bin itself stays date-math-free. chrono is already transitively
# present via data-server; this is the test-direct declaration.
chrono = "0.4"
[[bin]]
name = "swarm_runner"
path = "src/bin/swarm_runner.rs"
[[bin]]
name = "timeshard_runner"
path = "src/bin/timeshard_runner.rs"
```
- [ ] **Step 3: Verify the crate builds and m5.1/m5.2 stay green (no regression)**
Run: `cargo test --manifest-path ail-embed/Cargo.toml --lib`
Expected: PASS — `kernel_run_sums_prices` + `adapter::tests::tick_to_px_is_mid` `ok`; `0 failed`. (`fold_window` is additive; `[[bin]] timeshard_runner` references a not-yet-created file but `--lib` does not build bins, so this isolates the adapter/lib regression check. The bin + full-suite gate is Task 3 Step 4.)
---
### Task 2: The time-shard E2E test (RED — bin does not exist yet)
**Files:**
- Create: `ail-embed/tests/timeshard.rs`
- [ ] **Step 1: Write `ail-embed/tests/timeshard.rs`**
```rust
//! Time-shard boundary-invisibility E2E (spec Architecture §3
//! second assertion / Testing strategy §3). One symbol, N disjoint
//! single-month windows, one thread per window (each owning its
//! `Kernel` — `Ctx: !Send`). Asserts, in spec strength order:
//! (a) each shard's `(acc,n)` == a single-thread host fold of THAT
//! EXACT window in stream order, BIT-EXACT — the
//! boundary-invisibility proof (the kernel result for a window
//! is independent of how data-server chunked it);
//! (b) the host-summed `(Σacc,Σn)` == the identically-summed
//! reference partials, bit-exact by construction;
//! (c) the whole-window single-thread total within an
//! f64-reassociation tolerance only (cross-shard re-association
//! is host arithmetic, NOT a kernel property — asserting it
//! bit-exact would be WRONG, per the spec).
//! Plus the global leak-Σ (deterministic since dbd76e5) and a
//! journal-only friction timing capture (NO bench gate).
//!
//! Concrete pin: EURUSD, months 2017-03 / 2017-04 / 2017-05 (all
//! present on /mnt, contiguous, no gaps — recon-verified). data-server
//! has no public month-listing API, so the months are hard-pinned.
//! Skips when /mnt absent — data-server's own precedent.
use std::process::Command;
use std::sync::Arc;
use chrono::{NaiveDate, TimeZone, Utc};
use data_server::{DataServer, DEFAULT_DATA_PATH};
const SYMBOL: &str = "EURUSD";
/// Three disjoint single-month windows. Contiguous so the whole-window
/// union is exactly 2017-03-01 .. 2017-05-31 23:59:59.999.
const SHARDS: [(i32, u32); 3] = [(2017, 3), (2017, 4), (2017, 5)];
/// Relative tolerance for the (c) whole-window cross-check. Derivation:
/// recursive f64 summation of n terms has error <= (n-1)*eps*Σ|x_i|.
/// One EURUSD month ~250k ticks => 3 shards ~750k adds of O(1) mids;
/// Σ|x| ~ 8.3e5, (n-1)*eps ~ 8.3e-11 => abs err ~ 6.8e-5, rel ~ 8e-11.
/// 1e-6 is ~4 orders of safety yet still catches any real bug
/// (a genuine miscount is off by >> 1 ppm).
const REL_TOL: f64 = 1e-6;
fn skip_if_no_data() -> bool {
!std::path::Path::new(DEFAULT_DATA_PATH).exists()
}
/// Inclusive Unix-ms bounds for the whole of calendar month `(y, m)`.
/// `filter_files` windows at month granularity via
/// `unix_ms_to_year_month`, and `filter_chunk` is record-level
/// inclusive `[from, to]`; first-instant .. last-instant of the month
/// selects exactly that month's file and all its records.
fn month_window(y: i32, m: u32) -> (i64, i64) {
let from = Utc
.from_utc_datetime(
&NaiveDate::from_ymd_opt(y, m, 1)
.unwrap()
.and_hms_opt(0, 0, 0)
.unwrap(),
)
.timestamp_millis();
let (ny, nm) = if m == 12 { (y + 1, 1) } else { (y, m + 1) };
let next_from = Utc
.from_utc_datetime(
&NaiveDate::from_ymd_opt(ny, nm, 1)
.unwrap()
.and_hms_opt(0, 0, 0)
.unwrap(),
)
.timestamp_millis();
(from, next_from - 1)
}
/// Single-thread host reference: fold the mid stream over EXACTLY the
/// inclusive `[from_ms, to_ms]` window in data-server stream order
/// (`acc += mid; n += 1`). Bit-exact with `adapter::fold_window`
/// because the f64-add order is identical.
fn reference_window(server: &Arc<DataServer>, from_ms: i64, to_ms: i64) -> (f64, i64) {
let mut it = server
.stream_tick_windowed(SYMBOL, Some(from_ms), Some(to_ms))
.expect("EURUSD has a tick stream in the pinned window");
let (mut acc, mut n) = (0.0_f64, 0_i64);
while let Some(chunk) = it.next_chunk() {
for r in chunk.iter() {
acc += (r.ask + r.bid) / 2.0;
n += 1;
}
}
(acc, n)
}
#[test]
fn timeshard_boundary_invisibility_and_leak_free() {
if skip_if_no_data() {
eprintln!("skipping: {DEFAULT_DATA_PATH} absent (mirrors data-server)");
return;
}
let windows: Vec<(i64, i64)> =
SHARDS.iter().map(|&(y, m)| month_window(y, m)).collect();
// Spawn the time-shard runner: <data_dir> <symbol> then one
// "from:to" arg per shard. It folds each window on its own thread
// (own Kernel) and prints `RESULT <idx> <from> <to> <accbits> <n>`
// + a final `TIMING total_ticks=<N> elapsed_ms=<M>` line.
let mut cmd = Command::new(env!("CARGO_BIN_EXE_timeshard_runner"));
cmd.arg(DEFAULT_DATA_PATH).arg(SYMBOL);
for (f, t) in &windows {
cmd.arg(format!("{f}:{t}"));
}
let out = cmd
.env("AILANG_RC_STATS", "1")
.output()
.expect("spawn timeshard_runner");
let stdout = String::from_utf8_lossy(&out.stdout);
let stderr = String::from_utf8_lossy(&out.stderr);
assert!(
out.status.success(),
"timeshard_runner exited {:?}\nstdout:\n{stdout}\nstderr:\n{stderr}",
out.status.code()
);
let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
// Parse RESULT lines into (idx -> (from,to,acc,n)).
let mut shards: Vec<(usize, i64, i64, f64, i64)> = Vec::new();
for line in stdout.lines().filter(|l| l.starts_with("RESULT ")) {
let mut t = line.split_whitespace();
let _ = t.next(); // RESULT
let idx: usize = t.next().unwrap().parse().unwrap();
let from: i64 = t.next().unwrap().parse().unwrap();
let to: i64 = t.next().unwrap().parse().unwrap();
let acc = f64::from_bits(
u64::from_str_radix(t.next().unwrap(), 16).unwrap(),
);
let n: i64 = t.next().unwrap().parse().unwrap();
shards.push((idx, from, to, acc, n));
}
shards.sort_by_key(|s| s.0);
assert_eq!(
shards.len(),
SHARDS.len(),
"expected {} shard RESULT lines, got {}",
SHARDS.len(),
shards.len()
);
// (a) Per-shard BIT-EXACT vs a single-thread host fold of that
// exact window. THE boundary-invisibility proof.
let mut sum_acc = 0.0_f64;
let mut sum_n = 0_i64;
let mut ref_sum_acc = 0.0_f64;
let mut ref_sum_n = 0_i64;
for (i, &(idx, from, to, acc, n)) in shards.iter().enumerate() {
assert_eq!(idx, i, "shard indices contiguous 0..N");
assert_eq!((from, to), windows[i], "shard {i} window matches pin");
let (r_acc, r_n) = reference_window(&server, from, to);
assert_eq!(
acc.to_bits(),
r_acc.to_bits(),
"shard {i} ({from}..{to}): kernel acc BIT-EXACT vs \
single-thread host fold of that exact window \
(boundary-invisibility)"
);
assert_eq!(n, r_n, "shard {i}: tick count == reference");
assert!(r_n > 0, "shard {i}: non-empty window");
sum_acc += acc;
sum_n += n;
ref_sum_acc += r_acc;
ref_sum_n += r_n;
}
// (b) Host-summed partials == identically-summed reference
// partials, BIT-EXACT by construction (same values, same
// host summation order).
assert_eq!(
sum_acc.to_bits(),
ref_sum_acc.to_bits(),
"Σ shard acc bit-exact vs Σ reference partials"
);
assert_eq!(sum_n, ref_sum_n, "Σ shard n == Σ reference n");
// (c) Whole-window single-thread total within REL_TOL only —
// cross-shard re-association is host arithmetic, NOT a kernel
// property; asserting bit-exact here would be WRONG (spec §3).
let whole_from = windows.first().unwrap().0;
let whole_to = windows.last().unwrap().1;
let (whole_acc, whole_n) = reference_window(&server, whole_from, whole_to);
assert_eq!(
whole_n, sum_n,
"whole-window tick count == Σ shard n (windows are an exact \
disjoint partition of the union)"
);
let rel = ((whole_acc - sum_acc) / whole_acc).abs();
assert!(
rel < REL_TOL,
"whole-window total {whole_acc} vs Σ partials {sum_acc}: \
relative diff {rel:e} must be < {REL_TOL:e} (f64 \
reassociation only — NOT a kernel discrepancy)"
);
// Global leak-Σ across ALL ailang_rc_stats: lines (deterministic
// since dbd76e5: atomic global counters + build.rs runtime-track).
let mut allocs: u64 = 0;
let mut frees: u64 = 0;
let mut seen = 0usize;
for line in stderr.lines().filter(|l| l.starts_with("ailang_rc_stats:")) {
seen += 1;
for tok in line.split_whitespace() {
if let Some(v) = tok.strip_prefix("allocs=") {
allocs += v.parse::<u64>().expect("allocs= u64");
} else if let Some(v) = tok.strip_prefix("frees=") {
frees += v.parse::<u64>().expect("frees= u64");
}
}
}
assert!(seen > 0, "no ailang_rc_stats line; stderr:\n{stderr}");
assert_eq!(
allocs, frees,
"globally leak-free: Σallocs==Σfrees across {seen} stat line(s)"
);
// Friction harvest: journal-only observation, NO assertion (the
// value feeds the P2 batch-crossing decision; it is not a gate).
if let Some(tl) = stdout.lines().find(|l| l.starts_with("TIMING ")) {
eprintln!("m5.3 friction-harvest: {tl} (Σn={sum_n})");
}
}
```
- [ ] **Step 2: Run the test — verify it is RED (bin does not exist)**
Run: `cargo test --manifest-path ail-embed/Cargo.toml --test timeshard`
Expected: FAIL to compile — `error: environment variable
\`CARGO_BIN_EXE_timeshard_runner\` not defined at compile time`
(the `[[bin]] timeshard_runner` target's source file does not exist
yet, so Cargo does not set the env var `env!` reads). This is the
deterministic RED.
---
### Task 3: The `timeshard_runner` bin (GREEN)
**Files:**
- Create: `ail-embed/src/bin/timeshard_runner.rs`
- [ ] **Step 1: Write `ail-embed/src/bin/timeshard_runner.rs`**
```rust
//! Time-shard swarm harness (spec §3). Run as a subprocess by
//! `tests/timeshard.rs` under `AILANG_RC_STATS=1`.
//!
//! argv: `<data_dir> <symbol> <from:to> [<from:to> ...]` — one
//! inclusive Unix-ms window per shard. Date math lives in the test
//! (chrono); this bin is date-math-free. Spawns one thread per
//! window, each building its OWN `Kernel` (`Ctx: !Send` forces
//! one-ctx-per-thread — a shared-ctx design is `error[E0277]`).
//! Prints, in shard order, `RESULT <idx> <from> <to> <accbits:016x>
//! <n>`, then one `TIMING total_ticks=<Σn> elapsed_ms=<wall>` line
//! (the friction-harvest observation; the test records it in the
//! close-out journal, it is not asserted).
use std::sync::Arc;
use std::time::Instant;
use ail_embed::adapter::fold_window;
use data_server::DataServer;
fn main() {
let mut args = std::env::args().skip(1);
let data_dir = args.next().expect("argv[1] = data dir");
let symbol = args.next().expect("argv[2] = symbol");
let windows: Vec<(i64, i64)> = args
.map(|a| {
let (f, t) = a.split_once(':').expect("window arg is from:to");
(
f.parse::<i64>().expect("from_ms i64"),
t.parse::<i64>().expect("to_ms i64"),
)
})
.collect();
assert!(
windows.len() >= 2,
"need >= 2 disjoint windows for a time-shard, got {}",
windows.len()
);
let server = Arc::new(DataServer::new(&data_dir));
let started = Instant::now();
let handles: Vec<_> = windows
.iter()
.copied()
.enumerate()
.map(|(idx, (from, to))| {
let srv = Arc::clone(&server);
let sym = symbol.clone();
// Each thread builds its own Kernel inside the closure —
// Kernel/Ctx are !Send, so this is the ONLY shape that
// compiles (one ctx per thread, type-enforced).
std::thread::spawn(move || {
let (acc, n) = fold_window(&srv, &sym, from, to)
.expect("window has a tick stream");
(idx, from, to, acc, n)
})
})
.collect();
let mut results: Vec<(usize, i64, i64, f64, i64)> = handles
.into_iter()
.map(|h| h.join().expect("shard thread panicked"))
.collect();
let elapsed_ms = started.elapsed().as_millis();
results.sort_by_key(|r| r.0);
let mut total_ticks: i64 = 0;
for (idx, from, to, acc, n) in &results {
total_ticks += *n;
println!("RESULT {idx} {from} {to} {:016x} {n}", acc.to_bits());
}
println!("TIMING total_ticks={total_ticks} elapsed_ms={elapsed_ms}");
}
```
- [ ] **Step 2: Run the time-shard test — verify GREEN**
Run: `cargo test --manifest-path ail-embed/Cargo.toml --test timeshard`
Expected: PASS — `test timeshard_boundary_invisibility_and_leak_free
... ok`; `1 passed; 0 failed`. The `eprintln!` shows the
friction-harvest `TIMING` line (captured for the journal).
- [ ] **Step 3: Determinism — run the time-shard test ×5**
Run: `for i in 1 2 3 4 5; do cargo test --manifest-path ail-embed/Cargo.toml --test timeshard 2>&1 | grep 'test result:'; done`
Expected: `test result: ok. 1 passed; 0 failed; 0 ignored; ...`
**every** run (5/5). The leak-Σ is deterministic post-`dbd76e5`;
per-shard bit-exactness is deterministic by same-order f64 adds.
Any FAIL here is a hard stop (the leak-proof saga's lesson: one
green run is insufficient).
- [ ] **Step 4: Full ail-embed suite + the M5 acceptance gates**
Run: `cargo test --manifest-path ail-embed/Cargo.toml`
Expected: PASS — lib (`kernel_run_sums_prices`,
`adapter::tests::tick_to_px_is_mid`), `smoke`
(`hermetic_smoke_data_server_roundtrip`), `swarm`
(`symbol_fan_swarm_bit_exact`, `symbol_fan_swarm_leak_free`),
`timeshard` (`timeshard_boundary_invisibility_and_leak_free`),
`rt_archive_freshness` all `ok`; **0 failed, 0 ignored** across all
test binaries (the m5.2 leak-proof stays green; nothing
re-quarantined).
- [ ] **Step 5: Invariant 1 + no-compiler-surface-diff gate**
Run: `cargo metadata --format-version 1 --manifest-path Cargo.toml --no-deps 2>/dev/null | grep -c data-server | cat`
Expected: `0` — the AILang workspace graph has no `data-server`
(adding `chrono` as an `ail-embed` dev-dep does not enter it;
`ail-embed` is its own workspace root).
Run: `git status --porcelain | grep -E '^\?\?|^ ?M' | grep -vE '^.. (ail-embed/|docs/|bench/orchestrator-stats/)'`
Expected: empty output — m5.3 touches **only** `ail-embed/**`,
`docs/`, and the stats file. **Zero** diff to `crates/ailang-*`,
`crates/ail/`, `runtime/`, `examples/*.ail`, root `Cargo.toml`.
- [ ] **Step 6: Isolated runtime RED still green (no regression)**
Run: `cargo test -p ail --test embed_rc_global_stats_race 2>&1 | grep 'test result:'`
Expected: `test result: ok. 1 passed; 0 failed; ...` — the
runtime/build are untouched by m5.3; the audit-trail RED stays GREEN.
---
## Boss decisions (recorded for the audit)
- **Friction-harvest is journal-only.** `timeshard_runner` emits a
`TIMING total_ticks=N elapsed_ms=M` line; the test `eprintln!`s it
and the **Boss records the derived per-tick-FFI cost (elapsed /
total_ticks) in the m5.3 close-out journal** as the P2
flat-array-decision input. There is NO timing assertion and NO
`bench/` baseline touched (recon confirmed no bench harness is
invoked by `ail-embed`) — a wall-clock gate would flake; the spec
(`:302-305`) is explicit it is an observation, not a perf gate.
- **`chrono` is a dev-dependency only.** The date math lives in the
test; the bin takes raw `from:to` ms args and is date-math-free.
Invariant 1 is unaffected (compiler crates untouched; the AILang
workspace graph stays `data-server`/`chrono`-free since `ail-embed`
is its own workspace root — gated in Task 3 Step 5).
- **New bin, not an arg-mode on `swarm_runner`.** Keeps the
now-green m5.2 symbol-fan path byte-untouched (no regression
surface) and each runner single-responsibility; its own
`CARGO_BIN_EXE_timeshard_runner` is the established subprocess
wiring.
- **(c) is a tolerance check, never bit-exact.** Cross-shard
re-association is host f64 arithmetic, not a kernel property; the
spec and the m5-spec self-review are explicit. `REL_TOL = 1e-6` is
derived from the recursive-summation error bound with ~4 orders of
safety margin (shown in the test).
- **Pre-existing DESIGN.md drift out of scope.** `docs/DESIGN.md:2358-2360`
("additive M4 concern") is pre-existing M4-retirement drift, NOT
m5.3's to fix; the audit must not charge it to m5.3 (spec §"Out of
scope"). m5.3 ships zero `runtime/`/`crates/`/DESIGN.md change.
- **Milestone close next.** m5.3 is the last functional M5 iteration;
after Boss-verification the milestone closes and `audit` (mandatory)
runs separately.
## Self-review (planner Step 5)
1. **Spec coverage.** Architecture §3 time-shard second assertion →
Task 2/3 (per-shard bit-exact (a), Σ bit-exact (b), whole-window
tolerance (c)). Testing §3 → `tests/timeshard.rs`. Friction-harvest
(`:302-305`, Acceptance `:331-332`) → `TIMING` line + journal
record (Boss decision). Leak-Σ acceptance (`:323-324`) → the
leak-Σ block. Invariant 1 / no-surface-diff (`:309-314`,
`:328-330`) → Task 3 Step 5. skip-if-absent (`:326-327`) →
`skip_if_no_data`. Symbol-fan / hermetic smoke already shipped
(m5.2/m5.1) — not re-implemented; verified non-regressed in Task 3
Step 4. Audit runs separately (not an m5.3 task).
2. **Placeholder scan.** No "TBD/TODO/implement later/similar
to/add appropriate". `SYMBOL`/`SHARDS`/`REL_TOL` are concrete
pinned constants with recon-verified data + a derived tolerance,
not placeholders.
3. **Type/name consistency.** `fold_window`, `MidPriceStream`,
`Kernel`, `timeshard_runner`, `CARGO_BIN_EXE_timeshard_runner`,
`RESULT <idx> <from> <to> <accbits:016x> <n>` (written in Task 3 /
parsed in Task 2), `TIMING total_ticks=<N> elapsed_ms=<M>`,
`month_window`, `reference_window`, `SHARDS=[(2017,3),(2017,4),(2017,5)]`,
`SYMBOL="EURUSD"`, `REL_TOL=1e-6`,
`timeshard_boundary_invisibility_and_leak_free` — identical across
tasks and verification filters.
4. **Step granularity.** Each step is one file write or one command;
2-5 min. Largest paste (Task 2 Step 1) is a single verbatim test.
5. **No commit steps.** None. Work stays in the working tree.
6. **Pin/replacement substring contiguity.** The one verbatim
replacement (Task 1 Step 2, the `[dev-dependencies]`/`[[bin]]`
block) is not paired with any `contains(...)`/grep over its own
body. The greps (Task 3 Step 5) assert over `cargo metadata` /
`git status` output, not over a verbatim text body this plan
writes — no contiguity hazard.
7. **Compile-gate vs deferred-caller.** No signature change to an
existing function (only the additive `fold_window`; `fold_symbol`
and all callers untouched). Task 1 Step 3 uses `--lib` precisely
because the `[[bin]] timeshard_runner` path does not exist until
Task 3 — a full `cargo test` in Task 1 would fail to build the
declared bin. The full-suite/bin gate is correctly placed in
Task 3 Step 4 (after the bin exists). This ordering is deliberate
and internally consistent — `--lib` is satisfiable at Task 1
(bins are not built by `--lib`); the RED in Task 2 Step 2 is the
*intended* `env!` compile failure, not a plan defect.
8. **Verification filter strings resolve.** `--lib`,
`--test timeshard`, `--test embed_rc_global_stats_race`,
`timeshard_boundary_invisibility_and_leak_free` are all defined
*by this plan* (or are the shipped isolated RED) in the same
tasks that filter on them — resolve by construction. Task 3
Steps 4/5/6 use the unfiltered suite or explicit `grep -c … == 0`
/ empty-output / `1 passed` assertions, so "nothing ran" cannot
masquerade as success. Task 2 Step 2's "RED" is a *compile*
failure (deterministic, not a 0-filtered false-green).