90c1de841d
Public-API research deep-dive driving the just-shipped GER40 real-data
session-breakout demo (5b5f034) through the escalating sequence a serious
researcher attempts next: scale to multi-year, walk-forward, structural-param
sweep, multi-instrument compare. Standalone consumer crate (path-deps only;
public interface = ledger + glossary + example corpus + cargo doc; no
crates/*/src read). All runs against the real /mnt/tickdata/Pepperstone archive.
Findings: 3 bugs, 1 friction, 3 spec_gap, 1 working. Triaged to the tracker:
- spec_gap: the flagship breakout ships as a raw FlatGraph the World API cannot
consume (sweep / walk_forward / compare all forced a hand re-author to a
Composite, which reproduced the FlatGraph numbers EXACTLY) — refs #94.
- bug: process residency is O(window) (6->173 MiB across 1mo->10y) despite the
documented O(one-chunk) claim; the aura ring stays <=1024 but data-server
retention scales, and streaming_seam.rs only measures the ring — refs #95.
- bug: the bar-period is one knob split across Resample (tunable param) and
Session (baked) and desyncs silently to 0.0 pips; Delay.lag leaks into the
default param_space (the C34 deform-vs-tune discriminator) — refs #96.
- spec_gap: the walk_forward empty-space idiom for param-free strategies is
undefined — refs #97.
- spec_gap (no per-instrument pip registry) and friction (one-directional
ns->ms conversion) re-confirm the known #22 and #80.
- working: C1 determinism over 7y / 148k bars, the lazy streaming source, and
the sweep/WFO cores held under real multi-year load; the Composite re-author
was behaviour-preserving (C23).
Scratch consumer crate + run transcripts committed under fieldtests/ per the
fieldtest convention (matching cycle-0049). The crate path-deps the engine and
reads the real archive; its bins are the repros cited in the issues above.
144 lines
7.1 KiB
Rust
144 lines
7.1 KiB
Rust
// Task 3 (deep dive) — the silent period DESYNC.
|
|
//
|
|
// The coupling claim from t3_sweep: Resample's period IS a swept param, but
|
|
// Session's period is BAKED, so they cannot move together. This probe shows the
|
|
// SHARP edge of that: a researcher who sweeps `resample.period_minutes` to 30
|
|
// (expecting "30-minute bars") gets a graph where the BARS are 30m but the
|
|
// SESSION still indexes 15m slots. `entry_bar.target == 3` then no longer means
|
|
// "the 3rd 30m bar"; the two clocks silently disagree and the strategy computes a
|
|
// DIFFERENT, unintended thing — with NO error, NO warning. That is the danger of a
|
|
// coupling the param system cannot enforce.
|
|
//
|
|
// We run the SAME entry/exit-bar config at period_minutes = 15 (coupled, correct)
|
|
// vs period_minutes = 30 (desynced) and show both bootstrap+run happily, but the
|
|
// 30m run is semantically broken (Session still ticks 15m).
|
|
//
|
|
// PUBLIC SURFACE ONLY. The Composite + builders are the public re-authoring; the
|
|
// desync is a property of Session BAKING its period (its rustdoc) while Resample
|
|
// exposes its own.
|
|
|
|
use std::sync::mpsc;
|
|
use std::sync::Arc;
|
|
|
|
use aura_core::{Cell, Firing, Scalar, ScalarKind, Timestamp};
|
|
use aura_engine::{summarize, BlueprintNode, Composite, Edge, Role, RunManifest, RunReport, Target};
|
|
use aura_ingest::{M1Field, M1FieldSource};
|
|
use aura_std::{And, Delay, EqConst, Gt, Latch, Recorder, Resample, Session, SimBroker};
|
|
use chrono::TimeZone;
|
|
use chrono_tz::Europe::Berlin;
|
|
use data_server::{DataServer, DEFAULT_DATA_PATH};
|
|
|
|
const SYMBOL: &str = "GER40";
|
|
// The SESSION period is baked here — it never moves no matter what the sweep does
|
|
// to resample.period_minutes. THIS is the coupling the param system cannot keep.
|
|
const SESSION_PERIOD_BAKED: i64 = 15;
|
|
|
|
fn breakout(
|
|
session_period: i64,
|
|
) -> (Composite, mpsc::Receiver<(Timestamp, Vec<Scalar>)>, mpsc::Receiver<(Timestamp, Vec<Scalar>)>) {
|
|
let (tx_eq, rx_eq) = mpsc::channel();
|
|
let (tx_held, rx_held) = mpsc::channel();
|
|
let nodes: Vec<BlueprintNode> = vec![
|
|
Resample::builder().into(),
|
|
Delay::builder().into(),
|
|
Gt::builder().into(),
|
|
Session::builder(9, 0, Berlin, session_period).into(),
|
|
EqConst::builder().named("entry_bar").into(),
|
|
EqConst::builder().named("exit_bar").into(),
|
|
And::builder().into(),
|
|
Latch::builder().into(),
|
|
SimBroker::builder(1.0).into(),
|
|
Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_eq).into(),
|
|
Recorder::builder(vec![ScalarKind::F64], Firing::Any, tx_held).into(),
|
|
];
|
|
let edges = vec![
|
|
Edge { from: 0, to: 2, slot: 0, from_field: 3 },
|
|
Edge { from: 0, to: 1, slot: 0, from_field: 1 },
|
|
Edge { from: 1, to: 2, slot: 1, from_field: 0 },
|
|
Edge { from: 0, to: 3, slot: 0, from_field: 3 },
|
|
Edge { from: 3, to: 4, slot: 0, from_field: 0 },
|
|
Edge { from: 3, to: 5, slot: 0, from_field: 0 },
|
|
Edge { from: 2, to: 6, slot: 0, from_field: 0 },
|
|
Edge { from: 4, to: 6, slot: 1, from_field: 0 },
|
|
Edge { from: 6, to: 7, slot: 0, from_field: 0 },
|
|
Edge { from: 5, to: 7, slot: 1, from_field: 0 },
|
|
Edge { from: 7, to: 8, slot: 0, from_field: 0 },
|
|
Edge { from: 0, to: 8, slot: 1, from_field: 3 },
|
|
Edge { from: 8, to: 9, slot: 0, from_field: 0 },
|
|
Edge { from: 7, to: 10, slot: 0, from_field: 0 },
|
|
];
|
|
let roles = (0..4)
|
|
.map(|slot| Role {
|
|
name: ["open", "high", "low", "close"][slot as usize].into(),
|
|
targets: vec![Target { node: 0, slot }],
|
|
source: Some(ScalarKind::F64),
|
|
})
|
|
.collect();
|
|
(Composite::new("breakout", nodes, edges, roles, vec![]), rx_eq, rx_held)
|
|
}
|
|
|
|
fn run(server: &Arc<DataServer>, resample_period: i64, session_period: i64, from_ms: i64, to_ms: i64) -> (RunReport, usize) {
|
|
let (bp, rx_eq, rx_held) = breakout(session_period);
|
|
let space = bp.param_space();
|
|
// Build the param vector in param_space() order.
|
|
let point: Vec<Cell> = space
|
|
.iter()
|
|
.map(|ps| match ps.name.as_str() {
|
|
"resample.period_minutes" => Scalar::I64(resample_period).cell(),
|
|
"delay.lag" => Scalar::I64(1).cell(),
|
|
"entry_bar.target" => Scalar::I64(3).cell(),
|
|
"exit_bar.target" => Scalar::I64(5).cell(),
|
|
other => panic!("unexpected slot {other}"),
|
|
})
|
|
.collect();
|
|
let mut h = bp.bootstrap_with_cells(&point).expect("kind-checked");
|
|
let fields = [M1Field::Open, M1Field::High, M1Field::Low, M1Field::Close];
|
|
let srcs: Vec<Box<dyn aura_engine::Source>> = fields
|
|
.iter()
|
|
.map(|&f| Box::new(M1FieldSource::open(server, SYMBOL, Some(from_ms), Some(to_ms), f).unwrap()) as Box<dyn aura_engine::Source>)
|
|
.collect();
|
|
h.run(srcs);
|
|
drop(h);
|
|
let eq: Vec<(Timestamp, f64)> = rx_eq.try_iter().map(|(t, r)| (t, r[0].as_f64())).collect();
|
|
let n_bars = eq.len();
|
|
let ex: Vec<(Timestamp, f64)> = rx_held.try_iter().map(|(t, r)| (t, r[0].as_f64())).collect();
|
|
let rep = RunReport {
|
|
manifest: RunManifest { commit: "desync".into(), params: vec![], window: (Timestamp(0), Timestamp(0)), seed: 0, broker: "sim".into() },
|
|
metrics: summarize(&eq, &ex),
|
|
};
|
|
(rep, n_bars)
|
|
}
|
|
|
|
fn main() {
|
|
let server = Arc::new(DataServer::new(DEFAULT_DATA_PATH));
|
|
if !server.has_symbol(SYMBOL) {
|
|
println!("skip: no local {SYMBOL} data");
|
|
return;
|
|
}
|
|
let from = chrono::Utc.with_ymd_and_hms(2024, 9, 1, 0, 0, 0).unwrap().timestamp_millis();
|
|
let to = chrono::Utc.with_ymd_and_hms(2024, 10, 1, 0, 0, 0).unwrap().timestamp_millis() - 1;
|
|
|
|
println!("=== Task 3 deep dive: the silent period desync ===");
|
|
println!("entry_bar=3, exit_bar=5 fixed; Session period BAKED at {SESSION_PERIOD_BAKED}m.\n");
|
|
|
|
// Coupled (correct): resample 15m == session 15m. This is the shipped baseline.
|
|
let (coupled, n15) = run(&server, 15, SESSION_PERIOD_BAKED, from, to);
|
|
println!("resample=15m, session=15m (COUPLED, correct): bars={n15:<6} total_pips={:.1}", coupled.metrics.total_pips);
|
|
|
|
// Desynced: a researcher sweeps resample to 30m, EXPECTING 30m bars. Session
|
|
// still ticks 15m (baked). entry_bar==3 now indexes a 15m slot on a 30m bar
|
|
// stream — a different, unintended signal. NO error is raised.
|
|
let (desync, n30) = run(&server, 30, SESSION_PERIOD_BAKED, from, to);
|
|
println!("resample=30m, session=15m (DESYNCED, silent): bars={n30:<6} total_pips={:.1}", desync.metrics.total_pips);
|
|
|
|
// Contrast: to ACTUALLY get a coherent 30m strategy the researcher must REBUILD
|
|
// the Composite with session_period=30 (a different blueprint — topology, not a
|
|
// swept param). That run is coherent but is NOT reachable from the sweep.
|
|
let (rebuilt, n30b) = run(&server, 30, 30, from, to);
|
|
println!("resample=30m, session=30m (REBUILT blueprint): bars={n30b:<6} total_pips={:.1}", rebuilt.metrics.total_pips);
|
|
|
|
println!("\nObservation: all three bootstrap+run with NO error. The desynced");
|
|
println!("middle run is silently wrong — the param system cannot couple the two");
|
|
println!("periods, and only a blueprint rebuild (not a sweep) gives a coherent 30m.");
|
|
}
|