Compare commits

...

12 Commits

Author SHA1 Message Date
claude 84e1075176 fix(test): anchor the knob-lab scaffold under CARGO_TARGET_TMPDIR
built_scale_project() scaffolded ~66 MB (built node crate) into a
pid-suffixed tempdir, so its wipe-before-scaffold never hit a previous
run's copy: one leaked directory per test-binary run. 159 of those
(~10 GB) filled the host's 16 GB /tmp tmpfs to 100%, breaking every
std::env::temp_dir() consumer. A fixed name under CARGO_TARGET_TMPDIR
(real disk, reclaimed by cargo clean) makes the existing wipe reclaim
the previous run — steady state is exactly one directory. Trade-off:
two concurrent cargo-test invocations over the same checkout collide
on the fixed name, the documented pre-existing caveat class for this
suite (#223).

closes #257
2026-07-13 16:47:57 +02:00
claude a895891ab1 perf(ingest,cli): derive archive bounds from the monthly file index — probe_window drain retired
probe_window drained a source's entire close column just to learn its
first and last bar timestamp; both live callers (open_real_source and
campaign_window_ms on the campaign trunk) now resolve through
aura-ingest's new archive_extent(): list the symbol's SYM_YYYY_MM.m1
files, load only the boundary months via the existing loader seam, and
walk across gap months forward/backward — O(2 file loads) typical
instead of O(archive). Empty-window refusal semantics are unchanged
(pinned). Two characterization tests captured the resolved windows
against the OLD implementation over the synthetic per-test archive and
stay green unchanged after the swap.

Measured: the probe itself drops 366ms -> 13ms (~28x) on the full
2014-2026 GER40 archive; end-to-end command wall-clock moves only
~2-3% because the sim loop dominates — the fix removes an
O(archive-size) term, not the dominant cost.

closes #252
2026-07-13 16:47:57 +02:00
claude 479c6620e2 perf(cli): synthetic walkforward pre-flight validates axes without running members
blueprint_walkforward_family's pre-flight executed the full first-IS-
window sweep via blueprint_sweep_over and discarded the result, purely
to surface axis errors once before the parallel window fan-out (#177).
validate_axis_grid now drives the sweep terminal's own resolve/arity/
kind checks with a constant placeholder report — same single-sourced
rejection wording, no member run, no roller derivation needed.

Scope correction against the issue text: this path serves only the
synthetic (non---real) walkforward. The --real trunk routes through
verb_sugar into aura-campaign's [Sweep, WalkForward] process, whose
leading full-window sweep is a separate, larger duplication — re-filed
with corrected attribution in the issue's closing comment.

closes #253
2026-07-13 16:45:43 +02:00
claude a04fda3a4c fix(cli): resolve a relative paths.data against the project root
Env::data_path() handed paths.data to the data layer verbatim, so a
relative value resolved against the invoking process's cwd — aura run
from a project subdirectory silently read a different archive location
than from the root, while runs/ stayed anchored. Relative values now
join onto the project root, matching runs_root() and the [nodes]
pointer resolution; absolute values and the no-project default pass
through unchanged. RED-first unit test pins the root-anchored
resolution.

closes #254
2026-07-13 16:44:28 +02:00
claude c31e946bb7 test(cli): synthetic per-test M1 archive — no-window generalize tests go hostless
The two generalize E2E tests whose property is no-window span
resolution (shared-window fallback, intersection semantics) no longer
stream the 6.6 GB host archive: fresh_project_with_data() generates a
deterministic two-symbol M1 archive (SYMA 2024-01..08, SYMB 2024-03..06
strictly inside it, so the intersection differs from symbols[0]'s
window on both bounds) in the exact zip/48-byte-record format
data-server reads, plus geometry sidecars, under the per-test project's
paths.data. Both tests drop their local_data_present gates — they now
run on any host, data mount or not.

~35s and ~50s become 0.74s and 1.14s; the full cli_run binary lands at
~25s (211s at the branch base). Full workspace suite: 247s -> 45.7s.

refs #250
2026-07-13 16:44:28 +02:00
claude 25b8354959 test(cli): per-test project dirs via a [nodes] pointer — project_lock retired
Every E2E test that mutated the shared demo-project fixture store now
mints its own tempdir project whose 2-line Aura.toml points at the
once-built fixture crate by absolute [nodes] pointer; the runs/ store
follows the project root, so no two tests contend and libtest's
default thread parallelism applies. project_load keeps driving the
fixture directly (the suite's only proof of the inline-crate routing
arm); its single store-mutating test is that binary's only one, so no
lock is needed there either. This also closes the documented
cross-process race window (#223): there is no shared mutable store
left to race on.

Measured on the data-full host: cli_run 211s -> 57.2s (140 tests),
research_docs 21.1s -> 2.1s (73 tests), project_load green, clippy
clean.

refs #250
2026-07-13 16:43:16 +02:00
claude 00cc2fe927 build: compile dependencies at opt-level 2 in the dev profile
The E2E suite spawns the debug aura binary, whose wall-clock is
dominated by dependency code (zip inflate of the tick archive, sha2
over the project dylib, serde). Workspace crates stay at opt-level 0
for fast rebuilds and debugging; every pinned float is computed by
workspace code, so the byte-identity anchors are untouched.

refs #250
2026-07-13 16:42:18 +02:00
claude 038e1702b3 test(cli): bound three whale E2E windows instead of streaming full archives
The two dissolved-walkforward flag tests assert window-independent
properties (plateau acceptance, per-flag stop defaulting) and now use
the ~135-day window the reproduce e2e already uses. The mc roller-fit
test replaces its full-archive probe sweep with a fixed ~30-day window
inside that same span: the property needs a data-carrying window far
shorter than the 90+30-day roller, not a live-span derivation, and the
archive only grows forward. Measured on the data-full host: 19.6s,
19.6s and 14.5s+ of serialized real-data compute drop to 1.1-1.9s each.

refs #250
2026-07-13 16:42:18 +02:00
claude 1ebb94c1b8 feat(engine,cli): run manifests stamp untouched bound defaults (closes #249)
RunManifest gains defaults: Vec<(String, Scalar)> — the wrap-prefixed
bound_param_space() of the signal, read after axis reopening, so a
bound param an axis overrode has already left the space and flows
through params instead (disjoint by construction; verified end to end:
a sweep member's overridden fast.length sits in params while
slow.length/bias.scale sit in defaults). params keeps its "what varied"
semantics and stays the reproduce input. One-directional serde
widening (#[serde(default)]) per the selection/instrument/topology_hash
idiom — old records deserialize with an empty defaults; unlike the
Option fields it always serializes, mirroring params. ~20
struct-literal sites across five crates gained the field
(compile-mandated breadth, no behaviour change at those sites).

The C14 ledger records the underlying decision (2026-07-13): generated
outcome records spend redundancy on direct readability (single writer,
cannot drift); authored intent artifacts admit none (every redundancy
is a drift site) — so the fix lands in the manifest, never the
blueprint.

Verification: RED test run_manifest_stamps_untouched_bound_defaults
green; cargo build --workspace; cargo test --workspace green; clippy
-D warnings on the touched crates; binary-level sweep exclusivity
check.
2026-07-13 14:20:23 +02:00
claude 4e6d5d4688 test(cli): RED — run manifest stamps untouched bound defaults (refs #249)
Executable spec for the defaults field: aura run on the fully bound
examples/r_sma.json must report a manifest whose params stays [] (the
"what varied" record) and whose new defaults field carries the
untouched bound params as wrap-prefixed (name, Scalar) pairs in
bound_param_space() order — fast.length I64 2, slow.length I64 4,
bias.scale F64 0.5. Asserts on the stdout JSON (C14: stdout shape ==
on-disk shape), so it compiles against the current RunManifest and
fails on the absent key.
2026-07-13 13:41:21 +02:00
claude 487431d97d test(cli): migrate the _open references — closed twins for sweeps, fixtures for open semantics (closes #248)
The references to the relocated _open blueprints follow the plan's
two-way split: tests that only need a sweepable blueprint sweep the
closed twins via bound-override axes (#246) with their axis strings
byte-unchanged (the wrap prefix is the blueprint's internal name,
identical across twins); the 12 tests that genuinely exercise
open-param semantics (run/mc closed-guard refusals, subset-axes
MissingKnob, --list-axes open/bound line mix, gang dissolution and
campaign paths) read the fixtures under tests/fixtures/. The
research_docs/project_load store seeds sweep the closed twin (campaign
axis references resolve via the #246 reopen path); the INDEX.md history
note records the relocation.

Two deviations from the plan, both verified against the tree: doc
comments referencing the bare filename (no examples/ prefix) escaped
the global path swap and were reworded to stay truthful; the seed
variable open_bp was renamed closed_bp (the plan kept it, but the
rename touches only the binding and its uses and removes a
misdescription).

Verification: cargo test -p aura-cli --test cli_run (140 passed, 0
failed, real data exercised); --test research_docs; --test
project_load; the four byte-pinned exact grades unchanged; grep-clean
for examples/r_*_open.json across crates+docs; full workspace suite
green.
2026-07-13 13:36:31 +02:00
claude 6d3da424f4 test(cli): relocate the _open example twins to tests/fixtures (refs #248)
The four _open blueprints leave the user-facing gallery
(crates/aura-cli/examples/) and become test fixtures; the closed twins
are now the only example corpus (bound params are overridable defaults
since #246, so the closed files serve both run and sweep).

Compile-time embeds follow the move: the emit_* generators and the
shipped_*_serialize byte-pins target tests/fixtures/ (the open fixtures
stay builder-generated and byte-pinned — regenerating after the move
left the tree clean), load_open_r_sma and the axis-probe read the
fixture, and the walk-forward refit test plus the `aura graph` default
sample switch to the closed r_sma.json. graph_construct's open-fixture
tests swap example() -> fixture(), their names dropping the now-false
"shipped example" claim; a new e2e test pins the relocation property
(all four fixtures load and introspect, the gallery holds exactly the
four closed examples).

Intermediate state: cli_run.rs / research_docs.rs / project_load.rs
still reference the old example paths and fail at runtime until the
follow-up migration commit (MIGRATE sites move to the closed twins,
NEEDS-OPEN sites to the fixtures).

Verification: cargo build --workspace; cargo test -p aura-cli --bin
aura; cargo test -p aura-cli --test graph_construct; the three ignored
emit generators re-run byte-stable.
2026-07-13 11:42:03 +02:00
41 changed files with 1463 additions and 566 deletions
Generated
+2
View File
@@ -138,6 +138,7 @@ dependencies = [
"serde_json",
"sha2",
"toml",
"zip",
]
[[package]]
@@ -181,6 +182,7 @@ dependencies = [
"chrono",
"chrono-tz",
"data-server",
"zip",
]
[[package]]
+9
View File
@@ -36,3 +36,12 @@ serde = { version = "1", features = ["derive"] }
# parsed back. Exercised (with a 1-ULP canary value) by aura-cli's
# `f64_blueprint_param_survives_store_round_trip_bit_identically`.
serde_json = { version = "1", features = ["float_roundtrip"] }
# Dependencies at opt-level 2, workspace crates at the dev default (0): the E2E
# suite spawns the debug `aura` binary whose wall-clock is dominated by dependency
# code (zip inflate, sha2 dylib hashing, serde) — optimizing deps is multiplicative
# there, while workspace crates stay fast-to-rebuild and debuggable. Float pins are
# unaffected: every pinned float is computed by workspace code (IEEE semantics are
# opt-level-independent in Rust regardless).
[profile.dev.package."*"]
opt-level = 2
+1
View File
@@ -554,6 +554,7 @@ fn placeholder_report(params: &[(String, Scalar)], window_ms: (i64, i64)) -> Run
manifest: RunManifest {
commit: String::new(),
params: params.to_vec(),
defaults: Vec::new(),
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "faulted-member-placeholder".to_string(),
+2
View File
@@ -339,6 +339,7 @@ mod tests {
manifest: RunManifest {
commit: "test".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(0), Timestamp(1)),
seed: 0,
broker: "sim".to_string(),
@@ -1019,6 +1020,7 @@ mod wf_tests {
manifest: RunManifest {
commit: "wf-fake".to_string(),
params: params.to_vec(),
defaults: Vec::new(),
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "fake".to_string(),
+1
View File
@@ -58,6 +58,7 @@ fn planted_report(cell: &CellSpec, params: &[(String, Scalar)], window_ms: (i64,
manifest: RunManifest {
commit: "fake".to_string(),
params: params.to_vec(),
defaults: Vec::new(),
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "fake".to_string(),
@@ -46,6 +46,7 @@ impl MemberRunner for FixedRunner {
manifest: RunManifest {
commit: "e2e".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(window_ms.0), Timestamp(window_ms.1)),
seed: 0,
broker: "test".to_string(),
@@ -98,6 +99,7 @@ fn rankable_metrics_are_all_per_member_resolvable() {
manifest: RunManifest {
commit: "e2e".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(0), Timestamp(1)),
seed: 0,
broker: "test".to_string(),
+7
View File
@@ -53,3 +53,10 @@ toml = "0.8"
# long-option abbreviation. Research-side CLI only (invariant 8): a dev-loop
# compile tax, never a frozen-artifact tax.
clap = { version = "4", features = ["derive"] }
[dev-dependencies]
# zip: hand-packs a tiny synthetic M1 archive (tests/common/mod.rs) in the
# exact on-disk format data-server reads — already transitive via the
# data-server dependency above; declared directly here so test code may
# `use zip::...` (#250, no new dependency actually enters the build graph).
zip = "2"
+1
View File
@@ -1058,6 +1058,7 @@ mod tests {
manifest: RunManifest {
commit: "test".to_string(),
params,
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
+121 -33
View File
@@ -127,6 +127,7 @@ fn sim_optimal_manifest(
RunManifest {
commit: option_env!("AURA_COMMIT").unwrap_or("unknown").to_string(),
params,
defaults: Vec::new(),
window,
seed,
broker: format!("sim-optimal(pip_size={pip_size})"),
@@ -137,6 +138,24 @@ fn sim_optimal_manifest(
}
}
/// The wrap-prefixed BOUND-param defaults of `signal` (#249): every
/// `bound_param_space()` entry as a `(<signal.name()>.<param>, value)` pair, in
/// `bound_param_space()` order — the `RunManifest.defaults` field. Must be read
/// off `signal` BEFORE it is consumed by `wrap_r`/reopening: an axis-reopened
/// bound param has already left `bound_param_space()` by construction
/// (`Composite::reopen` forgets the bound value), so a caller that reopens
/// overrides before calling this naturally excludes them — no separate filter
/// needed. Same prefixing rule as `wrapped_bound_names`, kept separate because
/// that helper discards the value this one needs.
fn wrapped_bound_defaults(signal: &Composite) -> Vec<(String, Scalar)> {
let prefix = format!("{}.", signal.name());
signal
.bound_param_space()
.into_iter()
.map(|b| (format!("{prefix}{}", b.name), b.value))
.collect()
}
/// Map any byte outside the portable directory-name charset `[A-Za-z0-9._-]` to
/// `_`. The single source of filesystem-portability for an on-disk path component
/// (valid on Linux / Windows / macOS, also URL-path- and cloud-sync-safe). Used by
@@ -620,17 +639,8 @@ fn probe_window(
to_ms: Option<i64>,
env: &project::Env,
) -> (Timestamp, Timestamp) {
let mut probe = aura_ingest::open_columns(server, symbol, from_ms, to_ms, &[aura_ingest::M1Field::Close])
aura_ingest::archive_extent(server, std::path::Path::new(&env.data_path()), symbol, from_ms, to_ms)
.unwrap_or_else(|| no_data_in_window(symbol, from_ms, to_ms, env))
.pop()
.expect("open_columns yields one source per requested field");
let first =
aura_engine::Source::peek(probe.as_ref()).unwrap_or_else(|| no_data_in_window(symbol, from_ms, to_ms, env));
let mut last = first;
while let Some((t, _)) = aura_engine::Source::next(&mut *probe) {
last = t;
}
(first, last)
}
/// Open the real M1 sources for a recorded symbol over an optional window — one
@@ -1691,6 +1701,7 @@ fn run_signal_r(
.iter()
.map(|p| p.name.clone())
.collect();
let defaults = wrapped_bound_defaults(&signal); // read before `signal` is consumed below
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
@@ -1710,6 +1721,7 @@ fn run_signal_r(
let named_params: Vec<(String, Scalar)> =
names.into_iter().zip(params.iter().copied()).collect();
let mut manifest = sim_optimal_manifest(named_params, window, seed, pip_size);
manifest.defaults = defaults;
manifest.broker = r_sma_broker_label(pip_size);
manifest.topology_hash = Some(topo);
manifest.project = env.provenance();
@@ -1739,6 +1751,7 @@ fn run_blueprint_member(
binding: &binding::ResolvedBinding,
cost: &[aura_research::CostSpec],
) -> RunReport {
let defaults = wrapped_bound_defaults(&signal); // read before `signal` is consumed below
let (tx_eq, rx_eq) = mpsc::channel();
let (tx_ex, rx_ex) = mpsc::channel();
let (tx_r, rx_r) = mpsc::channel();
@@ -1779,6 +1792,7 @@ fn run_blueprint_member(
named.push((format!("cost[{k}].{knob}"), Scalar::f64(v)));
}
let mut manifest = sim_optimal_manifest(named, window, seed, pip);
manifest.defaults = defaults;
manifest.broker = r_sma_broker_label(pip);
manifest.topology_hash = Some(topo.to_string());
manifest.project = env.provenance();
@@ -2075,6 +2089,52 @@ fn run_oos_blueprint(
(Vec::new(), report)
}
/// A constant, zero-compute `RunReport` for [`validate_axis_grid`]'s sweep-terminal
/// probe. `SweepBinder::sweep_with_lattice`'s own `resolve_axes`/arity/kind checks all
/// run BEFORE this closure is invoked per grid point, so its body never influences the
/// validation outcome — only its signature (`Fn(&[Cell]) -> RunReport`) needs to
/// satisfy the terminal, at O(1) cost per point instead of a full member run.
fn axis_grid_probe_report() -> RunReport {
RunReport {
manifest: RunManifest {
commit: String::new(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "wf-axis-preflight-placeholder".to_string(),
selection: None,
instrument: None,
topology_hash: None,
project: None,
},
metrics: RunMetrics { total_pips: 0.0, max_drawdown: 0.0, bias_sign_flips: 0, r: None },
}
}
/// Validate the `--axis` grid against `doc`'s wrapped param space WITHOUT running any
/// member (#253). Reuses the SAME strict, erroring axis-name check
/// `blueprint_sweep_over` performs (`override_paths` — single-sourced, so the two
/// paths cannot drift to differently-worded rejections) to derive the #246 override
/// set, then drives the sweep terminal's own `resolve_axes`/arity/kind checks
/// (`SweepBinder::sweep_with_lattice`) with [`axis_grid_probe_report`] standing in for
/// the run closure — no data access, no sim engine tick. Axis resolution is
/// window-agnostic, so the caller derives or passes no window at all.
fn validate_axis_grid(
doc: &str, axes: &[(String, Vec<Scalar>)], raw_space: &[ParamSpec], probe_signal: &Composite,
env: &project::Env,
) -> Result<(), BindError> {
let overrides = override_paths(axes, raw_space, probe_signal).map_err(BindError::UnknownKnob)?;
let probe = blueprint_axis_probe_reopened(doc, env, &overrides);
let mut iter = axes.iter();
let (first_name, first_vals) = iter.next().expect("a blueprint walk-forward declares >= 1 axis");
let mut binder = probe.axis(first_name, first_vals.clone());
for (n, vals) in iter {
binder = binder.axis(n, vals.clone());
}
binder.sweep_with_lattice(|_| axis_grid_probe_report()).map(|_| ())
}
/// The loaded-blueprint IS-refit walk-forward: per IS window, re-optimize the
/// blueprint over the user `--axis` grid, select by `sqn_normalized`, run the
/// winner OOS, reusing the generic `walk_forward` driver + `select_winner`;
@@ -2124,15 +2184,10 @@ fn blueprint_walkforward_family(
// (which resolves its axes a single time before any member runs). `walk_forward` fans
// the per-window closure out across the windows in parallel, so a `BindError` raised
// *inside* the closure would `eprintln!`+`exit(2)` from several windows before any one
// exit lands — a racy, duplicated rejection (#177). Axis resolution is window-agnostic,
// so pre-flighting the first IS window here surfaces the error exactly once (it fails in
// `resolve_axes`, before any member runs); a per-window resolve then cannot re-raise.
let first_is = WindowRoller::new(span, is_len, oos_len, step, RollMode::Rolling)
.expect("roller config validated just above")
.next()
.expect("roller yields >= 1 window (validated above)")
.is;
if let Err(e) = blueprint_sweep_over(doc, axes, first_is.0, first_is.1, data, env, &binding) {
// exit lands — a racy, duplicated rejection (#177). Axis resolution is window-agnostic
// (#253): `validate_axis_grid` resolves the SAME grid the sweep terminal would, without
// running a single member — no IS window (or a second roller) is needed here at all.
if let Err(e) = validate_axis_grid(doc, axes, &raw_space, &probe_signal, env) {
eprintln!("aura: {e:?}");
std::process::exit(2);
}
@@ -3378,7 +3433,7 @@ fn dispatch_graph(a: GraphCmd, env: &project::Env) {
}
None if a.blueprint.is_none() => {
let bp = blueprint_from_json(
include_str!("../examples/r_sma_open.json"),
include_str!("../examples/r_sma.json"),
&|t| env.resolve(t),
)
.expect("the shipped r-sma example reloads into a renderable blueprint");
@@ -4053,10 +4108,10 @@ mod tests {
)
.expect("write examples/r_breakout.json");
std::fs::write(
"examples/r_breakout_open.json",
"tests/fixtures/r_breakout_open.json",
blueprint_to_json(&r_breakout_signal(None)).expect("serialize open r_breakout"),
)
.expect("write examples/r_breakout_open.json");
.expect("write tests/fixtures/r_breakout_open.json");
}
/// The shipped examples are a faithful serialisation of the carved signal (#159 cut 2):
@@ -4071,7 +4126,7 @@ mod tests {
);
assert_eq!(
blueprint_to_json(&r_breakout_signal(None)).expect("serialize open"),
include_str!("../examples/r_breakout_open.json"),
include_str!("../tests/fixtures/r_breakout_open.json"),
);
}
@@ -4104,10 +4159,10 @@ mod tests {
)
.expect("write examples/r_meanrev.json");
std::fs::write(
"examples/r_meanrev_open.json",
"tests/fixtures/r_meanrev_open.json",
blueprint_to_json(&r_meanrev_signal(None, None)).expect("serialize open r_meanrev"),
)
.expect("write examples/r_meanrev_open.json");
.expect("write tests/fixtures/r_meanrev_open.json");
}
/// Regenerates the shipped r_channel examples from the carved signal. Run by hand:
@@ -4123,10 +4178,10 @@ mod tests {
)
.expect("write examples/r_channel.json");
std::fs::write(
"examples/r_channel_open.json",
"tests/fixtures/r_channel_open.json",
blueprint_to_json(&r_channel_signal(None)).expect("serialize open r_channel"),
)
.expect("write examples/r_channel_open.json");
.expect("write tests/fixtures/r_channel_open.json");
}
/// The shipped examples are a faithful serialisation of the carved signal
@@ -4140,7 +4195,7 @@ mod tests {
);
assert_eq!(
blueprint_to_json(&r_channel_signal(None)).expect("serialize open"),
include_str!("../examples/r_channel_open.json"),
include_str!("../tests/fixtures/r_channel_open.json"),
);
}
@@ -4222,7 +4277,7 @@ mod tests {
);
assert_eq!(
blueprint_to_json(&r_meanrev_signal(None, None)).expect("open"),
include_str!("../examples/r_meanrev_open.json"),
include_str!("../tests/fixtures/r_meanrev_open.json"),
);
}
@@ -4718,7 +4773,7 @@ mod tests {
/// Loads the shipped open r-sma example (both SMA lengths free) through the
/// public `blueprint_from_json` path.
fn load_open_r_sma() -> Composite {
blueprint_from_json(include_str!("../examples/r_sma_open.json"), &|t| std_vocabulary(t))
blueprint_from_json(include_str!("../tests/fixtures/r_sma_open.json"), &|t| std_vocabulary(t))
.expect("loads")
}
@@ -5161,7 +5216,7 @@ mod tests {
// wraps the signal (name "sma_signal") so the names are prefixed —
// exactly what `--axis` binds.
let env = project::Env::std();
let open = include_str!("../examples/r_sma_open.json");
let open = include_str!("../tests/fixtures/r_sma_open.json");
let space = blueprint_axis_probe(open, &env).param_space();
let names: Vec<&str> = space.iter().map(|p| p.name.as_str()).collect();
assert_eq!(names, ["sma_signal.fast.length", "sma_signal.slow.length"]);
@@ -5174,9 +5229,10 @@ mod tests {
#[test]
fn blueprint_walkforward_family_refits_each_window() {
// The open fixture's two SMA lengths are re-fit per IS window over a 2x2 grid.
// The closed blueprint's two bound SMA lengths are re-fit per IS window over
// a 2x2 grid via the #246 bound-override reopen path (bound = overridable default).
let env = project::Env::std();
let doc = include_str!("../examples/r_sma_open.json");
let doc = include_str!("../examples/r_sma.json");
let axes = vec![
("sma_signal.fast.length".to_string(), vec![Scalar::i64(2), Scalar::i64(3)]),
("sma_signal.slow.length".to_string(), vec![Scalar::i64(4), Scalar::i64(6)]),
@@ -5496,6 +5552,38 @@ mod tests {
assert!(!err.contains("UnknownKnob"), "must not leak the debug render: {err}");
}
/// #249: an axis-reopened bound param flows through `params` ("what
/// varied") and must NOT also appear in `defaults` ("what was held") — the
/// two are disjoint by construction. Sweeping `fast.length` on the fully
/// bound r_sma example leaves `slow.length`/`bias.scale` untouched (they
/// stay in `defaults`), while `fast.length` moves to `params` and drops
/// out of `defaults` entirely.
#[test]
fn sweep_override_excludes_the_reopened_default_from_the_manifest() {
let env = project::Env::std();
let closed = load_closed_r_sma();
let doc = blueprint_to_json(&closed).expect("serializes");
let axes = vec![("sma_signal.fast.length".to_string(), vec![Scalar::i64(2)])];
let fam = blueprint_sweep_family(&doc, &axes, &DataSource::Synthetic, &env)
.expect("a bound param is a default — the axis overrides it");
assert_eq!(fam.points.len(), 1);
let manifest = &fam.points[0].report.manifest;
let param_names: Vec<&str> = manifest.params.iter().map(|(n, _)| n.as_str()).collect();
assert!(param_names.contains(&"sma_signal.fast.length"), "swept axis is a param: {param_names:?}");
let default_names: Vec<&str> = manifest.defaults.iter().map(|(n, _)| n.as_str()).collect();
assert!(
!default_names.contains(&"sma_signal.fast.length"),
"the reopened default must not also appear in defaults: {default_names:?}"
);
assert_eq!(
default_names,
["sma_signal.slow.length", "sma_signal.bias.scale"],
"the untouched bound params stay in defaults: {default_names:?}"
);
}
/// Property: an MC family's `aura reproduce` lines carry the member's own `seed=<N>`
/// label. MC members hold no tuning params (the params-join is empty), so the line would
/// otherwise print a BLANK member label; the seed is each draw's realization identity and
+29 -3
View File
@@ -182,12 +182,23 @@ impl Env {
TraceStore::open(self.runs_root())
}
/// The data archive root: `paths.data` inside a project when set,
/// the data-server default otherwise.
/// The data archive root: `paths.data` inside a project when set
/// (relative values resolved against the project root, same as
/// `runs_root()` and the `[nodes]` pointers), the data-server default
/// otherwise.
pub fn data_path(&self) -> String {
self.project
.as_ref()
.and_then(|p| p.toml.paths.data.clone())
.and_then(|p| {
p.toml.paths.data.as_ref().map(|data| {
let path = Path::new(data);
if path.is_absolute() {
data.clone()
} else {
p.root.join(path).to_string_lossy().to_string()
}
})
})
.unwrap_or_else(|| data_server::DEFAULT_DATA_PATH.to_string())
}
@@ -615,6 +626,21 @@ mod tests {
std::fs::remove_dir_all(&tmp).ok();
}
/// #254: a relative `paths.data` must resolve against the project root
/// (`p.root`), matching `runs_root()`'s and the `[nodes]` pointers'
/// resolution — not against the invoking process's cwd, which silently
/// changes the data location depending on the caller's working directory.
#[test]
fn data_path_resolves_a_relative_paths_data_against_the_project_root() {
let tmp = std::env::temp_dir().join(format!("aura-datarel-{}", std::process::id()));
std::fs::create_dir_all(&tmp).unwrap();
std::fs::write(tmp.join("Aura.toml"), "[paths]\ndata = \"data\"\n").unwrap();
let p = load(&tmp, false).expect("data-only load");
let env = Env::with_project(p);
assert_eq!(env.data_path(), tmp.join("data").to_string_lossy().to_string());
std::fs::remove_dir_all(&tmp).ok();
}
#[test]
fn two_nodes_pointers_refuse_with_multi_crate() {
let tmp = std::env::temp_dir().join(format!("aura-multi-{}", std::process::id()));
+1 -1
View File
@@ -236,7 +236,7 @@ mod tests {
fn render_html_is_self_contained_and_embeds_the_model() {
let env = crate::project::Env::std();
let bp = aura_engine::blueprint_from_json(
include_str!("../examples/r_sma_open.json"),
include_str!("../examples/r_sma.json"),
&|t| env.resolve(t),
)
.expect("the shipped r-sma example reloads into a renderable blueprint");
File diff suppressed because it is too large Load Diff
+222 -16
View File
@@ -3,14 +3,11 @@
//! and drive the `tests/fixtures/demo-project` fixture through the real
//! `aura` binary (#223).
//!
//! **Process-local serialization only.** [`project_lock`]'s `Mutex` is a
//! per-process `static`: it serializes test THREADS within one test-binary
//! process, but grants no cross-process exclusion. A process-parallel test
//! runner (e.g. `cargo-nextest`, which forks one process per test binary) —
//! or simply two concurrent `cargo test` invocations over the same working
//! tree — still race on the shared `tests/fixtures/demo-project/runs` store.
//! That boundary is a known, documented limitation (#223), not something
//! this module claims to close.
//! **Per-test project directories, no shared store.** [`fresh_project`] mints
//! a unique tempdir per test, wired to the shared, once-built fixture crate
//! via an absolute `[nodes]` pointer (`project.rs`'s pointer-arm routing), so
//! no two tests ever race on the same `runs/` store — libtest's default
//! thread parallelism applies without a serializing lock (#250).
//!
//! Each of the three test binaries compiles this file as its own `mod
//! common`, so an item unused by one binary trips that binary's `-D
@@ -18,7 +15,8 @@
//! rather than deleting a helper just because one binary doesn't need it.
use std::path::{Path, PathBuf};
use std::sync::{Mutex, MutexGuard, OnceLock};
use std::sync::OnceLock;
use std::sync::atomic::{AtomicUsize, Ordering};
/// The demo-project fixture (`Aura.toml` present), built once per test-binary
/// process. Each test binary has its own `OnceLock` (this module is compiled
@@ -42,13 +40,82 @@ pub fn built_project() -> &'static PathBuf {
})
}
/// Serializes every test that touches the shared demo-project fixture store
/// (they remove/re-seed `<fixture>/runs`, so parallel test threads would race
/// on it). A poisoned lock is taken over: one failed test must not cascade
/// into unrelated lock panics.
pub fn project_lock() -> MutexGuard<'static, ()> {
static LOCK: Mutex<()> = Mutex::new(());
LOCK.lock().unwrap_or_else(|e| e.into_inner())
/// A per-test project directory, removed on drop (including mid-test panic).
#[allow(dead_code)]
pub struct FreshProject {
root: PathBuf,
}
impl Drop for FreshProject {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.root);
}
}
/// Mints a unique, empty tempdir under `std::env::temp_dir()`, named
/// `aura-e2e-<pid>-<counter>` so concurrent test threads (same process) and
/// concurrent `cargo test` processes never collide.
fn mint_tempdir() -> PathBuf {
static COUNTER: AtomicUsize = AtomicUsize::new(0);
let root = std::env::temp_dir().join(format!(
"aura-e2e-{}-{}",
std::process::id(),
COUNTER.fetch_add(1, Ordering::Relaxed)
));
std::fs::create_dir_all(&root).expect("create fresh project tempdir");
root
}
/// Writes the `Aura.toml` + `demo_signal.json` common to every fresh project:
/// an absolute `[nodes]` pointer at the shared, once-built fixture crate, and
/// — when `data_dir` is `Some` — a `[paths] data` override pointing at it
/// (relative, resolved against the project root — `root` itself — since
/// `Env::data_path` joins a relative `paths.data` onto the project root,
/// same as `runs_root()` and the `[nodes]` pointers, #254).
fn write_project_files(root: &Path, data_dir: Option<&str>) {
let fixture = built_project();
let data_line = data_dir.map(|d| format!("data = \"{d}\"\n")).unwrap_or_default();
std::fs::write(
root.join("Aura.toml"),
format!(
"[paths]\nruns = \"runs\"\n{data_line}\n[nodes]\ncrates = [\"{}\"]\n",
fixture.display()
),
)
.expect("write fresh project Aura.toml");
std::fs::copy(fixture.join("demo_signal.json"), root.join("demo_signal.json"))
.expect("copy demo_signal.json into the fresh project");
}
/// Mint a unique tempdir holding a 2-line data-level project (`Aura.toml`
/// with an absolute `[nodes]` pointer at the shared, once-built fixture
/// crate) plus a copy of `demo_signal.json`. Every test gets its own `runs/`
/// store under this root, so no two tests ever contend for the same store —
/// this is what lets `project_lock` disappear (#250). Bind the guard for the
/// whole test body: `let (dir, _g) = fresh_project();`.
#[allow(dead_code)]
pub fn fresh_project() -> (PathBuf, FreshProject) {
let root = mint_tempdir();
write_project_files(&root, None);
(root.clone(), FreshProject { root })
}
/// Like [`fresh_project`], but additionally generates a tiny synthetic M1
/// archive under `<root>/data/` (two symbols, `SYMA` spanning 2024-01..08 and
/// `SYMB` spanning 2024-03..06 — `SYMB` lies strictly inside `SYMA`) and
/// points `[paths] data` at it. For tests whose property is archive-window
/// (span) semantics, not archive size — no host tick-data mount needed, and
/// no `--real`-symbol data refusal is ever reachable, so gates on data
/// presence are dead code once a test switches to this helper (#250).
#[allow(dead_code)]
pub fn fresh_project_with_data() -> (PathBuf, FreshProject) {
let root = mint_tempdir();
write_project_files(&root, Some("data"));
let data_dir = root.join("data");
std::fs::create_dir_all(&data_dir).expect("create synthetic archive dir");
synthetic_data::write_symbol_archive(&data_dir, "SYMA", (2024, 1), (2024, 8));
synthetic_data::write_symbol_archive(&data_dir, "SYMB", (2024, 3), (2024, 6));
(root.clone(), FreshProject { root })
}
/// A scratch filesystem entry a test writes under the git-tracked
@@ -78,3 +145,142 @@ impl Drop for ScratchGuard {
}
}
}
/// A tiny, deterministic synthetic M1 archive generator: hand-packs the exact
/// on-disk format `data-server` reads (its `loader.rs` and `records.rs`) —
/// one zip per `SYMBOL_YYYY_MM.m1` holding a single `.bin` entry of packed
/// 48-byte `RawM1Record`s, plus a `SYMBOL.meta.json` geometry sidecar — so
/// the two no-window `generalize` E2E tests get a real, tiny, hostless
/// archive instead of streaming the 6.6 GB host archive (#250).
#[allow(dead_code)]
mod synthetic_data {
use std::io::Write;
use std::path::Path;
/// Days since the Unix epoch for a proleptic-Gregorian civil date.
/// Howard Hinnant's `days_from_civil` — dependency-free calendar math so
/// this module needs no `chrono` (data-server's own dependency, not
/// aura-cli's).
fn days_from_civil(y: i32, m: u32, d: u32) -> i64 {
let y = i64::from(if m <= 2 { y - 1 } else { y });
let era = if y >= 0 { y } else { y - 399 } / 400;
let yoe = y - era * 400; // [0, 399]
let mp = (i64::from(m) + 9) % 12; // [0, 11], Mar=0 .. Feb=11
let doy = (153 * mp + 2) / 5 + i64::from(d) - 1; // [0, 365]
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; // [0, 146096]
era * 146_097 + doe - 719_468
}
/// Unix milliseconds at `y-m-d hh:mm:00` UTC.
fn unix_ms(y: i32, m: u32, d: u32, hh: u32, mm: u32) -> i64 {
days_from_civil(y, m, d) * 86_400_000 + (i64::from(hh) * 3600 + i64::from(mm) * 60) * 1000
}
/// 0 = Sunday .. 6 = Saturday (1970-01-01 was a Thursday, day 0).
fn weekday(y: i32, m: u32, d: u32) -> i64 {
(days_from_civil(y, m, d) % 7 + 4 + 7) % 7
}
fn days_in_month(y: i32, m: u32) -> u32 {
match m {
1 | 3 | 5 | 7 | 8 | 10 | 12 => 31,
4 | 6 | 9 | 11 => 30,
2 if (y % 4 == 0 && y % 100 != 0) || y % 400 == 0 => 29,
2 => 28,
_ => unreachable!("month out of range: {m}"),
}
}
/// A deterministic price at continuous minute offset `t` (minutes since
/// 2024-01-01T00:00 UTC, shared across both symbols so their overlapping
/// span carries the same underlying curve): a gentle upward trend plus a
/// 180-minute sine — short enough against a day's 480 tradeable minutes to
/// produce several SMA(3)/SMA(12) crossovers per day.
fn price_at(t_minutes: f64) -> f64 {
let hours = t_minutes / 60.0;
100.0 + 0.01 * hours + 5.0 * (2.0 * std::f64::consts::PI * t_minutes / 180.0).sin()
}
/// Packs one `RawM1Record`-shaped 48-byte little-endian record: `time`
/// (Delphi `TDateTime`, days since 1899-12-30 — the inverse of
/// `data_server::records::delphi_to_unix_ms`), `open/high/low/close`,
/// `spread` (f32), `volume` (i32).
fn pack_record(unix_ms_time: i64, open: f64, high: f64, low: f64, close: f64) -> [u8; 48] {
const DELPHI_EPOCH_OFFSET_DAYS: f64 = 25569.0;
const MS_PER_DAY: f64 = 86_400_000.0;
let dt = unix_ms_time as f64 / MS_PER_DAY + DELPHI_EPOCH_OFFSET_DAYS;
let mut rec = [0u8; 48];
rec[0..8].copy_from_slice(&dt.to_le_bytes());
rec[8..16].copy_from_slice(&open.to_le_bytes());
rec[16..24].copy_from_slice(&high.to_le_bytes());
rec[24..32].copy_from_slice(&low.to_le_bytes());
rec[32..40].copy_from_slice(&close.to_le_bytes());
rec[40..44].copy_from_slice(&0.5_f32.to_le_bytes());
rec[44..48].copy_from_slice(&100_i32.to_le_bytes());
rec
}
/// One month's worth of weekday, 08:00-16:00 UTC, 1-minute bars (~10k).
fn month_records(year: i32, month: u32) -> Vec<[u8; 48]> {
let epoch_ms = unix_ms(2024, 1, 1, 0, 0);
let mut records = Vec::new();
for day in 1..=days_in_month(year, month) {
if !(1..=5).contains(&weekday(year, month, day)) {
continue; // weekend
}
for hh in 8..16 {
for mm in 0..60 {
let ms = unix_ms(year, month, day, hh, mm);
let t = (ms - epoch_ms) as f64 / 60_000.0;
let (open, close) = (price_at(t), price_at(t + 1.0));
let high = open.max(close) + 0.2;
let low = open.min(close) - 0.2;
records.push(pack_record(ms, open, high, low, close));
}
}
}
records
}
/// Writes one `SYMBOL_YYYY_MM.m1` zip (a single `<SYMBOL>.bin` entry of
/// packed records — mirrors `data-server`'s own `create_test_m1_zip` test
/// helper) under `data_dir`.
fn write_month_zip(data_dir: &Path, symbol: &str, year: i32, month: u32) {
let path = data_dir.join(format!("{symbol}_{year:04}_{month:02}.m1"));
let file = std::fs::File::create(&path).expect("create synthetic m1 zip");
let mut zip = zip::ZipWriter::new(file);
zip.start_file::<String, ()>(format!("{symbol}.bin"), Default::default())
.expect("start synthetic m1 zip entry");
for rec in month_records(year, month) {
zip.write_all(&rec).expect("write synthetic m1 record");
}
zip.finish().expect("finish synthetic m1 zip");
}
/// Writes every month in `[from, to]` (inclusive, `(year, month)` pairs)
/// for `symbol`, plus its `<SYMBOL>.meta.json` geometry sidecar —
/// GER40-shaped (digits 1, pip/tick/lot sized like an index CFD) so the
/// R/stop machinery never hits "no recorded geometry".
pub fn write_symbol_archive(data_dir: &Path, symbol: &str, from: (i32, u32), to: (i32, u32)) {
let (mut y, mut m) = from;
loop {
write_month_zip(data_dir, symbol, y, m);
if (y, m) == to {
break;
}
m += 1;
if m > 12 {
m = 1;
y += 1;
}
}
std::fs::write(
data_dir.join(format!("{symbol}.meta.json")),
format!(
"{{\"schemaVersion\":2,\"symbol\":\"{symbol}\",\"digits\":1,\"pipSize\":1,\
\"tickSize\":0.1,\"lotSize\":1,\"baseAsset\":\"{symbol}\",\"quoteAsset\":\"EUR\"}}"
),
)
.expect("write synthetic geometry sidecar");
}
}
+41 -16
View File
@@ -471,7 +471,7 @@ fn graph_register_is_idempotent() {
fn graph_params_lists_raw_axis_namespace() {
let dir = temp_cwd("params");
let (stdout, stderr, code) =
run_in(&dir, &["graph", "introspect", "--params", &example("r_sma_open.json")]);
run_in(&dir, &["graph", "introspect", "--params", &fixture("r_sma_open.json")]);
assert_eq!(code, Some(0), "stdout: {stdout} stderr: {stderr}");
assert_eq!(stdout, "fast.length:I64\nslow.length:I64\n", "the raw open params, in order");
}
@@ -512,7 +512,7 @@ fn graph_introspect_mode_guard_still_exits_two_on_zero_or_two_modes() {
#[test]
fn graph_params_by_content_id_matches_params_by_file() {
let dir = temp_cwd("params-by-id");
let bp = example("r_sma_open.json");
let bp = fixture("r_sma_open.json");
let (reg_out, reg_err, reg_code) = run_in(&dir, &["graph", "register", &bp]);
assert_eq!(reg_code, Some(0), "register: {reg_out} {reg_err}");
let id = registered_id(&reg_out);
@@ -533,7 +533,7 @@ fn graph_params_by_content_id_matches_params_by_file() {
#[test]
fn graph_params_tolerates_content_prefix_on_target() {
let dir = temp_cwd("params-content-prefix");
let bp = example("r_sma_open.json");
let bp = fixture("r_sma_open.json");
let (reg_out, reg_err, reg_code) = run_in(&dir, &["graph", "register", &bp]);
assert_eq!(reg_code, Some(0), "register: {reg_out} {reg_err}");
let id = registered_id(&reg_out);
@@ -744,7 +744,7 @@ fn shipped_r_sma_example_is_genuinely_closed() {
assert_eq!(stdout, "", "the closed example must leave zero params unbound");
}
/// Property (#159): the shipped open example (`examples/r_sma_open.json`) is a
/// Property (#159): the open fixture (`tests/fixtures/r_sma_open.json`) is a
/// usable sweep-axis template from its public gallery location, not just a
/// byte-identical artifact — `graph introspect --params` lists exactly the two
/// unbound SMA lengths, in lowering order, matching the raw axis namespace the
@@ -753,10 +753,10 @@ fn shipped_r_sma_example_is_genuinely_closed() {
/// shipped copy is actually introspectable/bindable from where a consumer runs
/// it.
#[test]
fn shipped_r_sma_open_example_lists_its_axis_namespace() {
fn open_r_sma_fixture_lists_its_axis_namespace() {
let dir = temp_cwd("example-open-params");
let (stdout, stderr, code) =
run_in(&dir, &["graph", "introspect", "--params", &example("r_sma_open.json")]);
run_in(&dir, &["graph", "introspect", "--params", &fixture("r_sma_open.json")]);
assert_eq!(code, Some(0), "stdout: {stdout} stderr: {stderr}");
assert_eq!(
stdout, "fast.length:I64\nslow.length:I64\n",
@@ -764,6 +764,31 @@ fn shipped_r_sma_open_example_lists_its_axis_namespace() {
);
}
/// Property (#248): the four `_open` blueprint twins were RELOCATED out of the
/// public gallery, not deleted — each stem is present under `tests/fixtures/`
/// (the internal sweep-axis fixture location) and absent from `examples/` (the
/// public, user-facing gallery). This is the acceptance criterion's core claim
/// ("`examples/` ships no `_open` variants") stated as a filesystem fact, so a
/// future example addition that reintroduces a twin — or a migration that moves
/// the reference but forgets the file — fails here even though every
/// `include_str!`/path-string call site already resolves.
#[test]
fn open_twins_are_relocated_from_examples_to_fixtures_not_deleted() {
let manifest_dir = env!("CARGO_MANIFEST_DIR");
for stem in ["r_sma", "r_breakout", "r_meanrev", "r_channel"] {
let gallery_path = format!("{manifest_dir}/examples/{stem}_open.json");
assert!(
!std::path::Path::new(&gallery_path).exists(),
"{stem}_open.json must not ship in the public examples/ gallery (#248)"
);
let fixture_path = format!("{manifest_dir}/tests/fixtures/{stem}_open.json");
assert!(
std::path::Path::new(&fixture_path).exists(),
"{stem}_open.json must survive relocated under tests/fixtures/ (#248)"
);
}
}
/// Property (#159 cut 2): the shipped closed example (`examples/r_breakout.json`) is
/// genuinely closed — `graph introspect --params` reports zero unbound params.
#[test]
@@ -775,14 +800,14 @@ fn shipped_r_breakout_example_is_genuinely_closed() {
assert_eq!(stdout, "", "the closed example must leave zero params unbound");
}
/// Property (#159 cut 2): the shipped open example (`examples/r_breakout_open.json`)
/// Property (#159 cut 2): the open fixture (`tests/fixtures/r_breakout_open.json`)
/// lists its ONE ganged channel axis (#61: the two rolling windows are structurally
/// one knob).
#[test]
fn shipped_r_breakout_open_example_lists_its_axis_namespace() {
fn open_r_breakout_fixture_lists_its_axis_namespace() {
let dir = temp_cwd("r-breakout-example-open-params");
let (stdout, stderr, code) =
run_in(&dir, &["graph", "introspect", "--params", &example("r_breakout_open.json")]);
run_in(&dir, &["graph", "introspect", "--params", &fixture("r_breakout_open.json")]);
assert_eq!(code, Some(0), "stdout: {stdout} stderr: {stderr}");
assert_eq!(
stdout, "channel_length:I64\n",
@@ -805,7 +830,7 @@ fn shipped_r_breakout_open_example_lists_its_axis_namespace() {
/// member) would be caught here even if the internal builder-level
/// equivalence still holds.
#[test]
fn shipped_r_breakout_open_gang_axis_matches_the_closed_example() {
fn open_r_breakout_fixture_gang_axis_matches_the_closed_example() {
let closed_dir = temp_cwd("r-breakout-gang-axis-closed");
let (closed_stdout, closed_stderr, closed_code) = run_in(&closed_dir, &["run", &example("r_breakout.json")]);
assert_eq!(closed_code, Some(0), "closed run stderr: {closed_stderr}");
@@ -815,7 +840,7 @@ fn shipped_r_breakout_open_gang_axis_matches_the_closed_example() {
let sweep_dir = temp_cwd("r-breakout-gang-axis-sweep");
let (sweep_stdout, sweep_stderr, sweep_code) = run_in(
&sweep_dir,
&["sweep", &example("r_breakout_open.json"), "--axis", "r_breakout_signal.channel_length=3"],
&["sweep", &fixture("r_breakout_open.json"), "--axis", "r_breakout_signal.channel_length=3"],
);
assert_eq!(sweep_code, Some(0), "sweep stderr: {sweep_stderr}");
let lines: Vec<&str> = sweep_stdout.lines().collect();
@@ -840,13 +865,13 @@ fn shipped_r_meanrev_example_is_genuinely_closed() {
assert_eq!(stdout, "", "the closed example must leave zero params unbound");
}
/// Property (#159 cut 3): the shipped open example (`examples/r_meanrev_open.json`)
/// Property (#159 cut 3): the open fixture (`tests/fixtures/r_meanrev_open.json`)
/// lists its raw axis namespace, in lowering (node-add) order.
#[test]
fn shipped_r_meanrev_open_example_lists_its_axis_namespace() {
fn open_r_meanrev_fixture_lists_its_axis_namespace() {
let dir = temp_cwd("r-meanrev-example-open-params");
let (stdout, stderr, code) =
run_in(&dir, &["graph", "introspect", "--params", &example("r_meanrev_open.json")]);
run_in(&dir, &["graph", "introspect", "--params", &fixture("r_meanrev_open.json")]);
assert_eq!(code, Some(0), "stdout: {stdout} stderr: {stderr}");
assert_eq!(
stdout, "window:I64\nband.factor:F64\n",
@@ -863,7 +888,7 @@ fn shipped_r_meanrev_open_example_lists_its_axis_namespace() {
/// param positions could silently mis-bind the co-present un-ganged axis
/// instead (or vice-versa) even though each axis alone still resolves.
#[test]
fn shipped_r_meanrev_open_gang_plus_plain_axis_matches_the_closed_example() {
fn open_r_meanrev_fixture_gang_plus_plain_axis_matches_the_closed_example() {
let closed_dir = temp_cwd("r-meanrev-gang-axis-closed");
let (closed_stdout, closed_stderr, closed_code) = run_in(&closed_dir, &["run", &example("r_meanrev.json")]);
assert_eq!(closed_code, Some(0), "closed run stderr: {closed_stderr}");
@@ -874,7 +899,7 @@ fn shipped_r_meanrev_open_gang_plus_plain_axis_matches_the_closed_example() {
let (sweep_stdout, sweep_stderr, sweep_code) = run_in(
&sweep_dir,
&[
"sweep", &example("r_meanrev_open.json"),
"sweep", &fixture("r_meanrev_open.json"),
"--axis", "r_meanrev_signal.window=3",
"--axis", "r_meanrev_signal.band.factor=2.0",
],
+4 -5
View File
@@ -9,7 +9,7 @@ use std::process::{Command, Output};
use std::time::{Duration, SystemTime, UNIX_EPOCH};
mod common;
use common::{built_project, project_lock};
use common::built_project;
fn fixture_dir() -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/demo-project")
@@ -98,17 +98,16 @@ fn outside_a_project_the_demo_blueprint_is_unknown() {
/// rather than shell-relative (C17).
#[test]
fn project_registry_anchors_at_discovered_root_not_invocation_cwd() {
let _fixture = project_lock();
let dir = built_project();
let sub = dir.join("src");
let runs_dir = dir.join("runs");
std::fs::remove_dir_all(&runs_dir).ok();
let open_bp =
format!("{}/examples/r_sma_open.json", env!("CARGO_MANIFEST_DIR"));
let closed_bp =
format!("{}/examples/r_sma.json", env!("CARGO_MANIFEST_DIR"));
let out = Command::new(env!("CARGO_BIN_EXE_aura"))
.args([
"sweep",
&open_bp,
&closed_bp,
"--axis",
"sma_signal.fast.length=2,4",
"--axis",
@@ -63,11 +63,15 @@ fn aura_in(dir: &Path, args: &[&str]) -> Output {
/// `blueprints/scaled_open.json`. Returns the project dir. `OnceLock` so the
/// expensive scaffold+build happens once per test binary; the node crate's
/// path-dep resolves into this checkout (the baked engine root), like
/// `project_new.rs`.
/// `project_new.rs`. The base is a FIXED name under `CARGO_TARGET_TMPDIR`
/// (real disk, reclaimed by `cargo clean`), not a pid-suffixed tempdir: the
/// wipe below must hit the PREVIOUS run's copy, else every test-binary run
/// leaks the ~66 MB built scaffold — 159 of those filled the 16 GB /tmp
/// tmpfs to 100 % (#257).
fn built_scale_project() -> &'static PathBuf {
static BUILT: OnceLock<PathBuf> = OnceLock::new();
BUILT.get_or_init(|| {
let base = std::env::temp_dir().join(format!("aura-235-{}", std::process::id()));
let base = Path::new(env!("CARGO_TARGET_TMPDIR")).join("aura-235-knob-lab");
let _ = std::fs::remove_dir_all(&base);
std::fs::create_dir_all(&base).expect("create scaffold base");
File diff suppressed because it is too large Load Diff
+1
View File
@@ -1763,6 +1763,7 @@ mod tests {
manifest: RunManifest {
commit: "test".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
+2
View File
@@ -239,6 +239,7 @@ mod tests {
manifest: RunManifest {
commit: "test".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window,
seed,
broker: "test".to_string(),
@@ -266,6 +267,7 @@ mod tests {
manifest: RunManifest {
commit: "t".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: i as u64,
broker: "t".to_string(),
+20 -6
View File
@@ -34,6 +34,16 @@ pub struct RunManifest {
/// self-describing [`Scalar`], so the param's kind (an `i64` length vs an
/// `f64` scale) survives into the record instead of collapsing to `f64`.
pub params: Vec<(String, Scalar)>,
/// The bound params this run did NOT vary — the wrap-prefixed
/// `bound_param_space()` of the signal, EXCLUDING any bound param an axis
/// reopened (those flow through `params` instead, #246). Complements
/// `params` ("what varied") with "what was held": the two are disjoint by
/// construction. Empty for every pre-#249 record. One-directional serde
/// widening (C14/C18), identical idiom to `selection`/`instrument`/
/// `topology_hash` — except a `Vec`, not an `Option`, so it always
/// serializes (mirroring `params`), never skipped when empty.
#[serde(default)]
pub defaults: Vec<(String, Scalar)>,
/// The data-window: inclusive `(from, to)` epoch-ns bounds (C12).
pub window: (Timestamp, Timestamp),
/// The RNG seed (C12 seed-as-input). `0` for a seed-free synthetic run.
@@ -298,7 +308,7 @@ mod tests {
#[test]
fn runmanifest_without_selection_serialises_without_the_key() {
let m = RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: None, instrument: None,
topology_hash: None,
project: None,
@@ -317,7 +327,7 @@ mod tests {
#[test]
fn runmanifest_instrument_round_trips_and_omits_when_none() {
let with = RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: None, instrument: Some("GER40".into()),
topology_hash: None,
project: None,
@@ -328,7 +338,7 @@ mod tests {
assert_eq!(back.instrument, Some("GER40".to_string()));
let without = RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: None, instrument: None,
topology_hash: None,
project: None,
@@ -370,7 +380,7 @@ mod tests {
commit: Some("deadbeef".into()),
};
let m = RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: None, instrument: None,
topology_hash: None,
project: Some(p.clone()),
@@ -393,7 +403,7 @@ mod tests {
neighbourhood_score: None, n_neighbours: None,
};
let m = RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: Some(sel.clone()), instrument: None,
topology_hash: None,
project: None,
@@ -531,6 +541,7 @@ mod tests {
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(0.5)),
],
defaults: vec![],
window: (Timestamp(1), Timestamp(5)),
seed: 0,
broker: "sim-optimal(pip_size=0.0001)".to_string(),
@@ -637,6 +648,7 @@ mod tests {
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(1.0)),
],
defaults: vec![],
window: (Timestamp(1), Timestamp(6)),
seed: 0,
broker: "sim-optimal(pip_size=1.0)".to_string(),
@@ -654,7 +666,7 @@ mod tests {
};
assert_eq!(
report.to_json(),
r#"{"manifest":{"commit":"abc123","params":[["sma_fast",{"I64":2}],["sma_slow",{"I64":4}],["bias_scale",{"F64":1.0}]],"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12.0,"max_drawdown":1.0,"bias_sign_flips":1}}"#,
r#"{"manifest":{"commit":"abc123","params":[["sma_fast",{"I64":2}],["sma_slow",{"I64":4}],["bias_scale",{"F64":1.0}]],"defaults":[],"window":[1,6],"seed":0,"broker":"sim-optimal(pip_size=1.0)"},"metrics":{"total_pips":12.0,"max_drawdown":1.0,"bias_sign_flips":1}}"#,
);
}
@@ -669,6 +681,7 @@ mod tests {
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(1.0)),
],
defaults: vec![],
window: (Timestamp(1), Timestamp(6)),
seed: 0,
broker: "sim-optimal(pip_size=1.0)".to_string(),
@@ -693,6 +706,7 @@ mod tests {
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(1.0)),
],
defaults: vec![],
window: (Timestamp(1), Timestamp(6)),
seed: 0,
broker: "sim-optimal(pip_size=1.0)".to_string(),
+2
View File
@@ -722,6 +722,7 @@ mod tests {
manifest: RunManifest {
commit: "test".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
@@ -877,6 +878,7 @@ mod tests {
manifest: RunManifest {
commit: point.iter().map(|c| c.i64().to_string()).collect::<Vec<_>>().join(","),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
+1
View File
@@ -288,6 +288,7 @@ mod tests {
manifest: RunManifest {
commit: "t".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "t".to_string(),
@@ -99,6 +99,7 @@ fn run_point(point: &[Cell]) -> RunReport {
manifest: RunManifest {
commit: "list-sweep-e2e".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
@@ -112,6 +112,7 @@ fn run_point(point: &[Cell]) -> RunReport {
manifest: RunManifest {
commit: "random-sweep-e2e".to_string(),
params: Vec::new(),
defaults: Vec::new(),
window: (Timestamp(0), Timestamp(0)),
seed: 0,
broker: "test".to_string(),
+7
View File
@@ -16,6 +16,13 @@ data-server = { git = "http://192.168.178.103:3000/Brummel/data-server.git", bra
aura-engine = { path = "../aura-engine" }
[dev-dependencies]
# zip: hand-packs a tiny synthetic M1 archive (lib.rs's own unit tests, #252's
# archive_extent gap-walk coverage) in the exact on-disk format data-server
# reads — already transitive via the data-server dependency above (a normal,
# non-dev dependency there); declared directly here so test code may `use
# zip::...` (mirrors aura-cli's tests/common/mod.rs, no new dependency
# actually enters the build graph).
zip = "2"
# the integration tests bootstrap a sample harness + fold a RunReport
aura-std = { path = "../aura-std" }
# chrono / chrono-tz build the Berlin-local-wall-clock window bounds + the
@@ -63,6 +63,7 @@ fn run_point(server: &Arc<DataServer>, point: &[Cell], from: Timestamp, to: Time
manifest: RunManifest {
commit: "ger40-breakout-sweep".to_string(),
params: vec![],
defaults: vec![],
window: (from, to),
seed: 0,
broker: "sim-optimal(pip_size=1)".to_string(),
@@ -73,6 +73,7 @@ fn run_point(
manifest: RunManifest {
commit: "ger40-breakout-wfo".to_string(),
params: vec![],
defaults: vec![],
window: (from, to),
seed: 0,
broker: "sim-optimal(pip_size=1)".to_string(),
@@ -396,6 +396,7 @@ pub fn report_from_trace(trace: &[BarTrace], from: Timestamp, to: Timestamp) ->
("entry_bar".to_string(), Scalar::i64(3)),
("exit_bar".to_string(), Scalar::i64(5)),
],
defaults: vec![],
window: (from, to),
seed: 0,
broker: "sim-optimal(pip_size=1)".to_string(),
+312
View File
@@ -23,6 +23,7 @@
use aura_core::{Scalar, Timestamp};
use data_server::records::M1Parsed;
use data_server::SymbolChunkIter;
use std::path::Path;
/// Re-export of the data-server archive entry points (#81): a real-data source
/// builds from `aura-ingest` alone — a consumer never names the external
@@ -411,10 +412,168 @@ pub fn instrument_geometry(server: &Arc<DataServer>, symbol: &str) -> Option<Ins
server.symbol_meta(symbol)
}
/// Days since 1970-01-01 for the first of a proleptic-Gregorian civil
/// `(year, month, day)` — Howard Hinnant's closed-form `days_from_civil`, so
/// [`month_start_ms`] needs no calendar library. Already precedented in this
/// codebase (`aura-cli`'s `tests/common::synthetic_data` carries the identical
/// formula for its own synthetic archive generator); duplicated here rather
/// than shared because that copy lives in a test-only module of a different
/// crate.
fn days_from_civil(y: i64, m: u32, d: u32) -> i64 {
let y = if m <= 2 { y - 1 } else { y };
let era = if y >= 0 { y } else { y - 399 } / 400;
let yoe = (y - era * 400) as u64; // [0, 399]
let mp = (u64::from(m) + 9) % 12; // [0, 11]: Mar=0 .. Feb=11
let doy = (153 * mp + 2) / 5 + u64::from(d) - 1; // [0, 365]
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; // [0, 146096]
era * 146_097 + doe as i64 - 719_468
}
/// UTC epoch-ms of `(year, month)`'s first instant — matches
/// `data_server::records::unix_ms_to_year_month`'s own UTC calendar (chrono's
/// `DateTime<Utc>`), the reference [`archive_extent`] derives a
/// single-candidate-month probe window against.
fn month_start_ms(year: u16, month: u8) -> i64 {
days_from_civil(i64::from(year), u32::from(month), 1) * 86_400_000
}
/// UTC epoch-ms of the instant immediately after `(year, month)`'s last
/// millisecond (the exclusive upper bound [`archive_extent`] clamps a
/// forward-search probe's `to_ms` against, so a single query loads at most
/// one month's file).
fn month_end_ms_exclusive(year: u16, month: u8) -> i64 {
let (ny, nm) = if month == 12 { (year + 1, 1) } else { (year, month + 1) };
month_start_ms(ny, nm)
}
/// The sorted `(year, month)` pairs that exist as `<symbol>_YYYY_MM.m1` files
/// directly under `data_path` — mirrors data-server's own file-naming
/// convention (`SymbolIndex::scan`, already load-bearing in its loader) via a
/// plain directory listing scoped to the one already-known `symbol`, so this
/// needs no general symbol-boundary regex and reads no private index (#252:
/// no new data-server surface). Empty when `data_path` doesn't exist or holds
/// no matching file for `symbol` — the same "no archive for this symbol"
/// condition an absent `DataServer` symbol represents.
fn list_m1_months(data_path: &Path, symbol: &str) -> Vec<(u16, u8)> {
let prefix = format!("{symbol}_");
let mut months = Vec::new();
let Ok(entries) = std::fs::read_dir(data_path) else {
return months;
};
for entry in entries.flatten() {
let file_name = entry.file_name();
let Some(name) = file_name.to_str() else { continue };
let Some(rest) = name.strip_prefix(prefix.as_str()) else { continue };
let Some(year_month) = rest.strip_suffix(".m1") else { continue };
let Some((year_str, month_str)) = year_month.split_once('_') else { continue };
if year_str.len() != 4
|| month_str.len() != 2
|| !year_str.bytes().all(|b| b.is_ascii_digit())
|| !month_str.bytes().all(|b| b.is_ascii_digit())
{
continue;
}
let (Ok(year), Ok(month)) = (year_str.parse::<u16>(), month_str.parse::<u8>()) else {
continue;
};
if (1..=12).contains(&month) {
months.push((year, month));
}
}
months.sort_unstable();
months
}
/// Derive the archive's `(first, last)` bar timestamp inside `[from_ms, to_ms]`
/// (inclusive Unix-ms, `None` = open-ended) at O(1) file loads per boundary
/// instead of draining every bar in the window (#252). Lists the symbol's
/// monthly files directly off `data_path` ([`list_m1_months`]), then loads
/// ONLY the boundary month through the existing `stream_m1_windowed`/
/// [`M1FieldSource`] seam — walking forward (for the first bar) or backward
/// (for the last) across gap months whose candidate file yields nothing in
/// range, exactly like the cross-file skip-ahead a full-window drain already
/// did internally, just narrowed to one file per candidate instead of every
/// file in the window. `server` is the caller's own `Arc<DataServer>` (shares
/// its file cache — the two boundary files loaded here are exactly the ones a
/// subsequent real run reads first/last, so this is not wasted work, only
/// reordered). Returns `None` exactly when no month in `[from_ms, to_ms]` has
/// a bar — no archive for the symbol, or a window with zero matching bars —
/// the same refusal condition a drain-based emptiness check detects.
pub fn archive_extent(
server: &Arc<DataServer>,
data_path: &Path,
symbol: &str,
from_ms: Option<i64>,
to_ms: Option<i64>,
) -> Option<(Timestamp, Timestamp)> {
let months = list_m1_months(data_path, symbol);
if months.is_empty() {
return None;
}
let from_ym = from_ms.map(data_server::records::unix_ms_to_year_month);
let to_ym = to_ms.map(data_server::records::unix_ms_to_year_month);
let in_range = |&(y, m): &(u16, u8)| {
from_ym.is_none_or(|fym| (y, m) >= fym) && to_ym.is_none_or(|tym| (y, m) <= tym)
};
// Forward walk: the first candidate month (ascending) whose file, probed
// over just that month's span, yields a bar — a gap or an empty boundary
// file simply advances to the next one.
let first = months.iter().filter(|ym| in_range(ym)).find_map(|&(y, m)| {
let month_cap = month_end_ms_exclusive(y, m) - 1;
let probe_to = to_ms.map_or(month_cap, |t| t.min(month_cap));
let src = M1FieldSource::open(server, symbol, from_ms, Some(probe_to), M1Field::Close)?;
aura_engine::Source::peek(&src)
})?;
// Backward walk: the last candidate month (descending) whose file,
// probed over just that month's span, yields a bar — fully drained (a
// single file) to find its own last matching record.
let last = months.iter().rev().filter(|ym| in_range(ym)).find_map(|&(y, m)| {
let month_floor = month_start_ms(y, m);
let probe_from = from_ms.map_or(month_floor, |f| f.max(month_floor));
let mut src = M1FieldSource::open(server, symbol, Some(probe_from), to_ms, M1Field::Close)?;
let mut last_ts = aura_engine::Source::peek(&src)?;
while let Some((t, _)) = aura_engine::Source::next(&mut src) {
last_ts = t;
}
Some(last_ts)
})?;
Some((first, last))
}
#[cfg(test)]
mod tests {
use super::*;
/// A unique scratch directory under the OS temp root, removed on drop —
/// mirrors `aura-cli`'s own `tests/common::mint_tempdir` pattern (unique
/// name via pid + an atomic counter) so this crate's tests need no
/// `tempfile` dev-dependency for a handful of throwaway fixture dirs.
struct ScratchDir(std::path::PathBuf);
impl ScratchDir {
fn new(tag: &str) -> Self {
static COUNTER: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0);
let n = COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let dir = std::env::temp_dir()
.join(format!("aura-ingest-test-{tag}-{}-{n}", std::process::id()));
std::fs::create_dir_all(&dir).expect("create scratch dir");
Self(dir)
}
fn path(&self) -> &std::path::Path {
&self.0
}
}
impl Drop for ScratchDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
#[test]
fn instrument_geometry_none_for_unknown_symbol() {
// A symbol with no sidecar yields None — the accessor surfaces the reader's
@@ -447,6 +606,159 @@ mod tests {
}
}
/// [`month_start_ms`] anchored against the same 2017-03-01 UTC reference
/// [`unix_ms_to_epoch_ns_scales_ms_to_ns`] pins, plus the Feb/Dec month-end
/// edges [`month_end_ms_exclusive`] must get right (leap-year length, and
/// the December -> next-January year rollover) — the two properties
/// [`archive_extent`]'s per-candidate-month probe bound depends on.
#[test]
fn month_start_and_end_ms_bracket_known_months() {
assert_eq!(month_start_ms(2017, 3), 1_488_326_400_000);
// 2024 is a leap year: February runs through the 29th, so March 1
// 00:00 UTC is exactly 29 days after February 1 00:00 UTC.
assert_eq!(
month_end_ms_exclusive(2024, 2) - month_start_ms(2024, 2),
29 * 86_400_000
);
// 2023 is not a leap year: 28 days.
assert_eq!(
month_end_ms_exclusive(2023, 2) - month_start_ms(2023, 2),
28 * 86_400_000
);
// December rolls over into the next year's January.
assert_eq!(month_end_ms_exclusive(2024, 12), month_start_ms(2025, 1));
// A month's end is exclusive: the next month starts immediately after.
assert_eq!(month_end_ms_exclusive(2024, 3), month_start_ms(2024, 4));
}
/// [`list_m1_months`] scans `data_path` for `<symbol>_YYYY_MM.m1` files —
/// mirroring data-server's own `SymbolIndex::scan` regex (`test_symbol_index_scan`
/// in its `lib.rs`) but scoped to one already-known symbol, so it needs no
/// general-purpose parser. Sorted ascending, and never confused by another
/// symbol's file or a non-M1 extension sharing the same directory.
#[test]
fn list_m1_months_lists_and_sorts_the_one_symbols_files() {
let dir = ScratchDir::new("list-months");
for name in [
"EURUSD_2020_06.m1",
"EURUSD_2019_12.m1",
"EURUSD_2020_01.m1",
"EURUSD_2020_01.tick", // different format, same symbol+month: ignored
"GBPUSD_2020_01.m1", // different symbol: ignored
"EURUSD_2020_1.m1", // malformed month (one digit): ignored
"not_a_data_file.txt",
] {
std::fs::write(dir.path().join(name), b"").expect("write stub file");
}
assert_eq!(
list_m1_months(dir.path(), "EURUSD"),
vec![(2019, 12), (2020, 1), (2020, 6)]
);
}
#[test]
fn list_m1_months_empty_for_unknown_symbol_or_missing_dir() {
let dir = ScratchDir::new("list-months-empty");
std::fs::write(dir.path().join("EURUSD_2020_01.m1"), b"").expect("write stub file");
assert!(list_m1_months(dir.path(), "NOSYM").is_empty());
assert!(list_m1_months(Path::new("/nonexistent-archive-path"), "EURUSD").is_empty());
}
/// Writes a minimal one-record `<symbol>_YYYY_MM.m1` zip at `unix_ms_time`
/// (mirrors `data-server`'s own `write_m1_file` test helper — duplicated
/// here, not imported, since that helper lives in its private `#[cfg(test)]`
/// module). Deliberately zip/binary-shaped, not a stub, since
/// [`archive_extent`] must actually load this file through the real
/// `stream_m1_windowed`/`M1FieldSource` seam, not just see its filename.
fn write_one_bar_month(dir: &Path, symbol: &str, year: u16, month: u8, unix_ms_time: i64) {
use std::io::Write;
const DELPHI_EPOCH_OFFSET_DAYS: f64 = 25569.0;
const MS_PER_DAY: f64 = 86_400_000.0;
let dt = unix_ms_time as f64 / MS_PER_DAY + DELPHI_EPOCH_OFFSET_DAYS;
let mut rec = [0u8; 48];
rec[0..8].copy_from_slice(&dt.to_le_bytes());
rec[8..16].copy_from_slice(&1.0_f64.to_le_bytes()); // open
rec[16..24].copy_from_slice(&1.0_f64.to_le_bytes()); // high
rec[24..32].copy_from_slice(&1.0_f64.to_le_bytes()); // low
rec[32..40].copy_from_slice(&1.0_f64.to_le_bytes()); // close
rec[40..44].copy_from_slice(&0.0_f32.to_le_bytes()); // spread
rec[44..48].copy_from_slice(&0_i32.to_le_bytes()); // volume
let path = dir.join(format!("{symbol}_{year:04}_{month:02}.m1"));
let file = std::fs::File::create(&path).expect("create synthetic m1 zip");
let mut zip = zip::ZipWriter::new(file);
zip.start_file::<String, ()>(format!("{symbol}.bin"), Default::default())
.expect("start synthetic m1 zip entry");
zip.write_all(&rec).expect("write synthetic m1 record");
zip.finish().expect("finish synthetic m1 zip");
}
/// Property (#252): a requested window whose starting month has NO archive
/// file (a gap) still resolves the first bar, by walking forward to the
/// next available month that actually has one — not refusing outright.
/// Three files (Jan, MISSING Feb, Mar 2021), each holding one bar; a
/// window opening in the missing February must land on March's bar.
#[test]
fn archive_extent_walks_forward_across_a_gap_month() {
let dir = ScratchDir::new("extent-forward-gap");
let jan_ms = month_start_ms(2021, 1) + 3_600_000; // 2021-01-01 01:00 UTC
let mar_ms = month_start_ms(2021, 3) + 3_600_000; // 2021-03-01 01:00 UTC
write_one_bar_month(dir.path(), "GAPSYM", 2021, 1, jan_ms);
write_one_bar_month(dir.path(), "GAPSYM", 2021, 3, mar_ms);
let server = Arc::new(DataServer::new(dir.path()));
// From = the start of the missing February; no upper bound.
let from_ms = month_start_ms(2021, 2);
let (first, last) = archive_extent(&server, dir.path(), "GAPSYM", Some(from_ms), None)
.expect("March's bar is reachable by walking past the empty February");
assert_eq!(first, unix_ms_to_epoch_ns(mar_ms));
assert_eq!(last, unix_ms_to_epoch_ns(mar_ms));
}
/// Property (#252): symmetric to the forward walk, an upper bound landing
/// in a gap month walks BACKWARD to find the last bar of the nearest
/// earlier available month.
#[test]
fn archive_extent_walks_backward_across_a_gap_month() {
let dir = ScratchDir::new("extent-backward-gap");
let jan_ms = month_start_ms(2021, 1) + 3_600_000;
let mar_ms = month_start_ms(2021, 3) + 3_600_000;
write_one_bar_month(dir.path(), "GAPSYM", 2021, 1, jan_ms);
write_one_bar_month(dir.path(), "GAPSYM", 2021, 3, mar_ms);
let server = Arc::new(DataServer::new(dir.path()));
// To = the end of the missing February; no lower bound.
let to_ms = month_end_ms_exclusive(2021, 2) - 1;
let (first, last) = archive_extent(&server, dir.path(), "GAPSYM", None, Some(to_ms))
.expect("January's bar is reachable by walking back past the empty February");
assert_eq!(first, unix_ms_to_epoch_ns(jan_ms));
assert_eq!(last, unix_ms_to_epoch_ns(jan_ms));
}
/// Property (#252): an unknown symbol (no monthly files at all) yields
/// `None` — the same file-level-absence contract [`open_columns`] and
/// [`instrument_geometry`] already share.
#[test]
fn archive_extent_none_for_unknown_symbol() {
let dir = ScratchDir::new("extent-unknown-symbol");
let server = Arc::new(DataServer::new(dir.path()));
assert!(archive_extent(&server, dir.path(), "NOSYM", None, None).is_none());
}
/// Property (#252): a window entirely before the archive's first month (or
/// entirely after its last) yields `None` rather than snapping to the
/// nearest available month — [`archive_extent`]'s `in_range` bound must
/// reject every candidate, not just prefer the closest one.
#[test]
fn archive_extent_none_when_window_misses_every_month() {
let dir = ScratchDir::new("extent-misses-every-month");
write_one_bar_month(dir.path(), "GAPSYM", 2021, 3, month_start_ms(2021, 3) + 3_600_000);
let server = Arc::new(DataServer::new(dir.path()));
// Entirely before the one archived month.
assert!(archive_extent(&server, dir.path(), "GAPSYM", None, Some(month_start_ms(2021, 1))).is_none());
// Entirely after it.
assert!(archive_extent(&server, dir.path(), "GAPSYM", Some(month_start_ms(2021, 5)), None).is_none());
}
#[test]
fn open_columns_propagates_file_level_absence_as_none() {
// The open_ohlc contract, generalized: file-level absence (unknown
@@ -65,6 +65,7 @@ fn run_point(server: &Arc<DataServer>, point: &[Cell], from: Timestamp, to: Time
manifest: RunManifest {
commit: "ger40-breakout-world".to_string(),
params: vec![],
defaults: vec![],
window: (from, to),
seed: 0,
broker: "sim-optimal(pip_size=1)".to_string(),
+1
View File
@@ -86,6 +86,7 @@ fn run_sample_over(prices: Vec<(Timestamp, Scalar)>) -> RunReport {
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(0.5)),
],
defaults: vec![],
window,
seed: 0,
broker: "sim-optimal(pip_size=0.0001)".to_string(),
@@ -86,6 +86,7 @@ fn run_sample(window: (Timestamp, Timestamp), source: Box<dyn Source>) -> RunRep
("sma_slow".to_string(), Scalar::i64(4)),
("bias_scale".to_string(), Scalar::f64(0.5)),
],
defaults: vec![],
window,
seed: 0,
broker: "sim-optimal(pip_size=0.0001)".to_string(),
+5 -1
View File
@@ -29,6 +29,8 @@ pub(crate) struct RunReportRead {
struct RunManifestRead {
commit: String,
params: Vec<(String, ScalarRead)>,
#[serde(default)]
defaults: Vec<(String, ScalarRead)>,
window: (Timestamp, Timestamp),
seed: u64,
broker: String,
@@ -73,12 +75,14 @@ impl<'de> Deserialize<'de> for ScalarRead {
impl From<RunReportRead> for RunReport {
fn from(r: RunReportRead) -> Self {
let RunManifestRead {
commit, params, window, seed, broker, selection, instrument, topology_hash, project,
commit, params, defaults, window, seed, broker, selection, instrument, topology_hash,
project,
} = r.manifest;
RunReport {
manifest: RunManifest {
commit,
params: params.into_iter().map(|(name, v)| (name, v.0)).collect(),
defaults: defaults.into_iter().map(|(name, v)| (name, v.0)).collect(),
window,
seed,
broker,
+2 -1
View File
@@ -816,6 +816,7 @@ mod tests {
manifest: RunManifest {
commit: "c".to_string(),
params: vec![("p".to_string(), Scalar::f64(1.0))],
defaults: vec![],
window: (Timestamp(1), Timestamp(2)),
seed: 0,
broker: "b".to_string(),
@@ -1182,7 +1183,7 @@ mod tests {
params: vec![],
report: RunReport {
manifest: RunManifest {
commit: "c".into(), params: vec![], window: (Timestamp(0), Timestamp(0)),
commit: "c".into(), params: vec![], defaults: vec![], window: (Timestamp(0), Timestamp(0)),
seed: 0, broker: "b".into(), selection: None, instrument: None,
topology_hash: None,
project: None,
+1
View File
@@ -299,6 +299,7 @@ mod tests {
RunManifest {
commit: "c".to_string(),
params: vec![("p".to_string(), Scalar::f64(1.0))],
defaults: vec![],
window: (Timestamp(1), Timestamp(5)),
seed: 0,
broker: "sim-optimal(pip_size=0.0001)".to_string(),
@@ -30,6 +30,7 @@ fn tiny_report() -> RunReport {
manifest: RunManifest {
commit: "c".to_string(),
params: vec![],
defaults: vec![],
window: (Timestamp(1), Timestamp(2)),
seed: 0,
broker: "b".to_string(),
+14 -1
View File
@@ -1032,7 +1032,8 @@ SQN comparability. Each swept member's manifest records the fixed R-defining par
beside the floated knobs (reproducible from its own manifest, C18).
[HISTORY — the built-in `--strategy` sweep surface was retired with the demos →
blueprint-data (#159, cuts 1b-4); the rankable R sweep now runs as `aura sweep
<blueprint.json> --axis …` over examples/r_sma_open.json.]
<blueprint.json> --axis …` over r_sma_open.json (an example then; relocated to
`crates/aura-cli/tests/fixtures/` by #248).]
**Realization (cycle 0067, #130 + #135).** **#130 (SQN100):** `RMetrics` gains
`sqn_normalized = (mean_R/stdev_R)·√(min(n, 100))` — the n-normalized "SQN score"
@@ -1210,6 +1211,18 @@ every hand-rolled usage line reads `Usage: aura <verb> …` (#179, cycle 0101);
refusal diagnostics stay unprefixed (diagnostics are not usage lines). The
machine-first help surface (JSON/manifest help, stdin op-scripts) stays on the
#157/C21 track, not this cycle (settled as human/GNU convention compliance).
**Amendment (2026-07-13, #249 — two artifact classes, two redundancy budgets).**
The data layer the programmatic face emits is a public interface read raw (by
humans and LLMs), and its records split into two classes with opposite
redundancy budgets: **generated outcome records** (manifests, metrics, family
reports) have a single writer at run time — redundancy there cannot drift and
is deliberately spent on direct readability; **authored intent artifacts**
(blueprints, op-scripts, campaign documents) are author-maintained — every
redundancy is a drift site and stays out. Consequence shipped with #249: a run
manifest stamps the untouched bound defaults (`defaults`, one-directionally
widened beside `params`, which keeps its "what varied" reproduce semantics) so
a raw reader of a fully-bound run no longer sees a misleading `"params": []`;
the blueprint itself carries no such duplication.
### C15 — Resampling-as-node; sessions/calendars
**Guarantee.** A resampler is a node (finer stream → coarser bar stream),