ad4249f9e627efea5b4468084cde045f53b7884e
42 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 claude
|
d3b1a1aead |
feat(campaign,registry,cli): per-cell fault isolation — a failed cell is recorded, never a global abort
closes #272 A member fault (no-data, bind, run, or a caught panic) is now a recorded per-cell outcome instead of aborting the whole campaign and discarding every already-computed cell. The incident that motivated this (a 22-instrument campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap) now completes: the healthy cells persist, the gap cell is recorded as failed, and the run exits 3. Direction (owner decision 2026-07-14): run to completion and report compromised results; no coverage preflight, no window synthesis. Containment granularity: - The CELL for a sweep-stage member fault (a grid hole structurally compromises winner selection, so the whole cell fails). - The FOLD for a walk_forward member fault (independent time windows): the surviving folds pool into the family, failed folds are recorded as StageRealization.window_faults, and the summary names the ratio. - aura-registry: additive CellFault / CellFaultKind (closed: no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus fault/coverage fields on CellRealization and window_faults on StageRealization — all serde-default-skipped, so pre-#272 campaign_runs lines parse and round-trip byte-identical. - aura-campaign: run_cell returns a fault-annotated CellRealization instead of Err (execute's accumulate-then-append-once tail is unchanged and now persists every healthy cell + the one run record); a `contain` split keeps ExecFault::Registry and doc-shape preflight faults global while Member/Window become per-cell/per-fold. Member panics are caught with catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS) and recorded as MemberFault::Panic — a member panic no longer aborts the process. The wf stage partitions Registry faults (global) from Member/Window (per-fold) and filters faulted-fold placeholders (the "faulted-member-placeholder" broker sentinel) out of the persisted family. - aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage derives effective bounds + interior gap months from the #264 archive primitives; present_campaign prints per-cell failure notes + a completion summary and threads the failed-cell count; a run with >=1 failed cell exits 3 ("completed with failed cells") uniformly across `aura campaign run` and the dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result). Usage stays 2, refused-before-running stays 1, clean stays 0. Tests: the global-abort pins flip to containment (execute + the two wf fault tests → fold-containment + all-folds-fail-the-cell); new panic-containment tests on both the sweep path (PanicRunner) and the wf path (this commit adds the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3 register; a pre-#272-line byte-identical round-trip guard. Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean. Decision log: #272 comments (fork rationale, the fold Registry/Member split, the placeholder sentinel, uniform exit-3). Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry though the additive fields also touch aura-campaign's exec.rs literals — the loop absorbed it mechanically; a future plan for a cross-crate additive-field change should scope every crate's construction sites in the first task. |
||
|
claude
|
bb0b0aeac2 |
feat(analysis,campaign,registry,cli): bootstrap net R through the process pipeline
The OOS bootstrap conduit RMetrics.trade_rs becomes net_trade_rs and carries the COST-NETTED per-trade R (r − cost_in_r, trade order). Every conduit consumer is thereby net-when-costed: the monte-carlo pooled-OOS and per-survivor bootstraps, the walk-forward oos_r pooling, and the deflation null-max — which previously compared a net observed statistic against a gross-resampled null whenever a costed campaign selected on net_expectancy_r. An uncosted run is bit-identical (empty cost stream ⇒ cost 0.0 per trade), so every existing golden pin stays green. Design: one conduit, no knob — an explicit net: knob would let a costed campaign silently produce a gross headline again (the exact misreading of the issue's evidence). Per-member RMetrics scalar fields stay gross; net_expectancy_r keeps its meaning. Wire shape unchanged (serde(skip)). The net series is materialized as a separate expression in summarize_r; the pinned-float expressions (net_sum, the SQN pair, the lockstep r_metrics_from_rs copies) keep their tokens verbatim. Fork decisions and rationales: issue #259 comments. New coverage, both hostless over the synthetic SYMA archive: a costed campaign twin shifts the pooled-OOS bootstrap (sweep→gate→wf→mc) and every per-survivor bootstrap (sweep→mc) below its gross sibling, with the trade population unchanged; a unit test pins the conduit as the cost-netted series with the empty-stream degeneracy. Existing conduit tests renamed with the field. Verified: full workspace suite green (71 result blocks), clippy clean, zero bare trade_rs tokens remain, ledger conduit prose updated in place. closes #259 |
||
|
claude
|
1ccce9ad32 |
fix(test): anchor all test sandboxes on reclaimable pid-free names
Every hand-rolled test-sandbox helper keyed its directory on
std::process::id() under env::temp_dir(), so the pre-create wipe never
matched a prior run's path: one directory stranded per helper site per
test-binary invocation, >50k dirs / 15.5 GiB on the 16 GiB tmpfs by
2026-07-13.
The remedy extends commit 84e1075's knob-lab pattern to every site:
fixed, tag-keyed, PID-FREE names under the build-tree tmp anchor, each
site keeping its pre-create remove_dir_all — which now genuinely
reclaims the previous run. Integration/bench targets use
env!("CARGO_TARGET_TMPDIR"); src-internal unit-test helpers (cargo sets
that var only for integration/bench targets) derive the same anchor
from env!("CARGO_MANIFEST_DIR") + ../../target/tmp. Residue is bounded
to one directory per site, on disk instead of the tmpfs, per checkout.
One deliberate exception: project_new.rs's tmp() stays under
env::temp_dir() because two of its tests (new_outside_a_work_tree_*)
need a base genuinely detached from any git work tree, and target/
lives inside the checkout's work tree. Its name carries a per-checkout
hash discriminator instead, so concurrently running checkouts (which
share /tmp) cannot race on the fixed name while each run still
reclaims its predecessor.
tempfile-RAII was considered and rejected (issue #258 decision log):
it cannot cover the process-lifetime OnceLock scaffolds (statics never
drop), adds a dependency, and still strands kill-leftovers on the
tmpfs.
Verified: full workspace suite green (incl. the new RED regression
test temp_cwd_sandbox_does_not_leak_under_env_temp_dir), clippy clean,
sandboxes observed under target/tmp with stable names across runs.
Stale pre-fix PID-keyed dirs in /tmp are not swept by this change and
age out only by manual cleanup.
closes #258
|
||
|
claude
|
1ebb94c1b8 |
feat(engine,cli): run manifests stamp untouched bound defaults (closes #249)
RunManifest gains defaults: Vec<(String, Scalar)> — the wrap-prefixed bound_param_space() of the signal, read after axis reopening, so a bound param an axis overrode has already left the space and flows through params instead (disjoint by construction; verified end to end: a sweep member's overridden fast.length sits in params while slow.length/bias.scale sit in defaults). params keeps its "what varied" semantics and stays the reproduce input. One-directional serde widening (#[serde(default)]) per the selection/instrument/topology_hash idiom — old records deserialize with an empty defaults; unlike the Option fields it always serializes, mirroring params. ~20 struct-literal sites across five crates gained the field (compile-mandated breadth, no behaviour change at those sites). The C14 ledger records the underlying decision (2026-07-13): generated outcome records spend redundancy on direct readability (single writer, cannot drift); authored intent artifacts admit none (every redundancy is a drift site) — so the fix lands in the manifest, never the blueprint. Verification: RED test run_manifest_stamps_untouched_bound_defaults green; cargo build --workspace; cargo test --workspace green; clippy -D warnings on the touched crates; binary-level sweep exclusivity check. |
||
|
claude
|
e902a0f31d |
feat(cli,registry): bound-param axis overrides reach the campaign/--real trunk
Task 5 of the bound-override cycle. The campaign executor and its gates accept an axis naming a bound param everywhere the open surface was already accepted: CliMemberRunner::run_member re-opens the override set on the cell's probe space and raw reload (raw_bound_overrides_of — the campaign document's axis names live in strategy coordinates, matched against bound_param_space directly), the P3 preflight in validate_before_register threads the same set so the executable-shape check passes, and aura-registry's validate_campaign_refs treats a bound name as a valid axis (open space wins the arm order; the kind check still faults a mismatched axis over a bound path). The CLI-side helper family is named by coordinate system now: wrapped_bound_overrides_of (wrap-prefixed, sweep/wf/reproduce) beside raw_bound_overrides_of (campaign). A fresh post-block quality review found no correctness defect; its two stale-comment findings (renamed helpers cited under their old names) are fixed in this commit. refs #246 |
||
 Brummel
|
43b1c7ff5d |
feat(research,registry,cli): campaign data.bindings — the 6b rebind seam (#231 task 5)
DataSection gains an additive bindings block (role -> column; serde default + skip-if-empty, so binding-less documents keep their content ids — pinned). Validation is two-tier along the existing line: the intrinsic tier checks binding VALUES against the column vocabulary (DocFault::UnknownBindingColumn, alias-annotated display register); the resolver tier checks binding KEYS against the loaded strategies' input_roles() (RefFault::BindingRoleUnknown — a key must name a role of at least one campaign strategy). A cross-surface pin keeps the doc-tier vocabulary and the CLI binding module from drifting. CliMemberRunner threads the campaign overrides into resolve_binding on BOTH halves (column opening and wrap), keeping the C1 drift alarm comparing like-with-like; the verb sugar passes no bindings (name defaults rule). Proof: an archive-gated campaign e2e rebinds price -> open and yields different realized metrics from the close-bound run; refusal prose exact-pinned at both tiers. Verified: full workspace suite green, clippy -D warnings clean; independent quality review, all findings repaired. refs #231 |
||
|
Brummel
|
a1551a6097 |
fix(campaign): thread the risk regime through trace persistence
The persist-side C1 drift alarm re-ran every nominee under the hardcoded
default vol-stop because CellRealization dropped the regime at the record
boundary — a non-default risk regime + persist taps false-failed C1 and
refused to persist a correct trace; two regimes over the same cell would
also have collided in one trace dir.
Fix at the record boundary: CellRealization carries regime + regime_ordinal
(serde-default + skip-serializing keeps pre-existing stored records
byte-stable), populated where the cell is realized; the regime->StopRule
binding is single-sourced in stop_rule_for_regime, shared by the member
runner and the persist re-run so the two sites cannot drift apart again;
campaign_cell_key appends -r{regime_ordinal} for non-default regimes
(default keys unchanged, no stored dir renamed).
closes #219, closes #212
|
||
|
Brummel
|
a0d8aec3f7 |
feat(campaign): expand the campaign matrix over risk regimes (#210 T3)
The regime becomes the fourth structural coordinate of the campaign matrix, a
kept-separate axis (a peer of window, not aggregated-over like instrument):
- aura-campaign: CellSpec gains `regime` + `regime_ordinal`; the executor
resolves the regime list (absent/empty risk -> one default cell, ordinal 0),
runs a cell per (strategy, instrument, window, regime), and keys the nominee
map by the 3-tuple including the regime ordinal — so generalize aggregates
across instruments WITHIN a regime, never across regimes. Family names gain a
`-r{regime_ordinal}` discriminator so two regimes' families cannot collide.
- aura-registry: `CampaignGeneralization` gains `regime_ordinal` (with
`#[serde(default)]`, so pre-feature stored records still parse — pinned by a
new backward-compat test).
The existing single-regime execute tests shift their family ids by `-r0`; a new
test pins that two regimes yield distinct `-r0`/`-r1` families. `CellSpec.regime`
is write-only in this slice — its sole reader is the deferred member runner
(the next slice) — a held quality nit, seam-prep as the plan prescribes. Full
workspace suite green.
refs #210
|
||
|
Brummel
|
5256b3d0ca |
refactor(registry): single-source the deflation-bootstrap constants (#199)
DEFLATION_N_RESAMPLES (1000) and DEFLATION_BLOCK_LEN (5) were duplicated as independent copies in aura-cli and aura-campaign — a silent-drift risk (an edit in one crate but not the other refuses nothing, it just produces a wrong deflation result). Promote a single definition into aura-registry, which owns optimize_deflated (the sole consumer of these counts); aura-cli imports it and aura-campaign re-exports it to keep its public path stable. DEFLATION_SEED stays cli-local (no campaign twin). Values unchanged; behaviour-preserving, suite green unchanged (1043/0). closes #199 |
||
|
Brummel
|
b8ef77b1c7 |
feat(research,registry,campaign): tap vocabulary + trace_name record contract (0109 tasks 1-2)
Task 1: the tap namespace is a closed vocabulary — tap_vocabulary()
(the wrap convention's four sink names; escalation = new entry or an
authored blueprint sink, never an open node-path namespace) with
DocFault::UnknownTap in validate_campaign (index-addressed, prose
enumerating the vocabulary via the single source), SlotKind::TapKinds
on the persist_taps slot, open-slot hint naming the taps. The 0107
deferral seam test split into its two 0109 pins (unknown tap refuses
at validate; a valid tap passes validate and hits the member-data
seam) — the deferral eprintln itself survives until task 3.
Task 2: CampaignRunRecord.trace_name lands serde-default sparse; the
claim contract holds across the seam — execute stamps the sentinel iff
persist_taps is non-empty, append_campaign_run composes
Some("{campaign8}-{run}") onto the stored line (safe prefix take at
the registry seam), execute mirrors the same derivation onto the
returned copy; None stays None; the hand-seeded byte-identity dump pin
survives (absent -> skipped).
Gates: workspace 1037/0, clippy -D warnings clean.
refs #201
|
||
|
Brummel
|
ab778f2361 |
feat(campaign,registry,engine): record widenings + preflight v2 fault set (0108 tasks 1-2)
Task 1: RBootstrap and Generalization gain serde derives; lineage.rs gains StageBootstrap (per_survivor | pooled_oos) and CampaignGeneralization; StageRealization.bootstrap and CampaignRunRecord.generalizations land serde-default sparse (a pre-0108 campaign_runs.jsonl line still parses — pinned), threaded None/empty through every literal site. No behaviour change. Task 2: ExecFault::UnsupportedStage is gone; the preflight now admits the v2 shape sweep [gate]* [wf]? [mc]? [generalize]? via a monotone rank walk (precise per-pair PipelineShape prose) and statically refuses: single-instrument generalize (GeneralizeNeedsInstruments), non-R generalize metrics via check_r_metric (GeneralizeNonRMetric — no fourth roster site), zero mc params (ZeroBootstrapParam). CLI prose swapped accordingly. The loop's E2E phase added a tier-boundary pin: process validate accepts [sweep, mc, walk_forward] while campaign run refuses it data-free. KNOWN RED (documented in the plan, flips in task 5): crates/aura-cli/tests/research_docs.rs::campaign_run_v1_boundary_refuses_mc_process still pins the retired 'not executable in v1' prose. Interim gates: registry 57/0, campaign 33/0, cli --bin 111/0, workspace build clean. refs #200 |
||
|
Brummel
|
2968fbef40 |
feat(registry,cli): referential tier requires full open-param coverage; bind refusal speaks the raw namespace
validate == runnable (fieldtest 0107 F7): validate_campaign_refs now
checks the reverse direction — every open param of the resolved
strategy must be bound by some campaign axis — emitting the new
RefFault::ParamNotCovered { strategy, param } with the RAW
param_space() path; ref_fault_prose phrases it ('open param "X" is
bound by no campaign axis'). The executor's every-open-knob-required
rule is thereby mirrored at validate time, and campaign run's
referential gate catches an uncovered knob before any member binds.
The e2e fixture that legitimately under-covered (fast only) gains the
slow.length axis.
Second half, RED-first: the member-bind refusal no longer leaks the
wrapped bind-time path — it names the raw campaign-axis namespace with
the wrapped path as a parenthetical ('open param "threshold" ...
(wrapped path: sma.threshold)').
RED evidence: E0599 on the pinned ParamNotCovered test (tdd-author
handoff), then the flipped bind_axes prose pin observed failing on the
wrapped form.
Gates: workspace 988/0, clippy -D warnings clean.
closes #203
|
||
|
Brummel
|
b43def7d75 |
feat(engine,registry): ListSpace + campaign-run realization store (0107 tasks 2-3)
aura-engine: ListSpace — explicit point set implementing Space beside GridSpace/RandomSpace (a gate's survivor subset has no cartesian structure); GridSpace-identical arity/kind validation reusing the existing SweepError variants (KindMismatch.value_index carries the point ordinal for a list); empty point list is valid and sweeps to an empty family; re-exported at the crate root. aura-registry: CampaignRunRecord/CellRealization/StageRealization/ StageSelection — the thin campaign-level linking record over untouched family records (#198 decision 4) — with append_campaign_run (assigns run via next_campaign_run, a parallel counter beside next_run) and load_campaign_runs (missing file => empty) on the campaign_runs.jsonl sibling store; blueprint_path and find_blueprint_by_identity promoted to pub (the campaign executor resolves the same refs the referential tier validates). Gates: engine 290/0, registry 54/0, clippy -D warnings clean. refs #198 |
||
|
Brummel
|
db09e5de52 |
feat(cli): 0106 task 10 — aura process verb family (validate/introspect/register)
New research_docs.rs module: role-addressed process verbs with house-style fault prose (Display-free enums phrased at the seam), the introspect exactly-one-of usage guard (exit 2), and content-addressed registration. Seven binary seam tests pin intrinsic-ok/refusal prose, vocabulary/block/ unwired/content-id introspection, exit codes, and the stored file's location under the runs root. Quality-gate finding folded in (and back into the plan): register no longer re-derives the store path from runs_root — aura-registry now exposes process_path/campaign_path, the same single mapping its put/get pair routes through, so the printed path cannot drift from the store layout. Verification: research_docs seam tests 7/0, registry 49/0, clippy clean. refs #189 |
||
|
Brummel
|
ef3bec5844 |
feat(research): 0106 tasks 1-9 — document layer, content-id move, stores + referential tier
The aura-research leaf crate: ProcessDoc/CampaignDoc with strict parsing (hand-rolled StageBlock deserializer — deny_unknown_fields does not compose with internally-tagged enums; Axis declares its ScalarKind once with bare values, per the #189 user veto), canonical form + content_id_of (the SHA-256 primitive moved from aura-cli, which now delegates — the id pin set incl. the independent sweep-store recompute stayed green untouched), intrinsic validation for both document types (P1 constraints structural), and the introspection contract (schema tables single-source parse strictness AND vocabulary/describe/open-slots — the Blockly litmus made checkable). aura-registry: content-addressed processes/ + campaigns/ stores on the blueprint-store pattern (Ok(None) treat-as-empty) and the referential tier (validate_campaign_refs: process/strategy refs incl. identity-scan, axis-name + declared-kind checks against param_space). Plan deviations, verified by hand and folded back into the plan file: - Task 9's fixture could not use aura-composites as planned: every shipped composite with an open param routes through LinComb, which the zero-arg std_vocabulary roster deliberately excludes, so none round-trips through blueprint_from_json with the by-type-name resolver. The test hand-builds a minimal Bias composite instead (generic over param_space, as planned); the unused pro-forma aura-composites dev-dep from the repair pass is dropped. This is why the loop reported task 9 BLOCKED (review-loop-exhausted on the literal code block); the tree itself is complete and green. - A plan-verbatim test assertion false-matched ("deflate" as substring of "deflated-positive"); tightened to the JSON key. - OpenSlot doc comment backticked (rustdoc read strategies[0] as a link; doc gate back to 0 warnings). Verification: cargo test --workspace 898/0; clippy -D warnings clean; cargo doc --no-deps 0 warnings. Tasks 10-12 (CLI verb families, final gates) follow. refs #189 |
||
|
Brummel
|
4928e289f7 |
feat(project): the project-as-crate load boundary (cycle 0102)
A research project is now a loadable external cdylib crate. Inside a directory whose ancestry holds an Aura.toml, aura discovers the project root cargo-style, locates the compiled dylib via cargo metadata (debug default, --release opt-in), loads it load-and-hold, and refuses mismatches before trusting anything: the AURA_PROJECT descriptor (aura-core::project, #[repr(C)]) carries a C-ABI stamp prefix (rustc + aura-core version, baked per consuming build by the new aura-core build.rs) validated before any Rust-ABI field is read. The vocabulary charter gates the merged resolution: project type ids are ::-namespaced (std stays bare), duplicates refuse, and the enumerable type-id list must agree with the resolver, so introspection can never silently omit a project type. All blueprint verbs resolve through the merged project + std vocabulary via a per-invocation Env threaded through the dispatch chains; registry, trace-store, and data paths anchor at the project runs root (Aura.toml [paths], paths-only by design — instrument geometry stays the recorded sidecar, C15). RunManifest gains the Tier-1 project provenance field (namespace + dylib sha256 + best-effort commit), stamped beside topology_hash on the blueprint-run paths; pre-0102 registry lines load unchanged. Default node names strip the namespace, so :: never reaches the param-path address space. Proven by the demo-project fixture (built by the e2e via cargo, path-dep on this workspace): run twice bit-identical, provenance recorded, introspection lists demo::* beside std, registry anchors at the discovered root from a subdirectory; the badcharter fixture proves the charter refusal through the real libloading path; a never-built project refuses with a cargo-build hint. Outside a project every path collapses to the previous literals — goldens and manifest pins byte-identical. Verification: cargo build --workspace clean; cargo test --workspace 862 passed / 0 failed (incl. 7 project_load e2e); clippy -D warnings clean (one precedent-matching allow(too_many_arguments) on run_oos_blueprint, whose arity the Env threading raised to 8); doc build unchanged. Docs/ledger aligned: Aura.toml field lists are paths-only in project-layout.md, glossary, C16/C17; new C13 realization note records the per-invocation-reload reading and the load-and-hold one-shot scope boundary. New deps, per-case review (aura-cli leaf binary only, never the frozen artifact): libloading, toml. refs #180 |
||
|
Brummel
|
008692c043 |
feat(0094): content-addressed reproduction — aura reproduce <family-id> (iter 1)
Restore C18 reproducibility for generated (blueprint-sweep) runs (#158, acc 2): a sweep's topology is stored once, content-addressed by the topology_hash the manifest already carries, and `aura reproduce <family-id>` re-derives every persisted member bit-identically (C1) from the stored blueprint. - aura-registry: a dumb content-addressed bytes store — Registry::put_blueprint / get_blueprint / blueprints_dir over runs/blueprints/<hash>.json. No sha2 dep (caller owns the hash; reproduction's bit-identical compare is the integrity check); absent id -> Ok(None) (treat-as-empty, like load). - aura-cli: run_blueprint_member extracted from blueprint_sweep_family's member closure (behaviour-preserving; keystone stays green) so the sweep AND reproduce re-run the identical reduce-mode path — bit-identity by construction. The store write hooks the sweep persist seam (one blueprint per family, shared topo). reproduce_family_in loads each member's blueprint by hash, reconstructs the point from the recorded params (point_from_params, inverse of zip_params, over the WRAPPED signal's param_space so the prefixed knob names match), re-runs, and compares metrics. `["reproduce", id]` dispatch arm; refuse-don't-guess on an unknown id / missing stored blueprint (exit 2), DIVERGED -> exit 1. Dependency decision (per-case review, ratified): enabled serde_json `float_roundtrip`. Stored member metrics round-trip through families.jsonl as f64 JSON; the default parser can be 1 ULP off, which would make disk-loaded != recomputed and "bit-identical" (C1) physically impossible. The feature guarantees exact f64 round-trip. Blast-radius clean (full workspace suite green); it serves determinism, the invariant reproduction rests on. Canary: f64_blueprint_param_survives_store_round_trip_bit_identically. Two plan bugs caught + fixed by the implement loop and verified: reproduce's param_space must come from the wrapped signal (prefixed names, the #167 lockstep), and the family id is 0-indexed (live-0 / smacross-0, not -1). Verified: full workspace suite green (51 suites); clippy -D warnings clean; and the binary end-to-end — `aura sweep` (3 members, one shared stored blueprint) then `aura reproduce live-0` (3/3 bit-identical incl the open-at-end member), unknown id -> exit 2. acc 1 (graph introspect --content-id) + acc 3 (Tier-1 id stability) land in iteration 2 (plan 0094b). refs #158 |
||
|
Brummel
|
ff4a1b3d4a |
feat(0092): run-from-blueprint infrastructure — restructure + topology_hash + shared seam (Tasks 1-3)
Cycle-1 infrastructure for #165 (World/C21). Behaviour-preserving + workspace green; the CLI arm + tests (Tasks 4-5) follow. Task 1 — restructured stage1_r_graph into stage1_signal() (the serializable signal leg: SMA-cross -> Bias, a `price` input-role + a `bias` output) + wrap_stage1r(signal, ...) (broker/sinks/exec/cost around a nested signal), with stage1_r_graph() = wrap_stage1r(stage1_signal(...)). DEVIATION from the plan's "callers untouched" claim, verified behaviour-preserving: nesting the signal prefixes its param-space names (stage1_signal.fast.length), which broke the sweep's bare .axis("fast.length"); fixed exactly as the #137 stop-knob machinery does — FAST_LENGTH_SUFFIX/SLOW_LENGTH_SUFFIX suffix-resolution + stage1_r_friendly_name arms mapping the prefixed names back to the bare manifest names. All observable behaviour (manifest names, member keys, metrics, traces) byte-identical; the flat graph and every metric unchanged (full suite green, incl. sweep/walkforward/MC). Task 2 — RunManifest.topology_hash: Option<String> (Tier-1 optional, serde default/skip — old manifests byte-identical, #156), threaded None into all 24 literal sites across 7 crates. Also fixed the aura-registry RunManifestRead read-mirror to carry topology_hash (it was hardcoding None on load) — else a stored hash would silently drop on the registry round-trip once Task 3 stores a real one. Task 3 — the shared run_signal_stage1r seam (hash the signal, wrap, compile with params, bootstrap, run, manifest with params from param_space + topology_hash) + the sha256_hex topology_hash helper (sha2 in aura-cli, off the frozen engine, invariant 8). run_stage1_r now carries its topology_hash too (every run becomes self-identifying, #158). The seam is unwired until Task 4 (transient #[allow(dead_code)], retired there); RED-first seam tests added. Verified: cargo test --workspace green (51 suites, 0 failures); cargo clippy --workspace --all-targets -D warnings clean. refs #165 |
||
|
Brummel
|
0a06afed29 |
feat(0078): cross-instrument generalization as a validation step
Last piece of the Inferential-validation milestone (#146): a new `aura generalize` subcommand grades how consistently a single candidate holds across a set of instruments — the across-instrument axis of the same anti-false-discovery concern as trials-deflation (#144, within one family) and plateau-over-peak (#145, within one instrument's surface). Aggregator/validator, not a third selector (user-ratified): it runs a brought candidate (a single-cell stage1-r grid — --fast/--slow/--stop- length/--stop-k pinned to single values) across an instrument list (--real SYM1,SYM2,...), collects per-instrument R-metrics, and reduces them to a worst-case floor (min over instruments) + a sign-agreement count + the per-instrument breakdown. R-only (C10 — R is the only account/instrument-agnostic unit); the worst-case min is the cross- instrument sibling of PlateauMode::Worst and satisfies the non- domination constraint by construction (a strong instrument cannot lift a min). Rejected: a cross-instrument t-stat mean/std*sqrt(M), which reintroduces the pooled mean the milestone warns against. - RunManifest gains a first-class `instrument` lineage field (serde- widened, C14/C18 — legacy lines load as None, existing manifest bytes unchanged); the compat read-mirror threads it in lockstep. - FamilyKind::CrossInstrument (C12 comparison axis); the M per-instrument runs persist as a CrossInstrument family, each member self-identifying via its stamped instrument. The aggregate is recomputable from the members (McAggregate precedent), printed live. - The generalization reduction lives registry-side beside optimize_ deflated/optimize_plateau (C9), reusing resolve_metric/metric_value/ is_r_metric; check_r_metric is the data-free pre-check the CLI runs before evaluating any instrument. - Additive: existing sweep/walkforward/mc/standalone paths stay byte- identical (C23 — instrument omitted when None via skip_serializing_if). Verified by the orchestrator: cargo test --workspace green (0 failed — the new generalization_*, check_r_metric, parse_generalize_*, and the gated cross-instrument E2E all pass; the E2E ran the real GER40/USDJPY path on this host, exit 0); cargo clippy --workspace --all-targets -D warnings clean. closes #146 |
||
|
Brummel
|
cb32658fd1 |
feat(0077): prefer a robust parameter plateau over the in-sample peak
Add an opt-in plateau selection objective for walk-forward, recorded on the same RunManifest.selection carrier #144 introduced. Default argmax is byte-identical (C23); plateau is reached only via a new --select flag. What landed: - FamilySelection / SelectionMode reshape (report.rs): the selection RULE (mode) and its ANNOTATION are orthogonal — deflation fields become Option (present iff Argmax), plus PlateauMean/PlateauWorst variants and neighbourhood_score/n_neighbours (present iff Plateau*). Legacy lines still load: the deflation scalars deserialize from bare values via serde default (C14/C18). compat.rs embeds FamilySelection by value — reshape flows through untouched. - GridSpace::axis_lens() + SweepBinder::sweep_with_lattice (engine): the grid radixes in param_space()/odometer order. sweep() delegates to sweep_with_lattice with the lattice dropped, so every existing .sweep() caller is byte-unchanged. - optimize_plateau + PlateauMode + closed_neighbourhood (aura-registry, C9): each member scores as the mean/worst of its closed mixed-radix grid neighbourhood's metric_value; the winner is the smoothed argmax by the metric's direction (earliest-odometer tie, as optimize). Pure, no RNG (C1); in-sample only (C2). A higher_is_better helper now sources the per-metric direction once, shared by metric_cmp and optimize_plateau. - CLI --select <argmax|plateau:mean|plateau:worst> (default argmax; unknown token exits 2). walkforward_family dispatches via a select_winner helper; sweep_over/stage1_r_sweep_over carry the lattice as Option<Vec<usize>>. runs-family display gains a plateau(<mode>)=<score> over <n> cells line. RandomSpace-refuse: walkforward has no --random producer, so the plateau-without-lattice guard is structural — select_winner returns the refuse on a None lattice (the caller prints the message and exits 2), unit-tested at the helper, not via a --random E2E (decided + recorded on #145 comment 1974, with the lattice-seam fork). Tests: plateau-picks-plateau-not-spike, mean-vs-worst, lower-is-better direction-flip, closed-neighbourhood golden, single-member degeneracy, C1 determinism; CLI select-parse, select_winner refuse; E2E argmax-byte-identical (C23), plateau:mean and plateau:worst provenance. Full workspace suite green; clippy --all-targets -D warnings clean. closes #145 |
||
|
Brummel
|
ecc9541a56 |
0076.tidy: backfill spec-promised deflation tests + guard the dispersion-floor sign
Audit (architect, cycle 0076) found three tests the spec's testing strategy promised but the implement-loop did not land, plus an unguarded sign assumption: - optimize_deflated_centres_the_null_so_a_uniform_edge_is_not_called_overfit: the load-bearing protection for the mean-subtraction. A family where every member shares the same +R edge reads as NOT overfit only because the null is centred; an uncentred null would give ≈0.5. Removing the centring now flips a test (it previously passed every test — the architect's headline finding). - optimize_deflated_all_noise_family_has_high_overfit_probability: the high-p companion to the existing clear-edge low-p test. - optimize_deflated_provenance_reflects_only_the_passed_family: C2 at the registry boundary (n_trials == family size, raw == the family's own argmax), pinning that the record encodes only in-sample information. - A debug_assert in the dispersion-floor arm: it subtracts inflation, valid only for higher-is-better metrics; lower-is-better (max_drawdown/bias_sign_flips) would deflate wrong-signed. Unreached from the two shipped call sites, but optimize_deflated is public — guard the assumption rather than leave it implicit. cargo test -p aura-registry (33) + clippy green. refs #144 |
||
|
Brummel
|
a2959050a4 |
feat(0076): deflate the sweep winner's metric for the number of trials
Adds optimize_deflated beside optimize in aura-registry: the in-sample walk-forward selection now records, on each OOS winner's manifest, how much the winner's metric is inflated by the size of the search it won — a deflated score and (R arm) an overfit probability — without changing which member wins (additive, C23; a regression test pins optimize_deflated's winner == optimize's). - R arm (sqn_normalized/expectancy_r/...): a centred moving-block reality-check. Each member's per-trade R series is mean-subtracted to impose the no-edge null, resampled via the r_bootstrap kernel (extracted to a shared resample_block, byte-identical), and the best-of-K null-max gives overfit_probability = (count(null >= raw)+1)/(n+1) and deflated_score = raw - p95(null). Deterministic given the seed (C1). - total_pips arm: a closed-form expected-max-of-K dispersion floor (inv_norm_cdf + expected_max_of_normals), no probability. The record (FamilySelection on RunManifest) rides the proven serde(default, skip_serializing_if) widening + a compat.rs mirror, so legacy runs.jsonl/families.jsonl lines load unchanged and unstamped sweep/mc/standalone manifests stay byte-identical (C14/C18). The SelectionMode enum reserves a Plateau slot for #145. CLI: walk-forward stamps both arms; runs family <id> rank surfaces a human-readable deflated/overfit line. Quality review (re-dispatched after the implement-loop's quality gate exhausted) caught a real defect: on the R arm with positive resamples but no member carrying trade_rs (a family loaded from disk — trade_rs is serde-skipped), the null was all-NEG_INFINITY, turning deflated_score into +inf. Fixed by collapsing "no usable null" (zero resamples OR no trade_rs) into one degenerate floor (deflated == raw, overfit == 1.0); pinned by optimize_deflated_no_trade_rs_floors_instead_of_infinity. Verified: cargo test --workspace and cargo clippy --workspace --all-targets -D warnings both green. Frictionless Stage-1 R; n_eff advisory and reload-time recompute deferred (spec 0076 Out of scope). closes #144 |
||
|
Brummel
|
f286bb85f7 |
feat(0075): walk-forward strategy-selectable + R-reporting (iter 1)
Make `aura walkforward --strategy stage1-r [--real <SYM>]` roll IS->OOS windows, sweep the stage1-r grid in-sample, pick the winner by an R metric (sqn_normalized), run it out-of-sample, and report per-window + pooled OOS R-metrics. The bare SMA walkforward path is byte-identical (its dispatch arm is verbatim today's body; the pooled `oos_r` block is emitted only when a window carries an `r` block). Engine: - RMetrics gains an in-memory `trade_rs: Vec<f64>` (realised R per closed trade), excluded from serde (`#[serde(skip)]`) and from a hand-written PartialEq, so the C18 wire shape and every existing equality assertion / round-trip stay unchanged. summarize_r now retains the per-trade R vector it used to drop. - New `r_metrics_from_rs(&[f64])` reduces a flat (pooled across-window) R series to RMetrics. Its R-distribution arithmetic is copied verbatim from summarize_r (the byte-pinned floats must not be algebraically refactored); the two copies are guarded in lockstep by a cross-reducer equality test. net_expectancy_r = expectancy_r (exact at the Stage-1 cost=0 invariant); conviction_terciles_r = [0,0,0] (per-trade conviction is not pooled). CLI: - walkforward gains --strategy + the four stage1-r grid flags (reusing parse_csv_list / Stage1RGrid); walkforward_family is strategy-dispatched. - Windowed stage1-r helpers: stage1_r_sweep_over (reduce-mode folded IS sweep, O(trades)/member) and run_oos_r (non-reduce OOS run, for the stitched pip-equity curve), plus stage1_r_space. Frictionless Stage-1 R (costs are Stage-2). Verified: cargo build clean, full `cargo test --workspace` green (SMA walkforward, synthetic mc, stage1_r_single_run_output_golden, and the C18 round-trips all preserved), clippy -D warnings clean. Monte-Carlo R-bootstrap is iter 2. refs #139 |
||
|
Brummel
|
89d967a940 |
feat(0067): SQN100 metric + stage1-r sweep --trace
Two additive follow-ups to the cycle-0065/0066 R-sweep surface. Part A (#130) — n-normalized SQN (SQN100): RMetrics gains sqn_normalized = (mean_R/stdev_R)·√(min(n,100)), a turnover-robust ranking objective comparable across sweep members with different trade counts. The raw sqn is kept byte-identical (computed sqrt(n)*mean/sd, NOT via a factored shared ratio, so the existing metric does not drift even by 1 ULP). The field carries #[serde(default)] for C18 back-compat (a pre-0067 r: block deserialises with sqn_normalized = 0.0). The registry learns Metric::SqnNormalized (opt-in; the default ranker and raw sqn are unchanged), so `runs family rank sqn_normalized` works. The 0066 single-run golden is re-baselined (the r: block gains the field; for n=3 ≤ 100 the cap is inactive so sqn_normalized == sqn). Part B (#135) — wire --trace for the stage1-r sweep: stage1_r_sweep_family now honours --trace, persisting each member's equity/exposure/r_equity under runs/traces/<n>/<member_key>/ via the existing persist_traces_r (mirroring momentum_sweep_family); the interim run_sweep refusal guard is dropped. --trace is now symmetric across all three sweep strategies and a swept member is chartable (chart <n>/<member> --tap r_equity). Tests: SQN100 cap / below-cap / empty + serde back-compat (engine); rank-by sqn_normalized + unknown-metric message (registry); an E2E guard that both SQN values fold from recorded records and agree below the cap (the cross-crate column seam); inverted stage1-r --trace persistence + a CLI rank-by- sqn_normalized integration. Full workspace suite green (527 passed, 0 failed); clippy --all-targets -D warnings clean. Derived fork decisions logged on #130 and #135 (cap = fixed const 100; opt-in metric, default ranker unchanged; serde(default); Part B mirrors momentum_sweep_family + reuses persist_traces_r). closes #130 closes #135 |
||
|
Brummel
|
b8ed6f4858 |
feat(stage1-r): R sweep families + rank-by-SQN
Realizes #133 (cycle 0066): SQN becomes the single-number objective for ranking a Stage-1 sweep family by R signal quality. aura-registry — metric_cmp learns three higher-is-better R metrics (sqn, expectancy_r, net_expectancy_r) that reach into RunReport.metrics.r; a member with r:None sorts to the worst position (treated as -inf via total_cmp), so a pip-only family ranked by an R metric degrades to ordinal order without error. The UnknownMetric "known:" list is extended. No persistence change — the family store already carries the r block. aura-cli — `aura sweep --strategy stage1-r` sweeps the stage1-r harness over a fast×slow SIGNAL grid (4 members), each folding the dense R-record via summarize_r so its RunReport carries r:Some(..). The grid varies only the signal: the stop and sizing are held fixed because risk_budget is R-invariant (#128) and bias.scale is sign-only under flat-1R (both degenerate axes for the ranked metric), and because the stop defines the R unit — varying it would break cross-member SQN comparability (the motivation for the deferred SQN100, #130). The stage1-r topology is extracted into a shared stage1_r_graph(.., fast_len, slow_len) helper: the single run binds Some(2)/Some(4) (output byte-unchanged), only the sweep floats the knobs. Per-member --trace tracing is deferred (a follow-up): rather than accept the flag and silently write nothing (a quality-review catch), `aura sweep --strategy stage1-r --trace` is refused explicitly (exit 2); --name records the rankable family. Verification — a golden characterization test (stage1_r_single_run_output_golden, added against HEAD before the refactor) verifies the helper extraction is byte-preserving; full workspace suite 521 passed / 0 failed; clippy --all-targets -D warnings clean. The grounding-check that signed the spec first BLOCKed a false "already byte-pinned" claim, which is why the golden is added rather than assumed. All load-bearing forks (signal-only grid, None-sorts-worst, --trace refusal, helper extraction, golden shape) are recorded on the #133 decision log. closes #133 refs #117 #130 |
||
|
Brummel
|
c6c1d3bb10 |
feat(stage1-r): rename the exposure_sign_flips metric to bias_sign_flips (#126)
Complete the typed-field half of the exposure->bias rename - the one the cycle-0065
close audit flagged as aging worst (a serde-stable on-disk key).
- RunMetrics.exposure_sign_flips -> bias_sign_flips with
#[serde(alias = "exposure_sign_flips")], so legacy runs.jsonl still deserialises
(pinned by the legacy-read tests, which keep the old key); new output serialises
bias_sign_flips (the byte-pin tests updated to the new key).
- The registry rank metric: Metric::ExposureSignFlips -> BiasSignFlips, the rank
string accepts BOTH "bias_sign_flips" and "exposure_sign_flips" (CLI back-compat),
the error/listing updated; the rank tests exercise both names.
- The MC aggregate field renamed too (McAggregate is hand-rendered to JSON, not
serde-derived - no alias needed).
Scope held to the typed metric. The strategy-output PARAM NAMESPACE - the
.named("exposure") Bias instance, its exposure.scale knob path (Composite::param_space
derives the knob from the node name), and the exposure_scale manifest label - is a
coherent ~30-site, sweep-pinned surface (the implement loop correctly flagged renaming
the knob as an unscoped expansion touching walkforward + the cli_run byte-pins). Kept
uniformly as "exposure" and deferred to a follow-on so the cluster renames as one
consistent unit. SimBroker's pre-reframe exposure concept and the on-disk "exposure"
tap label stay by the #117 fork decision.
cargo test --workspace green; clippy --workspace --all-targets -D warnings clean; a
live `aura run --harness stage1-r` emits "bias_sign_flips".
closes #126
refs #117
|
||
|
Brummel
|
b4e84335c4 |
feat(stage1-r): Sizer + RiskExecutor + R-metric enrichment (iter 2)
Iteration 2 of cycle 0065 (Stage-1 R signal quality) — the node + metric layer. Builds on iter-1's PositionManagement dense R-record + core summarize_r. What ships: - PositionManagement gains a 4th `size` input (slot 3 -> col 10), fed by the Sizer. R is computed size-INVARIANTLY (pure stop-distance ratio), so size never touches realized_r — pinned by a RED test (scaling size leaves every realized_r unchanged) at the node and again end-to-end through the executor. - Sizer (aura-std): `size = risk_budget / stop_distance` — flat-1R (risk_budget 1.0 => one risk unit per trade, size inversely proportional to the stop, not a constant). Reads bias (firing/presence) + stop_distance; the Stage-2 fixed-fractional sizer slots in unchanged (swap risk_budget for risk_fraction*equity, same node shape). - summarize_r enrichment: SQN (sqrt(n)*mean_R/sample-stdev_R; n<2 or zero-variance -> 0), conviction_terciles_r (E[R] by |bias_at_entry| tercile), net_expectancy_r (gross minus one round-trip cost per trade, charged in R via the latched_dist recovered from the entry_price/stop_price columns). summarize_r gains a `round_trip_cost` param (price units). The net-of-cost recovery is tested through the real producer->consumer seam, not just hand-built rows. - RunMetrics.r: Option<RMetrics> with #[serde(default, skip_serializing_if = "Option::is_none")] — legacy runs.jsonl (no `r` key) deserialise to None and a pip-only run's on-disk shape stays byte-unchanged (C14/C18 back-compat). Every RunMetrics literal threaded (report.rs, aura-registry, aura-engine/mc). - RiskExecutor (aura-engine integration-test fixture, sibling of vol_stop_composite): FixedStop -> Sizer -> PositionManagement behind open bias+price input roles, price fanned to both the stop and PM, the Sizer's size into PM. Bias is produced in-graph from the single price source (a second bias *source* would k-way-merge into separate cycles and mark stale prices). The Veto is a DOCUMENTED SEAM, not a runtime node (a pass-through identity is what C19/C23 DCE deletes). Tests: the composite bootstraps + runs + folds to the documented hand value; R invariant under risk_budget while the size column scales; a live-folded RMetrics survives the RunMetrics serde round-trip. The dense-record size column (10) is brought under the cross-crate layout guard (stage1_r_e2e r_col_indices_match_producer_field_layout) so the executor fixture's size-invariance read is drift-protected like the others. Scope: the CLI/recording surface (#129) is sub-split into a separate iteration 3 and is NOT in this commit. Verified: cargo build --workspace clean; cargo test --workspace 500 passed, 0 failed; cargo clippy --workspace --all-targets -D warnings clean. refs #117 #127 #128 #129 |
||
|
Brummel
|
9b2adcbb1b |
feat(trace-store): name classification, family resolver, write-guard
aura-registry gains the read-side of total trace-name resolution and the
write-guard that makes it total:
- NameKind {Run, Family, NotFound} + TraceStore::name_kind — top-level index.json
is a single run; else ≥1 member subdir with index.json is a family; else absent.
- read_family -> Vec<FamilyMember{key, traces}> — every member of a family, read
via the existing read("<name>/<key>"), sorted by key (deterministic, C1); an
absent/empty family reads as empty (treat-as-absent), a malformed member
propagates Parse.
- ensure_name_free(name, WriteKind) + a TraceStoreError::NameTaken variant —
refuses only cross-kind name reuse, so the one ambiguous on-disk state (a name
used by both a run and a family) is unreachable; same-kind overwrite stays Ok.
Task 1 of plan 0061. Engine untouched (C9/C14). Tests: cargo test -p aura-registry
green (18 passed, incl. the 3 new), clippy -D warnings clean.
Note: the implement-loop quality gate exhausted its re-loop oscillating on the
method NAME (ensure_name_free "implies unused" vs a rename diverging from the
ratified plan/callers). Orchestrator overrule: the ratified name stands — it is
fixed by the spec/plan/#107 and the Task-3 call sites, and the doc comment already
states that Ok means "writable here", not "unused". The implementation is green,
clippy-clean, and spec-faithful.
refs #107
|
||
|
Brummel
|
8bb5256041 |
feat(trace-charts): persist recorder taps to disk (iter 1)
Iteration 1 (persist half) of the visual face — spec 0056 / plan 0055:
tap data from a graph and save it to disk so it can later be charted
(the serve/chart half is iteration 2).
- ColumnarTrace (aura-engine): a drained tap <-> columnar SoA form
({tap, kinds, ts, columns}); values coerced to f64 for plotting with
the base type preserved in `kinds` (C7); to_rows rebuilds uniformly-f64
rows for the serve-side join. Pure (C1), beside join_on_ts/summarize.
- TraceStore (aura-registry): a per-run directory under
runs/traces/<name>/ — one <tap>.json per tap + an index.json (manifest
+ tap order), beside the families.jsonl sibling store. Missing run =
NotFound (no panic); corrupt file = Parse{file}.
- CLI: a shared persist_traces helper threaded into run_sample /
run_macd / run_sample_real via an orthogonal `trace: Option<&str>`;
new arms `aura run [--macd] --trace <name>` and `--trace <name>` on
`run --real`. The default (no --trace) run is byte-for-byte unchanged.
Verified by the orchestrator: `cargo test --workspace` green (incl. the
4 columnar_trace, 3 trace_store, 4 cli_run e2e, and the 4-tuple
parse_real_args tests); `cargo clippy --workspace --all-targets -D
warnings` exit 0. Implementer deviations folded in, all
compiler-prescribed: #[derive(Debug)] on RunTraces (the RED tests
panic-format a Result over it), #[allow(clippy::type_complexity)] on
the parse_real_args 4-tuple (mirrors the crate's sample_harness idiom).
Default-run byte-identity stays pinned by run_prints_json_and_exits_zero.
refs #101
|
||
|
Brummel
|
03f80f0a95 |
docs(aura-registry): note Registry::open owns two directory-co-located stores
Registry::append_family / load_family_members write+read a fixed-name families.jsonl sibling via self.path.with_file_name(...), so isolation between registries is per-directory, not per-filename — two Registry::open calls with different runs filenames in one directory share a single family store. The sibling was documented on append_family but not on Registry::open, where a caller binds the path; this records the per-directory-isolation consequence at the bind point. Pure rustdoc ratify of existing (cycle 0045) behaviour. closes #82 |
||
|
Brummel
|
58bff32e6a |
fix(aura-registry): Registry::load tolerates legacy pre-0047 bare-float params
The cycle-0047 typed-Scalar params migration ( |
||
|
Brummel
|
20511d5d5c |
test: red for #83 — Registry::load must read a legacy pre-0047 bare-float params line
RED test pinning the real defect of bug F5 (surfaced by the milestone
"The World, part II" fieldtest). Registry::load cannot deserialize a
runs.jsonl line written before the cycle-0047 typed-Scalar params
migration, where each param value is a bare JSON float (e.g.
["sma_cross.fast",2.0]) rather than the tagged {"F64":2.0}. The test
feeds a minimal autonomous legacy line and asserts it loads — the bare
float read back via the documented f64 coercion — instead of erroring at
the first param.
Fails RED at the diagnosed cause (Parse "expected value" on the bare
float). GREEN side (back-compat read, no change to the typed-Scalar
forward format) follows.
refs #83
|
||
|
Brummel
|
86746e3d5d |
refactor(aura-core): Scalar as a native tagged enum, disjoint from Cell; typed RunManifest.params
Scalar was `struct { kind: ScalarKind, cell: Cell }` — "a Cell wearing a kind hat." The recorded reason for that shape was migration ease, which is not a design rationale (CLAUDE.md: rationale != effort), so the struct had no substantive defense. Redefine it as the native tagged union it conceptually is:
enum Scalar { I64(i64), F64(f64), Bool(bool), Timestamp(Timestamp) }
This is substantively better on four axes: kind/bits skew becomes unrepresentable (illegal states gone); accessors panic loudly on the wrong variant instead of a release-mode silent bit reinterpret; PartialEq derives the documented IEEE-754 / cross-kind value semantics (the hand-roll, needed only because the struct inherited Cell's bitwise compare, is gone); and serde is a plain derive emitting the externally-tagged wire form ({"I64":10}/{"F64":2.5}) — the private ScalarRepr shadow enum that motivated this was never needed. It is also C7-honest: erased-on-the-hot-path (Cell) and self-describing-at-the-edge (Scalar) are two disjoint types bridged by explicit conversion.
The whole public API is preserved (Scalar's fields were private), so call sites do not churn: the enum change is contained to scalar.rs, where cell() now encodes and from_cell() decodes per kind. Cell and ScalarKind are untouched.
With Scalar serializable, lift RunManifest.params from Vec<(String, f64)> to Vec<(String, Scalar)>: the param's kind (an i64 length vs an f64 scale) now survives into the C18 record (runs.jsonl) and the CLI JSON instead of collapsing to f64. scalar_as_param_f64 is deleted; sim_optimal_manifest passes typed params through. This is a deliberate wire-shape change — params now render as tagged scalars; the JSON-asserting tests are updated to the new shape on purpose.
Hand-authored manifest params across the CLI's single-run/mc/macd sites use honest kinds (lengths -> i64, scales -> f64) so they match the sweep path (which already derives correct kinds via zip_params); their in-binary JSON assertions are re-tagged accordingly.
Walk-forward fork resolves with no code change: WindowRun.chosen_params stays Vec<Scalar> (the serializable record carrier), reduced to f64 only at the param_stability statistic boundary (scalar_as_f64 retained). Glossary 'cell' entry updated to describe Scalar as the disjoint tagged union, not 'a cell plus its kind tag'.
Gates: build --all-targets, test --workspace (incl. new scalar_serde_round_trips), clippy -D warnings, doc --no-deps — all clean.
|
||
|
Brummel
|
43716be10e |
refactor(aura-engine): Cell into the construction path — param-plane base/frontend split
Cell becomes the carrier of the construction path; Scalar narrows to the author/render boundaries. The validated/enumerated param point carries no redundant kind (it lives once, in the declared param-space); at the author edge the kind is a checksum — two independent sources, the typed value vs. the slot — so the self-describing Scalar stays there. The name->slot binding is dynamic (C23), so the check is necessarily a runtime one and the value must self-describe for it. Base/frontend split (the AnyColumn push/push_cell pattern one level up): compile_with_cells / bootstrap_with_cells are the kind-check-free base; compile_with_params / bootstrap_with_params are the frontend that adds only the per-value kind checksum, strips to cells (new Scalar::cell(), the partner of Scalar::from_cell), and delegates. lower_items loses its per-primitive kind-check; PrimitiveBuilder::build and the std node builders (sma/ema/exposure/lincomb) read cells. Boundary: construction -> Cell (PrimitiveBuilder::build, lower_items, GridSpace.axes/points, SweepPoint.params, the sweep closure). Author edges stay Scalar (GridSpace::new, bind, compile_with_params/bootstrap_with_params). walkforward chosen_params stays a self-describing Scalar report record (Option A — WalkForwardResult carries no space); the cell winner is reconstructed once at the WindowRun site via from_cell. injective-check moved ahead of the arity-check in the frontend to preserve the pre-split error order (DuplicateParamPath before ParamArity). The lossy i64->f64 param projection (scalar_as_f64 / scalar_as_param_f64) is deliberately untouched — a separate follow-up. Behaviour-preserving (C1): build --all-targets / test / clippy -D warnings / doc all clean across the workspace. |
||
|
Brummel
|
cd3d1ca9ed |
refactor(aura-core): split Scalar into a tag-free Cell + ScalarKind
Motivation
----------
`Scalar` was a tagged enum (I64/F64/Bool/Ts), so every scalar value
physically carried its own kind tag. But the kind is already known from
the schema/port/column the value flows through (C7: the type is a
property of the column, not of the value — the hot path is already
columnar `Column<T>`, and `AnyColumn::get` *reconstructs* the tag from
the column on the way out). The per-value tag was therefore redundant
with the kind the surrounding context already holds.
That redundancy had three costs:
* It baked an implicit `match` (a branch) into every function that read
a Scalar payload — even where the caller statically knew the type.
The tag could never be exploited away.
* Size: a tagged enum is tag + payload = 16 bytes (f64/i64 alignment),
twice the 8 bytes the value needs. A `Column<Scalar>` would be double
the memory and half the cache utilisation.
* It is the shared root of several downstream papercuts we keep hitting
— the lossy f64 manifest field, the `unreachable!` panic on a
non-numeric param, the serde-tag question — all symptoms of "the type
is baked into the value".
Change
------
Introduce `Cell`: a type-erased 64-bit word (`struct Cell(u64)`) that is
not readable without external type context. It is constructed per base
type (`from_i64/from_f64/from_bool/from_ts`) and read only by naming the
type at the call site (`i64()/f64()/bool()/ts()`) — each a branch-free
bit-cast. The hot path resolves the kind once at the boundary (from the
schema) and then reads natively, with no per-value branch. `Cell` knows
nothing of `Scalar` or `ScalarKind`; the dependency is strictly one-way,
and it lives in its own `cell.rs` (more is planned on top of it).
`Scalar` becomes `struct { kind: ScalarKind, cell: Cell }` — the
self-describing form for the dynamic boundaries (builder binding,
serialization, rendering), built on top of `Cell`. Its `as_*` accessors
now `debug_assert` the kind and return the native value (free in
release); calling the wrong accessor is a caller bug, not a checked
`Option`. The variant constructors `Scalar::I64(..)` become associated
fns `Scalar::i64(..)`.
`PartialEq` is hand-written (not derived) to preserve the former enum's
value semantics: kinds must match, then native payloads compare, so f64
keeps IEEE-754 behaviour (`NaN != NaN`, `+0.0 == -0.0`) and a kind
mismatch is never equal even when the raw words coincide. A fixture
(`scalar_eq_is_value_not_bitwise`) pins exactly the cases where bit- and
value-equality diverge, so it can't silently regress. `Cell`'s own
`Eq`/`Hash` stay bitwise — correct for a raw word.
The change is behaviour-preserving: Scalar's observable behaviour is
identical to the pre-Cell enum (the value-equality fixture proves it);
only the internal representation changed. The ~440 call sites across the
workspace are a mechanical constructor rename plus ~12 destructuring
sites (match-arms / `let`-patterns) rewritten to `kind()` + `as_*`.
Verified: cargo build --workspace --all-targets, cargo clippy --workspace
--all-targets -- -D warnings, cargo test --workspace — all green.
|
||
|
Brummel
|
fb8cabf13f |
feat(aura-engine): sweep named-binding — zip_params + SweepFamily.named_params (C12 #57)
Thread a derived named view of a sweep point so consumers stop hand-zipping param_space() names onto the bare &[Scalar]. The free function zip_params(space, point) -> Vec<(String, Scalar)> in aura-core is the one shared projection; GridSpace retains the ParamSpec list it already receives in new() (it was validated then discarded); SweepFamily carries it (stamped once in sweep_with_threads) and exposes named_params(i). The three CLI hand-zip sites collapse to one zip_params call each. The run-closure signature `Fn(&[Scalar]) -> RunReport` is byte-unchanged — the named view is a derived projection (C23: slot is identity, name is derived), not a closure-currency change — so #52's RandomSpace plugs into the same sweep() execution layer with zero signature reconciliation (#71 firewall held). sim_optimal_manifest now takes typed Vec<(String, Scalar)> and does the lossy f64 collapse internally (one place; the manifest's f64-precursor field owns its own lossiness — the typed-manifest upgrade stays the deferred typed-param-space item, a non-goal here). All eight call sites migrated: three sweep closures pass zip_params, five hand-listing callers (run_sample, run_sample_real, mc_family, run_macd, run_sample_seeded) pass Scalar::F64 literals. Behaviour-preserving: the collapse output is identical to the old per-site zip, so aura sweep / walkforward / run output is byte-identical; SweepFamily.space threaded into the aura-registry optimize test literal. Verified by the orchestrator: cargo test --workspace green (4 new tests: zip_params x2, sweep_family_carries_param_space, family_named_params_round_trips), cargo clippy --workspace --all-targets -D warnings clean. (rust-analyzer emitted stale mid-edit diagnostics; the real cargo build/test is clean.) refs #57 |
||
|
Brummel
|
c2756732d4 |
refactor(aura-registry): split FamilyRunRecord family_id into {family, run} + derived id
Behaviour-preserving normalisation of the lineage record. The fused
`family_id: String` ("{name}-{counter}") is split into its two factors —
`family: String` + `run: usize` — and the user-facing handle becomes a derived
`family_id()` method ("{family}-{run}"), never stored or parsed.
Why: the fused field forced the counter logic to generate-and-check candidate
strings (to stay robust to '-' inside a name) rather than read it; with `run` a
plain int, `next_run` is a clean numeric max+1 over the field. The split is more
normalised (one fact per field) and the only property the fused token bought — a
single paste-able CLI handle — is recovered by deriving it. Storage and all
user-visible output stay byte-identical: append_family still returns
"{name}-{run}", Family.id is still "{name}-{run}", and the CLI prints the same
"family_id" lines (the json! key + the returned String, both unchanged).
Internal-only: FamilyRunRecord and group_families (now keyed on the (family,
run) tuple, first-seen order preserved) change; lib.rs re-exports, the
extractors, and all of aura-cli are untouched (everything downstream goes
through Family.id + append_family's returned String). The on-disk
families.jsonl key changes ({"family","run"} vs {"family_id"}) — a gitignored
runtime artifact with no committed fixture, and tests round-trip within a temp
dir, so no migration is owed. Doc reconciliation: the lineage/lib module headers
and the C18 ledger realization note now describe the split shape.
Verification: a workflow ran the refactor in an implementer agent, then three
adversarial lenses (behaviour-preservation, new-logic correctness,
completeness/doc-lag) tried to refute it — the first two found nothing, the
third flagged the four doc-lag sites now fixed here. Self-run gates:
cargo test --workspace green (registry 11, incl. a strengthened round-trip test
pinning the split fields + derived handle), cargo clippy --workspace
--all-targets -D warnings clean, cargo doc clean.
|
||
|
Brummel
|
1d9555c468 |
feat(aura-engine,aura-registry): producer-supplied window + registry lineage (0045 iter 1)
Iteration 1 of spec 0045 — the engine + registry core for storing orchestration families as linked records. aura-engine: add Source::bounds() — the inclusive (from, to) data extent a producer will stream, known without materialization (#71 firewall) — and a free window_of() that folds the union extent across a run's sources. This is the producer-supplied replacement for the prices.first()/.last() Vec scans at the CLI call sites (those migrate in iteration 2); a lazy producer now reports its window so stored lineage stays byte-identical whether a source is eager or streamed. aura-ingest: M1FieldSource stores its requested [from_ms, to_ms] window at open and reports it via bounds() (normalized to epoch-ns; None when open-ended — the archive-extent query is a deferred non-goal). aura-registry: new lineage module — FamilyKind / FamilyRunRecord (a RunReport stamped with family_id + kind + ordinal) / Family, plus Registry::append_family (assigns family_id = "{name}-{counter}" via a per-name generate-and-check counter, writes members to a sibling families.jsonl) and load_family_members; group_families re-derives the families (the round-trip), and the three *_member_reports extractors pull the per-kind member reports. The flat runs.jsonl store and its append/load/rank_by/optimize API are byte-for-byte unchanged (a test pins the two stores' disjointness). C9 preserved: aura-engine gains no registry dependency. New dep: serde derive on aura-registry (per-case policy, same basis as serde_json already is). Verification: cargo build --workspace; cargo test --workspace (228 green, incl. 3 new engine bounds/window_of, 1 data-gated ingest bounds run against real archive data, 5 new registry lineage round-trip/counter/disjointness tests); cargo clippy --workspace --all-targets -D warnings clean. The CLI surface (aura mc, family-aware aura runs, the window_of migration) is iteration 2. refs #70 |
||
|
Brummel
|
682e459554 |
audit: cycle 0041 — drift-clean (Source ingestion seam)
Architect drift review over 8b330e3..HEAD (the Source-seam cycle #71 plus the interim #67 optimize work and refactors). No semantic or contract drift: the seam faithfully preserves C3 (ms→epoch-ns at the one ingestion boundary), C4 (tie-by-source-index, byte-for-byte), and C7 (field-per-source SoA); the full suite is byte-identical green and the gated streaming tests pass on real data. Regression gate: `cargo test --workspace` green (the project declares no dedicated regression script; the suite is the gate). Tidied stale prose the cycle left behind (no behaviour change): - docs/design/INDEX.md (C12): replaced the cycle-0011 "deliberate eager gap" status note with a cycle-0041 realization note — the producer `Source` seam + streaming `M1FieldSource` now deliver per-source O(one-chunk) streaming; precisely scopes what remains open (cross-*sim* Arc<[T]> window sharing, still unbuilt until the orchestration families #66/#68/#69 consume it). - aura-core/src/lib.rs: dropped the now-shipped "Source trait + data-server ingestion" from the module doc's "still to come" list (aura-engine's twin line was fixed in the cycle; aura-core's was missed). - aura-ingest/src/lib.rs: module header now documents both coexisting paths (eager load_m1_window/M1Columns vs lazy streaming M1FieldSource). - aura-registry/src/lib.rs: rank_by/optimize doc comments linked to the private `metric_cmp` (a public→private intra-doc-link warning from the #67 commit); demoted to plain code spans, so `cargo doc --workspace` is now warning-clean. |
||
|
Brummel
|
2b56e090a9 |
feat(aura-registry): optimize — argmax over a SweepFamily by a named metric
Ships C12 orchestration axis 2 (optimize): pick the single best point of a sweep family by a named RunMetrics field. The GREEN half of the RED-first iteration whose executable-spec landed in the previous commit. optimize(&SweepFamily, metric) -> Result<SweepPoint, RegistryError> returns the whole winning SweepPoint — its params coordinate AND its RunReport — because the caller needs the params that won, not just the metric. "Best" is fixed per metric (total_pips higher-is-better; max_drawdown / exposure_sign_flips lower-is-better); ties resolve to the earliest enumeration-order point (the family is in odometer order, and only a strictly-better later point displaces the incumbent). Single source of truth for "best", as #67 required ("reuse rank_by; do not fork"): rank_by's per-metric direction is extracted into a private `metric_cmp` helper that resolves the boundary metric *name* to a closed `enum Metric` once and returns one comparator closure. Both rank_by (sort by it) and optimize (argmax via `reduce`, strictly-better-displaces) now call it, so the per-metric direction lives in exactly one match. rank_by is behaviour-preserving — its existing tests pass unchanged. The sweep executor is untouched. No caller-supplied `direction`: one definition of best per metric, consistent with the shipped rank_by invariant (a caller direction would admit incoherent asks like the worst max_drawdown). An unknown metric is UnknownMetric. A SweepFamily is non-empty by construction (EmptyAxis is rejected upstream), so the argmax always yields a winner. CLI surface: the "pick the best" view is already served by `aura runs rank <metric>`, whose first line is the optimum over the persisted registry; optimize() is the new *programmatic* axis-2 entry, for World programs that hold a live SweepFamily and need the winning params back in-process — which the registry/RunReport path cannot give. No redundant CLI command added. closes #67 |
||
|
Brummel
|
bdc383eb8b |
test(aura-registry): RED executable-spec for optimize argmax — refs #67
The RED half of a RED-first iteration for the optimize/argmax orchestration axis (C12 axis 2). Pins the headline behaviour before the feature exists: optimize(&SweepFamily, metric) returns the single winning SweepPoint — its params coordinate AND its RunReport — chosen by the metric's fixed per-metric sense of best (reusing rank_by's direction semantics), with ties resolving to the earliest enumeration-order point. No caller-supplied direction: that would contradict the shipped rank_by invariant that "best" is fixed by each metric's meaning, and would admit incoherent asks (e.g. the worst max_drawdown). One definition of best per metric. RED for the right reason — optimize is absent (E0425), the sole error; cargo build --workspace stays green (test-only module). GREEN lands in the following commit, which closes #67. |
||
|
Brummel
|
fe11cfea8a |
feat(engine,registry): SweepPoint carries RunReport; add aura-registry (cycle D / iter 2)
Iteration 2 of the run-registry cycle (#33): make the sweep family self-describing and add the registry store. Engine. SweepPoint.metrics (RunMetrics) becomes SweepPoint.report (RunReport), and the sweep closure bound is now Fn(&[Scalar]) -> RunReport. This closes the manifest-per-point gap cycle C explicitly deferred to #33 — each swept point now carries its full (manifest, metrics), the unit the registry indexes (C18). All engine-internal callers, the engine test fixture run_point, and the three sweep tests were rethreaded in the same iteration so the crate stays green; the CLI caller followed in the same change so `cargo test --workspace` is green again. CLI. The sweep_report closure builds a per-point RunReport (manifest params = param_space names zipped onto the point via scalar_as_param_f64; commit/window/ seed/broker as run_sample builds them) and prints via RunReport::to_json. The hand-rolled sweep_point_to_json/json_string/scalar_token are retired (the report renders itself), and the orphaned ParamSpec/SweepPoint imports dropped. Net: aura run, aura sweep, and (next iteration) aura runs all print the same RunReport shape. The two aura sweep goldens now pin the per-point manifest params, NOT the commit (it is the real git HEAD, volatile). Registry. New crate aura-registry: Registry::{open, append, load} over an append-only runs/runs.jsonl (one serde_json line per RunReport), free rank_by (best-first per metric — total_pips desc, max_drawdown/exposure_sign_flips asc), and RegistryError (Io / Parse{line} / UnknownMetric). Five unit tests cover append->load round-trip, missing-file = empty, corrupt-line = Parse{line}, per-metric ranking, and unknown-metric error. Built and unit-tested; NOT yet CLI-wired (aura sweep persistence + aura runs list/rank are iteration 3). Verified: cargo test --workspace green (aura-registry 5, aura-engine 93, aura-cli 7+8, others unchanged); clippy --workspace --all-targets -D warnings clean; aura run stdout shape unchanged; aura sweep now emits full RunReports. refs #33 |