5c2ac982bc2f07f6f89f1e8dada8df8362770760
38 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 claude
|
5c2ac982bc |
refactor: extract the member-run recipe into library crates (#295 part 1)
The shell no longer defines what an aura backtest is. Tasks 1-9 of the shell-boundary cycle — structural extraction, behaviour byte-identical: - aura-runner (new; the C28 assembly position): input binding (the C26 module, moved whole), the C1-load-bearing param<->config translators, harness assembly (wrap_r / run_signal_r / run_blueprint_member and the probe/reopen cluster), axis + risk-regime conventions (bind_axes et al.), the campaign family builders + MC guards, reproduce (process::exit -> RunnerError, shell remaps to identical bytes), the measurement-run orchestration, project loading (Env / cdylib load / charter / staleness, moved whole), the coverage gap-walk (deduplicated onto interior_gap_months), and DefaultMemberRunner — the public implementation of aura_campaign::MemberRunner (ex CliMemberRunner). The MemberRunner trait stays in aura-campaign; the column still imports no harness/data-binding machinery. - aura-measurement (new; seeds C28 rung 3): the IcMetrics/IcKey vocabulary + information_coefficient, verbatim incl. serde derives (#294 duplicate-timestamp semantics move as-is, unresolved). - aura-backtest: the pure per-run scaffold (point_from_params, wf_ms_sizes / fit_wf_ms_sizes, intersect_shared_window). - shell residue: main.rs / campaign_run.rs keep argv/dispatch, argv->document translation, and presentation only; walkforward_summary_json_from_reports now calls the public aura_engine::param_stability instead of its inline twin. - worked example (crates/aura-runner/examples/world_member_run.rs): a World program runs a member backtest through the library alone. Held quality findings, adjudicated: the plan's literal pub-visibility list for binding.rs kept (cosmetic); ~20 refusal sites inside aura-runner (family/member/measure/translate) still process::exit — behaviour-identical today, conversion to RunnerError is filed forward (refs #297); rustfmt line-width drift on re-pathed call sites left for a repo-wide fmt decision. Verification: cargo test --workspace green (1471 passed, 0 failed); cargo clippy --workspace --all-targets -D warnings clean. Byte-identity is pinned by the untouched shell E2E suites (cli_run, measure_ic, run_measurement, research_docs, run_refuses_unrunnable_blueprint, tap_recording — zero assertion edits). Remaining in this cycle: full-workspace c28_layering + shell-content check, ledger amendments (C28 assembly position, C25/C14 control-surface consequence, C26 realization note). refs #295 |
||
|
claude
|
a9d36ddd70 |
feat: measurement's first deflatable metric — the Information Coefficient
Give a measurement run a standalone post-run quality score: the Information
Coefficient (IC), corr(signal_t, forward_return_{t+h}), with a permutation
null model. Before this a measurement run persisted only tap names and series
(MeasurementReport) — inspectable but not rankable or deflatable. IC is the
first metric measurement supplies: the tap-to-scalar reduction the C28
process-column / metric-interface seam anticipates ("a named metric vocabulary
supplied by measurement and backtest instead of baked in R-only").
Surface: `aura measure ic <run> --signal <tap> --price <tap> [--horizon]
[--permutations] [--seed]`. It reads the run's two recorded tap traces, builds
the forward-return series offline over the recorded price spine, aligns
signal -> return by exact timestamp, computes the IC, and prints it with a
one-sided permutation-null overfit probability (Laplace-smoothed, mirroring the
R deflation arm's formula in optimize_deflated).
Placement follows the C28 ladder: the generic pieces — pearson_corr and a
Fisher-Yates permute over SplitMix64 — land in aura-analysis (the domain-free
statistics foundation, beside resample_block); the IC semantics (forward-return
horizon, the offline join) and the CLI verb land in aura-cli (the shell). The
run path (run_measurement), MeasurementReport, the trace store, and the registry
metric vocabulary are untouched — existing `aura run` output stays byte-identical
(the C18 golden run_prints_json_and_exits_zero stays green).
Causality (C2): the forward-return read close_{t+h} is over a completed run's
recorded trace, not an in-graph node. A look-ahead signal is structurally
impossible in a causal run, so the engineered-vs-noise acceptance property is
unit-tested over hand-built offline series (perfect signal -> IC ~= 1, p < 0.05;
a varying but exactly-uncorrelated signal -> IC = 0, not significant;
constant/degenerate -> the (0.0, 1.0) floor), while the E2E validates the CLI
wiring, report well-formedness, determinism, and the error paths over a real run.
This lands the second, structurally distinct null-model computation (permutation,
beside the moving-block bootstrap behind R). That makes the deferred registry
deflation-vocabulary abstraction (#147 item 2) demand-driven rather than
speculative — the next cycle.
Verified: cargo test --workspace green (incl. the new aura-analysis unit tests,
the aura-cli ic_* unit tests, and the measure_ic E2E); cargo clippy --workspace
--all-targets -D warnings clean; the C18 byte-identity golden unchanged.
closes #290
refs #147
|
||
|
claude
|
a56ab7859d |
refactor: cut the engine's backtest-metrics edge via RunReport<M>
C28 phase 2 (Stratification); realizes item 1 of the deferred #147. The engine's production surface no longer names a backtest-metric type: - RunReport becomes generic over its metric payload M; sweep/mc/walkforward/ blueprint thread the parameter (SweepPoint<M>, SweepFamily<M>, WindowRun<M>, WalkForwardResult<M>). RunManifest stays concrete and engine-owned (its selection: Option<FamilySelection> embeds the foundation-grade analysis type). - summarize and the MC assembly (McDraw/McFamily/McAggregate/RBootstrap/ r_bootstrap/monte_carlo) move to aura-backtest - McAggregate::from_draws reads RunMetrics fields by name, so generifying it is the phase-6 metric-vocabulary abstraction (#147 item 2), still deferred; wholesale relocation is the honest cut. The concrete instantiation lives in aura-backtest as `type RunReport = aura_engine::RunReport<RunMetrics>` + sibling aliases. - the statistics kernel (MetricStats/quantile/resample_block/SplitMix64) moves to the aura-analysis foundation; the engine re-imports it (inner->foundation, legal) and re-exports it so existing consumers stay source-compatible. Dependency inversion in one commit: aura-engine drops aura-backtest from [dependencies] (back to dev-deps for its SimBroker/RunMetrics test fixtures); aura-backtest gains aura-engine. Cycle-free for lib targets - the cycle closes only through the engine's dev-dep edge, the pattern aura-vocabulary already uses. aura-backtest reaches the kernel transitively through the engine re-export, so no aura-backtest -> aura-analysis edge exists (the C28 ladder permits backtest -> {core, engine} only). run_indexed / SplitMix64::next_f64 widened pub(crate) -> pub for cross-crate use. Consumers (registry/campaign/cli/composites/ingest/bench) rewired by import path only, no call-site logic changed. The c28_layering structural test extends to the full ladder: aura-analysis (no aura-* deps), aura-engine ⊆ {core, analysis}, aura-backtest ⊆ {core, engine}. Behaviour-preserving: 1448/0 tests, clippy -D warnings clean, serde shapes byte-identical (C18 - RunReport<M> keeps field order manifest,metrics; the CLI pre-serialized-splice contract unchanged), moved code traceable via git rename detection. Cycle-introduced broken intra-doc links fixed. closes #292 |
||
|
claude
|
94aaa4cde8 |
refactor: split aura-analysis into statistics and backtest metrics
C28 phase 5 (Stratification): the backtest reductions - RunMetrics, RMetrics, summarize_r, r_metrics_from_rs, and the position-event table - move verbatim into aura-backtest::metrics, beside the position_management producer whose record layout they read (the r_col/cost_col lockstep contract is now intra-crate; its stale "aura-std" comment corrected). aura-analysis is reduced to the domain-free half: the multiple-comparison hurdle math (inv_norm_cdf, expected_max_of_normals) and the selection-provenance types (FamilySelection/SelectionMode - kept beside the statistics so RunManifest.selection embeds a foundation-grade type), [dependencies] = serde only. The engine re-export surface is name-unchanged (report.rs re-imports from both crates), so no indirect consumer needed a source edit; aura-backtest moves from aura-engine's dev-dependencies into [dependencies] as a commented TRANSIENT widening of the C28 violation, removed by the phase-2 edge cut on this branch. The campaign drift guard imports its pinned types from their new source crates. Behaviour-preserving: 1448/0 tests, clippy -D warnings clean, serde shapes byte-identical (C18), moved code traceable via git copy detection. closes #291 |
||
|
claude
|
b39fd63396 |
refactor: split the aura-std roster into C28 layer crates
Phase 4 of the Stratification milestone. aura-std held four C28 ladder layers in one roster; this cuts them into layer-aligned, aura-core-only node crates so the import direction is enforced by the crate graph: - aura-std — engine nodes only (arithmetic/logic/rolling + sinks) - aura-market — session, resample - aura-strategy — bias, stops, sizer, cost-model machinery - aura-backtest — sim_broker, position_management - aura-vocabulary — the relocated closed std_vocabulary roster Node modules move verbatim (byte-identical renames); consumers are rewired by import path only. A new structural test (aura-vocabulary/tests/c28_layering.rs) asserts each node crate's [dependencies] stay within its C28-permitted inner set, catching the acyclic-but-outward violation the compiler misses. Behaviour byte-identical: full workspace suite green (1448 tests), no golden edited, clippy -D warnings clean. C28 Status block updated. closes #288 |
||
|
claude
|
13d8500402 |
audit: cycle-close tidy for #251 — fingerprint honesty, lockstep + surface-list guards
Architect drift review (cycle bfb8648..0389399): no domain-invariant breach. What holds: aura-bench is bin-only dev tooling with zero production reverse deps (invariant 8/C13); every committed surface runs on seeded synthetic data in scratch temp projects (invariant 9); run_reps enforces cross-rep fingerprint identity and the campaign fingerprint carries integer ordinals across the child boundary, consistent with C1 and its cross-command ULP carve-out; zip/clap per-case dependency notes in place (C16), and the ledger correctly gains no entry — tooling lives in the project facts + README. Three drift items, resolved in this commit: - [medium] the winner_fingerprint doc claimed walk-forward regressions fail the bench; the walk-forward stage in fact emits an empty realization record, so coverage is only indirect (via the bootstrap trade count). Doc now states the gap honestly; the engine-side fix is parked on #279. - [low] the fingerprint's lenient JSON parse would silently erode on a registry-schema rename: a new lockstep test builds a real CampaignRunRecord from the aura-registry/aura-analysis/aura-engine types (new dev-deps) and asserts the extracted fingerprint, so a rename breaks this crate's tests instead. - [low] LIBRARY_SURFACES was hand-synced with measure_surface: a new test pins that every listed surface measures without a binary path. Regression gates on the closing tree: cargo test --workspace 1379 green, clippy -D warnings clean, doc build clean, aura-bench run exit 0 with five 'fingerprint OK' at <=2% drift. No baseline moved in this commit — the bench baselines were pinned and verified in the feature commit. refs #251 |
||
|
claude
|
0b116e4105 |
feat(bench): scaffold aura-bench — baseline compare core + synthetic data plumbing
Tasks 1-2 of the bench-harness plan (refs #251): the new dev-only workspace bin crate with the report-only comparison core (BaselineDoc serde, per-metric relative drift, 10% NOTICE threshold, fingerprint-equality exit code) and the deterministic synthetic inputs (seeded LCG price walk, the proven 48-byte Delphi-record M1 zip archive writer, self-deleting scratch dirs). The archive writer is deliberately a third copy of the test-fixture packer (aura-cli tests / aura-ingest each carry one), byte-compatible by construction and pinned here by a round-trip test through the real ingest reader. The binary refuses debug builds (exit 2) — wall-clock numbers from a debug profile measure the profile, not the code — pinned by an E2E spawn test. Temporary #![allow(dead_code)] markers cover the not-yet-wired core until the CLI glue lands; the surfaces and driver follow. |
||
|
claude
|
cf94377f30 |
feat(campaign): parallel cell loop on the shared rayon pool
The flip: within each K-instrument chunk (the structural residency bound from the previous commit), all cells run concurrently on rayon's process- global pool — the same pool the engine's member/window fan-out already shares, so cells x members x windows feed one work-stealing scheduler and the per-cell serial seams (ingest transpose, single-threaded bootstrap) overlap with other cells' work. Results land in index-addressed slots and the ordered outputs are rebuilt in document order: byte-identical outcomes across worker counts and bounds (C1). Fault routing, restructured for a loop that cannot mid-iterate abort: member/window faults stay #272-contained inside run_cell; non-containable faults (incl. registry writes — infrastructure, not cell pathology) latch an atomic abort flag so unstarted cells never run, and after the join the lowest document-order fault among completed cells propagates as the run's error. Nothing partial is persisted at run level on the fatal path. Property pins: determinism across thread counts and bounds (1-thread/K=1 == 8-thread/K=2), distinct live instruments never exceed K (refcounting stub runner, 8-thread pool), a dead family store is run-fatal with no partial campaign-run record, member faults stay contained under the parallel loop. E2E fixtures (previous commit) pin the CLI prose and document-order persistence across K. Verified: workspace suite green, clippy -D warnings clean. rayon enters aura-campaign under the amended C16 per-case policy (comment in Cargo.toml, mirroring aura-engine). closes #277 |
||
|
claude
|
b048923f1c |
feat(engine): shared rayon pool for the disjoint-parallel core
Replace the per-call std::thread::scope fan-out in run_indexed with one process-wide rayon pool. run_indexed's callers nest — sweep (grid points), monte_carlo (seeds), and walk_forward (window bounds), where a walk_forward window runs an inner sweep — and each std::thread::scope spawned its own available_parallelism() OS-thread batch, so the two levels multiplied to ~available_parallelism()^2 threads (measured 150-240 concurrent on a 24-core box during a 42-cell campaign, ~10x oversubscription). rayon's pool is process-wide and persistent: a nested par_iter work-steals within the same N workers instead of spawning, so live workers never exceed the pool size. Determinism (C1) is preserved bit-for-bit: run_indexed collects via IndexedParallelIterator in enumeration order (not completion order), and the float aggregation (summarize_r / McAggregate / stitch) runs after the ordered collect, so IEEE-754 non-associativity introduces no variance. The existing _with_threads(1) == _with_threads(8) == public determinism tests stay green, now driving local rayon pools of 1 and N workers via ThreadPool::install. Structural inversion: the public sweep/monte_carlo/walk_forward become the core (calling run_indexed on the ambient pool through a shared assemble_* helper); the _with_threads variants collapse into thin, test-only wrappers (#[cfg(test)]) that run the same assembly inside a local pool. run_one stays Fn + Sync (no + Send): run_indexed borrows it in the map closure and the wrappers borrow it into install — passing it by value would move it into rayon's Map and force F: Send, a bound the public callers do not carry (hence the load-bearing #[allow(clippy::redundant_closure)] on run_indexed). Adds a nested-oversubscription regression test (asserts nested run_indexed never exceeds the pool size — the pre-rayon shape ran POOL^2) and an ignored wall-time benchmark. rayon admitted under the amended-C16 per-case dependency policy; it stays within C1 (parallelism across sims, never within one). This is the shared pool only — the first of three steps toward saturating the cores. The diagnosis found the larger loss is the idle valleys (~42% of the box idle: the sequential cell loop and the single-threaded bootstrap), which the pool alone does not fill. Two follow-ups are filed and recorded on #268: the Registry write path is not concurrency-safe (append_family races on a shared families.jsonl), the prerequisite for parallelizing the C1-disjoint cell loop with a RAM budget bounding concurrently-active instruments (the FileCache retains a symbol's parsed files while any reference is live). Verified: cargo test -p aura-engine (276 passed, the C1 determinism tests green), the new oversubscription test green, the benchmark runs (no overflow), clippy -D warnings clean, cargo test --workspace no failures. closes #268 |
||
|
claude
|
a895891ab1 |
perf(ingest,cli): derive archive bounds from the monthly file index — probe_window drain retired
probe_window drained a source's entire close column just to learn its first and last bar timestamp; both live callers (open_real_source and campaign_window_ms on the campaign trunk) now resolve through aura-ingest's new archive_extent(): list the symbol's SYM_YYYY_MM.m1 files, load only the boundary months via the existing loader seam, and walk across gap months forward/backward — O(2 file loads) typical instead of O(archive). Empty-window refusal semantics are unchanged (pinned). Two characterization tests captured the resolved windows against the OLD implementation over the synthetic per-test archive and stay green unchanged after the swap. Measured: the probe itself drops 366ms -> 13ms (~28x) on the full 2014-2026 GER40 archive; end-to-end command wall-clock moves only ~2-3% because the sim loop dominates — the fix removes an O(archive-size) term, not the dominant cost. closes #252 |
||
|
claude
|
c31e946bb7 |
test(cli): synthetic per-test M1 archive — no-window generalize tests go hostless
The two generalize E2E tests whose property is no-window span resolution (shared-window fallback, intersection semantics) no longer stream the 6.6 GB host archive: fresh_project_with_data() generates a deterministic two-symbol M1 archive (SYMA 2024-01..08, SYMB 2024-03..06 strictly inside it, so the intersection differs from symbols[0]'s window on both bounds) in the exact zip/48-byte-record format data-server reads, plus geometry sidecars, under the per-test project's paths.data. Both tests drop their local_data_present gates — they now run on any host, data mount or not. ~35s and ~50s become 0.74s and 1.14s; the full cli_run binary lands at ~25s (211s at the branch base). Full workspace suite: 247s -> 45.7s. refs #250 |
||
 Brummel
|
aeb0366aed |
feat(cli): aura campaign run — the executor verb over the MemberRunner driver (0107 tasks 8-9)
campaign_run.rs: target resolution (file is register-then-run sugar; bare 64-hex is the canonical id — #198 decision 1), project gate before any store write, zero-fault referential gate, process fetch + v1 preflight, then aura_campaign::execute with the CLI MemberRunner over the shipped loaded-blueprint convention: per-member blueprint reload, reduce-mode wrap, unique suffix-join of raw campaign axis names onto the wrapped param_space, windowed real data via the shipped ms->ns source seam (absent archive/zero-bar windows -> NoData). Emission: family_table / selection_report lines gated on the doc's emit list (names debug_asserted against emit_vocabulary), the campaign_run record line always, persist_taps deferred LOUDLY on stderr before execution (F7 lesson), zero-survivor cells noted on stderr with exit 0. exec_fault_prose/member_fault_prose keep the Debug-free house seam. Seam tests: 8 campaign_run_* e2e tests incl. outside-project, bogus target, unknown id, v1-boundary (mc process registers fine, run refuses), persist_taps ordering, and the gated real-data sweep->gate->walkforward e2e — which ran its full assert path on this host (local GER40 2024-09 archive) rather than the data-less skip. Also: exec.rs campaign-id guard tightened to lowercase hex, matching is_content_id and the store's self-keyed form (review nit). Gates: workspace 975/0, clippy -D warnings clean. refs #198 |
||
|
Brummel
|
3a4f1c4597 |
feat(campaign): aura-campaign crate — types, member_metric, preflight (0107 tasks 4-5)
New leaf library crate: the campaign-execution semantics home (#198 home decision — reusable beyond the CLI; NOT C21's project-side World). CellSpec + the MemberRunner seam (the only thing a consumer implements), MemberFault/ExecFault (Display-free, prose at the binary seam), member_metric over the 14 per-member scalars (third roster site, refs #190 — drift fails safe as a preflight refusal), and preflight: v1 pipeline shape std::sweep [std::gate]* [std::walk_forward], rankable-metric and gate-metric rosters, plateau-in-wf and deflate+plateau refusals, i64-fit guard on wf lengths — all before any member runs. Post-loop: the held quality finding (untested i64-fit arm) verified real and closed by hand — preflight_refuses_wf_length_exceeding_i64. Gates: aura-campaign 14/0, clippy -D warnings clean. refs #198 |
||
|
Brummel
|
ef3bec5844 |
feat(research): 0106 tasks 1-9 — document layer, content-id move, stores + referential tier
The aura-research leaf crate: ProcessDoc/CampaignDoc with strict parsing (hand-rolled StageBlock deserializer — deny_unknown_fields does not compose with internally-tagged enums; Axis declares its ScalarKind once with bare values, per the #189 user veto), canonical form + content_id_of (the SHA-256 primitive moved from aura-cli, which now delegates — the id pin set incl. the independent sweep-store recompute stayed green untouched), intrinsic validation for both document types (P1 constraints structural), and the introspection contract (schema tables single-source parse strictness AND vocabulary/describe/open-slots — the Blockly litmus made checkable). aura-registry: content-addressed processes/ + campaigns/ stores on the blueprint-store pattern (Ok(None) treat-as-empty) and the referential tier (validate_campaign_refs: process/strategy refs incl. identity-scan, axis-name + declared-kind checks against param_space). Plan deviations, verified by hand and folded back into the plan file: - Task 9's fixture could not use aura-composites as planned: every shipped composite with an open param routes through LinComb, which the zero-arg std_vocabulary roster deliberately excludes, so none round-trips through blueprint_from_json with the by-type-name resolver. The test hand-builds a minimal Bias composite instead (generic over param_space, as planned); the unused pro-forma aura-composites dev-dep from the repair pass is dropped. This is why the loop reported task 9 BLOCKED (review-loop-exhausted on the literal code block); the tree itself is complete and green. - A plan-verbatim test assertion false-matched ("deflate" as substring of "deflated-positive"); tightened to the JSON key. - OpenSlot doc comment backticked (rustdoc read strategies[0] as a link; doc gate back to 0 warnings). Verification: cargo test --workspace 898/0; clippy -D warnings clean; cargo doc --no-deps 0 warnings. Tasks 10-12 (CLI verb families, final gates) follow. refs #189 |
||
|
Brummel
|
4928e289f7 |
feat(project): the project-as-crate load boundary (cycle 0102)
A research project is now a loadable external cdylib crate. Inside a directory whose ancestry holds an Aura.toml, aura discovers the project root cargo-style, locates the compiled dylib via cargo metadata (debug default, --release opt-in), loads it load-and-hold, and refuses mismatches before trusting anything: the AURA_PROJECT descriptor (aura-core::project, #[repr(C)]) carries a C-ABI stamp prefix (rustc + aura-core version, baked per consuming build by the new aura-core build.rs) validated before any Rust-ABI field is read. The vocabulary charter gates the merged resolution: project type ids are ::-namespaced (std stays bare), duplicates refuse, and the enumerable type-id list must agree with the resolver, so introspection can never silently omit a project type. All blueprint verbs resolve through the merged project + std vocabulary via a per-invocation Env threaded through the dispatch chains; registry, trace-store, and data paths anchor at the project runs root (Aura.toml [paths], paths-only by design — instrument geometry stays the recorded sidecar, C15). RunManifest gains the Tier-1 project provenance field (namespace + dylib sha256 + best-effort commit), stamped beside topology_hash on the blueprint-run paths; pre-0102 registry lines load unchanged. Default node names strip the namespace, so :: never reaches the param-path address space. Proven by the demo-project fixture (built by the e2e via cargo, path-dep on this workspace): run twice bit-identical, provenance recorded, introspection lists demo::* beside std, registry anchors at the discovered root from a subdirectory; the badcharter fixture proves the charter refusal through the real libloading path; a never-built project refuses with a cargo-build hint. Outside a project every path collapses to the previous literals — goldens and manifest pins byte-identical. Verification: cargo build --workspace clean; cargo test --workspace 862 passed / 0 failed (incl. 7 project_load e2e); clippy -D warnings clean (one precedent-matching allow(too_many_arguments) on run_oos_blueprint, whose arity the Env threading raised to 8); doc build unchanged. Docs/ledger aligned: Aura.toml field lists are paths-only in project-layout.md, glossary, C16/C17; new C13 realization note records the per-invocation-reload reading and the load-and-hold one-shot scope boundary. New deps, per-case review (aura-cli leaf binary only, never the frozen artifact): libloading, toml. refs #180 |
||
|
Brummel
|
366170aeab |
feat(0098): CLI clap migration (iteration 1) — scoped help, --version, GNU flags
Replace the hand-rolled aura-cli argv parser with a clap derive parser. Delivers, all from one declarative source: - scoped `aura <sub> --help` (stdout, exit 0) — each subcommand's own Options section; retires the #131 uniform-global-blob help - `--version` / `-V` → `aura 0.1.0` (stdout, exit 0) - GNU `--flag=value`, the `--` end-of-options terminator, and long-option abbreviation (`--harn` → `--harness`) Architecture: one root Cli/Command enum, one `*Cmd` struct per subcommand. The four dual-grammar subcommands (run/sweep/walkforward/mc) map the loaded-blueprint branch as an optional `[blueprint]` positional + a post-parse `is_file()` dispatch on a single-sourced predicate; each built-in handler re-asserts a stray-positional guard (refuse-don't-guess) so clap's optional positional cannot silently swallow a typo (pinned by two new E2E tests). The execution layer (run_*/emit_*/runs_*/reproduce_*) is untouched — only arg-plumbing changes, via thin `*_from` adapters reusing the existing value helpers. Exit codes behaviour-preserved: clap parse errors exit 2 (matching the old usage-error=2); domain refusals stay exit 2; reproduce-diverged stays exit 1. The exit-code split (2=usage / 1=runtime) is iteration 2. clap is admitted under the C16 per-case review (research-side CLI, invariant 8 untouched — a dev-loop compile tax, not a frozen-artifact tax). Forced plan corrections (implementer; verified against the diff): - Step 6's deletion of the `parse_*_args` fns orphaned ~34 grammar unit tests that called them; those were deleted and that grammar coverage moved to the renegotiated cli_run.rs E2E pins (`parse_select` kept + retested). - run_malformed_cost_value_...: a bad cost value is now a clap value-parse error (no "Usage:" line); renegotiated to pin exit 2 + empty stdout + the flag name "--cost-per-trade"; the units note stays pinned via `run --help`. - `allow_hyphen_values` on the three cost flags so `--cost-per-trade -0.5` reaches the non-negativity guard (else clap rejects -0.5 as an unknown option). - Deleted the now-dead `RealWindowGrammar` (clippy -D warnings). Orchestrator additions after the loop: - Enabled long-option abbreviation (`infer_long_args` on the root — one attribute, it propagates to subcommands) + a test, closing spec acceptance criterion 4 which the plan's RED tests did not cover. Decided to deliver rather than amend the spec: the two ratified decisions conflict on this low-tier item — Fork B (GNU compliance, getopt_long abbreviates) vs F5 (LLM/automation-first, human-only niceties deprioritized) — and the cost tiebreaker (one attribute, not per-struct work) resolves to deliver. Carry-on debt (not this iteration): the error-message "Usage:"/"usage:"/bare casing is now mixed (clap's capitalized "Usage:" beside preserved lowercase aura messages); a deliberate quality-hold since the casings are pinned by renegotiated/preserved tests. Cosmetic; a candidate for the iteration-2 exit-code-split cleanup. Verified green (orchestrator, this session): cargo build --workspace, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace all clean (0 failed across every test binary, including the renegotiated cli_run pins and the new --version / scoped-help / GNU-flag / dual-grammar tests). refs #175 |
||
|
Brummel
|
ff4a1b3d4a |
feat(0092): run-from-blueprint infrastructure — restructure + topology_hash + shared seam (Tasks 1-3)
Cycle-1 infrastructure for #165 (World/C21). Behaviour-preserving + workspace green; the CLI arm + tests (Tasks 4-5) follow. Task 1 — restructured stage1_r_graph into stage1_signal() (the serializable signal leg: SMA-cross -> Bias, a `price` input-role + a `bias` output) + wrap_stage1r(signal, ...) (broker/sinks/exec/cost around a nested signal), with stage1_r_graph() = wrap_stage1r(stage1_signal(...)). DEVIATION from the plan's "callers untouched" claim, verified behaviour-preserving: nesting the signal prefixes its param-space names (stage1_signal.fast.length), which broke the sweep's bare .axis("fast.length"); fixed exactly as the #137 stop-knob machinery does — FAST_LENGTH_SUFFIX/SLOW_LENGTH_SUFFIX suffix-resolution + stage1_r_friendly_name arms mapping the prefixed names back to the bare manifest names. All observable behaviour (manifest names, member keys, metrics, traces) byte-identical; the flat graph and every metric unchanged (full suite green, incl. sweep/walkforward/MC). Task 2 — RunManifest.topology_hash: Option<String> (Tier-1 optional, serde default/skip — old manifests byte-identical, #156), threaded None into all 24 literal sites across 7 crates. Also fixed the aura-registry RunManifestRead read-mirror to carry topology_hash (it was hardcoding None on load) — else a stored hash would silently drop on the registry round-trip once Task 3 stores a real one. Task 3 — the shared run_signal_stage1r seam (hash the signal, wrap, compile with params, bootstrap, run, manifest with params from param_space + topology_hash) + the sha256_hex topology_hash helper (sha2 in aura-cli, off the frozen engine, invariant 8). run_stage1_r now carries its topology_hash too (every run becomes self-identifying, #158). The seam is unwired until Task 4 (transient #[allow(dead_code)], retired there); RED-first seam tests added. Verified: cargo test --workspace green (51 suites, 0 failures); cargo clippy --workspace --all-targets -D warnings clean. refs #165 |
||
|
Brummel
|
94c88eb7e1 |
refactor(0079): extract the trading-analysis leaf into aura-analysis (refs #136)
aura-engine is documented (C16) and named as the reusable, domain-agnostic reactive SoA substrate, yet `report.rs` concentrated the trading-domain analysis layer inside it. This cycle relocates the pure-domain leaf into a new `aura-analysis` crate (deps: aura-core + serde only), erecting the engine/domain seam before the Stage-2 broker milestone deposits more currency/position-event code into the same module. Behaviour-preserving (C1): the full workspace suite stays green unchanged (665 passed), including the C18 on-disk byte-identity goldens (cli_run, the registry legacy-line loads). Moved to aura-analysis (bodies verbatim — no float expression re-associated): RunMetrics, RMetrics (+ its hand-written PartialEq), the private r_col column indices + SQN_CAP, summarize_r, r_metrics_from_rs, derive_position_events, PositionAction (+ From/TryFrom<i64>), PositionEvent, and the deflation stats helpers inv_norm_cdf / expected_max_of_normals. Their unit tests and private helpers (r_row / r_row_full / pm_row) relocate with them — tests live with their subject (the r_col helpers reach the private module directly, which a re-export cannot bridge). Stays in aura-engine for this cycle: the generic trace plumbing (ColumnarTrace, join_on_ts, JoinedRow, f64_field), RunManifest, RunReport, and summarize (it bridges recorded trace columns into RunMetrics, so it is trace-coupled). The mis-placed orchestration types FamilySelection / SelectionMode also stay (their relocation is entangled with the registry-vocabulary fork, deferred). report.rs pub-re-exports the moved symbols, so aura-engine's lib.rs re-export block and every `crate::report::X` / `aura_engine::X` consumer (aura-registry, aura-cli, aura-composites, aura-ingest) compile byte-unchanged; only the new crate, the workspace member list, aura-engine's Cargo.toml dep, and report.rs itself change. Deferred to later cycles (genuine design forks, route through specify when reached): making RunReport generic over a metric type M and re-bounding sweep/mc/walkforward; relocating the aura-registry Metric / metric_cmp / rank vocabulary; relocating FamilySelection / SelectionMode. Fork decisions logged on #136. |
||
|
Brummel
|
1088320571 |
fix: terminate cleanly on a broken stdout pipe across all family-emitting commands
GREEN for the broken-pipe RED test (
|
||
|
Brummel
|
d5e9c270d9 |
build(deps): bump data-server to the geometry-reader commit
data-server #3 (closed) shipped DataServer::symbol_meta -> InstrumentGeometry (the per-symbol .meta.json reader). Bump 0f5e665 -> 694f96f2 so aura-ingest can consume the neutral geometry for the #143 cross-check. The bump alone changes no behaviour (the new API is unused until the #143 code lands). refs #143 |
||
|
Brummel
|
1a6eafa056 |
refactor(composites): extract Stage-1 composite-builders into their own crate
Dissolve the aura-engine -> aura-std dependency by moving the vol_stop /
risk_executor composite-builders out of aura-engine into a new aura-composites
crate. This restores the engine's domain-free layering, which the
composites-in-engine placement (cycle 0065) had inverted.
Why: the engine routes type-erased Scalar records and never names a concrete
node -- summarize_r reads the PositionManagement record positionally, by column
index, without linking aura-std (which is exactly why aura-std was a dev-only
dependency before 0065). Parking the node-naming composite-builders inside
aura-engine forced the engine's production code to reference the node library
for the first time (aura-engine -> aura-std), an upside-down layering: the
engine is the foundation the node library builds upon, not the reverse.
The clean home is a dedicated crate that depends on BOTH the engine's
GraphBuilder and the std nodes, leaving each lower crate untouched:
aura-composites -> { aura-engine, aura-std } -> aura-core (acyclic)
aura-engine -> aura-core only (domain-free again)
aura-std -> aura-core only (lean node library)
aura-std reverts to a [dev-dependency] of aura-engine (the engine's own E2E
tests -- stage1_r_e2e, random_sweep_e2e, ger40_breakout -- still drive real
std nodes through a bootstrapped Harness; that test-only edge creates no cycle).
Moved with history (git mv):
- crates/aura-engine/src/composites.rs -> crates/aura-composites/src/lib.rs
- crates/aura-engine/tests/risk_executor.rs -> crates/aura-composites/tests/
- crates/aura-engine/tests/vol_stop_composite.rs -> crates/aura-composites/tests/
Consumers rewired: aura-cli imports risk_executor / StopRule from aura_composites;
the moved tests import the builders from the crate under test. The intra-doc link
to RollMode and the module rationale doc are updated for the new home.
Behaviour-preserving: workspace build clean; clippy --all-targets -D warnings
clean; full suite 513 passed / 0 failed, identical to baseline (the two moved
test files run unchanged under aura-composites); cargo doc --no-deps clean (no
broken intra-doc links).
|
||
|
Brummel
|
15ee6d5f4f |
feat(trace-charts): serve persisted traces as a web chart (iter 2)
Iteration 2 (serve half) of the visual face — spec 0056 / plan 0056:
read a persisted run's traces and serve them as a self-contained uPlot
HTML chart. Completes the goal "tap data from a graph, serve it as a
chart in the browser".
- chart-viewer.js (first-party): a pure buildCharts(traces, mode) that
maps the injected window.AURA_TRACES to uPlot configs (no DOM) — overlay
= one plot, every series on its own y-scale over the shared xs; panels =
one plot per series, all on the same timestamp x. A browser-only mount()
instantiates the vendored global uPlot. Guarded by a node .mjs DOM test
(+ a sparse/gaps variant) shelling out exactly like viewer_dot.rs.
- render_chart_html + ChartMode/ChartData/Series (aura-cli/render.rs):
mirrors render_html — self-contained page, uPlot (1.6.32, vendored
|
||
|
Brummel
|
5b5f034be9 |
feat(aura-ingest): GER40 session-breakout on real M1 bars (examples/)
Run the Phase-1 session-breakout node vocabulary over REAL GER40 M1 bars
from the data-server archive — the first real-data exercise of the shipped
strategy nodes (Resample/Delay/Gt/Session/EqConst/And/Latch + SimBroker).
It is a pure composition of shipped aura-std nodes (no project-specific
signal code), so it lives in the engine repo's examples/ carveout (C9), not
a separate consumer crate. The breakout FlatGraph is reused VERBATIM from
aura-engine/tests/ger40_breakout.rs; only the sources change — four real
M1FieldSource (open/high/low/close, in the fixed C4 merge order) replace the
synthetic VecSources, with pip_size = 1.0 (GER40 is an index).
- examples/ger40_breakout_real.rs — runnable demo over a Sept-2024 window:
prints the RunReport metrics plus a per-session entry/exit trace.
- examples/shared/breakout_real.rs — the single 13-node wiring (the proven
11 + two trace taps), shared by the example and the test via #[path] so it
is never duplicated.
- tests/ger40_breakout_real.rs — gated determinism test (mirrors
real_bars.rs): skips where the archive is absent; where present, asserts
finite metrics, binary Latch exposure (held in {0.0, 1.0}), and
bit-identical C1 reruns of both the report and the recorded series.
chrono / chrono-tz added as aura-ingest dev-dependencies (the Session
timezone and the UTC window bounds), test-only — never on the normal path.
|
||
|
Brummel
|
4187f3bbc1 |
test(aura-engine): GER40 session-breakout e2e — the milestone capstone
Hand-wires all seven new aura-std nodes (Resample, Delay, Gt, Session, EqConst x2, And, Latch) + SimBroker into a raw-index FlatGraph over one synthetic Frankfurt session, pinning the whole vocabulary composed end-to-end: - held = [0,0,1,1,0] over the 5 bar emissions: flat, latched bar3-close (the strict breakout landing on session bar 3) through bar4, exit at bar5-close; - equity = [0,0,0,3,8] pips (SimBroker lagged-exposure integration, pip=1.0); - a no-entry control (bar3 close == bar2 high) pins the strict-> semantic; - two disjoint runs byte-identical (C1). Both spec traps exercised (bar6 rollover closes bar5; bars 1-2 warm Delay[1]). Phase-1 C9 deliverable; build-step 8 (capstone) of milestone 'Strategy node vocabulary I'. closes #91 |
||
|
Brummel
|
17197fed91 |
feat(aura-std): Session — Frankfurt bars_since_open (DST-aware)
Maps ctx.now() (epoch-ns UTC) to a Frankfurt session bar index: emits bars_since_open:i64 = (local wall-clock minutes past the 09:00 open) / period_minutes, in tz-aware DST-correct local time. So local 09:45 reads 3 in both CEST (UTC+2) and CET (UTC+1) — the close-instant convention (spec 0050 §4.1). Trigger is an f64 Any input (value ignored, wired from close15) so the node fires once per completed bar. Open/tz/period are baked structural config, not scalar params (a timezone is not a scalar, C11). Admits chrono/chrono-tz to aura-std — vetted DST math, never hand-rolled (per-case dep policy). Last node; build-step 7 of milestone 'Strategy node vocabulary I'. closes #90 |
||
|
Brummel
|
1d9555c468 |
feat(aura-engine,aura-registry): producer-supplied window + registry lineage (0045 iter 1)
Iteration 1 of spec 0045 — the engine + registry core for storing orchestration families as linked records. aura-engine: add Source::bounds() — the inclusive (from, to) data extent a producer will stream, known without materialization (#71 firewall) — and a free window_of() that folds the union extent across a run's sources. This is the producer-supplied replacement for the prices.first()/.last() Vec scans at the CLI call sites (those migrate in iteration 2); a lazy producer now reports its window so stored lineage stays byte-identical whether a source is eager or streamed. aura-ingest: M1FieldSource stores its requested [from_ms, to_ms] window at open and reports it via bounds() (normalized to epoch-ns; None when open-ended — the archive-extent query is a deferred non-goal). aura-registry: new lineage module — FamilyKind / FamilyRunRecord (a RunReport stamped with family_id + kind + ordinal) / Family, plus Registry::append_family (assigns family_id = "{name}-{counter}" via a per-name generate-and-check counter, writes members to a sibling families.jsonl) and load_family_members; group_families re-derives the families (the round-trip), and the three *_member_reports extractors pull the per-kind member reports. The flat runs.jsonl store and its append/load/rank_by/optimize API are byte-for-byte unchanged (a test pins the two stores' disjointness). C9 preserved: aura-engine gains no registry dependency. New dep: serde derive on aura-registry (per-case policy, same basis as serde_json already is). Verification: cargo build --workspace; cargo test --workspace (228 green, incl. 3 new engine bounds/window_of, 1 data-gated ingest bounds run against real archive data, 5 new registry lineage round-trip/counter/disjointness tests); cargo clippy --workspace --all-targets -D warnings clean. The CLI surface (aura mc, family-aware aura runs, the window_of migration) is iteration 2. refs #70 |
||
|
Brummel
|
4764656062 |
feat(aura-engine): walk-forward family — WindowRoller + walk_forward (C12 axis 3)
The third C12 orchestration axis: walk-forward varies the data window. A
WindowRoller is a pure iterator of WindowBounds { is, oos } — bounds only, never
tick data (#71 firewall). walk_forward runs a caller closure per split disjointly
over the shared run_indexed core (C1), and stitches the OOS pip-equity segments
into one continuous curve (each segment offset by the running sum of prior
segments' finals; an empty segment contributes 0.0, mirroring summarize). C2
no-look-ahead is structural: the roller only ever emits oos.0 == is.1 + 1, pinned
by a zero-tick bounds test. The in-sample optimize (axis 2) is closure-supplied,
not called by the engine: aura-engine has no aura-registry dependency (C9), and
C12 forbids baking search policy into the primitive — the CLI bridges both crates.
Param-stability is on-demand, not stored (the R2 decision, recorded with
provenance on #69): WalkForwardResult stores only the raw per-window outcomes +
the stitched curve, and param_stability(&result) -> Vec<MetricStats> is a public
reduction over the retained per-window chosen params. A stored summary would be
recomputable from the windows (redundant) and would force one statistic canonical
when "stability" admits several; this mirrors SweepFamily (raw points + on-demand
optimize), not McFamily (stored aggregate). MetricStats::from_values is extracted
from McAggregate::from_draws (behaviour-preserving — the 7 mc tests stay green)
and gains serde so the CLI summary renders it and #70 lineage can persist it.
`aura walkforward` runs a built-in rolling walk-forward over a synthetic windowed
source: per window it sweeps the built-in grid in-sample, optimizes by total_pips,
runs the chosen params out-of-sample, persists each OOS RunReport (C18), and prints
a stitched summary line.
Two plan deviations, both compiler-forced and consistent with siblings: added
#[derive(Debug)] to WindowRoller (the RED tests' unwrap_err needs Ok: Debug; the
sibling types already derive Debug); removed a now-redundant test-module
SyntheticSpec import after Step 2 hoisted it to the top-level use.
Verification: cargo test --workspace green (aura-engine 152 incl. walkforward 9 +
mc 9, aura-cli 14); clippy --workspace --all-targets -D warnings exit 0; cargo doc
--workspace --no-deps clean.
refs #69
|
||
|
Brummel
|
0ce843bada |
feat(aura-cli): aura run --real <SYMBOL> backtests the sample on real M1 bars
The first real-data backtest from the CLI: `aura run --real <SYMBOL> [--from <ms>] [--to <ms>]` runs the existing built-in sample signal-quality harness over real M1 *close* bars instead of synthetic prices, printing the same RunReport JSON as `aura run`. Bars are streamed lazily through aura_ingest::M1FieldSource (a Box<dyn Source>) — dogfooding the #71 Source seam, O(one chunk) resident, never eager. aura-ingest + data-server enter aura-cli's deps (data-server git line mirrored from aura-ingest). M1 only this cycle, deliberately. The TickSource is deferred: raw ticks are the wrong input for the bar-semantic SMA sample (an SMA over ticks is microstructure noise until a resampler node exists, C2), and tick's real driver is the realistic broker's bid/ask execution (C10) — it should arrive with that consumer, not speculatively here. Tick data is also local to only EURUSD/XAUUSD/GER40, whereas M1 covers the test symbol AAPL.US. Shape: run_sample_real builds sample_harness(), guards has_symbol / M1FieldSource::open (unknown symbol or empty window -> 'aura: no local data' + exit 2, never a panic), and reuses sim_optimal_manifest + the run_sample fold. parse_real_args is a pure, unit-tested grammar (mandatory symbol, then --from/--to in any order; bad/missing/duplicate flag -> Err). Data path is data_server::DEFAULT_DATA_PATH (no --data-path flag this cycle). Known simplification: the manifest window is derived by draining a *separate* single-pass probe source for first/last ts, so an unbounded window streams the archive twice (once to bound, once to run). Acceptable and O(1)-resident for now; a future Harness::run could surface the first/last cycle ts to drop the probe pass. Verified: cargo build/test/clippy --workspace all green; the gated run_sample_real_streams_real_close_bars_deterministically RAN (not skipped) over AAPL.US 2006-08; manual CLI smoke of the success + two error paths. |
||
|
Brummel
|
5bded0ca83 |
feat(aura-cli): aura runs registry — persist on sweep, list + rank (cycle D / iter 3)
Final iteration of the run-registry cycle (#33): the read surface that makes a sweep family — and runs across separate invocations — comparable over time (C18's "compare experiments over time", which has no home in git or Gitea). - `aura sweep` now persists each point's RunReport to runs/runs.jsonl (append- only, one serde_json line per run) in addition to printing it. - `aura runs list` prints every stored record, in store order. - `aura runs rank <metric>` prints the stored runs best-first by the metric (total_pips desc; max_drawdown / exposure_sign_flips asc); an unknown metric is a usage error (exit 2), like every other aura arg error. sweep_report splits into a production sweep_family() (the pure run) and a #[cfg(test)] renderer kept for the in-bin goldens; the dispatch gains run_sweep / runs_list / runs_rank over default_registry() (runs/runs.jsonl under cwd). Process goldens run the binary in a temp cwd so persistence never dirties the repo; /runs/ is git-ignored as a backstop. Verified end-to-end: two `aura sweep` invocations accumulate 8 records; `aura runs list` shows all 8; `aura runs rank total_pips` orders them best-first; `aura runs rank bogus` exits 2 with the known-metrics hint. cargo test --workspace green (cli_run 11, registry 5, engine 93, …); clippy --workspace --all-targets -D warnings clean; no stray runs/ in the work tree. refs #33 |
||
|
Brummel
|
fe11cfea8a |
feat(engine,registry): SweepPoint carries RunReport; add aura-registry (cycle D / iter 2)
Iteration 2 of the run-registry cycle (#33): make the sweep family self-describing and add the registry store. Engine. SweepPoint.metrics (RunMetrics) becomes SweepPoint.report (RunReport), and the sweep closure bound is now Fn(&[Scalar]) -> RunReport. This closes the manifest-per-point gap cycle C explicitly deferred to #33 — each swept point now carries its full (manifest, metrics), the unit the registry indexes (C18). All engine-internal callers, the engine test fixture run_point, and the three sweep tests were rethreaded in the same iteration so the crate stays green; the CLI caller followed in the same change so `cargo test --workspace` is green again. CLI. The sweep_report closure builds a per-point RunReport (manifest params = param_space names zipped onto the point via scalar_as_param_f64; commit/window/ seed/broker as run_sample builds them) and prints via RunReport::to_json. The hand-rolled sweep_point_to_json/json_string/scalar_token are retired (the report renders itself), and the orphaned ParamSpec/SweepPoint imports dropped. Net: aura run, aura sweep, and (next iteration) aura runs all print the same RunReport shape. The two aura sweep goldens now pin the per-point manifest params, NOT the commit (it is the real git HEAD, volatile). Registry. New crate aura-registry: Registry::{open, append, load} over an append-only runs/runs.jsonl (one serde_json line per RunReport), free rank_by (best-first per metric — total_pips desc, max_drawdown/exposure_sign_flips asc), and RegistryError (Io / Parse{line} / UnknownMetric). Five unit tests cover append->load round-trip, missing-file = empty, corrupt-line = Parse{line}, per-metric ranking, and unknown-metric error. Built and unit-tested; NOT yet CLI-wired (aura sweep persistence + aura runs list/rank are iteration 3). Verified: cargo test --workspace green (aura-registry 5, aura-engine 93, aura-cli 7+8, others unchanged); clippy --workspace --all-targets -D warnings clean; aura run stdout shape unchanged; aura sweep now emits full RunReports. refs #33 |
||
|
Brummel
|
eec876975c |
feat(engine): serde foundation + amend C16 dependency policy (cycle D / iter 1)
Iteration 1 of the run-registry cycle (#33): lay the dependency + serde foundation the registry's typed read-path needs, and amend the contract that forbade it. Contract change (load-bearing — C18/C16). C16's blanket "zero-external- dependency by commitment" + "aura-ingest is the sole external-dependency firewall" framing is struck and replaced by a considered, per-case dependency policy: dependencies are admitted by deliberate review (what a crate pulls in vs. what it buys), with particular scrutiny for anything entering the frozen deploy artifact (C13); standard vetted crates (serde, rayon, ...) pass that review and are used wherever they do the job, including in the bot; hand-rolling what a vetted crate does is the anti-pattern. C16's engine/project split + three-tier node reuse are unchanged. The five source comments that asserted the struck framing (aura-engine/aura-ingest Cargo.toml + aura-ingest/report.rs docs) are rewritten to match; no live source or ledger text still asserts it. Per-case justification for serde (the policy now requires one): it gives the run-report types a typed (de)serialization path — exactly what ranking/compare over stored runs needs, and what the writer-only hand-rolled JSON (C14) could never provide; closes the typed-read-path gap (#17). Its closure is tiny, ubiquitous, heavily audited, and its output deterministic (C1-safe). serde_json is test-only this iteration (dev-dependency); no production serde_json yet. Changes: [workspace.dependencies] serde (derive) + serde_json; serde derives on Timestamp (aura-core) and RunMetrics/RunManifest/RunReport (aura-engine), with serde_json round-trip tests (window renders as a [from,to] integer array via the transparent Timestamp newtype). The hand-rolled to_json writers are untouched (still drive stdout). No aura-registry crate, no SweepPoint change — those are iterations 2 and 3. Verified: cargo test --workspace green (incl. the two new round-trip tests); clippy --workspace --all-targets -D warnings clean; no surviving assertion of the struck zero-dep framing in live source or the ledger. refs #33 |
||
|
Brummel
|
66dff88528 |
feat(aura-cli): render the graph as a self-contained WASM-Graphviz viewer; retire ascii-dag
Iteration 2 of cycle 0026 (graph render redesign). `aura graph` no longer emits ASCII: it now prints one self-contained `.html` to stdout — the ported prototype pin-graph viewer (drill / inline-expand / styled tooltips / breadcrumb) driven by the deterministic model serializer that shipped in iteration 1, with layout and SVG done in-browser by Graphviz compiled to WebAssembly. Closes the redesign opened by spec 0026; unblocked by cycle 0027 (input ports are now named, so the viewer labels every input pin from the model's real names instead of inventing "a"/"b"). What ships: - crates/aura-cli/assets/: the ported graph-viewer.js (prototype genDot/chrome verbatim + a normalizeModel adapter mapping aura's real model tuple shape — ins = [kind, firing, name], index node keys, `comp` refs, `src_<role>` — into the viewer's native shape), plus the vendored Graphviz-WASM (@viz-js/viz@3.7.0) and svg-pan-zoom@3.6.1 blobs and a refresh-assets.sh provenance script. - crates/aura-cli/src/render.rs: render_html(&Composite) -> String, a read-only (C9) assembly — model_to_json + include_str! of the inlined assets, no eval, no build, no serde (C14). The `["graph"]` arm calls it. - Retired: the ascii-dag adapter (graph.rs), the `ascii-dag` dependency, the `--compiled`/`--macd` graph flag plumbing, render_compiled, the color/terminal plumbing, and the 12 ascii-dag-asserting tests + the now-orphaned helpers. The cli_run integration test now pins the HTML page envelope. - crates/aura-core/src/node.rs: the two last ascii-dag prose justifications re-grounded (single-line label rule kept, re-justified generically; the param-generic rule re-justified on C23, not on a renderer limitation). - docs/design/INDEX.md: the cycle-0026 C9 realization note (both iterations). Vendoring decision (spec deferred it to the plan): the WASM/JS blobs are checked in, not fetched at build time. include_str! needs them at compile time, and a build-time unpkg fetch would make the build non-hermetic/non-offline and let the shipped artifact drift with the registry — against C1 (determinism) and C8 (frozen artifacts). ~1.4 MB is the price of a hermetic, frozen render asset. Three adaptations beyond the plan, each verified: (1) render.rs imports `aura_engine::Composite` (the path model_to_json takes), not aura_core; (2) macd_blueprint is now test-only — removing the `--macd` arm left it used solely by the param-space test, so it took `#[cfg(test)]` rather than removal or an `#[allow]` suppression (the production `aura run --macd` path is intact, verified emitting JSON); (3) the `grep ascii-dag` over crates/ matches only the new contract test, which names the term deliberately to document the retirement — no stale justification prose remains. Invariants held (verified independently, not on agent report): C9 (render path read-only), C10 (viewer is a render asset, no logic/DSL), C4 (four-colour palette = the four scalar base types), C14 (no serde). The iteration-1 model byte golden is unchanged and green. cargo build --workspace: 0 errors. cargo test --workspace: 157 passed, 0 failed (incl. the two new HTML tests + the unchanged model_golden). cargo clippy --workspace --all-targets -- -D warnings: clean. ascii-dag absent from `cargo tree -p aura-cli`. closes #51 |
||
|
Brummel
|
0a855c3943 |
feat(aura-cli): aura graph renders a wired graph as an ASCII DAG (#13)
Make a wired graph introspectable so a mis-wiring becomes visible. Two
selectable views, both authored from one built-in sample blueprint:
aura graph clustered blueprint view — composites drawn as named
ascii-dag cluster boxes (pre-inline, C9)
aura graph --compiled flat compilat view — composite boundaries dissolved
(C23); the graph the run loop actually runs
The payoff is intrinsic, param-carrying node labels: two SMAs read SMA(2) /
SMA(4), so a swapped fast/slow input reads back differently. This is realized
by a `label()` default method on the core Node trait — a non-load-bearing
render symbol the run loop never reads (wiring is by index), the symbol C23
already reserves citing #13. Recorded as a C8 refinement in the ledger.
Rejected the alternatives in brainstorm: a separate Describe trait (forces a
combined trait-object dance for a label the ledger calls non-load-bearing) and
extrinsic parallel labels (decoupled from the node, so an author can label the
slow SMA "fast" and mask the very bug the render exists to surface).
Mechanism:
- aura-core: Node::label() default method (additive, object-safe, single-line).
- aura-std: per-node overrides — SMA(n)/Exposure(s)/SimBroker(p) carry params;
Sub/Add/LinComb/Recorder are bare (their identity is not a mis-wiring axis).
- aura-engine: Composite gains an authored `name` (cluster title; dissolves at
inline) + read-only graph-as-data accessors on Blueprint/Composite. No
external dependency — the engine stays dependency-pure (C16); ascii-dag lives
only in aura-cli. The label-free index-wired compilat (C23) is unchanged.
- aura-cli: ascii-dag v0.9.1 + a graph.rs adapter (Vertical mode; labels
materialized into an owned Vec<String> that outlives the borrow-based Graph).
`aura run` is untouched (the sample duplication is dedup idea #14).
Tests: a concern-defining test (swapped sample renders != correct), structure
pins (cluster present in blueprint view, absent in compiled view), per-node
label disambiguation, and two frozen byte-goldens of the deterministic render.
The cycle-0012 bit-identical and run-sample non-regression tests stay green.
Verified by the orchestrator (not the agent report): cargo build --workspace
clean; cargo test --workspace all green; cargo clippy --workspace --all-targets
-D warnings clean; both views run live and render as expected.
closes #13
|
||
|
Brummel
|
a24729e97f |
feat(aura-ingest): data-server M1 ingestion at the one merge boundary
aura's first real data source (closes #7, Walking-skeleton milestone). A new aura-ingest workspace crate transposes data-server's AoS M1Parsed records into SoA base columns (C7), normalizes Unix-ms to canonical epoch-ns at the one ingestion boundary (C3), and feeds the existing k-way merge a real close-price stream so a backtest runs over real bars, deterministically (C1). Surface: - unix_ms_to_epoch_ns(time_ms) -> Timestamp (= ms * 1_000_000), the single C3 unit normalization. - M1Columns: the OHLCV bar as a bundle of base columns (open/high/low/close/ spread: f64, volume: i64, ts: epoch-ns) — SoA, C7. - transpose_m1(&[M1Parsed]) -> M1Columns: pure AoS->SoA (C1). - M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>: the price input the SMA-cross sample strategy consumes. - load_m1_window(server, symbol, from_ms, to_ms): drains data-server's chronological chunks, transposes once at the boundary. Design (per spec 0011): - Eager materialization, not a lazy/shared Source abstraction: C12's cross-sim Arc<[T]> sharing has no consumer until the multi-sim orchestration cycle; deferred, the transpose logic carries over. - aura-ingest is the external-dependency firewall. data-server transitively pulls chrono + regex + zip (+ their trees) into Cargo.lock; isolating it in this one crate keeps aura-core/std/engine zero-external-dependency and the frozen deploy artifact (C6 replays recorded streams, never re-ingests) clean. cargo test --workspace now needs a populated cargo cache (one Gitea fetch, done). - Scope: boundary + tests only, no aura run CLI arg surface (M1 first; tick and the CLI wiring are follow-ups). Tests: 6 hermetic unit tests on hand-built M1Parsed (normalization, field-wise transpose, purity, close_stream order, empty edges) + one gated integration test (tests/real_bars.rs) that runs the cycle-0007 sample harness over real AAPL.US 2006-08 close bars and asserts finite metrics + two-run bit-identical JSON (C1); it skips cleanly where /mnt/tickdata is absent. On this machine it ran the real path. Workspace gates green: test (86), clippy -D warnings, doc -D warnings. Minor: transient unused-import warnings in the new lib.rs across the additive build steps resolved once load_m1_window consumed the imports; final -D warnings gate clean. |
||
|
Brummel
|
559903a14e |
feat(aura-cli): aura run — end-to-end sample-harness CLI
The Walking-skeleton milestone's closing seam: a real `aura run` binary that
bootstraps a built-in sample harness, runs it deterministically, and prints the
cycle-0009 metrics+manifest report as canonical JSON to stdout — the headline
C14 "run a sim, emit structured metrics" move, end-to-end, from a binary for the
first time.
Two deliverables:
- aura-std::Recorder — a reusable recording sink node (the glossary *sink* role,
C8/C22): a pure consumer (output: vec![]) over kinds.len() input columns,
holding an mpsc::Sender<(Timestamp, Vec<Scalar>)> as its out-of-graph
destination. mpsc keeps the engine's purity invariant (C7): no Rc/RefCell.
Supports all four base scalar kinds; returns None until every column is warm.
Promotes the shape the #[cfg(test)] fixtures already proved into a shipped,
reusable block (the fixtures stay as historical snapshots; de-dup is a later
tidy).
- aura-cli `run` subcommand — synthetic_prices / sample_harness / run_sample /
main. The sample harness (synthetic source → SMA(2)/SMA(4) → Sub → Exposure →
SimBroker → two Recorder sinks) is authored in plain Rust over the raw
Harness::bootstrap(nodes, sources, edges) API (C17/C20) — the open
experiment-builder DSL thread is deliberately not committed this cycle. main
hand-parses one subcommand: `run` prints run_sample().to_json() (exit 0),
anything else prints a one-line usage to stderr (exit 2). aura-cli gains
aura-std + aura-core path deps; the workspace stays zero-(external-)dependency.
The built-in synthetic stream (7 ticks, rises then reverses) yields a non-trivial
demo trace: equity [0,0,0,0,-0.08,-0.17,-0.13] → total_pips -0.13, max_drawdown
0.17, exposure_sign_flips 1. The run_sample unit test pins the integer flip count
exactly and the two f64 metrics within 1e-9 (the real run's float dust is ~7e-15,
confirming the tolerance); determinism is pinned exactly (two runs → identical
JSON). tests/cli_run.rs drives the built binary for the observable exit/stdout
contract.
manifest.commit is filled from option_env!("AURA_COMMIT") (defaults to
"unknown"); real HEAD capture via build.rs is deferred. Non-goals untouched:
experiment-builder DSL, aura new, Aura.toml schema, the data-server source (#7).
One deviation from the plan's verbatim code: a scoped
#[allow(clippy::type_complexity)] on sample_harness (its (Harness, Receiver,
Receiver) return tuple trips the lint under -D warnings) — consistent with the
identical allow on the build_two_sink_harness fixture in aura-engine. Verified
myself: cargo test --workspace (61 green, 0 red), clippy --all-targets
-D warnings (clean), cargo doc -D warnings (clean), and both smoke runs.
closes #8
|
||
|
Brummel
|
dd5c3fad96 |
feat(engine): the deterministic single-source sim loop
Cycle 0003: aura-engine's first real content — a Sim that runs a wired DAG of
nodes deterministically, cycle by cycle. First point in the project where
authored nodes execute against data, not just under a hand-fed Ctx.
- `Sim` (aura-engine) — a bootstrapped, frozen root graph: a flat Vec<NodeBox>
(each node owns its input columns, the cycle-0002 shape) + an index edge
table, topologically ordered (Kahn). `bootstrap` sizes every input column from
its node's schema, kind-checks each edge and source target, and rejects
directed cycles — C7's "type check paid once at wiring" generalized to the
whole topology (`BootstrapError::{KindMismatch, BadIndex, Cycle}`).
- `run` — the deterministic loop: per record, forward the source value into its
target slots, evaluate nodes in topo order, capture the observed node's output
at eval time, and forward each `Some` output into its consumers' input columns
(a `None` forwards nothing — the structural seed of sample-and-hold). The loop
destructures `&mut self` into disjoint field borrows and allocates nothing on
the per-cycle path. This is the flat, monomorphized sharpening of RustAst's
reference-counted, interior-mutable observer push graph (C1/C7).
- `Edge` / `Target` (aura-engine) — producer->consumer and source->consumer wiring.
- `Sub` (aura-std) — a 2-input f64-difference node, so the loop is proven on a
real fan-out + join DAG (source -> {SMA(2), SMA(4)} -> Sub), with a determinism
assertion (C1: a second identical run is bit-identical).
Engine library depends only on aura-core; a test-only dev-dependency on aura-std
lets the integration test wire real Sma/Sub nodes. Sim carries a hand-written,
node-opaque `Debug` impl (Box<dyn Node> is not Debug; the impl prints
nodes.len() + the index/topology fields, needed by the bootstrap-rejection tests'
`unwrap_err`).
Deliberately deferred (recorded as decisions): freshness-gated recompute /
sample-and-hold (C5 -> cycle 0004, with the second source that makes it testable);
the explicit monotonic cycle_id counter (its first reader is freshness, 0004);
the Source trait + data-server ingestion + k-way merge (C3/C11); the builder API
(C19); the real sink + run registry (C18/C22).
Gates green: cargo build/test (26: 18 aura-core + 3 aura-std + 5 aura-engine)/
clippy -D warnings clean; surface-purity grep (no dyn-Any / Rc / RefCell) clean.
refs walking-skeleton
|
||
|
Brummel
|
b99912ca59 |
docs+scaffold: engine/project split (C16-C18), aura-std, usage doc
Refine the structure per interview: aura is the reusable engine, research projects are separate external repos depending on it (C16). Add aura-std (universal standard-node tier) to the workspace; remove nodes/ from the engine (project concept). Record authoring-surface = Claude Code + skills pipeline, no embedded coding-LLM (C17), and project management = one-repo-one-project + Aura-native run registry (C18). Add CLAUDE.md invariants 9-10, code_roots -> [crates], and docs/project-layout.md documenting the day-in-the-life and project repo layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Brummel
|
744c2f31a8 |
chore: scaffold aura workspace and wire the skills dev-cycle
Cargo workspace aura-core -> aura-engine -> aura-cli (bin `aura`) plus nodes/ for hot-reloadable cdylib node crates. Crate bodies are intentionally API-free; types arrive from the first spec. Adds the skills profile (.claude/dev-cycle-profile.yml), the project CLAUDE.md with the eight domain invariants, and the design-ledger skeleton. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |