Iteration 2 of the run-registry cycle (#33): make the sweep family
self-describing and add the registry store.
Engine. SweepPoint.metrics (RunMetrics) becomes SweepPoint.report (RunReport),
and the sweep closure bound is now Fn(&[Scalar]) -> RunReport. This closes the
manifest-per-point gap cycle C explicitly deferred to #33 — each swept point now
carries its full (manifest, metrics), the unit the registry indexes (C18). All
engine-internal callers, the engine test fixture run_point, and the three sweep
tests were rethreaded in the same iteration so the crate stays green; the CLI
caller followed in the same change so `cargo test --workspace` is green again.
CLI. The sweep_report closure builds a per-point RunReport (manifest params =
param_space names zipped onto the point via scalar_as_param_f64; commit/window/
seed/broker as run_sample builds them) and prints via RunReport::to_json. The
hand-rolled sweep_point_to_json/json_string/scalar_token are retired (the report
renders itself), and the orphaned ParamSpec/SweepPoint imports dropped. Net:
aura run, aura sweep, and (next iteration) aura runs all print the same
RunReport shape. The two aura sweep goldens now pin the per-point manifest
params, NOT the commit (it is the real git HEAD, volatile).
Registry. New crate aura-registry: Registry::{open, append, load} over an
append-only runs/runs.jsonl (one serde_json line per RunReport), free rank_by
(best-first per metric — total_pips desc, max_drawdown/exposure_sign_flips asc),
and RegistryError (Io / Parse{line} / UnknownMetric). Five unit tests cover
append->load round-trip, missing-file = empty, corrupt-line = Parse{line},
per-metric ranking, and unknown-metric error. Built and unit-tested; NOT yet
CLI-wired (aura sweep persistence + aura runs list/rank are iteration 3).
Verified: cargo test --workspace green (aura-registry 5, aura-engine 93, aura-cli
7+8, others unchanged); clippy --workspace --all-targets -D warnings clean; aura
run stdout shape unchanged; aura sweep now emits full RunReports.
refs #33
Iteration 1 of the run-registry cycle (#33): lay the dependency + serde
foundation the registry's typed read-path needs, and amend the contract that
forbade it.
Contract change (load-bearing — C18/C16). C16's blanket "zero-external-
dependency by commitment" + "aura-ingest is the sole external-dependency
firewall" framing is struck and replaced by a considered, per-case dependency
policy: dependencies are admitted by deliberate review (what a crate pulls in
vs. what it buys), with particular scrutiny for anything entering the frozen
deploy artifact (C13); standard vetted crates (serde, rayon, ...) pass that
review and are used wherever they do the job, including in the bot; hand-rolling
what a vetted crate does is the anti-pattern. C16's engine/project split +
three-tier node reuse are unchanged. The five source comments that asserted the
struck framing (aura-engine/aura-ingest Cargo.toml + aura-ingest/report.rs docs)
are rewritten to match; no live source or ledger text still asserts it.
Per-case justification for serde (the policy now requires one): it gives the
run-report types a typed (de)serialization path — exactly what ranking/compare
over stored runs needs, and what the writer-only hand-rolled JSON (C14) could
never provide; closes the typed-read-path gap (#17). Its closure is tiny,
ubiquitous, heavily audited, and its output deterministic (C1-safe). serde_json
is test-only this iteration (dev-dependency); no production serde_json yet.
Changes: [workspace.dependencies] serde (derive) + serde_json; serde derives on
Timestamp (aura-core) and RunMetrics/RunManifest/RunReport (aura-engine), with
serde_json round-trip tests (window renders as a [from,to] integer array via the
transparent Timestamp newtype). The hand-rolled to_json writers are untouched
(still drive stdout). No aura-registry crate, no SweepPoint change — those are
iterations 2 and 3.
Verified: cargo test --workspace green (incl. the two new round-trip tests);
clippy --workspace --all-targets -D warnings clean; no surviving assertion of
the struck zero-dep framing in live source or the ledger.
refs #33
aura's first real data source (closes#7, Walking-skeleton milestone). A new
aura-ingest workspace crate transposes data-server's AoS M1Parsed records into
SoA base columns (C7), normalizes Unix-ms to canonical epoch-ns at the one
ingestion boundary (C3), and feeds the existing k-way merge a real close-price
stream so a backtest runs over real bars, deterministically (C1).
Surface:
- unix_ms_to_epoch_ns(time_ms) -> Timestamp (= ms * 1_000_000), the single C3
unit normalization.
- M1Columns: the OHLCV bar as a bundle of base columns (open/high/low/close/
spread: f64, volume: i64, ts: epoch-ns) — SoA, C7.
- transpose_m1(&[M1Parsed]) -> M1Columns: pure AoS->SoA (C1).
- M1Columns::close_stream() -> Vec<(Timestamp, Scalar)>: the price input the
SMA-cross sample strategy consumes.
- load_m1_window(server, symbol, from_ms, to_ms): drains data-server's
chronological chunks, transposes once at the boundary.
Design (per spec 0011):
- Eager materialization, not a lazy/shared Source abstraction: C12's cross-sim
Arc<[T]> sharing has no consumer until the multi-sim orchestration cycle;
deferred, the transpose logic carries over.
- aura-ingest is the external-dependency firewall. data-server transitively
pulls chrono + regex + zip (+ their trees) into Cargo.lock; isolating it in
this one crate keeps aura-core/std/engine zero-external-dependency and the
frozen deploy artifact (C6 replays recorded streams, never re-ingests) clean.
cargo test --workspace now needs a populated cargo cache (one Gitea fetch,
done).
- Scope: boundary + tests only, no aura run CLI arg surface (M1 first; tick and
the CLI wiring are follow-ups).
Tests: 6 hermetic unit tests on hand-built M1Parsed (normalization, field-wise
transpose, purity, close_stream order, empty edges) + one gated integration
test (tests/real_bars.rs) that runs the cycle-0007 sample harness over real
AAPL.US 2006-08 close bars and asserts finite metrics + two-run bit-identical
JSON (C1); it skips cleanly where /mnt/tickdata is absent. On this machine it
ran the real path. Workspace gates green: test (86), clippy -D warnings, doc
-D warnings.
Minor: transient unused-import warnings in the new lib.rs across the additive
build steps resolved once load_m1_window consumed the imports; final -D warnings
gate clean.
Refine the structure per interview: aura is the reusable engine, research projects are separate external repos depending on it (C16). Add aura-std (universal standard-node tier) to the workspace; remove nodes/ from the engine (project concept). Record authoring-surface = Claude Code + skills pipeline, no embedded coding-LLM (C17), and project management = one-repo-one-project + Aura-native run registry (C18). Add CLAUDE.md invariants 9-10, code_roots -> [crates], and docs/project-layout.md documenting the day-in-the-life and project repo layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cargo workspace aura-core -> aura-engine -> aura-cli (bin `aura`) plus nodes/ for hot-reloadable cdylib node crates. Crate bodies are intentionally API-free; types arrive from the first spec. Adds the skills profile (.claude/dev-cycle-profile.yml), the project CLAUDE.md with the eight domain invariants, and the design-ledger skeleton.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brummel