Resolutions for the architect's four drift items (all fix/document, none
ratified away):
- [medium] execute enforced family-name uniqueness only via the CLI's
validate tier: preflight now refuses duplicate campaign instruments
itself (defense in depth for direct callers), RED-first
(execute_refuses_duplicate_instruments).
- [medium] the parallel cell loop's C1 relationship lived only in the
git-ignored spec: C1 gains a realization note (docs/design/INDEX.md)
recording the chunked instrument-major schedule, the structural
residency bound, and the two scheduling-dependent fatal-path carve-outs
(fault attribution among completed cells; already-written family lines)
— both inert and outside the success-path bit-identity.
- [low] the fatal-path orphan-line honesty is part of that note.
- [low] the --parallel-instruments zero-reject acceptance criterion had no
protecting test: campaign_run_rejects_a_zero_parallel_instruments_bound
pins clap's NonZeroUsize usage error (exit 2).
Gates re-verified: workspace suite green, clippy -D warnings clean.
refs #277
The flip: within each K-instrument chunk (the structural residency bound
from the previous commit), all cells run concurrently on rayon's process-
global pool — the same pool the engine's member/window fan-out already
shares, so cells x members x windows feed one work-stealing scheduler and
the per-cell serial seams (ingest transpose, single-threaded bootstrap)
overlap with other cells' work. Results land in index-addressed slots and
the ordered outputs are rebuilt in document order: byte-identical outcomes
across worker counts and bounds (C1).
Fault routing, restructured for a loop that cannot mid-iterate abort:
member/window faults stay #272-contained inside run_cell; non-containable
faults (incl. registry writes — infrastructure, not cell pathology) latch
an atomic abort flag so unstarted cells never run, and after the join the
lowest document-order fault among completed cells propagates as the run's
error. Nothing partial is persisted at run level on the fatal path.
Property pins: determinism across thread counts and bounds (1-thread/K=1
== 8-thread/K=2), distinct live instruments never exceed K (refcounting
stub runner, 8-thread pool), a dead family store is run-fatal with no
partial campaign-run record, member faults stay contained under the
parallel loop. E2E fixtures (previous commit) pin the CLI prose and
document-order persistence across K.
Verified: workspace suite green, clippy -D warnings clean. rayon enters
aura-campaign under the amended C16 per-case policy (comment in
Cargo.toml, mirroring aura-engine).
closes#277
Two pieces of the parallel-cell-loop iteration (spec: parallel-cell-loop),
still sequential — behaviour-identical results, suite green unchanged:
- validate_campaign refuses a campaign document listing the same instrument
twice (DocFault::DuplicateInstrument + CLI prose). A duplicate enumerates
two identical cells and is the one input that could make two cells share a
registry family name; with distinct instruments the per-cell family name
(all four cell axes + stage ordinal) is unique by construction, so the
upcoming parallel appends can never race one name's run-index assignment.
- aura_campaign::execute flattens the 4-nested cell loop into a document-
order-indexed cell list, groups it by instrument ordinal, and walks
sequential chunks of K instrument groups; results land in index-addressed
slots and the ordered outputs (outcomes, realizations, nominee groups) are
rebuilt in document order. K arrives as a new NonZeroUsize parameter
(default DEFAULT_PARALLEL_INSTRUMENTS = 4), exposed as
--parallel-instruments on aura campaign run and threaded through the
run_campaign chain; the dissolved-verb sugar paths pass the default.
The chunk walk is the structural RAM bound for the parallel flip that
follows: no cell of an instrument outside the current chunk can be in
flight at all (chosen over a semaphore gate — blocking inside pool tasks
risks worker starvation; decision log on #277). Verified: workspace suite
green unchanged, clippy -D warnings clean, E2E fixtures pin the CLI prose
and the document-order persistence across K.
refs #277
Architect drift review for the #276 cycle (b048923..HEAD). What holds:
C1 is preserved (the append_write_lock sits at the results-persistence
edge, outside the lock-free sim loop); the mechanism is correct (the
guard spans the full read-counter-then-write critical section in both
locked functions, no re-entrancy via the load paths, poison entered by
into_inner); both fixed paths are pinned by property tests.
One [medium] drift item, resolved here on the fix path: Registry::append
(the flat runs store, runs.jsonl) was the third identical unsynchronized
OpenOptions::append + writeln! path, left standing unguarded beside its
two newly-guarded siblings — and it is public API with external
consumers (C18), so a concurrent caller would hit exactly the #276
corruption. The append_write_lock doc also enumerated the two guarded
stores as if exhaustive. Fixed RED-first: the sibling property test
failed 2/2 against the unguarded path (same "trailing characters"
merged-line corruption), then append takes the same lock and the doc now
names all three JSONL append paths. No other finding: put_blueprint /
put_doc (content-addressed idempotent writes) and TraceStore::write
(per-name whole-file writes) are race-tolerant by shape and correctly
outside the append scope.
Regression gate: no dedicated regression scripts configured; the
project-facts gates all green — cargo test --workspace no failures (the
three concurrency property tests among them), clippy --workspace
--all-targets -D warnings clean, cargo doc --workspace --no-deps clean.
Cycle #276 is drift-clean.
refs #276
Registry::append_family and append_campaign_run did unsynchronized
OpenOptions::append + writeln! to their shared JSONL stores; concurrent
writers through one shared &Registry interleaved one append's content
and newline writes with another's, merging records onto one physical
line and corrupting the store (the #276 RED test failed 3/3 with a
"trailing characters" parse error). The planned parallel campaign cell
loop (#277) makes exactly that caller shape reachable, so this is its
prerequisite.
Mechanism: one internal append_write_lock: Mutex<()> on Registry,
acquired by both append functions and held across the whole
read-the-run-counter-then-write critical section — serializing the
write also serializes the read-before-write counter derivation, so
concurrent appends under one name cannot double-assign a run index.
Mutual exclusion around a per-cell-per-stage microsecond write, not a
barrier: cells never rendezvous, and the sim event loop stays lock-free
(C1's "without locking" governs the sims; this lock sits at the
results-persistence edge outside the hot path). A poisoned lock is
entered anyway (PoisonError::into_inner): every store line is
independent, so one panicked writer must not wedge all later appends.
Scope, per the decision log on #276: in-process synchronization owned
by the Registry type itself; cross-process locking stays out of
contract (the corrected doc comments now state the actual guarantee).
The serial per-cell-buffer alternative stays open to #277 as an
ordering decision on top — this lock does not preclude it.
Adds the sibling test pinning append_campaign_run under the same
concurrent load. Verified: the #276 headline test green 3/3 (was red
3/3), sibling green, cargo test --workspace no failures, clippy
--workspace --all-targets -D warnings clean.
closes#276
Pin the #276 headline behaviour RED-first: N threads sharing one
Registry handle and concurrently appending families must leave
families.jsonl well-formed — every member of every successful append
survives as its own complete, parseable JSON line (none torn, merged
into a neighbour, or lost). Today's unsynchronized OpenOptions::append
+ writeln! interleaves concurrent writers' content and newline writes,
so the test fails deterministically (verified 3/3 runs: "trailing
characters" merged-line parse error).
The sharing topology — one shared &Registry across scoped threads —
models the planned parallel campaign cell loop's caller shape and
admits any of the issue's candidate fixes.
refs #276
Replace the per-call std::thread::scope fan-out in run_indexed with one
process-wide rayon pool. run_indexed's callers nest — sweep (grid
points), monte_carlo (seeds), and walk_forward (window bounds), where a
walk_forward window runs an inner sweep — and each std::thread::scope
spawned its own available_parallelism() OS-thread batch, so the two
levels multiplied to ~available_parallelism()^2 threads (measured
150-240 concurrent on a 24-core box during a 42-cell campaign, ~10x
oversubscription). rayon's pool is process-wide and persistent: a nested
par_iter work-steals within the same N workers instead of spawning, so
live workers never exceed the pool size.
Determinism (C1) is preserved bit-for-bit: run_indexed collects via
IndexedParallelIterator in enumeration order (not completion order), and
the float aggregation (summarize_r / McAggregate / stitch) runs after
the ordered collect, so IEEE-754 non-associativity introduces no
variance. The existing _with_threads(1) == _with_threads(8) == public
determinism tests stay green, now driving local rayon pools of 1 and N
workers via ThreadPool::install.
Structural inversion: the public sweep/monte_carlo/walk_forward become
the core (calling run_indexed on the ambient pool through a shared
assemble_* helper); the _with_threads variants collapse into thin,
test-only wrappers (#[cfg(test)]) that run the same assembly inside a
local pool. run_one stays Fn + Sync (no + Send): run_indexed borrows it
in the map closure and the wrappers borrow it into install — passing it
by value would move it into rayon's Map and force F: Send, a bound the
public callers do not carry (hence the load-bearing
#[allow(clippy::redundant_closure)] on run_indexed).
Adds a nested-oversubscription regression test (asserts nested
run_indexed never exceeds the pool size — the pre-rayon shape ran
POOL^2) and an ignored wall-time benchmark. rayon admitted under the
amended-C16 per-case dependency policy; it stays within C1 (parallelism
across sims, never within one).
This is the shared pool only — the first of three steps toward
saturating the cores. The diagnosis found the larger loss is the idle
valleys (~42% of the box idle: the sequential cell loop and the
single-threaded bootstrap), which the pool alone does not fill. Two
follow-ups are filed and recorded on #268: the Registry write path is
not concurrency-safe (append_family races on a shared families.jsonl),
the prerequisite for parallelizing the C1-disjoint cell loop with a RAM
budget bounding concurrently-active instruments (the FileCache retains a
symbol's parsed files while any reference is live).
Verified: cargo test -p aura-engine (276 passed, the C1 determinism
tests green), the new oversubscription test green, the benchmark runs
(no overflow), clippy -D warnings clean, cargo test --workspace no
failures.
closes#268
refs #275
Drift review (architect) over ad4249f..HEAD found the cycle drift-clean at the core
and confirmed what holds: C1 preserved (`run()`'s k-way merge / event loop has zero
diff — `run_bound` only reorders the supply via `bind_sources` then delegates to
`run`, byte-identity pinned); the C4 realization and C23 refinement ledger notes are
accurate and tightly scoped (`SourceSpec.role` is read in exactly one place, sourced
only from the lowering — no leak); and all three binding-carrying production run sites
moved to `run_bound` + `key_supply`, every remaining `.run(` being `#[cfg(test)]`. No
regression harness exists, so the architect is the sole drift gate.
One medium drift item fixed:
- FIX: the CLI's `.expect()` on `run_bound` covered Missing/Extra/Unnamed feed but not
`DuplicateFeed`. The blueprint path does not enforce root-role-name uniqueness (only
the construction op-script path does), so a duplicate-role blueprint would reach
`bind_sources`' `DuplicateFeed` and panic behind the `.expect()` instead of refusing.
Closed at the binding boundary: `resolve_binding` now refuses a duplicate input-role
name — a named `aura:`-register refusal on every CLI run path (single run, sweep, and
campaign all resolve the binding upfront), matching its existing price/close-ambiguity
refusal. The `.expect()` invariant is now genuinely true on all paths. Pinned by
`binding::duplicate_role_name_refuses`.
Two low items:
- `SourceBindError::Display` had no test (its renderer stays dead until a
decoupled-supply caller lands, #124). Added `source_bind_error_display_names_the_role`
pinning the message strings; reachability is deferred by design.
- The reverted fieldtest `SourceSpec` sweep leaves the corpus one more revival break
against the current engine API — carried, as the corpus is pre-existing bit-rot
unrelated to this cycle (documented in 9e30805 and on #275).
Suite: `cargo test --workspace` green (1321 passed, 0 failed); `cargo clippy
--workspace --all-targets -- -D warnings` clean. Spec + plan (git-ignored working
files) discarded.
closes#275
The CLI half of by-name source binding, plus the ledger record.
Every production run site that carries a ResolvedBinding — `run_signal_r`,
`run_blueprint_member` (sweep / reproduction), and the campaign re-run/trace path —
now keys its opened sources by role name via `key_supply(binding, sources)` and
drives the harness through `Harness::run_bound` instead of the positional
`run(sources)`. `key_supply` pairs each opened column with its declared role from
`binding.entries()` — the one place open-order and role-order meet, made explicit so
`bind_sources` verifies the wiring↔supply role match by name rather than by a
maintained canonical-order convention.
Because the current single-binding CLI derives both the `SourceSpec` roles (via
`wrap_r`) and the supply roles (via `key_supply`) from the same `binding.entries()`,
the bind cannot fail on this path — so the call site asserts the invariant with
`.expect`, matching the adjacent `close_handle.expect("ResolvedBinding guarantees a
close entry")` idiom. The named `SourceBindError` refusal path lives in
`bind_sources` for future decoupled-supply callers (independently-built multi-feed /
recorded sources, #124), where a mismatch becomes reachable.
Ledger: a C4 realization note (supply resolved by name into declaration order, so
supply order is no longer load-bearing; the tie-break guarantee is unchanged) and a
scoped C23 refinement (a `SourceSpec.role` is load-bearing for source binding; every
other flat-graph name stays a non-load-bearing raw-index symbol).
The fieldtest-corpus `SourceSpec` sweep (planned Task 5) was reverted: the fieldtest
packages already fail to build against the current engine API for reasons unrelated
to `SourceSpec` (bootstrap arity, removed `InputSpec`, drifted
`Recorder`/`SimBroker`/`RMetrics`, renamed `Scalar` helpers) — pre-existing bit-rot
from earlier cycles. A partial `SourceSpec` migration neither revives nor further
breaks them, so the scope decision to touch them (premised on their being
otherwise-buildable) was withdrawn; reviving the corpus is separate from #275.
Verification: `cargo test --workspace` green (0 failed; the real-data OHLC channel
e2e exercises the migrated `run_bound` path byte-identically); `cargo build
--workspace --all-targets` clean; `cargo clippy --workspace --all-targets -D
warnings` clean.
refs #275
The engine-side half of by-name source binding. `SourceSpec` carries a load-bearing
`role: Option<String>` — `Some(name)` for a spec lowered from a bound `Role` (carried
through the blueprint lowering, previously dropped at compile), `None` for a
hand-built raw-index spec. A pure `bind_sources(specs, keyed_supply)` resolves a
`Vec<(role, Box<dyn Source>)>` into the positional `Vec` `run` consumes, in
`SourceSpec` declaration order, erroring named (`SourceBindError`:
Missing/Extra/Duplicate/Unnamed feed) on a mis-bind. `Harness::run_bound` is the
ergonomic method the by-name production path calls.
The raw-index `run(Vec)` primitive and the k-way merge / per-cycle event loop are
untouched: a positional run stays byte-identical (C1, pinned by
`run_bound_out_of_order_matches_positional_run`), and every existing `run(Vec)` caller
keeps working (it ignores `role`). A `BTreeMap` index makes the Duplicate/Extra
diagnostics deterministic.
C4's tie-break guarantee ("source declaration order") is unchanged — `bind_sources`
emits in `SourceSpec` order, so the merge's source-index tie-break resolves on
declaration order regardless of the caller's supply order. The narrow C23 amendment
(a `SourceSpec.role` is load-bearing for source binding, the rest of the flat graph
stays raw-index) is deferred to the ledger note.
The four error-path RED tests assert via `.map(|_| ()).unwrap_err()`: the `Ok` arm
`Vec<Box<dyn Source>>` is not `Debug` (`Source` has no `Debug` supertrait), so a bare
`.unwrap_err()` would not compile — a plan-byte defect corrected here.
Also sweeps the 45 raw-index `SourceSpec { .. }` literals to `SourceSpec::raw(..)`
across the workspace (mechanical, compiler-enumerated) and pins that a bound root
role's name survives lowering.
Verification: `cargo test --workspace` green (0 failed; incl. 5 `bind_sources` cases,
the `run_bound` byte-identity test, the role-survives-lowering pin, and the four
`.sources`-equality twins under the now-stricter `role` equality);
`cargo build --workspace --all-targets` clean; `cargo clippy -p aura-engine
--all-targets` clean.
Drift review (architect) over a55e4cf..HEAD found the cycle substantially
clean — C25 closed vocabulary, C1 determinism/behaviour-preservation, the
additive-serde widening, exhaustive StageBlock matches, and the uniform exit-3
convention all hold. Two drift items resolved:
- RATIFY: the `SilencedPanic` member-boundary panic-hook silencer (added by the
#272 implementer, not named in the spec) is a legitimate mechanism — it keeps
"recorded, campaign continues" observably true on stderr by suppressing the
default crash backtrace around each contained `catch_unwind`. Its mutex
serialises only the ref-count/hook-swap (O(1)), never member computation, so
C1 disjoint-parallel determinism is preserved. Documented in the ledger's
#272 realization paragraph rather than left as undocumented global state.
- FIX: `cell_fault_kind_label` hand-wrote the snake_case strings that must match
`CellFaultKind`'s serde `rename_all` (two sources of truth an aggregate over
campaign_runs.jsonl could silently diverge from). Pinned with a test asserting
each label equals the serialized form; the efficient `&'static str` stays.
(The #272 commit's own concern-driven doc fixes and the wf panic-containment
test landed in d3b1a1a; this commit carries only the two audit-phase items.)
Suite: cargo test --workspace green (1311 tests, 0 failed); clippy clean.
closes#272
A member fault (no-data, bind, run, or a caught panic) is now a recorded
per-cell outcome instead of aborting the whole campaign and discarding every
already-computed cell. The incident that motivated this (a 22-instrument
campaign lost ~36 healthy cells ~6.7 min in because Copper had an archive gap)
now completes: the healthy cells persist, the gap cell is recorded as failed,
and the run exits 3.
Direction (owner decision 2026-07-14): run to completion and report
compromised results; no coverage preflight, no window synthesis.
Containment granularity:
- The CELL for a sweep-stage member fault (a grid hole structurally
compromises winner selection, so the whole cell fails).
- The FOLD for a walk_forward member fault (independent time windows): the
surviving folds pool into the family, failed folds are recorded as
StageRealization.window_faults, and the summary names the ratio.
- aura-registry: additive CellFault / CellFaultKind (closed:
no_data|bind|run|panic|window) / WindowFault / CellCoverage, plus
fault/coverage fields on CellRealization and window_faults on
StageRealization — all serde-default-skipped, so pre-#272 campaign_runs
lines parse and round-trip byte-identical.
- aura-campaign: run_cell returns a fault-annotated CellRealization instead of
Err (execute's accumulate-then-append-once tail is unchanged and now
persists every healthy cell + the one run record); a `contain` split keeps
ExecFault::Registry and doc-shape preflight faults global while Member/Window
become per-cell/per-fold. Member panics are caught with
catch_unwind(AssertUnwindSafe) at all three member-run sites (sweep IS/OOS)
and recorded as MemberFault::Panic — a member panic no longer aborts the
process. The wf stage partitions Registry faults (global) from Member/Window
(per-fold) and filters faulted-fold placeholders (the
"faulted-member-placeholder" broker sentinel) out of the persisted family.
- aura-cli: exec_fault_prose gains the Panic arm; CliMemberRunner::window_coverage
derives effective bounds + interior gap months from the #264 archive
primitives; present_campaign prints per-cell failure notes + a completion
summary and threads the failed-cell count; a run with >=1 failed cell exits 3
("completed with failed cells") uniformly across `aura campaign run` and the
dissolved sweep/walkforward/mc/generalize verbs (exit_on_campaign_result).
Usage stays 2, refused-before-running stays 1, clean stays 0.
Tests: the global-abort pins flip to containment (execute + the two wf fault
tests → fold-containment + all-folds-fail-the-cell); new panic-containment
tests on both the sweep path (PanicRunner) and the wf path (this commit adds
the wf mirror the loop left uncovered); a new gapped-archive e2e (one covered
cell + one gap cell → exit 3); the ~14 CLI exit-1 pins move to the exit-3
register; a pre-#272-line byte-identical round-trip guard.
Suite: cargo test --workspace green (1309 tests, 0 failed); clippy clean.
Decision log: #272 comments (fork rationale, the fold Registry/Member split,
the placeholder sentinel, uniform exit-3).
Follow-up (minor, not blocking): the plan under-scoped Task 1 to aura-registry
though the additive fields also touch aura-campaign's exec.rs literals — the
loop absorbed it mechanically; a future plan for a cross-crate additive-field
change should scope every crate's construction sites in the first task.
closes#256
Fork B (owner decision 2026-07-14): the dissolved walkforward/mc
translations' leading sweep executed the full grid over the whole campaign
window and persisted a Sweep family, yet only the enumerated parameter
points ever crossed the stage seam (the wf stage re-sweeps them per IS
window itself). The leading stage is now the fieldless vocabulary block
std::grid: it enumerates, executes nothing, persists nothing.
- aura-research: StageBlock::Grid ({"block":"std::grid"}), schema-strict
parse arm (empty slot list: every key but "block" is refused by the
generic unknown-slot check), PROCESS_BLOCKS entry, intrinsic-tier no-op
arm; the vocabulary test's non-empty-slots guard carries a pinned
std::grid-only exception (a nominal slot would misdescribe the
vocabulary to describe_block consumers).
- aura-campaign: the inter-stage seam is a typed two-armed StageFlow
(points-only vs executed members); gate / mc-per-survivor fence the
points-only arm with defensive PipelineShape faults; preflight admits
std::grid only as the first stage and only immediately before
std::walk_forward (every other neighbor consumes executed reports).
- aura-cli: translate_walkforward / translate_mc lead with
StageBlock::Grid; the two family-shape E2E pins flip to zero Sweep
families; translate_generalize keeps its executed sweep(argmax) — its
generalize stage consumes the argmax winner report as the cell nominee.
- docs: dated #256 amendment in the ledger's verb-dissolution narrative;
the authoring-guide vocabulary transcript gains the std::grid line.
Behaviour preservation: the exact-grade real-data pins
(walkforward_real_e2e_pins_the_exact_current_grade,
mc_r_bootstrap_real_e2e_pins_the_exact_current_grade) pass unmodified —
survivor points reach the wf stage in the same odometer order as before.
Measured (the #256 acceptance measurement; debug build, real GER40 2025,
2x2 grid, `aura walkforward --real`, 3 runs each):
before (a55e4cf): 6.27 / 6.18 / 6.18 s
after: 4.52 / 4.49 / 4.45 s (~ -27% wall clock)
Suite: cargo test --workspace green (0 failed); clippy clean.
Decision log: #256 comments (fork rationale incl. the StageFlow seam and
the slot-guard exception).
Cycle-close audit over the feature trio 75bfb93..d1b3a3d (#261#260#262). Architect verdict: clean — C1 (both stop variants and the
per-instrument cost round-trip through the manifest), C2 (rollover-only
emission), C18 (scalar cost wire form byte-identical; VolTf additive),
the four-scalar discipline (SessionFrankfurt bakes the timezone, no
string kind), and the match-arm lockstep across all four RiskRegime/
StopRule sites all hold; commit bodies match the diff. One low drift
item — the repeated 60·10^9 ns-per-minute literal in vol_tf_stop.rs —
fixed here as NS_PER_MINUTE rather than carried.
No regression scripts are configured (the architect is the sole gate);
full workspace suite green, clippy clean. Spec and plan working files
discarded per the audit lifecycle.
refs #262
The second risk-regime variant vol_tf{period_minutes, length, k}
computes the vol estimator over completed time buckets: an H1 signal
gets an H1-matched stop in the research matrix instead of a hand-scaled
minute stop (the issue's k-inflation workaround retires). Additive
externally-tagged serde (stored Vol documents keep their content ids);
the executor arm wires the VolTfStop primitive via the builder+bind
chain; validate checks period_minutes/length/k positivity per regime.
The member-manifest round-trip holds for both variants (C1): vol_tf
members stamp stop_period_minutes/stop_length/stop_k and the reproduce
path re-derives the VolTf stop whenever stop_period_minutes is present
— never a silent default-Vol fallback. The Regimes slot label and the
unit notes name the new variant (period_minutes IS the stop's
timescale); glossary, authoring guide, and design ledger record the
second variant. Default regime and sugar paths byte-untouched.
Coverage: node units (rollover-only emission, bucket-delta math, the
constant-|delta| correspondence with the per-cycle regime), executor
fold, serde/validate units, the manifest round-trip unit, and a
hostless two-regime e2e (distinct ordinals; vol_tf members stamp their
timescale, vol members do not).
closes#262
k · sqrt(EMA(delta^2, length)) over completed period_minutes buckets —
the vol regime on a coarser clock, as a fused primitive in the Resample
mold: the in-progress bucket lives in node state, the finished bucket's
close is differenced against the previous one on rollover, and the stop
distance is emitted ONCE per completed bucket (C2: a partial bucket
never exists downstream; Firing::Any consumers hold the last value).
EMA follows Ema's exact convention (SMA seed over the first length
deltas, alpha = 2/(length+1)), pinned by a constant-|delta|
correspondence test against the per-cycle vol regime. Not rostered in
the std vocabulary (stop primitives are risk-executor internals).
First slice of #262; the regime variants, executor arm, and manifest
round-trip follow.
refs #262
Each cost component's knob (cost_per_trade / slip_vol_mult /
carry_per_cycle) now accepts one number for every cell — today's form,
byte-identical on the wire so stored documents keep their content ids
(CostValue's custom serde, the Axis precedent) — or an instrument-keyed
map resolved per cell at member construction. A mixed-scale matrix
(GER40 ~2e4, EURUSD ~1.1) keeps consistent cost units in ONE document:
the N-copies workaround retires and generalize-under-constant-costs
becomes expressible again.
Validation is strict both ways at the intrinsic tier: a map missing a
campaign instrument, or naming one the campaign does not list, refuses
at validate with prose naming the component and instruments (no silent
default — a partial map would reproduce exactly the silent unit
inconsistency the issue reports). The instrument threads through
cost_knob/cost_nodes_for/run_blueprint_member from the cell (run side
and persist re-run side receive the same value — the C1 drift-alarm
lockstep); the manifest stamps the resolved per-cell value under the
unchanged cost[k].<knob> key, and the sugar/reproduce paths pass an
inert instrument (their specs are scalar by construction). Fork
decisions and the discarded first attempt: issue #260 comments.
New coverage: serde round-trips (map form; scalar stays bare), validate
cross-check units, both prose directions pinned, and two hostless e2es
over the synthetic SYMA+SYMB archive — per-cell resolution visible in
the member manifests, and the validate refusal.
closes#260
A Frankfurt-baked zero-arg Session preset (open 09:00 Europe/Berlin as
literals) joins the std vocabulary as SessionFrankfurt, declaring
period_minutes: I64 (default 15) as its one scalar knob — the
single-knob roster pattern the cost nodes established. A data-only
blueprint can now reach a session/time-of-day stream (bars_since_open),
unblocking time-of-day conditioners for the confirm/refute research
model. Roster count pins bump 28 -> 29 in both conscious-act sites.
Fork decision on the issue: the preset variant over a param-vocabulary
extension — a timezone param would need a string kind, breaching the
four-scalar discipline; the preset keeps the vocabulary closed and is
the additive extension the roster-exclusion comment anticipates. The
4-arg Session::builder stays untouched; in_session/session_open_ts
stay deferred (#154).
closes#261
A data-only op-script naming the roster id SessionFrankfurt must
resolve to a Frankfurt-baked Session preset (09:00 Europe/Berlin baked,
one period_minutes: I64 knob, bars_since_open: I64 out) and build to a
finished Composite. Fails on std_vocabulary returning None — the preset
is absent from the roster; fork decision on the issue.
refs #261
Cycle-close audit over the /boss batch 84e1075..bdafbde (#258#259#264#266#247#265#269). Architect verdict: substance clean — C1
byte-identity (conduit serde-skipped, pinned floats verbatim, uncosted
bit-identical), C10 one-home-for-cost (all four conduit consumers read
the already-netted series), C18 (conduit off the wire, --version and
manifest single-sourced) all hold. Two medium debt items, both fixed
here rather than carried:
- render_bind_error's catch-all Debug-framed the remaining BindError
variants, making #269's no-identifier-on-stderr claim argv-gated
rather than structural. The match is now exhaustive: DuplicateBinding,
EmptyAxis, EmptyRange render as axis-usage prose; a Compile fault — a
blueprint defect, not a usage error — renders as a prose frame with
the compile detail explicitly labelled internal (per-variant
CompileError prose belongs to the graph-build surface, not this
boundary). Unit tests pin the arms.
- The #265 knob-unit notes covered constant and vol_slippage but not
carry.carry_per_cycle, which carries the identical price-unit-vs-R
trap; the note set and its pins now cover all three cost components.
No regression scripts are configured (the architect is the sole gate);
full workspace suite green, clippy clean. Spec and plan working files
discarded per the audit lifecycle.
refs #265 refs #269
render_bind_error's catch-all Debug-framed UnknownKnob although its
payload is already a fully-prosed message; the variant now renders the
payload alone. The two tests that pinned the frame verbatim repoint to
the prose fragments. Exit 2 unchanged. Completes the #247 defect
family: no Rust identifier reaches a user's stderr on either verb path.
closes#269
The rejection must reach stderr as the prose payload alone — no
UnknownKnob(..) Debug frame around it. Fails on today's frame; the
prose fragments already pass, so the sole pinned defect is the frame.
refs #269
campaign introspect --unwired now annotates the four silently
misreadable knobs with unit/semantics sub-lines: cost_per_trade is a
price-unit numerator charged as cost/|entry−stop| in R (not a cost in
R); slip_vol_mult multiplies the per-cycle vol estimate; vol.length
smooths the estimator and does not set the stop's timescale; vol.k
scales the stop distance (the risk-unit lever). Rendered additively
via a new OpenSlot.notes field, so the sibling tests pinning the slot
hint lines verbatim stay green byte-identically.
The glossary's cost-model and risk-regime entries mirror the same
semantics inline (two-sentence convention kept), the risk entry
pointing at #262 for the timescale-matched variant.
closes#265
The four silently-misreadable cost/risk knobs (cost_per_trade,
slip_vol_mult, vol.length, vol.k) must carry one-line unit/semantics
annotations in campaign introspect --unwired — blocking the price-vs-R
and length-as-timescale misreadings that produce no error today.
refs #265
The blueprint-sweep terminal rendered BindError via {:?}, leaking
KindMismatch { .. } and MissingKnob(..) Rust Debug structs where the
sibling diagnostic on the same verb ships prose. Both now render in
that register — the axis name, the expected-vs-supplied kinds resp.
the unbound knob, and the --list-axes pointer; exit stays 2.
Scoped out (same defect family, different verb path): the walkforward
path's UnknownKnob wrapper still prints its Debug-ish frame around an
already-prose payload and is pinned verbatim by two existing tests;
filed as a follow-up rather than widened into this slice.
closes#247
Both blueprint-sweep bind faults (wrong-kind axis value; an open param
no axis covers) must print prose in the unknown-axis sibling's register
— axis name, kind/knob specifics, a --list-axes pointer — never the raw
Debug structs KindMismatch{..}/MissingKnob(..). Exit stays 2. Fails on
the Debug leakage present today, not on scaffolding.
refs #247
aura --version now prints 'aura 0.1.0 (<commit>)', the commit sourced
from the same AURA_COMMIT build provenance the run manifest stamps
(option_env! at build time, 'unknown' fallback, dirty suffix included)
— a pre-run freshness probe: a stale binary is now distinguishable from
an engine bug at the CLI surface, and research docs pinning
'Engine: aura @ <sha>' can verify the binary they invoke.
The version string is memoized as a OnceLock-leaked &'static str:
clap 4.6's builder Str accepts only &'static str (no From<String>), so
the owned-String alternative a quality pass suggested does not build —
verified, held with evidence in the loop report.
closes#266
aura --version must print 'aura 0.1.0 (<commit>)', the commit sourced
from the same AURA_COMMIT build provenance the run manifest stamps —
a pre-run freshness probe distinguishing a stale binary from an engine
bug. Fails on the absent parenthesized commit, not on scaffolding.
refs #266
aura data list prints the archive's known symbols sorted, one per line
(DataServer::symbols() over the project-resolved archive root — the
discovery step before aura data coverage or scoping a campaign's
instrument matrix). An empty or absent archive prints a 'no symbols'
prose line and exits 0, per the verb corpus's empty-result register.
Pure report fn with unit tests; the headline e2e runs hostless over
the synthetic SYMA+SYMB archive.
Together with the coverage verb this closes the inventory gap: an
agent operating through the CLI alone can now discover instruments
and their per-month gaps before authoring a campaign.
closes#264
aura data list must print the archive's symbols sorted, one per line,
exit 0 — the discovery step before aura data coverage. Hostless RED over
the synthetic SYMA+SYMB archive; fails on the absent 'list' subcommand
(clap exit 2), not on scaffolding.
refs #264
aura data coverage <SYMBOL> prints the archive's monthly file-index
coverage: a span line (first..last present month) and either an explicit
'no gaps' line or one 'missing: YYYY-MM..YYYY-MM' line per interior
contiguous hole. The Copper failure mode — a campaign aborting mid-run
because a first/last-bounds check passed while the window start had no
data — is now visible before any run (the real archive reports
'Copper missing: 2017-04..2019-09'). An unknown symbol (no archive
files) refuses on stderr with exit 1; informational absence stays
prose + exit 0 per the verb corpus convention.
Mechanics: list_m1_months goes pub in aura-ingest (archive_extent
deliberately walks past gaps and cannot report them); a new two-level
'aura data' clap namespace follows the Nodes precedent; the archive
root resolves through the normal project path (paths.data override,
data-server default otherwise). Pure report fn with unit tests for the
refusal, the gapless span, and the collapsed-range shape; the headline
e2e runs hostless over a new gapped synthetic-archive fixture.
refs #264 (cut 2, aura data list, follows separately)
aura data coverage <SYMBOL> must print the archive's first/last present
month and each interior missing-month range collapsed to one
'missing: YYYY-MM..YYYY-MM' line — the Copper failure mode (first/last
check passes while the window start has no data) made visible before a
campaign runs. Hostless RED over a new gapped synthetic-archive fixture
(fresh_project_with_gapped_data); fails on the absent 'data' subcommand
(clap exit 2), not on scaffolding.
refs #264
The OOS bootstrap conduit RMetrics.trade_rs becomes net_trade_rs and
carries the COST-NETTED per-trade R (r − cost_in_r, trade order). Every
conduit consumer is thereby net-when-costed: the monte-carlo pooled-OOS
and per-survivor bootstraps, the walk-forward oos_r pooling, and the
deflation null-max — which previously compared a net observed statistic
against a gross-resampled null whenever a costed campaign selected on
net_expectancy_r. An uncosted run is bit-identical (empty cost stream
⇒ cost 0.0 per trade), so every existing golden pin stays green.
Design: one conduit, no knob — an explicit net: knob would let a costed
campaign silently produce a gross headline again (the exact misreading
of the issue's evidence). Per-member RMetrics scalar fields stay gross;
net_expectancy_r keeps its meaning. Wire shape unchanged (serde(skip)).
The net series is materialized as a separate expression in summarize_r;
the pinned-float expressions (net_sum, the SQN pair, the lockstep
r_metrics_from_rs copies) keep their tokens verbatim. Fork decisions
and rationales: issue #259 comments.
New coverage, both hostless over the synthetic SYMA archive: a costed
campaign twin shifts the pooled-OOS bootstrap (sweep→gate→wf→mc) and
every per-survivor bootstrap (sweep→mc) below its gross sibling, with
the trade population unchanged; a unit test pins the conduit as the
cost-netted series with the empty-stream degeneracy. Existing conduit
tests renamed with the field.
Verified: full workspace suite green (71 result blocks), clippy clean,
zero bare trade_rs tokens remain, ledger conduit prose updated in place.
closes#259
Every hand-rolled test-sandbox helper keyed its directory on
std::process::id() under env::temp_dir(), so the pre-create wipe never
matched a prior run's path: one directory stranded per helper site per
test-binary invocation, >50k dirs / 15.5 GiB on the 16 GiB tmpfs by
2026-07-13.
The remedy extends commit 84e1075's knob-lab pattern to every site:
fixed, tag-keyed, PID-FREE names under the build-tree tmp anchor, each
site keeping its pre-create remove_dir_all — which now genuinely
reclaims the previous run. Integration/bench targets use
env!("CARGO_TARGET_TMPDIR"); src-internal unit-test helpers (cargo sets
that var only for integration/bench targets) derive the same anchor
from env!("CARGO_MANIFEST_DIR") + ../../target/tmp. Residue is bounded
to one directory per site, on disk instead of the tmpfs, per checkout.
One deliberate exception: project_new.rs's tmp() stays under
env::temp_dir() because two of its tests (new_outside_a_work_tree_*)
need a base genuinely detached from any git work tree, and target/
lives inside the checkout's work tree. Its name carries a per-checkout
hash discriminator instead, so concurrently running checkouts (which
share /tmp) cannot race on the fixed name while each run still
reclaims its predecessor.
tempfile-RAII was considered and rejected (issue #258 decision log):
it cannot cover the process-lifetime OnceLock scaffolds (statics never
drop), adds a dependency, and still strands kill-leftovers on the
tmpfs.
Verified: full workspace suite green (incl. the new RED regression
test temp_cwd_sandbox_does_not_leak_under_env_temp_dir), clippy clean,
sandboxes observed under target/tmp with stable names across runs.
Stale pre-fix PID-keyed dirs in /tmp are not swept by this change and
age out only by manual cleanup.
closes#258
The helper keys its sandbox on std::process::id() under env::temp_dir(),
so the pre-create wipe never matches a prior run's path: one directory
strands per test-binary invocation on the 16 GiB tmpfs (>50k dirs by
2026-07-13). The test pins the property at one representative site: the
returned path embeds no PID and lives off env::temp_dir().
refs #258
built_scale_project() scaffolded ~66 MB (built node crate) into a
pid-suffixed tempdir, so its wipe-before-scaffold never hit a previous
run's copy: one leaked directory per test-binary run. 159 of those
(~10 GB) filled the host's 16 GB /tmp tmpfs to 100%, breaking every
std::env::temp_dir() consumer. A fixed name under CARGO_TARGET_TMPDIR
(real disk, reclaimed by cargo clean) makes the existing wipe reclaim
the previous run — steady state is exactly one directory. Trade-off:
two concurrent cargo-test invocations over the same checkout collide
on the fixed name, the documented pre-existing caveat class for this
suite (#223).
closes#257
probe_window drained a source's entire close column just to learn its
first and last bar timestamp; both live callers (open_real_source and
campaign_window_ms on the campaign trunk) now resolve through
aura-ingest's new archive_extent(): list the symbol's SYM_YYYY_MM.m1
files, load only the boundary months via the existing loader seam, and
walk across gap months forward/backward — O(2 file loads) typical
instead of O(archive). Empty-window refusal semantics are unchanged
(pinned). Two characterization tests captured the resolved windows
against the OLD implementation over the synthetic per-test archive and
stay green unchanged after the swap.
Measured: the probe itself drops 366ms -> 13ms (~28x) on the full
2014-2026 GER40 archive; end-to-end command wall-clock moves only
~2-3% because the sim loop dominates — the fix removes an
O(archive-size) term, not the dominant cost.
closes#252
blueprint_walkforward_family's pre-flight executed the full first-IS-
window sweep via blueprint_sweep_over and discarded the result, purely
to surface axis errors once before the parallel window fan-out (#177).
validate_axis_grid now drives the sweep terminal's own resolve/arity/
kind checks with a constant placeholder report — same single-sourced
rejection wording, no member run, no roller derivation needed.
Scope correction against the issue text: this path serves only the
synthetic (non---real) walkforward. The --real trunk routes through
verb_sugar into aura-campaign's [Sweep, WalkForward] process, whose
leading full-window sweep is a separate, larger duplication — re-filed
with corrected attribution in the issue's closing comment.
closes#253
Env::data_path() handed paths.data to the data layer verbatim, so a
relative value resolved against the invoking process's cwd — aura run
from a project subdirectory silently read a different archive location
than from the root, while runs/ stayed anchored. Relative values now
join onto the project root, matching runs_root() and the [nodes]
pointer resolution; absolute values and the no-project default pass
through unchanged. RED-first unit test pins the root-anchored
resolution.
closes#254
The two generalize E2E tests whose property is no-window span
resolution (shared-window fallback, intersection semantics) no longer
stream the 6.6 GB host archive: fresh_project_with_data() generates a
deterministic two-symbol M1 archive (SYMA 2024-01..08, SYMB 2024-03..06
strictly inside it, so the intersection differs from symbols[0]'s
window on both bounds) in the exact zip/48-byte-record format
data-server reads, plus geometry sidecars, under the per-test project's
paths.data. Both tests drop their local_data_present gates — they now
run on any host, data mount or not.
~35s and ~50s become 0.74s and 1.14s; the full cli_run binary lands at
~25s (211s at the branch base). Full workspace suite: 247s -> 45.7s.
refs #250
Every E2E test that mutated the shared demo-project fixture store now
mints its own tempdir project whose 2-line Aura.toml points at the
once-built fixture crate by absolute [nodes] pointer; the runs/ store
follows the project root, so no two tests contend and libtest's
default thread parallelism applies. project_load keeps driving the
fixture directly (the suite's only proof of the inline-crate routing
arm); its single store-mutating test is that binary's only one, so no
lock is needed there either. This also closes the documented
cross-process race window (#223): there is no shared mutable store
left to race on.
Measured on the data-full host: cli_run 211s -> 57.2s (140 tests),
research_docs 21.1s -> 2.1s (73 tests), project_load green, clippy
clean.
refs #250
The E2E suite spawns the debug aura binary, whose wall-clock is
dominated by dependency code (zip inflate of the tick archive, sha2
over the project dylib, serde). Workspace crates stay at opt-level 0
for fast rebuilds and debugging; every pinned float is computed by
workspace code, so the byte-identity anchors are untouched.
refs #250
The two dissolved-walkforward flag tests assert window-independent
properties (plateau acceptance, per-flag stop defaulting) and now use
the ~135-day window the reproduce e2e already uses. The mc roller-fit
test replaces its full-archive probe sweep with a fixed ~30-day window
inside that same span: the property needs a data-carrying window far
shorter than the 90+30-day roller, not a live-span derivation, and the
archive only grows forward. Measured on the data-full host: 19.6s,
19.6s and 14.5s+ of serialized real-data compute drop to 1.1-1.9s each.
refs #250
RunManifest gains defaults: Vec<(String, Scalar)> — the wrap-prefixed
bound_param_space() of the signal, read after axis reopening, so a
bound param an axis overrode has already left the space and flows
through params instead (disjoint by construction; verified end to end:
a sweep member's overridden fast.length sits in params while
slow.length/bias.scale sit in defaults). params keeps its "what varied"
semantics and stays the reproduce input. One-directional serde
widening (#[serde(default)]) per the selection/instrument/topology_hash
idiom — old records deserialize with an empty defaults; unlike the
Option fields it always serializes, mirroring params. ~20
struct-literal sites across five crates gained the field
(compile-mandated breadth, no behaviour change at those sites).
The C14 ledger records the underlying decision (2026-07-13): generated
outcome records spend redundancy on direct readability (single writer,
cannot drift); authored intent artifacts admit none (every redundancy
is a drift site) — so the fix lands in the manifest, never the
blueprint.
Verification: RED test run_manifest_stamps_untouched_bound_defaults
green; cargo build --workspace; cargo test --workspace green; clippy
-D warnings on the touched crates; binary-level sweep exclusivity
check.
Executable spec for the defaults field: aura run on the fully bound
examples/r_sma.json must report a manifest whose params stays [] (the
"what varied" record) and whose new defaults field carries the
untouched bound params as wrap-prefixed (name, Scalar) pairs in
bound_param_space() order — fast.length I64 2, slow.length I64 4,
bias.scale F64 0.5. Asserts on the stdout JSON (C14: stdout shape ==
on-disk shape), so it compiles against the current RunManifest and
fails on the absent key.
The references to the relocated _open blueprints follow the plan's
two-way split: tests that only need a sweepable blueprint sweep the
closed twins via bound-override axes (#246) with their axis strings
byte-unchanged (the wrap prefix is the blueprint's internal name,
identical across twins); the 12 tests that genuinely exercise
open-param semantics (run/mc closed-guard refusals, subset-axes
MissingKnob, --list-axes open/bound line mix, gang dissolution and
campaign paths) read the fixtures under tests/fixtures/. The
research_docs/project_load store seeds sweep the closed twin (campaign
axis references resolve via the #246 reopen path); the INDEX.md history
note records the relocation.
Two deviations from the plan, both verified against the tree: doc
comments referencing the bare filename (no examples/ prefix) escaped
the global path swap and were reworded to stay truthful; the seed
variable open_bp was renamed closed_bp (the plan kept it, but the
rename touches only the binding and its uses and removes a
misdescription).
Verification: cargo test -p aura-cli --test cli_run (140 passed, 0
failed, real data exercised); --test research_docs; --test
project_load; the four byte-pinned exact grades unchanged; grep-clean
for examples/r_*_open.json across crates+docs; full workspace suite
green.
The four _open blueprints leave the user-facing gallery
(crates/aura-cli/examples/) and become test fixtures; the closed twins
are now the only example corpus (bound params are overridable defaults
since #246, so the closed files serve both run and sweep).
Compile-time embeds follow the move: the emit_* generators and the
shipped_*_serialize byte-pins target tests/fixtures/ (the open fixtures
stay builder-generated and byte-pinned — regenerating after the move
left the tree clean), load_open_r_sma and the axis-probe read the
fixture, and the walk-forward refit test plus the `aura graph` default
sample switch to the closed r_sma.json. graph_construct's open-fixture
tests swap example() -> fixture(), their names dropping the now-false
"shipped example" claim; a new e2e test pins the relocation property
(all four fixtures load and introspect, the gallery holds exactly the
four closed examples).
Intermediate state: cli_run.rs / research_docs.rs / project_load.rs
still reference the old example paths and fail at runtime until the
follow-up migration commit (MIGRATE sites move to the closed twins,
NEEDS-OPEN sites to the fixtures).
Verification: cargo build --workspace; cargo test -p aura-cli --bin
aura; cargo test -p aura-cli --test graph_construct; the three ignored
emit generators re-run byte-stable.
Per-cycle fieldtest over the #246 surface, consumer-perspective, public
interface only: the scaffold quickstart end to end (aura new -> run ->
--list-axes -> override sweep), --list-axes on a partially-open
blueprint, an override family reproduced bit-identically (3/3 members),
and the error surface (wrong axis name, kind-mismatched value — both
exit 2). Corpus under fieldtests/cycle-246-bound-override/.
Dispositions: the one bug (F6) is fixed in this commit — authoring-guide
§1 still asserted a bound param 'never shows up in --list-axes' and 'no
aura sweep --axis can reopen it', contradicting the shipped #246
semantics and the C12 amendment; the three-states paragraph now reads
open = must-bind, bound = overridable default. The friction finding (F5,
KindMismatch/MissingKnob printed as raw Debug structs beside polished
prose) is filed as #247.
refs #246
Architect drift review over 0ad8fc6..e4fb64d: the C12 amendment's words
match the shipped behaviour exactly (identity from the authored document,
run/mc closed-guards untouched, bound = overridable default; every
point-binding reload goes through the reopened probe). One medium item
fixed here: the unknown-axis test's rationale comment still described the
retired 'fully bound; nothing to sweep' refusal. Noted as low-priority
debt, unchanged: the four override-derivation helpers across the wrapped/
raw coordinate systems are candidates for consolidation when the next
family boundary appears.
refs #246
claude