Files
Aura/docs/specs/fieldtest-0007-signal-quality.md
T
Brummel f3cd2e1320 fieldtest: cycle-0007 — 4 examples, 7 findings
First fieldtest of the signal-quality loop. A standalone downstream-consumer crate
(fieldtests/cycle-0007-signal-quality/) path-depends on the engine crates and
exercises the post-0007 surface from the public interface only (rustdoc + specs +
ledger + glossary, never crates/*/src):

  1 single-signal quality backtest, end to end (SMA-cross -> Exposure -> SimBroker
    -> Recorder pip curve)
  2 Exposure clamp + sizing (hard ±1 saturation, sign preserved)
  3 sim-optimal integration: short-on-falling pays positive pips; pip_size=2 curve
    exactly halves pip_size=1
  4 north-star combine-two-signals (two MA-cross spreads summed into one exposure)

Findings: 3 working (carry-on), 2 spec_gap, 2 friction.
  - spec_gap: SimBroker firing policy + cold-exposure->0.0 warm-up emission shape
    not on the public surface (only in the feat commit body).
  - spec_gap: SimBroker input slot order (0=exposure, 1=price) only in C10 prose /
    commit body; a swapped wiring is not caught at bootstrap (both f64).
  - friction: the north-star "combine two signals" move needs a sum/LinComb
    combinator aura-std does not ship (hand-authored Add2 in the fixture).
  - friction: standalone consumer crate still needs the empty [workspace] table
    (carried from cycle-0006; tracked as #9).

Spec at docs/specs/fieldtest-0007-signal-quality.md feeds the next plan.

refs #4 #5

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 17:30:13 +02:00

12 KiB
Raw Blame History

Fieldtest — cycle-0007 (signal-quality loop) — 2026-06-04

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill)

Scope

Cycle 0007 realizes the C10 reframe (the signal-quality half) as two new aura-std nodes on the unchanged, domain-free engine:

  • Exposure { scale } — the decision/sizing node: clamp(signal / scale, -1, +1), one f64 exposure per fired cycle, None until its input is present.
  • SimBroker { pip_size } — the sim-optimal broker: a two-input node (exposure, price) that accumulates prev_exposure · (price prev_price) / pip_size and emits cumulative pip equity. pip_size is held reference metadata, never streamed.

Composed with the existing Sma / Sub and the cycle-0006 recording-node pattern (a node with output: vec![] pushing (ctx.now(), row) out of graph), these wire an end-to-end signal-quality backtest whose pip-equity curve is recorded via a sink. The fieldtest exercised this from a standalone downstream consumer crate (fieldtests/cycle-0007-signal-quality/) that path-depends on the engine crates and uses only the public surface (rustdoc + design ledger + glossary + project-layout, never crates/*/src).

Build exercised: all four binaries built and run from the current working tree via cargo run --manifest-path fieldtests/cycle-0007-signal-quality/Cargo.toml --bin <name> (debug profile, HEAD source recompiled each invocation — confirmed by the Compiling aura-core/std/engine lines).

Examples

fieldtests/cycle-0007-signal-quality/c0007_1_single_signal_quality.rs — single-signal quality backtest, end to end

  • Wires the full primary research loop: price → {SMA(2), SMA(4)} → Sub → Exposure(4.0) → SimBroker(1.0) → Recorder, price also fanning into the broker's price slot. Records the cumulative pip-equity curve over a rising price.
  • Fits the headline axis: composition into an end-to-end signal-quality harness + recording the equity curve (project-layout "A day in the life" step 3).
  • Outcome: built, ran. The pip VALUES matched the hand model exactly in the warm region; the recorded curve had three extra leading 0.0 rows (t=1,2,3) my first hand model did not predict — see the firing-shape spec_gap. Final assertion adjusted to the observed (principled) shape; passes.

fieldtests/cycle-0007-signal-quality/c0007_2_exposure_clamp.rs — Exposure shaping (clamp + sizing)

  • Feeds raw scores spanning below / at / above the saturation band, both signs, straight into Exposure(10.0) → Recorder. Asserts each output ∈ [-1,+1] and the exact clamp(score/scale, -1, +1) mapping (incl. hard saturation at ±1 and sign preservation).
  • Fits the "intent/exposure shaping via Exposure" axis.
  • Outcome: built, ran, matched expected [0.0, 0.5, 1.0, 1.0, -0.7, -1.0, 0.3] on the first try. Clean.

fieldtests/cycle-0007-signal-quality/c0007_3_pip_size_scale.rs — sim-optimal integration, short signal, pip_size divisor

  • Runs the same falling-price stream through two harnesses (pip_size 1.0 vs 2.0). Asserts (1) a short signal on a falling market makes positive pips (neg exposure · neg return > 0); (2) the pip_size=2.0 curve is exactly half the pip_size=1.0 curve, timestamp-for-timestamp.
  • Fits the "sim-optimal pip-equity integration via SimBroker" + "different pip_size" axes.
  • Outcome: built, ran. Final pips +3.0 (short pays); pip_size=2.0 curve exactly halved. Clean.

fieldtests/cycle-0007-signal-quality/c0007_4_combine_two_signals.rs — combine two signals into one exposure (north-star move)

  • Two MA-cross spreads (SMA2-SMA4 and SMA3-SMA6) summed by a hand-authored Add2 node into one combined score, shaped to exposure, fed to the broker. Asserts a warm equity region exists, equity is positive, and steady-state +1 (saturated) exposure earns +2 pips/step.
  • Fits the "combine two signals into one exposure and backtest the combination (the stated north-star research move)" axis (design INDEX C10).
  • Outcome: built, ran, final pips +4.0, steady-state slope +2/step. Passes — but the combine step required hand-writing a summing node aura-std does not ship (see friction).

Findings

[working] Exposure mapping is exactly as documented, bound is hard

  • Example: c0007_2.
  • What happened: clamp(score/scale, -1, +1) reproduced verbatim from the rustdoc one-line description — 0→0, 5/10→0.5, 25/10→saturates to 1.0 (not 2.5), 40/10→−1.0, sign preserved throughout. Every recorded value lay in [-1,+1].
  • Why working: the new surface was reached for, used as the rustdoc described, and correct on the first run with no surprises; the hard saturation (a downstream author's main safety question — "can exposure exceed ±1?") is empirically firm.
  • Recommended action: carry-on.

[working] Sim-optimal pip integration: sign, causality, and pip_size divisor all correct

  • Example: c0007_3 (and the warm region of c0007_1).
  • What happened: a short exposure on a falling price integrated to positive pips (+3.0), confirming the prev_exposure · Δprice sign convention; doubling pip_size exactly halved every pip value (the divisor relationship); the t-1 exposure earning t's return (causality, C2) showed up as the steady +1.0-pip/step slope under constant exposure and constant +2 price steps in c0007_1.
  • Why working: the broker's documented integration model held bit-for-bit against three independent hand computations (rising-long, falling-short, pip_size scaling).
  • Recommended action: carry-on.

[working] End-to-end signal-quality harness composes and records cleanly

  • Examples: c0007_1, c0007_4.
  • What happened: the full SMA-cross → Exposure → SimBroker → Recorder chain bootstrapped and ran with the cycle-0006 recording pattern unchanged; one source fanning into both the indicator legs and the broker's price slot wired without fuss; the recorded pip-equity curve drained from the channel exactly as in 0006.
  • Why working: the headline cycle deliverable — a runnable signal-quality backtest whose equity curve you can record — works from the public surface, including the north-star "combine two signals" composition.
  • Recommended action: carry-on.

[spec_gap] SimBroker's firing policy / warm-up emission shape is not on the public surface

  • Examples: c0007_1 (surfaced), c0007_3, c0007_4 (same shape).
  • What happened: I predicted the broker would emit its first equity row only once its exposure leg warmed (from t=4 in c0007_1). It actually emits on every price-fresh cycle from t=1, treating a still-cold exposure as 0.0 and recording leading 0.0 rows (t=1,2,3). Verbatim, c0007_1: recorded = [(1,0.0),(2,0.0),(3,0.0),(4,0.0),(5,1.0),(6,2.0),(7,3.0)] vs my first predicted = [(4,0.0),(5,1.0),(6,2.0),(7,3.0)]. The pip values matched once warm; only the emission shape differed.
  • Why spec_gap: the SimBroker rustdoc states the integration formula but not (a) its per-input firing policy (does it fire on price-fresh, exposure-fresh, or a barrier?), nor (b) that it emits 0.0-equity rows before the exposure leg warms. Both readings — "emit from first price tick, cold-exposure = flat" (what ships) and "emit only once exposure is defined" (what I guessed) — are plausible from the prose, and they change the recorded-curve length a downstream consumer reasons about. The feat commit body documents the cold-exposure→0.0 detail (Window::get(0) "or 0.0 when the exposure leg is cold"), but a commit body is not the public reference surface; the rustdoc / ledger C10 Realization is silent on it. I did not probe the engine internals to confirm why (that would end the test); the behaviour is the finding.
  • Recommended action: tighten the design ledger / rustdoc — state SimBroker's firing policy and the cold-exposure-emits-0.0 warm-up contract on the rustdoc (and, if it is a contract worth pinning, in C10's Realization note), so a consumer can predict the recorded curve's length and leading values.

[spec_gap] SimBroker's input slot order (exposure vs price) is only in the commit body / C10 prose, not the rustdoc

  • Examples: c0007_1, c0007_3, c0007_4 (all wire Target { node: broker, slot: 1 } for price and Edge { to: broker, slot: 0 } for exposure).
  • What happened: to wire the broker I had to know which input slot is exposure and which is price. The SimBroker rustdoc (struct.SimBroker.html, including its Node impl) does not render schema()'s body, so it does not state the slot order. I recovered it from design-ledger C10's Realization note ("a two-input node (exposure, price)") plus the feat commit body ("exposure (slot 0) + price (slot 1)"). I picked slot 0 = exposure, slot 1 = price; the pip signs came out correct, confirming the guess — but a wrong guess (swapping them) would also bootstrap successfully (both slots are f64, no kind mismatch to catch it) and silently produce a wrong-but-plausible equity curve.
  • Why spec_gap: a two-input node whose slots are not symmetric and not type- distinguishable needs its slot contract on the consumer-facing surface; the rustdoc omits it, and a mis-wiring is not caught at bootstrap (both f64). The ordering "exposure first" is only inferable from prose order in C10 / a commit message.
  • Recommended action: tighten the rustdoc — name the input slots on SimBroker's doc comment (slot 0 = exposure ∈ [-1,+1], slot 1 = price). Same applies to any future multi-f64-input node where slots are role-distinct but kind-identical.

[friction] The north-star "combine two signals" move needs a combinator aura-std does not ship

  • Example: c0007_4.
  • What happened: aura-std ships Sma, Sub, Exposure, SimBroker. To combine two signals into one exposure — the explicitly stated north-star research move ("backtest one signal, combine it with another", C10) — I needed to add (or weight-sum) two spread streams, and there is no Add / weighted-sum / mean combinator in aura-std. I hand-authored a project-local Add2 node (a schema with two f64 inputs + one f64 output, an eval summing the newest values), which is legitimate per C16 but is boilerplate every consumer attempting the headline move must rewrite. Sub exists (difference) but not its companion sum.
  • Why friction: the task completed, but the cycle's own stated primary research loop ("combine it with another") cannot be expressed with the shipped nodes alone; the most basic combinator for it is absent, so the consumer writes it by hand. This is the natural next-tidy candidate (cf. the cycle-0006 "recorder boilerplate" friction).
  • Recommended action: plan — add an Add / weighted-sum (or a small LinComb) combinator to aura-std so the stated north-star combination is expressible from shipped blocks. Pairs naturally with the existing Sub.

[friction] Standalone consumer crate still needs the empty [workspace] table (carried from cycle-0006)

  • Example: all (the fixture crate's Cargo.toml).
  • What happened: the same nested-consumer-crate resolver fight the cycle-0006 fieldtest recorded. I pre-emptively added the empty [workspace] table (Cargo's own keep-it-out-of-the-workspace fix) so the crate builds; without it the build fails with "current package believes it's in a workspace when it's not".
  • Why friction: it recurs for every downstream consumer crate created beside/under the engine repo (C16: "a project is always a Rust crate" depending on aura). Not new this cycle, but it bit again, confirming it is a standing onboarding wall, not a one-off.
  • Recommended action: plan — already noted in the cycle-0006 fieldtest; fold into the future aura new scaffolder / onboarding docs (open ledger thread). Carry-on is acceptable if 0006's item is already queued.

Recommendation summary

Finding Action
[working] Exposure clamp/sizing exact, hard bound carry-on
[working] SimBroker integration: sign / causality / pip_size divisor carry-on
[working] end-to-end harness composes + records (incl. combine) carry-on
[spec_gap] SimBroker firing policy / warm-up 0.0 emission undocumented tighten the design ledger / rustdoc
[spec_gap] SimBroker input slot order only in commit body / C10 prose tighten the rustdoc
[friction] no sum/weighted combinator in aura-std for the north-star move plan
[friction] consumer crate needs empty [workspace] (carried from 0006) plan (or carry-on if 0006 item queued)