Files
Aura/docs/specs/fieldtest-0007-signal-quality.md
T
Brummel f3cd2e1320 fieldtest: cycle-0007 — 4 examples, 7 findings
First fieldtest of the signal-quality loop. A standalone downstream-consumer crate
(fieldtests/cycle-0007-signal-quality/) path-depends on the engine crates and
exercises the post-0007 surface from the public interface only (rustdoc + specs +
ledger + glossary, never crates/*/src):

  1 single-signal quality backtest, end to end (SMA-cross -> Exposure -> SimBroker
    -> Recorder pip curve)
  2 Exposure clamp + sizing (hard ±1 saturation, sign preserved)
  3 sim-optimal integration: short-on-falling pays positive pips; pip_size=2 curve
    exactly halves pip_size=1
  4 north-star combine-two-signals (two MA-cross spreads summed into one exposure)

Findings: 3 working (carry-on), 2 spec_gap, 2 friction.
  - spec_gap: SimBroker firing policy + cold-exposure->0.0 warm-up emission shape
    not on the public surface (only in the feat commit body).
  - spec_gap: SimBroker input slot order (0=exposure, 1=price) only in C10 prose /
    commit body; a swapped wiring is not caught at bootstrap (both f64).
  - friction: the north-star "combine two signals" move needs a sum/LinComb
    combinator aura-std does not ship (hand-authored Add2 in the fixture).
  - friction: standalone consumer crate still needs the empty [workspace] table
    (carried from cycle-0006; tracked as #9).

Spec at docs/specs/fieldtest-0007-signal-quality.md feeds the next plan.

refs #4 #5

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 17:30:13 +02:00

198 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Fieldtest — cycle-0007 (signal-quality loop) — 2026-06-04
**Status:** Draft — awaiting orchestrator triage
**Author:** fieldtester (dispatched by fieldtest skill)
## Scope
Cycle 0007 realizes the C10 reframe (the signal-quality half) as two new
`aura-std` nodes on the unchanged, domain-free engine:
- `Exposure { scale }` — the decision/sizing node: `clamp(signal / scale, -1, +1)`,
one `f64` exposure per fired cycle, `None` until its input is present.
- `SimBroker { pip_size }` — the sim-optimal broker: a two-input node (exposure,
price) that accumulates `prev_exposure · (price prev_price) / pip_size` and
emits cumulative pip equity. `pip_size` is held reference metadata, never
streamed.
Composed with the existing `Sma` / `Sub` and the cycle-0006 recording-node
pattern (a node with `output: vec![]` pushing `(ctx.now(), row)` out of graph),
these wire an end-to-end signal-quality backtest whose pip-equity curve is
recorded via a sink. The fieldtest exercised this from a standalone downstream
consumer crate (`fieldtests/cycle-0007-signal-quality/`) that `path`-depends on
the engine crates and uses only the public surface (rustdoc + design ledger +
glossary + project-layout, never `crates/*/src`).
**Build exercised:** all four binaries built and run from the current working
tree via `cargo run --manifest-path fieldtests/cycle-0007-signal-quality/Cargo.toml
--bin <name>` (debug profile, HEAD source recompiled each invocation — confirmed
by the `Compiling aura-core/std/engine` lines).
## Examples
### fieldtests/cycle-0007-signal-quality/c0007_1_single_signal_quality.rs — single-signal quality backtest, end to end
- Wires the full primary research loop: `price → {SMA(2), SMA(4)} → Sub →
Exposure(4.0) → SimBroker(1.0) → Recorder`, price also fanning into the broker's
price slot. Records the cumulative pip-equity curve over a rising price.
- Fits the headline axis: composition into an end-to-end signal-quality harness +
recording the equity curve (project-layout "A day in the life" step 3).
- Outcome: built, ran. The pip VALUES matched the hand model exactly in the warm
region; the recorded curve had three *extra* leading `0.0` rows (t=1,2,3) my
first hand model did not predict — see the firing-shape spec_gap. Final
assertion adjusted to the observed (principled) shape; passes.
### fieldtests/cycle-0007-signal-quality/c0007_2_exposure_clamp.rs — Exposure shaping (clamp + sizing)
- Feeds raw scores spanning below / at / above the saturation band, both signs,
straight into `Exposure(10.0) → Recorder`. Asserts each output ∈ [-1,+1] and the
exact `clamp(score/scale, -1, +1)` mapping (incl. hard saturation at ±1 and sign
preservation).
- Fits the "intent/exposure shaping via Exposure" axis.
- Outcome: built, ran, matched expected `[0.0, 0.5, 1.0, 1.0, -0.7, -1.0, 0.3]`
on the first try. Clean.
### fieldtests/cycle-0007-signal-quality/c0007_3_pip_size_scale.rs — sim-optimal integration, short signal, pip_size divisor
- Runs the same falling-price stream through two harnesses (`pip_size` 1.0 vs 2.0).
Asserts (1) a short signal on a falling market makes positive pips
(`neg exposure · neg return > 0`); (2) the `pip_size=2.0` curve is exactly half
the `pip_size=1.0` curve, timestamp-for-timestamp.
- Fits the "sim-optimal pip-equity integration via SimBroker" + "different pip_size"
axes.
- Outcome: built, ran. Final pips +3.0 (short pays); pip_size=2.0 curve exactly
halved. Clean.
### fieldtests/cycle-0007-signal-quality/c0007_4_combine_two_signals.rs — combine two signals into one exposure (north-star move)
- Two MA-cross spreads (`SMA2-SMA4` and `SMA3-SMA6`) summed by a **hand-authored
`Add2` node** into one combined score, shaped to exposure, fed to the broker.
Asserts a warm equity region exists, equity is positive, and steady-state +1
(saturated) exposure earns +2 pips/step.
- Fits the "combine two signals into one exposure and backtest the combination
(the stated north-star research move)" axis (design INDEX C10).
- Outcome: built, ran, final pips +4.0, steady-state slope +2/step. Passes — but
the combine step required hand-writing a summing node aura-std does not ship
(see friction).
## Findings
### [working] Exposure mapping is exactly as documented, bound is hard
- Example: c0007_2.
- What happened: `clamp(score/scale, -1, +1)` reproduced verbatim from the rustdoc
one-line description — 0→0, 5/10→0.5, 25/10→saturates to 1.0 (not 2.5),
40/10→−1.0, sign preserved throughout. Every recorded value lay in [-1,+1].
- Why working: the new surface was reached for, used as the rustdoc described, and
correct on the first run with no surprises; the hard saturation (a downstream
author's main safety question — "can exposure exceed ±1?") is empirically firm.
- Recommended action: carry-on.
### [working] Sim-optimal pip integration: sign, causality, and pip_size divisor all correct
- Example: c0007_3 (and the warm region of c0007_1).
- What happened: a short exposure on a falling price integrated to **positive**
pips (+3.0), confirming the `prev_exposure · Δprice` sign convention; doubling
`pip_size` exactly halved every pip value (the divisor relationship); the t-1
exposure earning t's return (causality, C2) showed up as the steady +1.0-pip/step
slope under constant exposure and constant +2 price steps in c0007_1.
- Why working: the broker's documented integration model held bit-for-bit against
three independent hand computations (rising-long, falling-short, pip_size scaling).
- Recommended action: carry-on.
### [working] End-to-end signal-quality harness composes and records cleanly
- Examples: c0007_1, c0007_4.
- What happened: the full `SMA-cross → Exposure → SimBroker → Recorder` chain
bootstrapped and ran with the cycle-0006 recording pattern unchanged; one source
fanning into both the indicator legs and the broker's price slot wired without
fuss; the recorded pip-equity curve drained from the channel exactly as in 0006.
- Why working: the headline cycle deliverable — a runnable signal-quality backtest
whose equity curve you can record — works from the public surface, including the
north-star "combine two signals" composition.
- Recommended action: carry-on.
### [spec_gap] SimBroker's firing policy / warm-up emission shape is not on the public surface
- Examples: c0007_1 (surfaced), c0007_3, c0007_4 (same shape).
- What happened: I predicted the broker would emit its first equity row only once
its exposure leg warmed (from t=4 in c0007_1). It actually emits on **every
price-fresh cycle** from t=1, treating a still-cold exposure as 0.0 and recording
leading `0.0` rows (t=1,2,3). Verbatim, c0007_1:
`recorded = [(1,0.0),(2,0.0),(3,0.0),(4,0.0),(5,1.0),(6,2.0),(7,3.0)]` vs my first
`predicted = [(4,0.0),(5,1.0),(6,2.0),(7,3.0)]`. The pip *values* matched once
warm; only the emission *shape* differed.
- Why spec_gap: the `SimBroker` rustdoc states the integration formula but **not**
(a) its per-input firing policy (does it fire on price-fresh, exposure-fresh, or
a barrier?), nor (b) that it emits `0.0`-equity rows before the exposure leg
warms. Both readings — "emit from first price tick, cold-exposure = flat" (what
ships) and "emit only once exposure is defined" (what I guessed) — are plausible
from the prose, and they change the recorded-curve length a downstream consumer
reasons about. The feat commit body documents the cold-exposure→0.0 detail
(`Window::get(0)` "or 0.0 when the exposure leg is cold"), but a commit body is
not the public reference surface; the rustdoc / ledger C10 Realization is silent
on it. I did not probe the engine internals to confirm *why* (that would end the
test); the behaviour is the finding.
- Recommended action: tighten the design ledger / rustdoc — state `SimBroker`'s
firing policy and the cold-exposure-emits-`0.0` warm-up contract on the rustdoc
(and, if it is a contract worth pinning, in C10's Realization note), so a
consumer can predict the recorded curve's length and leading values.
### [spec_gap] SimBroker's input slot order (exposure vs price) is only in the commit body / C10 prose, not the rustdoc
- Examples: c0007_1, c0007_3, c0007_4 (all wire `Target { node: broker, slot: 1 }`
for price and `Edge { to: broker, slot: 0 }` for exposure).
- What happened: to wire the broker I had to know which input slot is exposure and
which is price. The `SimBroker` rustdoc (`struct.SimBroker.html`, including its
`Node` impl) does **not** render `schema()`'s body, so it does not state the slot
order. I recovered it from design-ledger C10's Realization note ("a two-input
node (exposure, price)") plus the feat commit body ("exposure (slot 0) + price
(slot 1)"). I picked slot 0 = exposure, slot 1 = price; the pip signs came out
correct, confirming the guess — but a wrong guess (swapping them) would also
bootstrap successfully (both slots are f64, no kind mismatch to catch it) and
silently produce a wrong-but-plausible equity curve.
- Why spec_gap: a two-input node whose slots are not symmetric and not type-
distinguishable needs its slot contract on the consumer-facing surface; the
rustdoc omits it, and a mis-wiring is *not* caught at bootstrap (both f64). The
ordering "exposure first" is only inferable from prose order in C10 / a commit
message.
- Recommended action: tighten the rustdoc — name the input slots on `SimBroker`'s
doc comment (slot 0 = exposure ∈ [-1,+1], slot 1 = price). Same applies to any
future multi-`f64`-input node where slots are role-distinct but kind-identical.
### [friction] The north-star "combine two signals" move needs a combinator aura-std does not ship
- Example: c0007_4.
- What happened: aura-std ships `Sma`, `Sub`, `Exposure`, `SimBroker`. To combine
two signals into one exposure — the explicitly stated north-star research move
("backtest one signal, combine it with another", C10) — I needed to *add* (or
weight-sum) two spread streams, and there is no `Add` / weighted-sum / mean
combinator in aura-std. I hand-authored a project-local `Add2` node (a `schema`
with two f64 inputs + one f64 output, an `eval` summing the newest values),
which is legitimate per C16 but is boilerplate every consumer attempting the
headline move must rewrite. `Sub` exists (difference) but not its companion sum.
- Why friction: the task completed, but the cycle's own stated primary research
loop ("combine it with another") cannot be expressed with the shipped nodes
alone; the most basic combinator for it is absent, so the consumer writes it by
hand. This is the natural next-tidy candidate (cf. the cycle-0006 "recorder
boilerplate" friction).
- Recommended action: plan — add an `Add` / weighted-sum (or a small `LinComb`)
combinator to aura-std so the stated north-star combination is expressible from
shipped blocks. Pairs naturally with the existing `Sub`.
### [friction] Standalone consumer crate still needs the empty `[workspace]` table (carried from cycle-0006)
- Example: all (the fixture crate's `Cargo.toml`).
- What happened: the same nested-consumer-crate resolver fight the cycle-0006
fieldtest recorded. I pre-emptively added the empty `[workspace]` table (Cargo's
own keep-it-out-of-the-workspace fix) so the crate builds; without it the build
fails with "current package believes it's in a workspace when it's not".
- Why friction: it recurs for every downstream consumer crate created beside/under
the engine repo (C16: "a project is always a Rust crate" depending on aura). Not
new this cycle, but it bit again, confirming it is a standing onboarding wall, not
a one-off.
- Recommended action: plan — already noted in the cycle-0006 fieldtest; fold into
the future `aura new` scaffolder / onboarding docs (open ledger thread). Carry-on
is acceptable if 0006's item is already queued.
## Recommendation summary
| Finding | Action |
|---|---|
| [working] Exposure clamp/sizing exact, hard bound | carry-on |
| [working] SimBroker integration: sign / causality / pip_size divisor | carry-on |
| [working] end-to-end harness composes + records (incl. combine) | carry-on |
| [spec_gap] SimBroker firing policy / warm-up `0.0` emission undocumented | tighten the design ledger / rustdoc |
| [spec_gap] SimBroker input slot order only in commit body / C10 prose | tighten the rustdoc |
| [friction] no sum/weighted combinator in aura-std for the north-star move | plan |
| [friction] consumer crate needs empty `[workspace]` (carried from 0006) | plan (or carry-on if 0006 item queued) |