fieldtest: World/C21 milestone — 4 examples, 9 findings (functional promise holds)
Milestone-close validation of the World/C21 blueprint-data family surface, run
from the PUBLIC interface only (design ledger + op-script grammar; blueprints
authored as a consumer via `aura graph build`, no crates/ source read). Binary
built from 4710fcc.
The milestone's FUNCTIONAL PROMISE HOLDS (3 working findings): a consumer-authored
blueprint round-trips the whole verb family (graph build → list-axes → sweep →
rank → reproduce; walkforward → reproduce; mc → reproduce); all three FamilyKinds
(Sweep 4/4, MonteCarlo 5/5, WalkForward 3/3) reproduce bit-identically through the
one shared content-addressed store (the C18 core promise); list-axes→sweep→rank
composes and self-corrects.
Findings surfaced for follow-on (NOT fixed here — filed to the tracker): 2 bugs
(aura run on an open blueprint PANICS exit 101 instead of a clean exit-2 like mc;
closed→walkforward --axis prints the rejection twice), 2 friction (sweep-side
closed error terser than mc's exemplary message; mc reproduce lines omit the
seed), 2 spec-gaps (every open knob is mandatory on sweep but the docs frame
list-axes as a menu; the whole blueprint-data verb family is invisible from
`--help` / no README — empirically confirms #159). Detail per finding in the spec.
fieldtests/world-c21-milestone/ carries the op-scripts, authored blueprints,
captured outputs, and run_all.sh (regenerates the gitignored runs/ telemetry).
refs #170
This commit is contained in:
@@ -0,0 +1,204 @@
|
||||
# Fieldtest — milestone World/C21 (run + orchestrate harness families from blueprint-data) — 2026-07-01
|
||||
|
||||
**Status:** Draft — awaiting orchestrator triage
|
||||
**Author:** fieldtester (dispatched by fieldtest skill)
|
||||
**Binary exercised:** `target/debug/aura`, built from HEAD `4710fcc` via
|
||||
`cargo build --workspace` (build clean). All invocations are the debug binary of
|
||||
the current working tree.
|
||||
|
||||
## Scope
|
||||
|
||||
The milestone promises that the **World** constructs and orchestrates *families*
|
||||
of harnesses from serialized **blueprint-data** — not just one backtest. A
|
||||
downstream researcher takes a loaded blueprint (a serialized signal graph) and
|
||||
drives a coherent verb family over it: `graph build` (author a blueprint from a
|
||||
JSON op-script), `sweep --list-axes` (discover sweepable knobs), `sweep --axis …`
|
||||
(grid families), `mc --seeds N` (Monte-Carlo families), `walkforward --axis …`
|
||||
(IS-refit walk-forward families, the capstone #173 / cycle 0097), `runs families`
|
||||
+ `runs family <id> rank <metric>` (list + rank), and `reproduce <id>`
|
||||
(content-addressed, bit-identical re-derivation across all three family kinds
|
||||
through one shared store, C18). This fieldtest drove all of it **from the public
|
||||
surface only** — the design ledger (C24/C18/C21/C9), the glossary, the documented
|
||||
op-script grammar, and the two shipped `.json` blueprint fixtures as example data.
|
||||
No implementation source (`crates/**/*.rs`) was read. Blueprints were **authored
|
||||
as a consumer** via `aura graph build` from op-scripts, not copied from fixtures.
|
||||
|
||||
## Examples
|
||||
|
||||
### fieldtests/world-c21-milestone/w1_open_crossover.ops.json — discover → sweep → rank
|
||||
- Op-script → `aura graph build` → an **open** SMA-crossover→Bias blueprint
|
||||
(`w1_open_crossover.bp.json`); then `sweep --list-axes` to discover the axis
|
||||
names, then `sweep --axis …` with exactly those names, then `runs families` /
|
||||
`runs family <id> rank <metric>`.
|
||||
- Fits the milestone's grid-family + discovery + ranking axes; exercises the
|
||||
whole list-axes→sweep→rank composition on a consumer-authored blueprint.
|
||||
- Outcome: **built** (exit 0, canonical, no trailing newline); `--list-axes`
|
||||
named 3 open knobs; a full sweep produced a 4-member `Sweep` family; `rank`
|
||||
ordered best-first by `sqn_normalized` / `expectancy_r`; `reproduce` re-derived
|
||||
4/4 bit-identically. One friction en route (see [spec_gap] mandatory-knobs).
|
||||
|
||||
### fieldtests/world-c21-milestone/w1 (reuse) — walk-forward → reproduce (CAPSTONE)
|
||||
- `aura walkforward w1_open_crossover.bp.json --axis … --select argmax` →
|
||||
`WalkForward` family, then `aura reproduce w2_wf-0`.
|
||||
- The capstone axis (#173): IS-refit walk-forward over a loaded blueprint +
|
||||
content-addressed reproduction of the third family kind.
|
||||
- Outcome: **ran** (exit 0). 3 OOS members + a 4th
|
||||
`{"walkforward":{oos_r, param_stability, stitched_total_pips, windows:3}}`
|
||||
summary line, exactly as promised; `reproduce` re-derived 3/3 bit-identically.
|
||||
|
||||
### fieldtests/world-c21-milestone/w3_closed_crossover.ops.json — Monte-Carlo → reproduce
|
||||
- Op-script → `aura graph build` → a **closed** (fully bound) blueprint; then
|
||||
`aura mc --seeds 5`, then `aura reproduce w3_mc-0`.
|
||||
- Fits the Monte-Carlo-family + reproduce axes; needs the open/closed distinction
|
||||
inverted from sweep (mc requires a closed blueprint).
|
||||
- Outcome: **ran** (exit 0). `--list-axes` empty (fully bound); `mc` produced a
|
||||
5-member `MonteCarlo` family (seeds 1..5, window [1,60], real trades);
|
||||
`reproduce` re-derived 5/5 bit-identically.
|
||||
|
||||
### fieldtests/world-c21-milestone/w4_* — cross-verb coherence & error actionability
|
||||
- Closed→sweep, open→mc, closed→walkforward, unknown-axis, unknown-family-id,
|
||||
subset-of-open-knobs, and `run` on an open blueprint.
|
||||
- Fits the "do the verbs compose / are errors actionable" axis.
|
||||
- Outcome: mostly clean exit-2 diagnostics (see findings), **plus one panic bug**
|
||||
(`aura run` on an open blueprint).
|
||||
|
||||
## Findings
|
||||
|
||||
### [working] Consumer-authored blueprint round-trips through the whole family surface
|
||||
- Examples: w1 (sweep), w2 (walkforward), w3 (mc).
|
||||
- Authoring a blueprint from the **documented op-script grammar** (`aura graph
|
||||
build < ops.json`) was friction-free: exit 0, canonical bytes, no trailing
|
||||
newline (`tail -c` confirmed). The same authored `.bp.json` then fed `run`,
|
||||
`sweep`, `mc`, and `walkforward` unchanged. `graph introspect --vocabulary` /
|
||||
`--node SMA` give a clean author-discovery surface (ports, kinds, bind hint).
|
||||
- This is the milestone's substrate working end-to-end from a real consumer path,
|
||||
not just the shipped fixtures.
|
||||
- Recommended action: carry-on.
|
||||
|
||||
### [working] All three FamilyKinds reproduce bit-identically (the C18 core promise)
|
||||
- Examples: w1 (`Sweep` 4/4), w2 (`WalkForward` 3/3), w3 (`MonteCarlo` 5/5).
|
||||
- `aura reproduce <id>` re-derived every member of every family kind
|
||||
bit-identically (exit 0, `reproduced N/N members bit-identically`). Each member
|
||||
manifest carries the shared `topology_hash`; sweep/wf members echo their params
|
||||
in the reproduce line. This is exactly "every family kind persists AND
|
||||
reproduces through one shared content-addressed store."
|
||||
- Recommended action: carry-on.
|
||||
|
||||
### [working] list-axes → sweep composes and is self-correcting; rank orders best-first
|
||||
- Example: w1.
|
||||
- `--list-axes` printed `<name>:<kind>` per open knob, prefixed by the
|
||||
blueprint's own root name (`graph.fast.length` for my `graph`-named blueprint;
|
||||
`stage1_signal.fast.length` for the fixture). Every printed name was accepted
|
||||
verbatim by `--axis`; a fully-bound knob is absent from the list (fixture's
|
||||
`scale`) and my unbound `scale` is present — the list tracks the actual open
|
||||
set. `runs family … rank sqn_normalized|expectancy_r` ordered members best-first.
|
||||
- Note: the C24 ledger text uses the fixture-specific `stage1_signal.` prefix as
|
||||
its example; `--list-axes` makes the real prefix explicit, so the composition is
|
||||
self-correcting even though the doc example is fixture-bound.
|
||||
- Recommended action: carry-on.
|
||||
|
||||
### [bug] `aura run <open-blueprint.json>` panics instead of erroring cleanly
|
||||
- Example: w4 (`w4_run_open.err` / `.exit`).
|
||||
- Verbatim:
|
||||
```
|
||||
thread 'main' (…) panicked at crates/aura-cli/src/main.rs:3107:10:
|
||||
signal binds + wraps to a valid harness: ParamArity { expected: 3, got: 0 }
|
||||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||||
```
|
||||
Exit 101. The sibling verb `aura mc` on the *same* open blueprint gives a clean,
|
||||
actionable exit-2 error ("mc requires a closed blueprint … 3 free knob(s) — bind
|
||||
them or use `aura sweep --axis`"). `run` should likewise refuse an open
|
||||
(free-knob) blueprint at the dispatch boundary, not `.expect()`-panic — the
|
||||
cycle-0097 audit's own robustness claim is "a malformed blueprint is rejected at
|
||||
the dispatch boundary (exit 2, no panic)". A consumer who authors an open
|
||||
blueprint and runs `aura run` on it (a natural first move) hits a raw panic +
|
||||
backtrace note.
|
||||
- Repro: `aura run <blueprint-with-≥1-free-knob>.json` → panic (exit 101).
|
||||
- Recommended downstream action: debug (RED test: `run` on an open blueprint exits
|
||||
2 with a "requires a closed/bound blueprint" message, no panic).
|
||||
|
||||
### [bug] closed→walkforward with an axis prints the same diagnostic twice
|
||||
- Example: w4 (`w4_4c_closed_to_wf.err`).
|
||||
- `aura walkforward <closed bp> --axis graph.fast.length=2,3` emits
|
||||
`aura: UnknownKnob("graph.fast.length")` **twice** (exit 2). Exit code is
|
||||
correct and there is no panic; the diagnostic is just duplicated. Cosmetic but
|
||||
observable output defect (the sweep / mc paths emit their rejection once).
|
||||
- Repro: `aura walkforward <closed bp> --axis <any>=<v>` → two identical stderr
|
||||
lines.
|
||||
- Recommended downstream action: debug (low severity — dedupe the emit; likely a
|
||||
per-window closure re-raising the same top-level error).
|
||||
|
||||
### [friction] closed→sweep error is terse where the mc-side equivalent is exemplary
|
||||
- Example: w4 (`w4_4a_closed_to_sweep.err` vs `w4_4b_open_to_mc.err`).
|
||||
- `sweep <closed bp> --axis graph.fast.length=…` → `UnknownKnob("graph.fast.length")`.
|
||||
The knob *exists* in the blueprint but is bound out, so "UnknownKnob" is
|
||||
technically-true-but-misleading. The inverse case (`mc <open bp>`) instead says
|
||||
"mc requires a closed blueprint (no free parameters); 3 free knob(s) — bind them
|
||||
or use `aura sweep --axis`" — which names the count and the fix. The sweep side
|
||||
deserves the symmetric "this blueprint is fully bound; nothing to sweep (use
|
||||
`--list-axes`)" message.
|
||||
- Why friction (not bug): task completes with the right exit code; the surface
|
||||
just under-explains a coherent rejection.
|
||||
- Recommended downstream action: plan (tidy the sweep-side open/closed message to
|
||||
match the mc-side quality).
|
||||
|
||||
### [friction] `mc reproduce` lines omit the seed (blank member label)
|
||||
- Example: w3 (`w3_reproduce.out`).
|
||||
- MC reproduce prints `w3_mc-0 member reproduced: bit-identical` — the member
|
||||
label between "member" and "reproduced" is blank, so a consumer cannot tell
|
||||
*which seed* reproduced (sweep/walkforward show the params). The `mc` **run**
|
||||
output does carry `"seed":N`, so the information exists; only the reproduce line
|
||||
drops it.
|
||||
- Why friction: reproduction succeeds; the audit line is just less legible for the
|
||||
seed-varying family kind.
|
||||
- Recommended downstream action: plan (label MC reproduce lines with `seed=N`).
|
||||
|
||||
### [spec_gap] Every open knob is MANDATORY on `sweep`/`walkforward`, but the docs frame list-axes as a menu
|
||||
- Examples: w1 (first attempt), w4 (`w4_4f_missingknob_subset.err`).
|
||||
- `--list-axes` presented 3 open knobs; sweeping the two "obvious" ones
|
||||
(`graph.fast.length`, `graph.slow.length`) failed with
|
||||
`MissingKnob("graph.bias.scale")` (exit 2). Every open knob must be assigned ≥1
|
||||
value; there is **no default** (consistent with C19's value-empty leaf / no
|
||||
baked default — but that consequence is nowhere in the family-verb docs). The
|
||||
glossary/ledger call `--list-axes` "discover sweepable knobs" and say the names
|
||||
are "exactly what `--axis` binds" — true, but silent that they are *all
|
||||
required*. I picked the reading "supply every listed knob; pin the ones you
|
||||
don't want to vary with a single-value axis" and it worked; an equally plausible
|
||||
reading — "unspecified knobs take a default / stay at their authored value" — is
|
||||
what the engine rejects. The diagnostic is actionable (names the missing knob).
|
||||
- Recommended downstream action: ratify + tighten the docs (state that a sweep
|
||||
binds the blueprint's *entire* open param-space; a subset needs the rest bound
|
||||
out at authoring time or pinned via single-value axes). Optionally reconsider
|
||||
whether an un-swept open knob could error at *parse* time listing all missing
|
||||
knobs at once rather than one-at-a-time.
|
||||
|
||||
### [spec_gap] The whole blueprint-data verb family is undiscoverable from `--help` (no README)
|
||||
- Examples: all — first thing a consumer runs is `aura --help`.
|
||||
- `aura --help`, `aura graph --help`, `aura graph build --help`, and
|
||||
`aura graph introspect --help` **all print the identical generic usage string**,
|
||||
which lists only the hard-wired `--harness`/`--strategy` forms. None of the
|
||||
milestone verbs appear: `graph build`, `graph introspect`, `run <bp.json>`,
|
||||
`sweep <bp> --list-axes`, `sweep <bp> --axis`, `mc <bp> --seeds`,
|
||||
`walkforward <bp> --axis`. The repo has **no README**. So the entire milestone
|
||||
surface is discoverable *only* from the design ledger (a design doc, not a user
|
||||
guide) and the op-script grammar block inside C24. A downstream consumer with
|
||||
just the binary + `--help` cannot find any of it.
|
||||
- This matches the deferred #159 (CLI discoverability of `build`/`introspect`),
|
||||
but empirically it means the milestone's own surface is invisible at the CLI.
|
||||
- Recommended downstream action: plan (surface the blueprint-data verbs in
|
||||
`--help` and/or a README; tracked as #159). Until then, treat the ledger as the
|
||||
only entry point — a real gap for anyone but the author.
|
||||
|
||||
## Recommendation summary
|
||||
|
||||
| Finding | Class | Action |
|
||||
|---|---|---|
|
||||
| Consumer-authored blueprint round-trips whole family surface | working | carry-on |
|
||||
| All three FamilyKinds reproduce bit-identically | working | carry-on |
|
||||
| list-axes→sweep composes; rank best-first | working | carry-on |
|
||||
| `aura run` on open blueprint panics (exit 101) | bug | debug |
|
||||
| closed→walkforward prints diagnostic twice | bug | debug |
|
||||
| closed→sweep error terser than mc-side equivalent | friction | plan |
|
||||
| mc reproduce lines omit the seed | friction | plan |
|
||||
| every open knob mandatory on sweep, undocumented | spec_gap | ratify + tighten docs |
|
||||
| blueprint-data verbs undiscoverable from `--help` / no README | spec_gap | plan (#159) |
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
# Milestone fieldtest — World/C21: run + orchestrate harness FAMILIES from
|
||||
# blueprint-data. Driven entirely from the public surface (design ledger C24/C18/
|
||||
# C21 + glossary + the documented op-script grammar). No source was read.
|
||||
#
|
||||
# Build first: cargo build --workspace (binary: target/debug/aura)
|
||||
# Then: ./run_all.sh
|
||||
set -u
|
||||
BIN="${AURA_BIN:-../../target/debug/aura}"
|
||||
|
||||
echo "===================================================================="
|
||||
echo "Scenario 1 — discover -> sweep -> list -> rank (OPEN blueprint)"
|
||||
echo "===================================================================="
|
||||
$BIN graph build < w1_open_crossover.ops.json > w1_open_crossover.bp.json
|
||||
$BIN sweep w1_open_crossover.bp.json --list-axes # discover the 3 open knobs
|
||||
# NOTE: every open knob is MANDATORY; a subset -> MissingKnob (see w4_4f).
|
||||
$BIN sweep w1_open_crossover.bp.json \
|
||||
--axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \
|
||||
--name w1_cross_sweep
|
||||
$BIN runs families
|
||||
$BIN runs family w1_cross_sweep-0 rank sqn_normalized
|
||||
$BIN reproduce w1_cross_sweep-0 # 4/4 bit-identical
|
||||
|
||||
echo "===================================================================="
|
||||
echo "Scenario 2 — walk-forward -> reproduce (CAPSTONE, OPEN blueprint)"
|
||||
echo "===================================================================="
|
||||
$BIN walkforward w1_open_crossover.bp.json \
|
||||
--axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \
|
||||
--select argmax --name w2_wf
|
||||
$BIN reproduce w2_wf-0 # 3/3 bit-identical
|
||||
|
||||
echo "===================================================================="
|
||||
echo "Scenario 3 — monte-carlo -> reproduce (CLOSED blueprint)"
|
||||
echo "===================================================================="
|
||||
$BIN graph build < w3_closed_crossover.ops.json > w3_closed_crossover.bp.json
|
||||
$BIN sweep w3_closed_crossover.bp.json --list-axes # empty: fully bound
|
||||
$BIN mc w3_closed_crossover.bp.json --seeds 5 --name w3_mc
|
||||
$BIN reproduce w3_mc-0 # 5/5 bit-identical
|
||||
|
||||
echo "===================================================================="
|
||||
echo "Scenario 4 — cross-verb coherence + error actionability"
|
||||
echo "===================================================================="
|
||||
$BIN sweep w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4a closed->sweep
|
||||
$BIN mc w1_open_crossover.bp.json --seeds 3 # 4b open->mc (exemplary err)
|
||||
$BIN walkforward w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4c (dup diagnostic)
|
||||
$BIN sweep w1_open_crossover.bp.json --axis graph.nonesuch.length=2 \
|
||||
--axis graph.slow.length=5 --axis graph.bias.scale=0.5 # 4d unknown axis
|
||||
$BIN reproduce no-such-family-99 # 4e unknown family
|
||||
$BIN run w1_open_crossover.bp.json # 4 BUG: PANICS (exit 101)
|
||||
@@ -0,0 +1 @@
|
||||
{"family_id":"w1_cross_sweep-0","kind":"Sweep","members":4}
|
||||
@@ -0,0 +1,3 @@
|
||||
graph.fast.length:I64
|
||||
graph.slow.length:I64
|
||||
graph.bias.scale:F64
|
||||
@@ -0,0 +1 @@
|
||||
{"format_version":1,"blueprint":{"name":"graph","nodes":[{"primitive":{"type":"SMA","name":"fast"}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub","name":"sub"}},{"primitive":{"type":"Bias","name":"bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}}
|
||||
@@ -0,0 +1,12 @@
|
||||
[
|
||||
{"op": "source", "role": "price", "kind": "F64"},
|
||||
{"op": "add", "type": "SMA", "name": "fast"},
|
||||
{"op": "add", "type": "SMA", "name": "slow"},
|
||||
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
|
||||
{"op": "add", "type": "Sub", "name": "sub"},
|
||||
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
|
||||
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
|
||||
{"op": "add", "type": "Bias", "name": "bias"},
|
||||
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
|
||||
{"op": "expose", "from": "bias.bias", "as": "bias"}
|
||||
]
|
||||
@@ -0,0 +1,4 @@
|
||||
{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":0.2146800000000105,"max_drawdown":0.19279999999999503,"bias_sign_flips":2,"r":{"expectancy_r":0.515750880540302,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":1.1439327691362082,"avg_loss_r":-0.7406128966515102,"profit_factor":3.089151631866538,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.6231985130629969,"sqn_normalized":0.6231985130629969,"net_expectancy_r":0.515750880540302,"conviction_terciles_r":[2.0771328641652427,0.21073267410717364,-0.7406128966515102]}}}
|
||||
{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.0023866666666611386,"max_drawdown":0.208293333333331,"bias_sign_flips":2,"r":{"expectancy_r":0.03753199703966933,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":1.127161364109783,"avg_loss_r":-0.5072826864953875,"profit_factor":1.1109795328290115,"max_r_drawdown":0.2739524763392649,"n_open_at_end":1,"sqn":0.0668754310632922,"sqn_normalized":0.0668754310632922,"net_expectancy_r":0.03753199703966933,"conviction_terciles_r":[-0.2739524763392649,1.127161364109783,-0.7406128966515102]}}}
|
||||
{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.20921999999999025,"max_drawdown":0.34265999999999847,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}
|
||||
{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.339953333333338,"max_drawdown":0.3845266666666764,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}
|
||||
@@ -0,0 +1,5 @@
|
||||
w1_cross_sweep-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
w1_cross_sweep-0 member graph.fast.length=2, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
w1_cross_sweep-0 member graph.fast.length=3, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
w1_cross_sweep-0 member graph.fast.length=3, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
reproduced 4/4 members bit-identically
|
||||
@@ -0,0 +1 @@
|
||||
aura: MissingKnob("graph.bias.scale")
|
||||
@@ -0,0 +1,4 @@
|
||||
{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":0.2146800000000105,"max_drawdown":0.19279999999999503,"bias_sign_flips":2,"r":{"expectancy_r":0.515750880540302,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":1.1439327691362082,"avg_loss_r":-0.7406128966515102,"profit_factor":3.089151631866538,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.6231985130629969,"sqn_normalized":0.6231985130629969,"net_expectancy_r":0.515750880540302,"conviction_terciles_r":[2.0771328641652427,0.21073267410717364,-0.7406128966515102]}}}}
|
||||
{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.20921999999999025,"max_drawdown":0.34265999999999847,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}}
|
||||
{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.0023866666666611386,"max_drawdown":0.208293333333331,"bias_sign_flips":2,"r":{"expectancy_r":0.03753199703966933,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":1.127161364109783,"avg_loss_r":-0.5072826864953875,"profit_factor":1.1109795328290115,"max_r_drawdown":0.2739524763392649,"n_open_at_end":1,"sqn":0.0668754310632922,"sqn_normalized":0.0668754310632922,"net_expectancy_r":0.03753199703966933,"conviction_terciles_r":[-0.2739524763392649,1.127161364109783,-0.7406128966515102]}}}}
|
||||
{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.339953333333338,"max_drawdown":0.3845266666666764,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}}
|
||||
@@ -0,0 +1,4 @@
|
||||
w2_wf-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
w2_wf-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
w2_wf-0 member graph.fast.length=2, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical
|
||||
reproduced 3/3 members bit-identically
|
||||
@@ -0,0 +1,4 @@
|
||||
{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[25,36],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":0.8444293887137961,"mode":"Argmax","deflated_score":0.8444293887137961,"overfit_probability":0.000999000999000999,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.029682254132962535,"max_drawdown":0.0442303121080338,"bias_sign_flips":2,"r":{"expectancy_r":0.01050668114243935,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":0.09881344876754977,"avg_loss_r":-0.1661068541077815,"profit_factor":1.1897576327998223,"max_r_drawdown":0.1661068541077815,"n_open_at_end":1,"sqn":0.11861590656751105,"sqn_normalized":0.11861590656751105,"net_expectancy_r":0.01050668114243935,"conviction_terciles_r":[0.11079586598682767,-0.1661068541077815,0.08683103154827189]}}}}
|
||||
{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[37,48],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":-2.17811914776625,"mode":"Argmax","deflated_score":-2.2204163800380603,"overfit_probability":1.0,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.07698666016143554,"max_drawdown":0.07698666016143554,"bias_sign_flips":2,"r":{"expectancy_r":-0.3935692692179748,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":0.38656029975355716,"avg_loss_r":-0.7836340537037407,"profit_factor":0.24664592989963363,"max_r_drawdown":0.7927601270548278,"n_open_at_end":1,"sqn":-1.0088923276440807,"sqn_normalized":-1.0088923276440807,"net_expectancy_r":-0.3935692692179748,"conviction_terciles_r":[0.38656029975355716,-0.7927601270548277,-0.7745079803526537]}}}}
|
||||
{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[49,60],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":-0.43277394607057,"mode":"Argmax","deflated_score":-1.0185052275567759,"overfit_probability":1.0,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.02907021601207918,"max_drawdown":0.02907021601207918,"bias_sign_flips":0,"r":{"expectancy_r":-0.5505542307007089,"n_trades":1,"win_rate":0.0,"avg_win_r":0.0,"avg_loss_r":-0.5505542307007089,"profit_factor":-0.0,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.0,"sqn_normalized":0.0,"net_expectancy_r":-0.5505542307007089,"conviction_terciles_r":[0.0,0.0,0.0]}}}}
|
||||
{"walkforward":{"oos_r":{"avg_loss_r":-0.5709822980539929,"avg_win_r":0.1947290657628856,"conviction_terciles_r":[0.0,0.0,0.0],"expectancy_r":-0.24282028498961644,"max_r_drawdown":1.8105378609141427,"n_open_at_end":0,"n_trades":7,"net_expectancy_r":-0.24282028498961644,"profit_factor":0.25578165876580955,"sqn":-1.3720083218957762,"sqn_normalized":-1.3720083218957762,"win_rate":0.42857142857142855},"param_stability":[{"mean":2.0,"p25":2.0,"p5":2.0,"p50":2.0,"p75":2.0,"p95":2.0},{"mean":6.666666666666667,"p25":5.0,"p5":5.0,"p50":5.0,"p75":7.5,"p95":9.5},{"mean":0.5,"p25":0.5,"p5":0.5,"p50":0.5,"p75":0.5,"p95":0.5}],"stitched_total_pips":0.0,"windows":3}}
|
||||
@@ -0,0 +1 @@
|
||||
{"format_version":1,"blueprint":{"name":"graph","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":4}}]}},{"primitive":{"type":"Sub","name":"sub"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":0.5}}]}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}}
|
||||
@@ -0,0 +1,12 @@
|
||||
[
|
||||
{"op": "source", "role": "price", "kind": "F64"},
|
||||
{"op": "add", "type": "SMA", "name": "fast", "bind": {"length": {"I64": 2}}},
|
||||
{"op": "add", "type": "SMA", "name": "slow", "bind": {"length": {"I64": 4}}},
|
||||
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
|
||||
{"op": "add", "type": "Sub", "name": "sub"},
|
||||
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
|
||||
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
|
||||
{"op": "add", "type": "Bias", "name": "bias", "bind": {"scale": {"F64": 0.5}}},
|
||||
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
|
||||
{"op": "expose", "from": "bias.bias", "as": "bias"}
|
||||
]
|
||||
@@ -0,0 +1,6 @@
|
||||
{"family_id":"w3_mc-0","seed":1,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":1,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.08581949682071963,"max_drawdown":0.1782160409053034,"bias_sign_flips":16,"r":{"expectancy_r":-0.12761687270091654,"n_trades":17,"win_rate":0.29411764705882354,"avg_win_r":0.9422351195309215,"avg_loss_r":-0.5733885361308491,"profit_factor":0.684697969583447,"max_r_drawdown":5.084202573801676,"n_open_at_end":1,"sqn":-0.5709680662988649,"sqn_normalized":-0.5709680662988649,"net_expectancy_r":-0.12761687270091654,"conviction_terciles_r":[-0.5315147387933803,0.07491264421171377,0.006435165463506309]}}}}
|
||||
{"family_id":"w3_mc-0","seed":2,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":2,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.09465506366012477,"max_drawdown":0.14566471662215458,"bias_sign_flips":21,"r":{"expectancy_r":-0.1955399496066683,"n_trades":23,"win_rate":0.2608695652173913,"avg_win_r":0.8069365813559868,"avg_loss_r":-0.5493551958287819,"profit_factor":0.5184280562437801,"max_r_drawdown":5.678578478310644,"n_open_at_end":1,"sqn":-1.256408534576408,"sqn_normalized":-1.256408534576408,"net_expectancy_r":-0.1955399496066683,"conviction_terciles_r":[-0.14554052889979213,-0.3686624230732036,-0.0661669692586496]}}}}
|
||||
{"family_id":"w3_mc-0","seed":3,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":3,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.1458394558178947,"max_drawdown":0.18901416622571146,"bias_sign_flips":17,"r":{"expectancy_r":-0.19288571424888643,"n_trades":18,"win_rate":0.2777777777777778,"avg_win_r":0.6783848238535024,"avg_loss_r":-0.5279897673651899,"profit_factor":0.49417101631666377,"max_r_drawdown":5.505838374215854,"n_open_at_end":1,"sqn":-1.1554023132748366,"sqn_normalized":-1.1554023132748366,"net_expectancy_r":-0.19288571424888643,"conviction_terciles_r":[-0.10609643833726563,-0.3594422393970695,-0.11311846501232425]}}}}
|
||||
{"family_id":"w3_mc-0","seed":4,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":4,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.007137445429146379,"max_drawdown":0.0743086741823305,"bias_sign_flips":15,"r":{"expectancy_r":-0.07823191112894709,"n_trades":16,"win_rate":0.3125,"avg_win_r":1.09899031269056,"avg_loss_r":-0.6133329219559956,"profit_factor":0.8144696515391392,"max_r_drawdown":3.291640293181073,"n_open_at_end":1,"sqn":-0.31198614113195705,"sqn_normalized":-0.31198614113195705,"net_expectancy_r":-0.07823191112894709,"conviction_terciles_r":[-0.03355532359322373,-0.09249635589718175,-0.10357536343518768]}}}}
|
||||
{"family_id":"w3_mc-0","seed":5,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":5,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.05516687451601755,"max_drawdown":0.10049029278117855,"bias_sign_flips":21,"r":{"expectancy_r":-0.13824742850239835,"n_trades":22,"win_rate":0.36363636363636365,"avg_win_r":0.7288739517681235,"avg_loss_r":-0.6337453600855538,"profit_factor":0.6572030775170173,"max_r_drawdown":3.882156009122663,"n_open_at_end":1,"sqn":-0.7376556894251807,"sqn_normalized":-0.7376556894251807,"net_expectancy_r":-0.13824742850239835,"conviction_terciles_r":[-0.23289767226966626,-0.013422680153997594,-0.16465012001088963]}}}}
|
||||
{"mc_aggregate":{"bias_sign_flips":{"mean":18.0,"p25":16.0,"p5":15.2,"p50":17.0,"p75":21.0,"p95":21.0},"max_drawdown":{"mean":0.1375387781433357,"p25":0.10049029278117855,"p5":0.07954499790210011,"p50":0.14566471662215458,"p75":0.1782160409053034,"p95":0.18685454116162986},"total_pips":{"mean":-0.07772366724878062,"p25":-0.09465506366012477,"p5":-0.1356025773863407,"p50":-0.08581949682071963,"p75":-0.05516687451601755,"p95":-0.016743331246520624}}}
|
||||
@@ -0,0 +1,6 @@
|
||||
w3_mc-0 member reproduced: bit-identical
|
||||
w3_mc-0 member reproduced: bit-identical
|
||||
w3_mc-0 member reproduced: bit-identical
|
||||
w3_mc-0 member reproduced: bit-identical
|
||||
w3_mc-0 member reproduced: bit-identical
|
||||
reproduced 5/5 members bit-identically
|
||||
@@ -0,0 +1 @@
|
||||
{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":0.34185000000002036,"max_drawdown":0.11139999999998655,"bias_sign_flips":2,"r":{"expectancy_r":1.2710005136982836,"n_trades":3,"win_rate":1.0,"avg_win_r":1.2710005136982836,"avg_loss_r":0.0,"profit_factor":0.0,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":3.141496526818299,"sqn_normalized":3.141496526818299,"net_expectancy_r":1.2710005136982836,"conviction_terciles_r":[0.9285858482198718,2.0771328641652427,0.8072828287097363]}}}
|
||||
@@ -0,0 +1 @@
|
||||
aura: UnknownKnob("graph.fast.length")
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1 @@
|
||||
aura: mc requires a closed blueprint (no free parameters); 3 free knob(s) — bind them or use `aura sweep --axis`
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1,2 @@
|
||||
aura: UnknownKnob("graph.fast.length")
|
||||
aura: UnknownKnob("graph.fast.length
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1 @@
|
||||
aura: UnknownKnob("graph.nonesuch.length")
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1 @@
|
||||
aura: no such family 'no-such-family-99'
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1 @@
|
||||
aura: MissingKnob("graph.bias.scale")
|
||||
@@ -0,0 +1 @@
|
||||
2
|
||||
@@ -0,0 +1,4 @@
|
||||
|
||||
thread 'main' (1296536) panicked at crates/aura-cli/src/main.rs:3107:10:
|
||||
signal binds + wraps to a valid harness: ParamArity { expected: 3, got: 0 }
|
||||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||||
@@ -0,0 +1 @@
|
||||
run-open EXIT 101
|
||||
@@ -0,0 +1,22 @@
|
||||
Add
|
||||
And
|
||||
Bias
|
||||
CarryCost
|
||||
ConstantCost
|
||||
Delay
|
||||
EMA
|
||||
EqConst
|
||||
FixedStop
|
||||
Gt
|
||||
Latch
|
||||
LongOnly
|
||||
Mul
|
||||
PositionManagement
|
||||
Resample
|
||||
RollingMax
|
||||
RollingMin
|
||||
Sizer
|
||||
SMA
|
||||
Sqrt
|
||||
Sub
|
||||
VolSlippageCost
|
||||
Reference in New Issue
Block a user