From 780d82306741bd69d3bf01d8d56a22a687479b14 Mon Sep 17 00:00:00 2001 From: Brummel Date: Wed, 1 Jul 2026 17:00:54 +0200 Subject: [PATCH] =?UTF-8?q?fieldtest:=20World/C21=20milestone=20=E2=80=94?= =?UTF-8?q?=204=20examples,=209=20findings=20(functional=20promise=20holds?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Milestone-close validation of the World/C21 blueprint-data family surface, run from the PUBLIC interface only (design ledger + op-script grammar; blueprints authored as a consumer via `aura graph build`, no crates/ source read). Binary built from 4710fcc. The milestone's FUNCTIONAL PROMISE HOLDS (3 working findings): a consumer-authored blueprint round-trips the whole verb family (graph build → list-axes → sweep → rank → reproduce; walkforward → reproduce; mc → reproduce); all three FamilyKinds (Sweep 4/4, MonteCarlo 5/5, WalkForward 3/3) reproduce bit-identically through the one shared content-addressed store (the C18 core promise); list-axes→sweep→rank composes and self-corrects. Findings surfaced for follow-on (NOT fixed here — filed to the tracker): 2 bugs (aura run on an open blueprint PANICS exit 101 instead of a clean exit-2 like mc; closed→walkforward --axis prints the rejection twice), 2 friction (sweep-side closed error terser than mc's exemplary message; mc reproduce lines omit the seed), 2 spec-gaps (every open knob is mandatory on sweep but the docs frame list-axes as a menu; the whole blueprint-data verb family is invisible from `--help` / no README — empirically confirms #159). Detail per finding in the spec. fieldtests/world-c21-milestone/ carries the op-scripts, authored blueprints, captured outputs, and run_all.sh (regenerates the gitignored runs/ telemetry). refs #170 --- ...stone-world-c21-blueprint-data-families.md | 204 ++++++++++++++++++ fieldtests/world-c21-milestone/run_all.sh | 49 +++++ .../world-c21-milestone/w1_families.out | 1 + .../world-c21-milestone/w1_list_axes.err | 0 .../world-c21-milestone/w1_list_axes.out | 3 + .../world-c21-milestone/w1_open_build.err | 0 .../w1_open_crossover.bp.json | 1 + .../w1_open_crossover.ops.json | 12 ++ fieldtests/world-c21-milestone/w1_rank.err | 0 fieldtests/world-c21-milestone/w1_rank.out | 4 + .../world-c21-milestone/w1_reproduce.err | 0 .../world-c21-milestone/w1_reproduce.out | 5 + fieldtests/world-c21-milestone/w1_sweep.err | 1 + fieldtests/world-c21-milestone/w1_sweep.out | 0 fieldtests/world-c21-milestone/w1_sweep2.err | 0 fieldtests/world-c21-milestone/w1_sweep2.out | 4 + .../world-c21-milestone/w2_reproduce.err | 0 .../world-c21-milestone/w2_reproduce.out | 4 + fieldtests/world-c21-milestone/w2_wf.err | 0 fieldtests/world-c21-milestone/w2_wf.out | 4 + .../world-c21-milestone/w3_closed_build.err | 0 .../w3_closed_crossover.bp.json | 1 + .../w3_closed_crossover.ops.json | 12 ++ fieldtests/world-c21-milestone/w3_mc.err | 0 fieldtests/world-c21-milestone/w3_mc.out | 6 + .../world-c21-milestone/w3_reproduce.err | 0 .../world-c21-milestone/w3_reproduce.out | 6 + fieldtests/world-c21-milestone/w3_run.err | 0 fieldtests/world-c21-milestone/w3_run.out | 1 + .../w4_4a_closed_to_sweep.err | 1 + .../w4_4a_closed_to_sweep.exit | 1 + .../w4_4a_closed_to_sweep.out | 0 .../world-c21-milestone/w4_4b_open_to_mc.err | 1 + .../world-c21-milestone/w4_4b_open_to_mc.exit | 1 + .../world-c21-milestone/w4_4b_open_to_mc.out | 0 .../w4_4c_closed_to_wf.err | 2 + .../w4_4c_closed_to_wf.exit | 1 + .../w4_4c_closed_to_wf.out | 0 .../w4_4d_unknown_axis.err | 1 + .../w4_4d_unknown_axis.exit | 1 + .../w4_4d_unknown_axis.out | 0 .../w4_4e_reproduce_unknown.err | 1 + .../w4_4e_reproduce_unknown.exit | 1 + .../w4_4e_reproduce_unknown.out | 0 .../w4_4f_missingknob_subset.err | 1 + .../w4_4f_missingknob_subset.exit | 1 + .../w4_4f_missingknob_subset.out | 0 .../world-c21-milestone/w4_run_open.err | 4 + .../world-c21-milestone/w4_run_open.exit | 1 + .../world-c21-milestone/w4_run_open.out | 0 fieldtests/world-c21-milestone/w4_vocab.err | 0 fieldtests/world-c21-milestone/w4_vocab.out | 22 ++ 52 files changed, 358 insertions(+) create mode 100644 docs/specs/fieldtest-milestone-world-c21-blueprint-data-families.md create mode 100755 fieldtests/world-c21-milestone/run_all.sh create mode 100644 fieldtests/world-c21-milestone/w1_families.out create mode 100644 fieldtests/world-c21-milestone/w1_list_axes.err create mode 100644 fieldtests/world-c21-milestone/w1_list_axes.out create mode 100644 fieldtests/world-c21-milestone/w1_open_build.err create mode 100644 fieldtests/world-c21-milestone/w1_open_crossover.bp.json create mode 100644 fieldtests/world-c21-milestone/w1_open_crossover.ops.json create mode 100644 fieldtests/world-c21-milestone/w1_rank.err create mode 100644 fieldtests/world-c21-milestone/w1_rank.out create mode 100644 fieldtests/world-c21-milestone/w1_reproduce.err create mode 100644 fieldtests/world-c21-milestone/w1_reproduce.out create mode 100644 fieldtests/world-c21-milestone/w1_sweep.err create mode 100644 fieldtests/world-c21-milestone/w1_sweep.out create mode 100644 fieldtests/world-c21-milestone/w1_sweep2.err create mode 100644 fieldtests/world-c21-milestone/w1_sweep2.out create mode 100644 fieldtests/world-c21-milestone/w2_reproduce.err create mode 100644 fieldtests/world-c21-milestone/w2_reproduce.out create mode 100644 fieldtests/world-c21-milestone/w2_wf.err create mode 100644 fieldtests/world-c21-milestone/w2_wf.out create mode 100644 fieldtests/world-c21-milestone/w3_closed_build.err create mode 100644 fieldtests/world-c21-milestone/w3_closed_crossover.bp.json create mode 100644 fieldtests/world-c21-milestone/w3_closed_crossover.ops.json create mode 100644 fieldtests/world-c21-milestone/w3_mc.err create mode 100644 fieldtests/world-c21-milestone/w3_mc.out create mode 100644 fieldtests/world-c21-milestone/w3_reproduce.err create mode 100644 fieldtests/world-c21-milestone/w3_reproduce.out create mode 100644 fieldtests/world-c21-milestone/w3_run.err create mode 100644 fieldtests/world-c21-milestone/w3_run.out create mode 100644 fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.err create mode 100644 fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.exit create mode 100644 fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.out create mode 100644 fieldtests/world-c21-milestone/w4_4b_open_to_mc.err create mode 100644 fieldtests/world-c21-milestone/w4_4b_open_to_mc.exit create mode 100644 fieldtests/world-c21-milestone/w4_4b_open_to_mc.out create mode 100644 fieldtests/world-c21-milestone/w4_4c_closed_to_wf.err create mode 100644 fieldtests/world-c21-milestone/w4_4c_closed_to_wf.exit create mode 100644 fieldtests/world-c21-milestone/w4_4c_closed_to_wf.out create mode 100644 fieldtests/world-c21-milestone/w4_4d_unknown_axis.err create mode 100644 fieldtests/world-c21-milestone/w4_4d_unknown_axis.exit create mode 100644 fieldtests/world-c21-milestone/w4_4d_unknown_axis.out create mode 100644 fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.err create mode 100644 fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.exit create mode 100644 fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.out create mode 100644 fieldtests/world-c21-milestone/w4_4f_missingknob_subset.err create mode 100644 fieldtests/world-c21-milestone/w4_4f_missingknob_subset.exit create mode 100644 fieldtests/world-c21-milestone/w4_4f_missingknob_subset.out create mode 100644 fieldtests/world-c21-milestone/w4_run_open.err create mode 100644 fieldtests/world-c21-milestone/w4_run_open.exit create mode 100644 fieldtests/world-c21-milestone/w4_run_open.out create mode 100644 fieldtests/world-c21-milestone/w4_vocab.err create mode 100644 fieldtests/world-c21-milestone/w4_vocab.out diff --git a/docs/specs/fieldtest-milestone-world-c21-blueprint-data-families.md b/docs/specs/fieldtest-milestone-world-c21-blueprint-data-families.md new file mode 100644 index 0000000..5a99a12 --- /dev/null +++ b/docs/specs/fieldtest-milestone-world-c21-blueprint-data-families.md @@ -0,0 +1,204 @@ +# Fieldtest — milestone World/C21 (run + orchestrate harness families from blueprint-data) — 2026-07-01 + +**Status:** Draft — awaiting orchestrator triage +**Author:** fieldtester (dispatched by fieldtest skill) +**Binary exercised:** `target/debug/aura`, built from HEAD `4710fcc` via +`cargo build --workspace` (build clean). All invocations are the debug binary of +the current working tree. + +## Scope + +The milestone promises that the **World** constructs and orchestrates *families* +of harnesses from serialized **blueprint-data** — not just one backtest. A +downstream researcher takes a loaded blueprint (a serialized signal graph) and +drives a coherent verb family over it: `graph build` (author a blueprint from a +JSON op-script), `sweep --list-axes` (discover sweepable knobs), `sweep --axis …` +(grid families), `mc --seeds N` (Monte-Carlo families), `walkforward --axis …` +(IS-refit walk-forward families, the capstone #173 / cycle 0097), `runs families` ++ `runs family rank ` (list + rank), and `reproduce ` +(content-addressed, bit-identical re-derivation across all three family kinds +through one shared store, C18). This fieldtest drove all of it **from the public +surface only** — the design ledger (C24/C18/C21/C9), the glossary, the documented +op-script grammar, and the two shipped `.json` blueprint fixtures as example data. +No implementation source (`crates/**/*.rs`) was read. Blueprints were **authored +as a consumer** via `aura graph build` from op-scripts, not copied from fixtures. + +## Examples + +### fieldtests/world-c21-milestone/w1_open_crossover.ops.json — discover → sweep → rank +- Op-script → `aura graph build` → an **open** SMA-crossover→Bias blueprint + (`w1_open_crossover.bp.json`); then `sweep --list-axes` to discover the axis + names, then `sweep --axis …` with exactly those names, then `runs families` / + `runs family rank `. +- Fits the milestone's grid-family + discovery + ranking axes; exercises the + whole list-axes→sweep→rank composition on a consumer-authored blueprint. +- Outcome: **built** (exit 0, canonical, no trailing newline); `--list-axes` + named 3 open knobs; a full sweep produced a 4-member `Sweep` family; `rank` + ordered best-first by `sqn_normalized` / `expectancy_r`; `reproduce` re-derived + 4/4 bit-identically. One friction en route (see [spec_gap] mandatory-knobs). + +### fieldtests/world-c21-milestone/w1 (reuse) — walk-forward → reproduce (CAPSTONE) +- `aura walkforward w1_open_crossover.bp.json --axis … --select argmax` → + `WalkForward` family, then `aura reproduce w2_wf-0`. +- The capstone axis (#173): IS-refit walk-forward over a loaded blueprint + + content-addressed reproduction of the third family kind. +- Outcome: **ran** (exit 0). 3 OOS members + a 4th + `{"walkforward":{oos_r, param_stability, stitched_total_pips, windows:3}}` + summary line, exactly as promised; `reproduce` re-derived 3/3 bit-identically. + +### fieldtests/world-c21-milestone/w3_closed_crossover.ops.json — Monte-Carlo → reproduce +- Op-script → `aura graph build` → a **closed** (fully bound) blueprint; then + `aura mc --seeds 5`, then `aura reproduce w3_mc-0`. +- Fits the Monte-Carlo-family + reproduce axes; needs the open/closed distinction + inverted from sweep (mc requires a closed blueprint). +- Outcome: **ran** (exit 0). `--list-axes` empty (fully bound); `mc` produced a + 5-member `MonteCarlo` family (seeds 1..5, window [1,60], real trades); + `reproduce` re-derived 5/5 bit-identically. + +### fieldtests/world-c21-milestone/w4_* — cross-verb coherence & error actionability +- Closed→sweep, open→mc, closed→walkforward, unknown-axis, unknown-family-id, + subset-of-open-knobs, and `run` on an open blueprint. +- Fits the "do the verbs compose / are errors actionable" axis. +- Outcome: mostly clean exit-2 diagnostics (see findings), **plus one panic bug** + (`aura run` on an open blueprint). + +## Findings + +### [working] Consumer-authored blueprint round-trips through the whole family surface +- Examples: w1 (sweep), w2 (walkforward), w3 (mc). +- Authoring a blueprint from the **documented op-script grammar** (`aura graph + build < ops.json`) was friction-free: exit 0, canonical bytes, no trailing + newline (`tail -c` confirmed). The same authored `.bp.json` then fed `run`, + `sweep`, `mc`, and `walkforward` unchanged. `graph introspect --vocabulary` / + `--node SMA` give a clean author-discovery surface (ports, kinds, bind hint). +- This is the milestone's substrate working end-to-end from a real consumer path, + not just the shipped fixtures. +- Recommended action: carry-on. + +### [working] All three FamilyKinds reproduce bit-identically (the C18 core promise) +- Examples: w1 (`Sweep` 4/4), w2 (`WalkForward` 3/3), w3 (`MonteCarlo` 5/5). +- `aura reproduce ` re-derived every member of every family kind + bit-identically (exit 0, `reproduced N/N members bit-identically`). Each member + manifest carries the shared `topology_hash`; sweep/wf members echo their params + in the reproduce line. This is exactly "every family kind persists AND + reproduces through one shared content-addressed store." +- Recommended action: carry-on. + +### [working] list-axes → sweep composes and is self-correcting; rank orders best-first +- Example: w1. +- `--list-axes` printed `:` per open knob, prefixed by the + blueprint's own root name (`graph.fast.length` for my `graph`-named blueprint; + `stage1_signal.fast.length` for the fixture). Every printed name was accepted + verbatim by `--axis`; a fully-bound knob is absent from the list (fixture's + `scale`) and my unbound `scale` is present — the list tracks the actual open + set. `runs family … rank sqn_normalized|expectancy_r` ordered members best-first. +- Note: the C24 ledger text uses the fixture-specific `stage1_signal.` prefix as + its example; `--list-axes` makes the real prefix explicit, so the composition is + self-correcting even though the doc example is fixture-bound. +- Recommended action: carry-on. + +### [bug] `aura run ` panics instead of erroring cleanly +- Example: w4 (`w4_run_open.err` / `.exit`). +- Verbatim: + ``` + thread 'main' (…) panicked at crates/aura-cli/src/main.rs:3107:10: + signal binds + wraps to a valid harness: ParamArity { expected: 3, got: 0 } + note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace + ``` + Exit 101. The sibling verb `aura mc` on the *same* open blueprint gives a clean, + actionable exit-2 error ("mc requires a closed blueprint … 3 free knob(s) — bind + them or use `aura sweep --axis`"). `run` should likewise refuse an open + (free-knob) blueprint at the dispatch boundary, not `.expect()`-panic — the + cycle-0097 audit's own robustness claim is "a malformed blueprint is rejected at + the dispatch boundary (exit 2, no panic)". A consumer who authors an open + blueprint and runs `aura run` on it (a natural first move) hits a raw panic + + backtrace note. +- Repro: `aura run .json` → panic (exit 101). +- Recommended downstream action: debug (RED test: `run` on an open blueprint exits + 2 with a "requires a closed/bound blueprint" message, no panic). + +### [bug] closed→walkforward with an axis prints the same diagnostic twice +- Example: w4 (`w4_4c_closed_to_wf.err`). +- `aura walkforward --axis graph.fast.length=2,3` emits + `aura: UnknownKnob("graph.fast.length")` **twice** (exit 2). Exit code is + correct and there is no panic; the diagnostic is just duplicated. Cosmetic but + observable output defect (the sweep / mc paths emit their rejection once). +- Repro: `aura walkforward --axis =` → two identical stderr + lines. +- Recommended downstream action: debug (low severity — dedupe the emit; likely a + per-window closure re-raising the same top-level error). + +### [friction] closed→sweep error is terse where the mc-side equivalent is exemplary +- Example: w4 (`w4_4a_closed_to_sweep.err` vs `w4_4b_open_to_mc.err`). +- `sweep --axis graph.fast.length=…` → `UnknownKnob("graph.fast.length")`. + The knob *exists* in the blueprint but is bound out, so "UnknownKnob" is + technically-true-but-misleading. The inverse case (`mc `) instead says + "mc requires a closed blueprint (no free parameters); 3 free knob(s) — bind them + or use `aura sweep --axis`" — which names the count and the fix. The sweep side + deserves the symmetric "this blueprint is fully bound; nothing to sweep (use + `--list-axes`)" message. +- Why friction (not bug): task completes with the right exit code; the surface + just under-explains a coherent rejection. +- Recommended downstream action: plan (tidy the sweep-side open/closed message to + match the mc-side quality). + +### [friction] `mc reproduce` lines omit the seed (blank member label) +- Example: w3 (`w3_reproduce.out`). +- MC reproduce prints `w3_mc-0 member reproduced: bit-identical` — the member + label between "member" and "reproduced" is blank, so a consumer cannot tell + *which seed* reproduced (sweep/walkforward show the params). The `mc` **run** + output does carry `"seed":N`, so the information exists; only the reproduce line + drops it. +- Why friction: reproduction succeeds; the audit line is just less legible for the + seed-varying family kind. +- Recommended downstream action: plan (label MC reproduce lines with `seed=N`). + +### [spec_gap] Every open knob is MANDATORY on `sweep`/`walkforward`, but the docs frame list-axes as a menu +- Examples: w1 (first attempt), w4 (`w4_4f_missingknob_subset.err`). +- `--list-axes` presented 3 open knobs; sweeping the two "obvious" ones + (`graph.fast.length`, `graph.slow.length`) failed with + `MissingKnob("graph.bias.scale")` (exit 2). Every open knob must be assigned ≥1 + value; there is **no default** (consistent with C19's value-empty leaf / no + baked default — but that consequence is nowhere in the family-verb docs). The + glossary/ledger call `--list-axes` "discover sweepable knobs" and say the names + are "exactly what `--axis` binds" — true, but silent that they are *all + required*. I picked the reading "supply every listed knob; pin the ones you + don't want to vary with a single-value axis" and it worked; an equally plausible + reading — "unspecified knobs take a default / stay at their authored value" — is + what the engine rejects. The diagnostic is actionable (names the missing knob). +- Recommended downstream action: ratify + tighten the docs (state that a sweep + binds the blueprint's *entire* open param-space; a subset needs the rest bound + out at authoring time or pinned via single-value axes). Optionally reconsider + whether an un-swept open knob could error at *parse* time listing all missing + knobs at once rather than one-at-a-time. + +### [spec_gap] The whole blueprint-data verb family is undiscoverable from `--help` (no README) +- Examples: all — first thing a consumer runs is `aura --help`. +- `aura --help`, `aura graph --help`, `aura graph build --help`, and + `aura graph introspect --help` **all print the identical generic usage string**, + which lists only the hard-wired `--harness`/`--strategy` forms. None of the + milestone verbs appear: `graph build`, `graph introspect`, `run `, + `sweep --list-axes`, `sweep --axis`, `mc --seeds`, + `walkforward --axis`. The repo has **no README**. So the entire milestone + surface is discoverable *only* from the design ledger (a design doc, not a user + guide) and the op-script grammar block inside C24. A downstream consumer with + just the binary + `--help` cannot find any of it. +- This matches the deferred #159 (CLI discoverability of `build`/`introspect`), + but empirically it means the milestone's own surface is invisible at the CLI. +- Recommended downstream action: plan (surface the blueprint-data verbs in + `--help` and/or a README; tracked as #159). Until then, treat the ledger as the + only entry point — a real gap for anyone but the author. + +## Recommendation summary + +| Finding | Class | Action | +|---|---|---| +| Consumer-authored blueprint round-trips whole family surface | working | carry-on | +| All three FamilyKinds reproduce bit-identically | working | carry-on | +| list-axes→sweep composes; rank best-first | working | carry-on | +| `aura run` on open blueprint panics (exit 101) | bug | debug | +| closed→walkforward prints diagnostic twice | bug | debug | +| closed→sweep error terser than mc-side equivalent | friction | plan | +| mc reproduce lines omit the seed | friction | plan | +| every open knob mandatory on sweep, undocumented | spec_gap | ratify + tighten docs | +| blueprint-data verbs undiscoverable from `--help` / no README | spec_gap | plan (#159) | diff --git a/fieldtests/world-c21-milestone/run_all.sh b/fieldtests/world-c21-milestone/run_all.sh new file mode 100755 index 0000000..2d56f9a --- /dev/null +++ b/fieldtests/world-c21-milestone/run_all.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# Milestone fieldtest — World/C21: run + orchestrate harness FAMILIES from +# blueprint-data. Driven entirely from the public surface (design ledger C24/C18/ +# C21 + glossary + the documented op-script grammar). No source was read. +# +# Build first: cargo build --workspace (binary: target/debug/aura) +# Then: ./run_all.sh +set -u +BIN="${AURA_BIN:-../../target/debug/aura}" + +echo "====================================================================" +echo "Scenario 1 — discover -> sweep -> list -> rank (OPEN blueprint)" +echo "====================================================================" +$BIN graph build < w1_open_crossover.ops.json > w1_open_crossover.bp.json +$BIN sweep w1_open_crossover.bp.json --list-axes # discover the 3 open knobs +# NOTE: every open knob is MANDATORY; a subset -> MissingKnob (see w4_4f). +$BIN sweep w1_open_crossover.bp.json \ + --axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \ + --name w1_cross_sweep +$BIN runs families +$BIN runs family w1_cross_sweep-0 rank sqn_normalized +$BIN reproduce w1_cross_sweep-0 # 4/4 bit-identical + +echo "====================================================================" +echo "Scenario 2 — walk-forward -> reproduce (CAPSTONE, OPEN blueprint)" +echo "====================================================================" +$BIN walkforward w1_open_crossover.bp.json \ + --axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \ + --select argmax --name w2_wf +$BIN reproduce w2_wf-0 # 3/3 bit-identical + +echo "====================================================================" +echo "Scenario 3 — monte-carlo -> reproduce (CLOSED blueprint)" +echo "====================================================================" +$BIN graph build < w3_closed_crossover.ops.json > w3_closed_crossover.bp.json +$BIN sweep w3_closed_crossover.bp.json --list-axes # empty: fully bound +$BIN mc w3_closed_crossover.bp.json --seeds 5 --name w3_mc +$BIN reproduce w3_mc-0 # 5/5 bit-identical + +echo "====================================================================" +echo "Scenario 4 — cross-verb coherence + error actionability" +echo "====================================================================" +$BIN sweep w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4a closed->sweep +$BIN mc w1_open_crossover.bp.json --seeds 3 # 4b open->mc (exemplary err) +$BIN walkforward w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4c (dup diagnostic) +$BIN sweep w1_open_crossover.bp.json --axis graph.nonesuch.length=2 \ + --axis graph.slow.length=5 --axis graph.bias.scale=0.5 # 4d unknown axis +$BIN reproduce no-such-family-99 # 4e unknown family +$BIN run w1_open_crossover.bp.json # 4 BUG: PANICS (exit 101) diff --git a/fieldtests/world-c21-milestone/w1_families.out b/fieldtests/world-c21-milestone/w1_families.out new file mode 100644 index 0000000..3c82983 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_families.out @@ -0,0 +1 @@ +{"family_id":"w1_cross_sweep-0","kind":"Sweep","members":4} diff --git a/fieldtests/world-c21-milestone/w1_list_axes.err b/fieldtests/world-c21-milestone/w1_list_axes.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_list_axes.out b/fieldtests/world-c21-milestone/w1_list_axes.out new file mode 100644 index 0000000..bc61fb8 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_list_axes.out @@ -0,0 +1,3 @@ +graph.fast.length:I64 +graph.slow.length:I64 +graph.bias.scale:F64 diff --git a/fieldtests/world-c21-milestone/w1_open_build.err b/fieldtests/world-c21-milestone/w1_open_build.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_open_crossover.bp.json b/fieldtests/world-c21-milestone/w1_open_crossover.bp.json new file mode 100644 index 0000000..83ba539 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_open_crossover.bp.json @@ -0,0 +1 @@ +{"format_version":1,"blueprint":{"name":"graph","nodes":[{"primitive":{"type":"SMA","name":"fast"}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub","name":"sub"}},{"primitive":{"type":"Bias","name":"bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}} \ No newline at end of file diff --git a/fieldtests/world-c21-milestone/w1_open_crossover.ops.json b/fieldtests/world-c21-milestone/w1_open_crossover.ops.json new file mode 100644 index 0000000..ccbeb1a --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_open_crossover.ops.json @@ -0,0 +1,12 @@ +[ + {"op": "source", "role": "price", "kind": "F64"}, + {"op": "add", "type": "SMA", "name": "fast"}, + {"op": "add", "type": "SMA", "name": "slow"}, + {"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]}, + {"op": "add", "type": "Sub", "name": "sub"}, + {"op": "connect", "from": "fast.value", "to": "sub.lhs"}, + {"op": "connect", "from": "slow.value", "to": "sub.rhs"}, + {"op": "add", "type": "Bias", "name": "bias"}, + {"op": "connect", "from": "sub.value", "to": "bias.signal"}, + {"op": "expose", "from": "bias.bias", "as": "bias"} +] diff --git a/fieldtests/world-c21-milestone/w1_rank.err b/fieldtests/world-c21-milestone/w1_rank.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_rank.out b/fieldtests/world-c21-milestone/w1_rank.out new file mode 100644 index 0000000..99ebd76 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_rank.out @@ -0,0 +1,4 @@ +{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":0.2146800000000105,"max_drawdown":0.19279999999999503,"bias_sign_flips":2,"r":{"expectancy_r":0.515750880540302,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":1.1439327691362082,"avg_loss_r":-0.7406128966515102,"profit_factor":3.089151631866538,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.6231985130629969,"sqn_normalized":0.6231985130629969,"net_expectancy_r":0.515750880540302,"conviction_terciles_r":[2.0771328641652427,0.21073267410717364,-0.7406128966515102]}}} +{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.0023866666666611386,"max_drawdown":0.208293333333331,"bias_sign_flips":2,"r":{"expectancy_r":0.03753199703966933,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":1.127161364109783,"avg_loss_r":-0.5072826864953875,"profit_factor":1.1109795328290115,"max_r_drawdown":0.2739524763392649,"n_open_at_end":1,"sqn":0.0668754310632922,"sqn_normalized":0.0668754310632922,"net_expectancy_r":0.03753199703966933,"conviction_terciles_r":[-0.2739524763392649,1.127161364109783,-0.7406128966515102]}}} +{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.20921999999999025,"max_drawdown":0.34265999999999847,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}} +{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.339953333333338,"max_drawdown":0.3845266666666764,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}} diff --git a/fieldtests/world-c21-milestone/w1_reproduce.err b/fieldtests/world-c21-milestone/w1_reproduce.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_reproduce.out b/fieldtests/world-c21-milestone/w1_reproduce.out new file mode 100644 index 0000000..97e76d8 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_reproduce.out @@ -0,0 +1,5 @@ +w1_cross_sweep-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical +w1_cross_sweep-0 member graph.fast.length=2, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical +w1_cross_sweep-0 member graph.fast.length=3, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical +w1_cross_sweep-0 member graph.fast.length=3, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical +reproduced 4/4 members bit-identically diff --git a/fieldtests/world-c21-milestone/w1_sweep.err b/fieldtests/world-c21-milestone/w1_sweep.err new file mode 100644 index 0000000..02d2372 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_sweep.err @@ -0,0 +1 @@ +aura: MissingKnob("graph.bias.scale") diff --git a/fieldtests/world-c21-milestone/w1_sweep.out b/fieldtests/world-c21-milestone/w1_sweep.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_sweep2.err b/fieldtests/world-c21-milestone/w1_sweep2.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w1_sweep2.out b/fieldtests/world-c21-milestone/w1_sweep2.out new file mode 100644 index 0000000..d6fb262 --- /dev/null +++ b/fieldtests/world-c21-milestone/w1_sweep2.out @@ -0,0 +1,4 @@ +{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":0.2146800000000105,"max_drawdown":0.19279999999999503,"bias_sign_flips":2,"r":{"expectancy_r":0.515750880540302,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":1.1439327691362082,"avg_loss_r":-0.7406128966515102,"profit_factor":3.089151631866538,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.6231985130629969,"sqn_normalized":0.6231985130629969,"net_expectancy_r":0.515750880540302,"conviction_terciles_r":[2.0771328641652427,0.21073267410717364,-0.7406128966515102]}}}} +{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.20921999999999025,"max_drawdown":0.34265999999999847,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}} +{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.0023866666666611386,"max_drawdown":0.208293333333331,"bias_sign_flips":2,"r":{"expectancy_r":0.03753199703966933,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":1.127161364109783,"avg_loss_r":-0.5072826864953875,"profit_factor":1.1109795328290115,"max_r_drawdown":0.2739524763392649,"n_open_at_end":1,"sqn":0.0668754310632922,"sqn_normalized":0.0668754310632922,"net_expectancy_r":0.03753199703966933,"conviction_terciles_r":[-0.2739524763392649,1.127161364109783,-0.7406128966515102]}}}} +{"family_id":"w1_cross_sweep-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":3}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.339953333333338,"max_drawdown":0.3845266666666764,"bias_sign_flips":1,"r":{"expectancy_r":-0.34908675226620095,"n_trades":2,"win_rate":0.5,"avg_win_r":0.5474739899790243,"avg_loss_r":-1.2456474945114262,"profit_factor":0.4395095662226312,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":-0.389362076452283,"sqn_normalized":-0.389362076452283,"net_expectancy_r":-0.34908675226620095,"conviction_terciles_r":[0.0,0.0,0.0]}}}} diff --git a/fieldtests/world-c21-milestone/w2_reproduce.err b/fieldtests/world-c21-milestone/w2_reproduce.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w2_reproduce.out b/fieldtests/world-c21-milestone/w2_reproduce.out new file mode 100644 index 0000000..e135410 --- /dev/null +++ b/fieldtests/world-c21-milestone/w2_reproduce.out @@ -0,0 +1,4 @@ +w2_wf-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical +w2_wf-0 member graph.fast.length=2, graph.slow.length=5, graph.bias.scale=0.5 reproduced: bit-identical +w2_wf-0 member graph.fast.length=2, graph.slow.length=10, graph.bias.scale=0.5 reproduced: bit-identical +reproduced 3/3 members bit-identically diff --git a/fieldtests/world-c21-milestone/w2_wf.err b/fieldtests/world-c21-milestone/w2_wf.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w2_wf.out b/fieldtests/world-c21-milestone/w2_wf.out new file mode 100644 index 0000000..4b33ffd --- /dev/null +++ b/fieldtests/world-c21-milestone/w2_wf.out @@ -0,0 +1,4 @@ +{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[25,36],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":0.8444293887137961,"mode":"Argmax","deflated_score":0.8444293887137961,"overfit_probability":0.000999000999000999,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.029682254132962535,"max_drawdown":0.0442303121080338,"bias_sign_flips":2,"r":{"expectancy_r":0.01050668114243935,"n_trades":3,"win_rate":0.6666666666666666,"avg_win_r":0.09881344876754977,"avg_loss_r":-0.1661068541077815,"profit_factor":1.1897576327998223,"max_r_drawdown":0.1661068541077815,"n_open_at_end":1,"sqn":0.11861590656751105,"sqn_normalized":0.11861590656751105,"net_expectancy_r":0.01050668114243935,"conviction_terciles_r":[0.11079586598682767,-0.1661068541077815,0.08683103154827189]}}}} +{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":5}],["graph.bias.scale",{"F64":0.5}]],"window":[37,48],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":-2.17811914776625,"mode":"Argmax","deflated_score":-2.2204163800380603,"overfit_probability":1.0,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.07698666016143554,"max_drawdown":0.07698666016143554,"bias_sign_flips":2,"r":{"expectancy_r":-0.3935692692179748,"n_trades":3,"win_rate":0.3333333333333333,"avg_win_r":0.38656029975355716,"avg_loss_r":-0.7836340537037407,"profit_factor":0.24664592989963363,"max_r_drawdown":0.7927601270548278,"n_open_at_end":1,"sqn":-1.0088923276440807,"sqn_normalized":-1.0088923276440807,"net_expectancy_r":-0.3935692692179748,"conviction_terciles_r":[0.38656029975355716,-0.7927601270548277,-0.7745079803526537]}}}} +{"family_id":"w2_wf-0","report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[["graph.fast.length",{"I64":2}],["graph.slow.length",{"I64":10}],["graph.bias.scale",{"F64":0.5}]],"window":[49,60],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","selection":{"selection_metric":"sqn_normalized","n_trials":4,"raw_winner_metric":-0.43277394607057,"mode":"Argmax","deflated_score":-1.0185052275567759,"overfit_probability":1.0,"n_resamples":1000,"block_len":5,"seed":3740379117},"topology_hash":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"},"metrics":{"total_pips":-0.02907021601207918,"max_drawdown":0.02907021601207918,"bias_sign_flips":0,"r":{"expectancy_r":-0.5505542307007089,"n_trades":1,"win_rate":0.0,"avg_win_r":0.0,"avg_loss_r":-0.5505542307007089,"profit_factor":-0.0,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":0.0,"sqn_normalized":0.0,"net_expectancy_r":-0.5505542307007089,"conviction_terciles_r":[0.0,0.0,0.0]}}}} +{"walkforward":{"oos_r":{"avg_loss_r":-0.5709822980539929,"avg_win_r":0.1947290657628856,"conviction_terciles_r":[0.0,0.0,0.0],"expectancy_r":-0.24282028498961644,"max_r_drawdown":1.8105378609141427,"n_open_at_end":0,"n_trades":7,"net_expectancy_r":-0.24282028498961644,"profit_factor":0.25578165876580955,"sqn":-1.3720083218957762,"sqn_normalized":-1.3720083218957762,"win_rate":0.42857142857142855},"param_stability":[{"mean":2.0,"p25":2.0,"p5":2.0,"p50":2.0,"p75":2.0,"p95":2.0},{"mean":6.666666666666667,"p25":5.0,"p5":5.0,"p50":5.0,"p75":7.5,"p95":9.5},{"mean":0.5,"p25":0.5,"p5":0.5,"p50":0.5,"p75":0.5,"p95":0.5}],"stitched_total_pips":0.0,"windows":3}} diff --git a/fieldtests/world-c21-milestone/w3_closed_build.err b/fieldtests/world-c21-milestone/w3_closed_build.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w3_closed_crossover.bp.json b/fieldtests/world-c21-milestone/w3_closed_crossover.bp.json new file mode 100644 index 0000000..7f87e4e --- /dev/null +++ b/fieldtests/world-c21-milestone/w3_closed_crossover.bp.json @@ -0,0 +1 @@ +{"format_version":1,"blueprint":{"name":"graph","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":4}}]}},{"primitive":{"type":"Sub","name":"sub"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":0.5}}]}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}} \ No newline at end of file diff --git a/fieldtests/world-c21-milestone/w3_closed_crossover.ops.json b/fieldtests/world-c21-milestone/w3_closed_crossover.ops.json new file mode 100644 index 0000000..d7732af --- /dev/null +++ b/fieldtests/world-c21-milestone/w3_closed_crossover.ops.json @@ -0,0 +1,12 @@ +[ + {"op": "source", "role": "price", "kind": "F64"}, + {"op": "add", "type": "SMA", "name": "fast", "bind": {"length": {"I64": 2}}}, + {"op": "add", "type": "SMA", "name": "slow", "bind": {"length": {"I64": 4}}}, + {"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]}, + {"op": "add", "type": "Sub", "name": "sub"}, + {"op": "connect", "from": "fast.value", "to": "sub.lhs"}, + {"op": "connect", "from": "slow.value", "to": "sub.rhs"}, + {"op": "add", "type": "Bias", "name": "bias", "bind": {"scale": {"F64": 0.5}}}, + {"op": "connect", "from": "sub.value", "to": "bias.signal"}, + {"op": "expose", "from": "bias.bias", "as": "bias"} +] diff --git a/fieldtests/world-c21-milestone/w3_mc.err b/fieldtests/world-c21-milestone/w3_mc.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w3_mc.out b/fieldtests/world-c21-milestone/w3_mc.out new file mode 100644 index 0000000..08107ea --- /dev/null +++ b/fieldtests/world-c21-milestone/w3_mc.out @@ -0,0 +1,6 @@ +{"family_id":"w3_mc-0","seed":1,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":1,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.08581949682071963,"max_drawdown":0.1782160409053034,"bias_sign_flips":16,"r":{"expectancy_r":-0.12761687270091654,"n_trades":17,"win_rate":0.29411764705882354,"avg_win_r":0.9422351195309215,"avg_loss_r":-0.5733885361308491,"profit_factor":0.684697969583447,"max_r_drawdown":5.084202573801676,"n_open_at_end":1,"sqn":-0.5709680662988649,"sqn_normalized":-0.5709680662988649,"net_expectancy_r":-0.12761687270091654,"conviction_terciles_r":[-0.5315147387933803,0.07491264421171377,0.006435165463506309]}}}} +{"family_id":"w3_mc-0","seed":2,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":2,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.09465506366012477,"max_drawdown":0.14566471662215458,"bias_sign_flips":21,"r":{"expectancy_r":-0.1955399496066683,"n_trades":23,"win_rate":0.2608695652173913,"avg_win_r":0.8069365813559868,"avg_loss_r":-0.5493551958287819,"profit_factor":0.5184280562437801,"max_r_drawdown":5.678578478310644,"n_open_at_end":1,"sqn":-1.256408534576408,"sqn_normalized":-1.256408534576408,"net_expectancy_r":-0.1955399496066683,"conviction_terciles_r":[-0.14554052889979213,-0.3686624230732036,-0.0661669692586496]}}}} +{"family_id":"w3_mc-0","seed":3,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":3,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.1458394558178947,"max_drawdown":0.18901416622571146,"bias_sign_flips":17,"r":{"expectancy_r":-0.19288571424888643,"n_trades":18,"win_rate":0.2777777777777778,"avg_win_r":0.6783848238535024,"avg_loss_r":-0.5279897673651899,"profit_factor":0.49417101631666377,"max_r_drawdown":5.505838374215854,"n_open_at_end":1,"sqn":-1.1554023132748366,"sqn_normalized":-1.1554023132748366,"net_expectancy_r":-0.19288571424888643,"conviction_terciles_r":[-0.10609643833726563,-0.3594422393970695,-0.11311846501232425]}}}} +{"family_id":"w3_mc-0","seed":4,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":4,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.007137445429146379,"max_drawdown":0.0743086741823305,"bias_sign_flips":15,"r":{"expectancy_r":-0.07823191112894709,"n_trades":16,"win_rate":0.3125,"avg_win_r":1.09899031269056,"avg_loss_r":-0.6133329219559956,"profit_factor":0.8144696515391392,"max_r_drawdown":3.291640293181073,"n_open_at_end":1,"sqn":-0.31198614113195705,"sqn_normalized":-0.31198614113195705,"net_expectancy_r":-0.07823191112894709,"conviction_terciles_r":[-0.03355532359322373,-0.09249635589718175,-0.10357536343518768]}}}} +{"family_id":"w3_mc-0","seed":5,"report":{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,60],"seed":5,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":-0.05516687451601755,"max_drawdown":0.10049029278117855,"bias_sign_flips":21,"r":{"expectancy_r":-0.13824742850239835,"n_trades":22,"win_rate":0.36363636363636365,"avg_win_r":0.7288739517681235,"avg_loss_r":-0.6337453600855538,"profit_factor":0.6572030775170173,"max_r_drawdown":3.882156009122663,"n_open_at_end":1,"sqn":-0.7376556894251807,"sqn_normalized":-0.7376556894251807,"net_expectancy_r":-0.13824742850239835,"conviction_terciles_r":[-0.23289767226966626,-0.013422680153997594,-0.16465012001088963]}}}} +{"mc_aggregate":{"bias_sign_flips":{"mean":18.0,"p25":16.0,"p5":15.2,"p50":17.0,"p75":21.0,"p95":21.0},"max_drawdown":{"mean":0.1375387781433357,"p25":0.10049029278117855,"p5":0.07954499790210011,"p50":0.14566471662215458,"p75":0.1782160409053034,"p95":0.18685454116162986},"total_pips":{"mean":-0.07772366724878062,"p25":-0.09465506366012477,"p5":-0.1356025773863407,"p50":-0.08581949682071963,"p75":-0.05516687451601755,"p95":-0.016743331246520624}}} diff --git a/fieldtests/world-c21-milestone/w3_reproduce.err b/fieldtests/world-c21-milestone/w3_reproduce.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w3_reproduce.out b/fieldtests/world-c21-milestone/w3_reproduce.out new file mode 100644 index 0000000..557a476 --- /dev/null +++ b/fieldtests/world-c21-milestone/w3_reproduce.out @@ -0,0 +1,6 @@ +w3_mc-0 member reproduced: bit-identical +w3_mc-0 member reproduced: bit-identical +w3_mc-0 member reproduced: bit-identical +w3_mc-0 member reproduced: bit-identical +w3_mc-0 member reproduced: bit-identical +reproduced 5/5 members bit-identically diff --git a/fieldtests/world-c21-milestone/w3_run.err b/fieldtests/world-c21-milestone/w3_run.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w3_run.out b/fieldtests/world-c21-milestone/w3_run.out new file mode 100644 index 0000000..ab76a0d --- /dev/null +++ b/fieldtests/world-c21-milestone/w3_run.out @@ -0,0 +1 @@ +{"manifest":{"commit":"4710fccf320e1f1fb4811c061b1ea0390d8b17ba","params":[],"window":[1,18],"seed":0,"broker":"sim-optimal+risk-executor(pip_size=0.0001)","topology_hash":"c485dfbd2a24f896f97dd05750c1237843dc324400d350dbe8a5de8447a67d39"},"metrics":{"total_pips":0.34185000000002036,"max_drawdown":0.11139999999998655,"bias_sign_flips":2,"r":{"expectancy_r":1.2710005136982836,"n_trades":3,"win_rate":1.0,"avg_win_r":1.2710005136982836,"avg_loss_r":0.0,"profit_factor":0.0,"max_r_drawdown":0.0,"n_open_at_end":1,"sqn":3.141496526818299,"sqn_normalized":3.141496526818299,"net_expectancy_r":1.2710005136982836,"conviction_terciles_r":[0.9285858482198718,2.0771328641652427,0.8072828287097363]}}} diff --git a/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.err b/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.err new file mode 100644 index 0000000..650c6e1 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.err @@ -0,0 +1 @@ +aura: UnknownKnob("graph.fast.length") diff --git a/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.exit b/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.out b/fieldtests/world-c21-milestone/w4_4a_closed_to_sweep.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_4b_open_to_mc.err b/fieldtests/world-c21-milestone/w4_4b_open_to_mc.err new file mode 100644 index 0000000..17deaed --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4b_open_to_mc.err @@ -0,0 +1 @@ +aura: mc requires a closed blueprint (no free parameters); 3 free knob(s) — bind them or use `aura sweep --axis` diff --git a/fieldtests/world-c21-milestone/w4_4b_open_to_mc.exit b/fieldtests/world-c21-milestone/w4_4b_open_to_mc.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4b_open_to_mc.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4b_open_to_mc.out b/fieldtests/world-c21-milestone/w4_4b_open_to_mc.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.err b/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.err new file mode 100644 index 0000000..8a9f15b --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.err @@ -0,0 +1,2 @@ +aura: UnknownKnob("graph.fast.length") +aura: UnknownKnob("graph.fast.length \ No newline at end of file diff --git a/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.exit b/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.out b/fieldtests/world-c21-milestone/w4_4c_closed_to_wf.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_4d_unknown_axis.err b/fieldtests/world-c21-milestone/w4_4d_unknown_axis.err new file mode 100644 index 0000000..92adde8 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4d_unknown_axis.err @@ -0,0 +1 @@ +aura: UnknownKnob("graph.nonesuch.length") diff --git a/fieldtests/world-c21-milestone/w4_4d_unknown_axis.exit b/fieldtests/world-c21-milestone/w4_4d_unknown_axis.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4d_unknown_axis.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4d_unknown_axis.out b/fieldtests/world-c21-milestone/w4_4d_unknown_axis.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.err b/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.err new file mode 100644 index 0000000..6c329d7 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.err @@ -0,0 +1 @@ +aura: no such family 'no-such-family-99' diff --git a/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.exit b/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.out b/fieldtests/world-c21-milestone/w4_4e_reproduce_unknown.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.err b/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.err new file mode 100644 index 0000000..02d2372 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.err @@ -0,0 +1 @@ +aura: MissingKnob("graph.bias.scale") diff --git a/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.exit b/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.exit new file mode 100644 index 0000000..0cfbf08 --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.exit @@ -0,0 +1 @@ +2 diff --git a/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.out b/fieldtests/world-c21-milestone/w4_4f_missingknob_subset.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_run_open.err b/fieldtests/world-c21-milestone/w4_run_open.err new file mode 100644 index 0000000..dcbde7d --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_run_open.err @@ -0,0 +1,4 @@ + +thread 'main' (1296536) panicked at crates/aura-cli/src/main.rs:3107:10: +signal binds + wraps to a valid harness: ParamArity { expected: 3, got: 0 } +note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace diff --git a/fieldtests/world-c21-milestone/w4_run_open.exit b/fieldtests/world-c21-milestone/w4_run_open.exit new file mode 100644 index 0000000..a15277f --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_run_open.exit @@ -0,0 +1 @@ +run-open EXIT 101 diff --git a/fieldtests/world-c21-milestone/w4_run_open.out b/fieldtests/world-c21-milestone/w4_run_open.out new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_vocab.err b/fieldtests/world-c21-milestone/w4_vocab.err new file mode 100644 index 0000000..e69de29 diff --git a/fieldtests/world-c21-milestone/w4_vocab.out b/fieldtests/world-c21-milestone/w4_vocab.out new file mode 100644 index 0000000..c63d6ae --- /dev/null +++ b/fieldtests/world-c21-milestone/w4_vocab.out @@ -0,0 +1,22 @@ +Add +And +Bias +CarryCost +ConstantCost +Delay +EMA +EqConst +FixedStop +Gt +Latch +LongOnly +Mul +PositionManagement +Resample +RollingMax +RollingMin +Sizer +SMA +Sqrt +Sub +VolSlippageCost