fieldtest: milestone-42 A/B — promise holds, cold start 9x slower to first family
Milestone-scope consumer run on the release binary at 2f1bace, three
scenarios: cold start (help-only) reaches a first single run in 5
commands and a first selecting 4-cell family in 28 with 5 dead ends;
the five retired verbs refuse without re-routing; the full retired
research scope (grid, walk-forward, monte-carlo bootstrap, generalize)
runs document-first over two years of GER40+USDJPY in 2.4s with
bit-identical reproduce (21/21). A/B against the sugar-era cycle-300
transcript: comparable to first run, ~9x slower to first family,
strictly better artifact — the dominant cost is an introspect --unwired
bug on present-but-empty presentation (fixed in-cycle, follow-up
commit) plus the missing pre-run coverage view (evidence on the
tracker). 4 bugs / 6 friction / 3 spec-gap / 4 working, routed on the
tracker.
refs #319
This commit is contained in:
@@ -0,0 +1,280 @@
|
||||
# Fieldtest transcript — milestone 42 (sugar retirement, document-first authoring)
|
||||
|
||||
Binary: `target/release/aura` built from the working tree at HEAD `2f1bace`
|
||||
(`cargo build --release -p aura-cli`, exit 0). Every command below was issued
|
||||
through that binary; no implementation source was read.
|
||||
|
||||
## Persona
|
||||
|
||||
A downstream consumer with no prior aura knowledge, holding only the CLI's own
|
||||
help, the scaffolded project's `CLAUDE.md`, and (from scenario 2 on) the public
|
||||
README / authoring guide. Scenario 2 additionally carries sugar-era muscle
|
||||
memory: `run` / `sweep` / `walkforward` / `mc` / `generalize`.
|
||||
|
||||
## Files
|
||||
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `coldstart-lab/` | scenario 1 — scaffolded by `aura new`, then grown to a 4-cell selecting sweep |
|
||||
| `coldstart-lab/documents/cold-process.json` | the process document authored from the `--unwired` ladder |
|
||||
| `coldstart-lab/documents/cold-campaign.json` | the campaign document authored from the `--unwired` ladder |
|
||||
| `spine-lab/` | scenario 3 — scaffolded, plus an op-script-built tapped blueprint |
|
||||
| `spine-lab/blueprints/m42_trend.ops.json` | the op-script (canonical authoring form) |
|
||||
| `spine-lab/blueprints/m42_trend.json` | `aura graph build` output, registered as label `m42_trend` |
|
||||
| `spine-lab/documents/spine-process.json` | grid → walk_forward → monte_carlo → generalize |
|
||||
| `spine-lab/documents/spine-campaign.json` | GER40 + USDJPY, 2 years, 6 axis points |
|
||||
| `logs/m42_1_unwired_ladder.sh` + `.txt` | replayable repro of the `--unwired` blind spot |
|
||||
| `logs/m42_2_retired_verbs.txt` | the five retired verbs + two retired flags, verbatim |
|
||||
| `logs/m42_3_spine.txt` | the spine run, reproduce, chart, tap refusals |
|
||||
| `logs/m42_3b_override_family.txt` | the whole `--override` refusal family, both legs |
|
||||
|
||||
`runs/` under each lab is gitignored by the scaffolder.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 1 — COLD START (help + scaffolded docs only)
|
||||
|
||||
### To a first successful single run — 4 commands, 0 dead ends
|
||||
|
||||
```
|
||||
1 $ aura --help
|
||||
→ about-text names the two layers, `exec` as the one executor, the
|
||||
document data plane (`aura process` / `aura campaign`), the
|
||||
`introspect --unwired` growth idiom, and `graph introspect --params`
|
||||
2 $ aura new --help
|
||||
3 $ aura new coldstart-lab
|
||||
created project "coldstart-lab" (data-only; attach native nodes later …)
|
||||
4 $ cat coldstart-lab/CLAUDE.md (scaffolded; 5 bullets, all correct)
|
||||
5 $ aura exec blueprints/signal.json
|
||||
{"manifest":{…,"topology_hash":"0d3c6940…"},"metrics":{…,"expectancy_r":1.271…}}
|
||||
EXIT=0
|
||||
```
|
||||
|
||||
Clean. The scaffolded `CLAUDE.md`'s first bullet is literally the command that
|
||||
works.
|
||||
|
||||
### To a first multi-cell family with a selection result — 24 commands, 5 dead ends
|
||||
|
||||
Discovery (7 commands, all productive):
|
||||
|
||||
```
|
||||
6 $ aura campaign --help
|
||||
7 $ echo '{}' > bare.json; aura campaign introspect --unwired bare.json
|
||||
8 $ aura process introspect --unwired bare.json
|
||||
9 $ aura process introspect --help
|
||||
10 $ aura process introspect --vocabulary → the 6 stage blocks + meanings
|
||||
11 $ aura process introspect --block std::sweep → metric | select | deflate
|
||||
12 $ aura graph introspect --params blueprints/signal.json
|
||||
fast.length:I64 default=2 / slow.length:I64 default=4 / bias.scale:F64 default=0.5
|
||||
13 $ aura data list → 30+ symbols incl. GER40
|
||||
14 $ aura data info GER40
|
||||
{"symbol":"GER40","digits":1,"pipSize":1.0,…} ← geometry only, no time coverage
|
||||
15 $ aura graph register blueprints/signal.json → 0d3c6940…
|
||||
16 $ aura process introspect --metrics → rankable | gate | generalize tags
|
||||
```
|
||||
|
||||
Growing the documents, with the refusals verbatim:
|
||||
|
||||
```
|
||||
17 $ aura process validate documents/cold-process.json # pipeline: [{"std::sweep":{…}}]
|
||||
aura: process document: a pipeline stage must carry a string "block" id EXIT=1
|
||||
← DEAD END 1. Neither `--unwired` ("pipeline (required, list of stage
|
||||
blocks)") nor `--block std::sweep` shows a stage's literal JSON shape.
|
||||
The refusal itself is precise and fixed it in one edit.
|
||||
|
||||
18 $ aura process validate (pipeline: [{"block":"std::sweep",…}]) EXIT=0
|
||||
19 $ aura process register documents/cold-process.json → 26079ba0…
|
||||
|
||||
20 $ aura campaign validate documents/cold-campaign.json # presentation: {}
|
||||
aura: campaign document: missing field `emit` EXIT=1
|
||||
← DEAD END 2. `campaign introspect --unwired` on this same file had
|
||||
reported no open slot for presentation at all (see logs/m42_1_*).
|
||||
|
||||
21 $ … "emit": "summary"
|
||||
aura: campaign document: invalid type: string "summary", expected a sequence EXIT=1
|
||||
← DEAD END 3 (the refusal names no legal value)
|
||||
|
||||
22 $ … "emit": []
|
||||
aura: campaign document: unknown variant `list`, expected one of `I64`, `F64`, `Bool`, `Timestamp`
|
||||
← DEAD END 4 — the axes `kind` slot; `--unwired` says only
|
||||
"axes (required, map: param name -> { kind, values })". EXIT=1
|
||||
|
||||
23 $ … axes kind → "I64", emit → ["nonsense"]
|
||||
aura: campaign document invalid:
|
||||
presentation.emit: unknown kind "nonsense" EXIT=1
|
||||
← DEAD END 5 (still no enumeration of the legal kinds)
|
||||
|
||||
24 $ aura campaign introspect --vocabulary → std::presentation exists
|
||||
25 $ aura campaign introspect --block std::presentation
|
||||
persist_taps required, list of: equity | exposure | r_equity | net_r_equity
|
||||
emit required, list of: family_table | selection_report ← the answer
|
||||
26 $ aura campaign validate documents/cold-campaign.json
|
||||
campaign document valid (intrinsic): 1 strategy(ies), 2 axes (4 points), … 1 cell(s)
|
||||
campaign document valid (referential): all references resolve, axes are in the param space
|
||||
campaign document valid (executable): pipeline shape and static guards pass EXIT=0
|
||||
27 $ aura campaign register documents/cold-campaign.json → dbf47283…
|
||||
28 $ aura exec documents/cold-campaign.json
|
||||
4 × {"family_id":"dbf47283-0-GER40-w0-r0-s0-0","report":{…}}
|
||||
{"selection_report":{…,"winner_ordinal":2,"params":[["fast.length",{"I64":3}],
|
||||
["slow.length",{"I64":8}]],"selection":{"selection_metric":"expectancy_r",
|
||||
"n_trials":4,"raw_winner_metric":0.008618…,"mode":"Argmax"}}}
|
||||
{"campaign_run":{…,"coverage":{"effective_from_ms":1704154499999,
|
||||
"effective_to_ms":1711659420000}}}
|
||||
aura: campaign run 0 recorded: 1 cells EXIT=0
|
||||
```
|
||||
|
||||
The guessed `from_ms`/`to_ms` (2024-01-01 … 2024-04-01) clipped silently and
|
||||
honestly to the archive's real coverage, which the `campaign_run` record then
|
||||
reports back as `coverage.effective_*`. There is no *pre*-run way to learn a
|
||||
symbol's time coverage: `aura data info` returns geometry only.
|
||||
|
||||
**Cold start total: 28 commands, 5 dead ends, all resolved from the surface.**
|
||||
|
||||
---
|
||||
|
||||
## Scenario 2 — RETURNING SUGAR USER
|
||||
|
||||
Full log: `logs/m42_2_retired_verbs.txt`. Condensed:
|
||||
|
||||
```
|
||||
$ aura run blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'run'
|
||||
tip: a similar subcommand exists: 'runs' EXIT=2
|
||||
|
||||
$ aura runs blueprints/m42_trend.json # following the only tip offered
|
||||
error: unrecognized subcommand 'blueprints/m42_trend.json' EXIT=2
|
||||
|
||||
$ aura sweep / walkforward / mc / generalize <bp>
|
||||
error: unrecognized subcommand '<verb>' # no tip, no retirement notice EXIT=2
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --sweep fast.length=2,3
|
||||
error: unexpected argument '--sweep' found
|
||||
tip: to pass '--sweep' as a value, use '-- --sweep' EXIT=2
|
||||
|
||||
$ aura graph introspect --list-axes blueprints/m42_trend.json
|
||||
error: unexpected argument '--list-axes' found EXIT=2
|
||||
```
|
||||
|
||||
Not one of the seven refusals names `exec`, a campaign document, or the fact
|
||||
that the verb was retired. Two of them actively point the wrong way (`runs`;
|
||||
`-- --sweep`). Re-routing costs exactly one extra command — `aura --help`,
|
||||
whose about-text is genuinely excellent — so the honest cost is **2 commands
|
||||
to re-orient, 0 commands of it attributable to the refusal**.
|
||||
|
||||
---
|
||||
|
||||
## Scenario 3 — VALIDATION SPINE (real data, document-first)
|
||||
|
||||
Full log: `logs/m42_3_spine.txt`; override family: `logs/m42_3b_override_family.txt`.
|
||||
|
||||
Authoring a tapped blueprint in the canonical op-script form worked first try:
|
||||
|
||||
```
|
||||
$ aura graph build --help # embeds the full op-list reference, incl. {"op":"tap",…}
|
||||
$ aura graph introspect --vocabulary ; --node SMA|Sub|Sign|Bias
|
||||
$ aura graph build < blueprints/m42_trend.ops.json > blueprints/m42_trend.json EXIT=0
|
||||
$ aura graph register blueprints/m42_trend.json --name m42_trend
|
||||
registered blueprint b657c714… ; label "m42_trend" -> b657c714…
|
||||
$ aura graph introspect --registered
|
||||
m42_trend b657c714… fast/slow SMA crossover direction, clamped into the bias contract
|
||||
```
|
||||
|
||||
`aura graph build` is reachable only from `aura graph --help` / the README —
|
||||
the scaffolded project `CLAUDE.md` never mentions how to author a *new*
|
||||
blueprint.
|
||||
|
||||
The methodology, as a process document (one dead end):
|
||||
|
||||
```
|
||||
$ aura campaign validate documents/spine-campaign.json # pipeline had a std::gate after walk_forward
|
||||
aura: campaign is not executable:
|
||||
process pipeline is not executable: stage 2: std::gate cannot follow std::walk_forward EXIT=1
|
||||
```
|
||||
|
||||
Precise (names the stage index and the illegal adjacency); it does not say what
|
||||
*may* follow. Dropping the gate:
|
||||
|
||||
```
|
||||
$ aura campaign validate documents/spine-campaign.json # 3 tiers green EXIT=0
|
||||
$ aura campaign register documents/spine-campaign.json → 0959eda8…
|
||||
$ aura exec documents/spine-campaign.json EXIT=0
|
||||
(42 member reports on stdout; 2.4 s wall for 2 instruments × 2 years × 21 WF windows)
|
||||
aura: campaign run 0 recorded: 2 cells
|
||||
aura: note: tap "net_r_equity" needs a cost model; add a cost block to the campaign document; skipped
|
||||
aura: traces persisted: 0959eda8-0 (1 tap(s) x 2 cell(s))
|
||||
```
|
||||
|
||||
The `campaign_run` record carries the whole retired research scope:
|
||||
|
||||
```
|
||||
cell GER40 stages [std::grid, std::walk_forward, std::monte_carlo]
|
||||
pooled_oos mean=-0.004729 p5=-0.0469 p95=0.0381 prob_le_zero=0.59 n=21910
|
||||
cell USDJPY stages [std::grid, std::walk_forward, std::monte_carlo]
|
||||
pooled_oos mean=-0.033809 p5=-0.0766 p95=0.0177 prob_le_zero=0.885 n=25258
|
||||
generalize worst_case=-0.251491 sign_agreement=0
|
||||
per_instrument=[GER40 -0.2515, USDJPY -0.0935] + per-instrument winners
|
||||
```
|
||||
|
||||
`aura runs family <id> rank expectancy_r` shows every walk-forward window with
|
||||
`deflated_score` / `overfit_probability` alongside; `aura reproduce
|
||||
0959eda8-0-GER40-w0-r0-s1-0` reports all 21 members **bit-identical**.
|
||||
|
||||
`presentation.emit` asked for `family_table` **and** `selection_report`; the run
|
||||
emitted 42 family-table lines and **zero** selection-report lines, with no note
|
||||
— contrast the `net_r_equity` tap, which announced its own skip.
|
||||
|
||||
### The `--override` what-if
|
||||
|
||||
```
|
||||
$ aura exec documents/spine-campaign.json --override bias.scale=0.25 EXIT=0
|
||||
manifest.params gains ["bias.scale",{"F64":0.25}]; manifest.defaults empties;
|
||||
topology_hash stays b657c714… (the registered blueprint's id)
|
||||
every R metric bit-identical to the baseline — correct: research runs flat-1R,
|
||||
only the bias *sign* gates firing (invariant 7).
|
||||
```
|
||||
|
||||
Same operation on the blueprint leg does **not** keep the id:
|
||||
|
||||
```
|
||||
$ aura exec blueprints/m42_trend.json → b657c714…
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=10 → f952a1c8…
|
||||
^ a NO-OP override: 10 is the bound default
|
||||
$ aura reproduce f952a1c8…
|
||||
aura: no such family 'f952a1c8519a2dbd…'
|
||||
```
|
||||
|
||||
### The `--override` refusal family (blueprint leg)
|
||||
|
||||
```
|
||||
--override fast.length EXIT=2 aura: exec: --override expects NODE.PARAM=VALUE, got `fast.length`
|
||||
--override fast.length=notanumber EXIT=2 (same, good)
|
||||
--override nosuch.param=1 EXIT=1 aura: axis nosuch.param: names no param of this blueprint (open or bound) — see `aura graph introspect --params <bp>`
|
||||
--override fast.length=0 EXIT=1 aura: SMA length must be >= 1 ← the #319 fix, working
|
||||
--override fast.length=-1 EXIT=1 aura: capacity overflow ← the same fix, not working
|
||||
--override fast.length=-5 EXIT=1 aura: capacity overflow
|
||||
--override bias.scale=2 EXIT=1 aura: this blueprint does not compile to a runnable harness: ParamKindMismatch { slot: 0, expected: F64, got: I64 }
|
||||
--override bias.scale=2.0 EXIT=0
|
||||
--override bias.scale=-5.0 EXIT=1 aura: Bias scale must be > 0
|
||||
```
|
||||
|
||||
and on the campaign leg for the same inputs:
|
||||
|
||||
```
|
||||
--override fast.length=7 EXIT=2 aura: exec: --override `fast.length` collides with a declared axis of the campaign; an override overrides a bound value, never an axis
|
||||
--override stop_k=1.0 EXIT=1 aura: campaign references do not resolve:
|
||||
strategy b657c714…: axis "stop_k" is not in the param space
|
||||
--override bias.scale=-5 EXIT=1 aura: campaign references do not resolve:
|
||||
strategy b657c714…: axis "bias.scale" declares a kind that is not the param's kind
|
||||
```
|
||||
|
||||
### Traces
|
||||
|
||||
```
|
||||
$ aura chart 0959eda8-0 EXIT=1 aura: no family member has a tap named 'equity'
|
||||
$ aura chart 0959eda8-0 --tap r_equity EXIT=0 (298 KB self-contained HTML on stdout)
|
||||
$ aura chart b657c714-GER40-w0 EXIT=1 aura: no recorded run or family 'b657c714-GER40-w0'
|
||||
under runs/traces (check the handle a sweep/walk-forward/campaign run printed for a
|
||||
typo — re-running with this handle as `--trace` will not create it)
|
||||
$ aura exec blueprints/signal.json --tap equity=record
|
||||
EXIT=1 aura: the tap plan names 'equity', but the blueprint declares no such tap — declared taps:
|
||||
```
|
||||
@@ -0,0 +1 @@
|
||||
/runs
|
||||
@@ -0,0 +1,4 @@
|
||||
# Static project context only (C17); paths only.
|
||||
[paths]
|
||||
runs = "runs"
|
||||
# data = "/path/to/archive" # the recorded-data root; defaults to the built-in path
|
||||
@@ -0,0 +1,24 @@
|
||||
# coldstart-lab — an aura research project (data-only)
|
||||
|
||||
This directory is an aura project: blueprints + research documents over the
|
||||
std vocabulary, anchored by `Aura.toml`. There is no crate and no build step.
|
||||
|
||||
- Run: `aura exec blueprints/signal.json` (single smoke run, synthetic stream
|
||||
— the starter is closed; all params bound; bound values are defaults —
|
||||
`--override NODE.PARAM=VALUE` may override one for this run (#246))
|
||||
- Campaign: `aura exec <campaign.json>` executes a registered campaign
|
||||
document (file or content id) — the multi-cell/axis surface
|
||||
- Axes: `aura graph introspect --params blueprints/signal.json` lists the
|
||||
open + bound-overridable axes, RAW `<node>.<param>` names (#328)
|
||||
- Native nodes: when the project needs its first project-specific node,
|
||||
`aura nodes new <name>` scaffolds a node crate beside this project and
|
||||
attaches it via `[nodes]` in `Aura.toml` (build it with `cargo build`).
|
||||
- Topology is data (`blueprints/*.json`); results land in `runs/`.
|
||||
- Execution model: a strategy emits a bias in [-1,+1] per cycle, held as the
|
||||
continuously-tracked target position; a protective stop defines the risk
|
||||
unit R, and quality metrics are R-based. Entry signals become held state
|
||||
via the signal-side latch/edge-pulse idiom (see
|
||||
`aura graph introspect --vocabulary`).
|
||||
- Data plane: process/campaign documents are authored directly with
|
||||
`aura process` / `aura campaign`, growing one from a bare `{}` via
|
||||
`aura campaign introspect --unwired`.
|
||||
@@ -0,0 +1 @@
|
||||
{"format_version":1,"blueprint":{"name":"coldstart_lab_signal","doc":"fast/slow SMA difference clamped into a directional bias","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":4}}]}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":0.5}}]}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}}
|
||||
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"kind": "campaign",
|
||||
"name": "cold-start-sweep",
|
||||
"description": "Cold-start 4-cell sweep over the starter blueprint's two lengths on GER40.",
|
||||
"data": {
|
||||
"instruments": [
|
||||
"GER40"
|
||||
],
|
||||
"windows": [
|
||||
{
|
||||
"from_ms": 1704067200000,
|
||||
"to_ms": 1711929600000
|
||||
}
|
||||
]
|
||||
},
|
||||
"strategies": [
|
||||
{
|
||||
"ref": {
|
||||
"content_id": "0d3c69401c8e6c9e56661fce348a4c288325a39b7167d7794b30b5141930ebf1"
|
||||
},
|
||||
"axes": {
|
||||
"fast.length": {
|
||||
"kind": "I64",
|
||||
"values": [
|
||||
2,
|
||||
3
|
||||
]
|
||||
},
|
||||
"slow.length": {
|
||||
"kind": "I64",
|
||||
"values": [
|
||||
8,
|
||||
16
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"process": {
|
||||
"ref": {
|
||||
"content_id": "26079ba0317aba8f07740f0fb0616ffed683467618925a3b483ba01d06f74a1f"
|
||||
}
|
||||
},
|
||||
"seed": 0,
|
||||
"presentation": {
|
||||
"persist_taps": [],
|
||||
"emit": [
|
||||
"family_table",
|
||||
"selection_report"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"kind": "process",
|
||||
"name": "cold-start-select",
|
||||
"description": "Sweep the axes space and pick the argmax expectancy_r member.",
|
||||
"pipeline": [
|
||||
{ "block": "std::sweep", "metric": "expectancy_r", "select": "argmax" }
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/bin/sh
|
||||
# m42_1 — the --unwired ladder's presentation blind spot, minimal replay.
|
||||
# Run from inside any scaffolded project. AURA=<path to the release binary>.
|
||||
set -u
|
||||
: "${AURA:=aura}"
|
||||
t=$(mktemp -d)
|
||||
|
||||
echo '{}' > "$t/a.json"
|
||||
echo '$ aura campaign introspect --unwired a.json # a.json = {}'
|
||||
"$AURA" campaign introspect --unwired "$t/a.json" | grep -E "^open slot: (data|presentation)"
|
||||
|
||||
cat > "$t/b.json" <<'J'
|
||||
{"format_version":1,"kind":"campaign","name":"x","seed":0,
|
||||
"data":{"instruments":["GER40"],"windows":[{"from_ms":1,"to_ms":2}]},
|
||||
"strategies":[{"ref":{"content_id":"0"},"axes":{}}],
|
||||
"process":{"ref":{"content_id":"0"}},
|
||||
"presentation":{}}
|
||||
J
|
||||
echo
|
||||
echo '$ aura campaign introspect --unwired b.json # data filled, presentation left {}'
|
||||
"$AURA" campaign introspect --unwired "$t/b.json" | grep -E "^open slot:"
|
||||
echo ' ^ only OPTIONAL slots are listed. The two REQUIRED sub-slots of an empty
|
||||
presentation section (persist_taps, emit) are never named — contrast an empty
|
||||
data section, which drills down to instruments/windows (see above).'
|
||||
|
||||
echo
|
||||
echo '$ aura campaign validate b.json # the slot the ladder never named'
|
||||
"$AURA" campaign validate "$t/b.json" 2>&1 | head -2
|
||||
|
||||
echo
|
||||
echo '$ aura campaign introspect --block std::presentation # where the answer actually lives'
|
||||
"$AURA" campaign introspect --block std::presentation
|
||||
rm -rf "$t"
|
||||
@@ -0,0 +1,23 @@
|
||||
$ aura campaign introspect --unwired a.json # a.json = {}
|
||||
open slot: data (required section: instruments + windows)
|
||||
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
|
||||
|
||||
$ aura campaign introspect --unwired b.json # data filled, presentation left {}
|
||||
open slot: description (optional, string — a one-line meaning (C29-gated when present))
|
||||
open slot: risk (optional, list of stop regimes { vol: { length, k } } | { vol_tf: { period_minutes, length, k } }; absent = one default regime)
|
||||
open slot: cost (optional, list of cost components { constant: { cost_per_trade } } | { vol_slippage: { slip_vol_mult } } | { carry: { carry_per_cycle } }; absent = zero costs (net == gross))
|
||||
^ only OPTIONAL slots are listed. The two REQUIRED sub-slots of an empty
|
||||
presentation section (persist_taps, emit) are never named — contrast an empty
|
||||
data section, which drills down to instruments/windows (see above).
|
||||
|
||||
$ aura campaign validate b.json # the slot the ladder never named
|
||||
aura: campaign document: missing field `persist_taps`
|
||||
|
||||
$ aura campaign introspect --block std::presentation # where the answer actually lives
|
||||
std::presentation — campaign section: taps to persist + tables to emit (data-level only)
|
||||
persist_taps required, list of: equity | exposure | r_equity | net_r_equity
|
||||
equity cumulative pip equity per cycle
|
||||
exposure signed position exposure per cycle
|
||||
r_equity cumulative gross R per cycle
|
||||
net_r_equity cumulative net R per cycle, after the cost model
|
||||
emit required, list of: family_table | selection_report
|
||||
@@ -0,0 +1,69 @@
|
||||
# m42_2 — returning sugar user: the five retired verbs, verbatim (release binary @ 2f1bace)
|
||||
|
||||
$ aura run blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'run'
|
||||
|
||||
tip: a similar subcommand exists: 'runs'
|
||||
|
||||
Usage: aura [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura sweep blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'sweep'
|
||||
|
||||
Usage: aura [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura walkforward blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'walkforward'
|
||||
|
||||
Usage: aura [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura mc blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'mc'
|
||||
|
||||
Usage: aura [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura generalize blueprints/m42_trend.json
|
||||
error: unrecognized subcommand 'generalize'
|
||||
|
||||
Usage: aura [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura runs blueprints/m42_trend.json # following the only tip offered
|
||||
error: unrecognized subcommand 'blueprints/m42_trend.json'
|
||||
|
||||
Usage: aura runs [OPTIONS] <COMMAND>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --sweep fast.length=2,3 # sugar-era flag muscle memory
|
||||
error: unexpected argument '--sweep' found
|
||||
|
||||
tip: to pass '--sweep' as a value, use '-- --sweep'
|
||||
|
||||
Usage: aura exec <TARGET>
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
|
||||
$ aura graph introspect --list-axes blueprints/m42_trend.json # retired discovery flag
|
||||
error: unexpected argument '--list-axes' found
|
||||
|
||||
Usage: aura graph introspect [OPTIONS]
|
||||
|
||||
For more information, try '--help'.
|
||||
EXIT=2
|
||||
@@ -0,0 +1,48 @@
|
||||
# m42_3 — the validation spine document-first, GER40 + USDJPY, 2023-01-01..2024-12-31
|
||||
# process pipeline: std::grid -> std::walk_forward(90/30/30 rolling, argmax expectancy_r) -> std::monte_carlo(200x5) -> std::generalize
|
||||
|
||||
$ aura campaign validate documents/spine-campaign.json
|
||||
campaign document valid (intrinsic): 1 strategy(ies), 2 axes (6 points), 2 instrument(s), 1 window(s), 1 regime(s) (default) — 2 cell(s)
|
||||
campaign document valid (referential): all references resolve, axes are in the param space
|
||||
campaign document valid (executable): pipeline shape and static guards pass
|
||||
EXIT=0
|
||||
|
||||
$ aura exec documents/spine-campaign.json # stdout: 42 member reports + campaign_run; stderr below
|
||||
EXIT=0
|
||||
aura: campaign run 2 recorded: 2 cells
|
||||
aura: note: tap "net_r_equity" needs a cost model; add a cost block to the campaign document; skipped
|
||||
aura: traces persisted: 0959eda8-2 (1 tap(s) x 2 cell(s))
|
||||
stdout lines: 43
|
||||
|
||||
# campaign_run record, trimmed to the validation payload:
|
||||
cell GER40 stages ['std::grid', 'std::walk_forward', 'std::monte_carlo']
|
||||
monte_carlo pooled_oos: mean=-0.004729 p5=-0.0469 p95=0.0381 prob_le_zero=0.59 n_trades=21910
|
||||
coverage {'effective_from_ms': 1672618499999, 'effective_to_ms': 1735592220000}
|
||||
cell USDJPY stages ['std::grid', 'std::walk_forward', 'std::monte_carlo']
|
||||
monte_carlo pooled_oos: mean=-0.033809 p5=-0.0766 p95=0.0177 prob_le_zero=0.885 n_trades=25258
|
||||
coverage {'effective_from_ms': 1672617660000, 'effective_to_ms': 1735603200000}
|
||||
generalize: metric=expectancy_r n_instruments=2 worst_case=-0.251491 sign_agreement=0 per_instrument=[['GER40', -0.2514905108214054], ['USDJPY', -0.09353171411968528]]
|
||||
NOTE: presentation.emit asked for family_table AND selection_report;
|
||||
stdout carried 42 family_table lines and ZERO selection_report lines, silently.
|
||||
|
||||
$ aura runs families
|
||||
{"family_id":"0959eda8-0-GER40-w0-r0-s1-0","kind":"WalkForward","members":21}
|
||||
{"family_id":"0959eda8-0-USDJPY-w0-r0-s1-0","kind":"WalkForward","members":21}
|
||||
|
||||
$ aura reproduce 0959eda8-0-GER40-w0-r0-s1-0 (first 2 of 21 members)
|
||||
0959eda8-0-GER40-w0-r0-s1-0 member fast.length=10, slow.length=100, stop_length=3, stop_k=2 reproduced: bit-identical
|
||||
0959eda8-0-GER40-w0-r0-s1-0 member fast.length=10, slow.length=50, stop_length=3, stop_k=2 reproduced: bit-identical
|
||||
|
||||
$ aura chart 0959eda8-0 # default tap is equity; the campaign persisted r_equity
|
||||
EXIT=1
|
||||
aura: no family member has a tap named 'equity'
|
||||
|
||||
$ aura chart b657c714-GER40-w0 # the literal directory name under runs/traces/0959eda8-0/
|
||||
EXIT=1
|
||||
aura: no recorded run or family 'b657c714-GER40-w0' under runs/traces (check the handle a sweep/walk-forward/campaign run printed for a typo — re-running with this handle as `--trace` will not create it)
|
||||
^ names a `--trace` flag that exists on no verb after the retirement.
|
||||
|
||||
$ aura exec blueprints/signal.json --tap equity=record # the scaffolded starter blueprint
|
||||
EXIT=1
|
||||
aura: the tap plan names 'equity', but the blueprint declares no such tap — declared taps:
|
||||
^ the scaffolded starter declares no taps at all; the list after the em dash is empty.
|
||||
@@ -0,0 +1,61 @@
|
||||
# m42_3b — the --override refusal family on both legs (release binary @ 2f1bace)
|
||||
# blueprint leg
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length
|
||||
EXIT=2
|
||||
aura: exec: --override expects NODE.PARAM=VALUE, got `fast.length`
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=notanumber
|
||||
EXIT=2
|
||||
aura: exec: --override expects NODE.PARAM=VALUE, got `fast.length=notanumber`
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override nosuch.param=1
|
||||
EXIT=1
|
||||
aura: axis nosuch.param: names no param of this blueprint (open or bound) — see `aura graph introspect --params <bp>`
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=0
|
||||
EXIT=1
|
||||
aura: SMA length must be >= 1
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=-1
|
||||
EXIT=1
|
||||
aura: capacity overflow
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=-5
|
||||
EXIT=1
|
||||
aura: capacity overflow
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override bias.scale=2
|
||||
EXIT=1
|
||||
aura: this blueprint does not compile to a runnable harness: ParamKindMismatch { slot: 0, expected: F64, got: I64 }
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override bias.scale=-5.0
|
||||
EXIT=1
|
||||
aura: Bias scale must be > 0
|
||||
|
||||
$ aura exec blueprints/m42_trend.json --override bias.scale=0.5
|
||||
EXIT=0
|
||||
|
||||
# campaign leg
|
||||
|
||||
$ aura exec documents/spine-campaign.json --override fast.length=7
|
||||
EXIT=2
|
||||
aura: exec: --override `fast.length` collides with a declared axis of the campaign; an override overrides a bound value, never an axis
|
||||
|
||||
$ aura exec documents/spine-campaign.json --override stop_k=1.0
|
||||
EXIT=1
|
||||
aura: campaign references do not resolve:
|
||||
strategy b657c7148655fe3774910614031a8dcae6be317f0bbe7de6cbabfeb5ccb1d7cf: axis "stop_k" is not in the param space
|
||||
|
||||
$ aura exec documents/spine-campaign.json --override bias.scale=-5
|
||||
EXIT=1
|
||||
aura: campaign references do not resolve:
|
||||
strategy b657c7148655fe3774910614031a8dcae6be317f0bbe7de6cbabfeb5ccb1d7cf: axis "bias.scale" declares a kind that is not the param's kind
|
||||
|
||||
# topology_hash under --override — blueprint leg vs campaign leg
|
||||
$ aura exec blueprints/m42_trend.json -> topology_hash:
|
||||
b657c7148655fe3774910614031a8dcae6be317f0bbe7de6cbabfeb5ccb1d7cf
|
||||
$ aura exec blueprints/m42_trend.json --override fast.length=10 (NO-OP: 10 is the bound default) -> topology_hash:
|
||||
f952a1c8519a2dbd30fac8facb6149a8774bf4094d552a11e50afecbad9f6ccc
|
||||
$ aura reproduce <that hash>
|
||||
aura: no such family 'f952a1c8519a2dbd30fac8facb6149a8774bf4094d552a11e50afecbad9f6ccc'
|
||||
@@ -0,0 +1 @@
|
||||
/runs
|
||||
@@ -0,0 +1,4 @@
|
||||
# Static project context only (C17); paths only.
|
||||
[paths]
|
||||
runs = "runs"
|
||||
# data = "/path/to/archive" # the recorded-data root; defaults to the built-in path
|
||||
@@ -0,0 +1,24 @@
|
||||
# spine-lab — an aura research project (data-only)
|
||||
|
||||
This directory is an aura project: blueprints + research documents over the
|
||||
std vocabulary, anchored by `Aura.toml`. There is no crate and no build step.
|
||||
|
||||
- Run: `aura exec blueprints/signal.json` (single smoke run, synthetic stream
|
||||
— the starter is closed; all params bound; bound values are defaults —
|
||||
`--override NODE.PARAM=VALUE` may override one for this run (#246))
|
||||
- Campaign: `aura exec <campaign.json>` executes a registered campaign
|
||||
document (file or content id) — the multi-cell/axis surface
|
||||
- Axes: `aura graph introspect --params blueprints/signal.json` lists the
|
||||
open + bound-overridable axes, RAW `<node>.<param>` names (#328)
|
||||
- Native nodes: when the project needs its first project-specific node,
|
||||
`aura nodes new <name>` scaffolds a node crate beside this project and
|
||||
attaches it via `[nodes]` in `Aura.toml` (build it with `cargo build`).
|
||||
- Topology is data (`blueprints/*.json`); results land in `runs/`.
|
||||
- Execution model: a strategy emits a bias in [-1,+1] per cycle, held as the
|
||||
continuously-tracked target position; a protective stop defines the risk
|
||||
unit R, and quality metrics are R-based. Entry signals become held state
|
||||
via the signal-side latch/edge-pulse idiom (see
|
||||
`aura graph introspect --vocabulary`).
|
||||
- Data plane: process/campaign documents are authored directly with
|
||||
`aura process` / `aura campaign`, growing one from a bare `{}` via
|
||||
`aura campaign introspect --unwired`.
|
||||
@@ -0,0 +1 @@
|
||||
{"format_version":1,"blueprint":{"name":"m42_trend","doc":"fast/slow SMA crossover direction, clamped into the bias contract","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":10}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":50}}]}},{"primitive":{"type":"Sub","name":"spread"}},{"primitive":{"type":"Sign","name":"dir"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":1.0}}]}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0},{"from":3,"to":4,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":4,"field":0,"name":"bias"}],"taps":[{"name":"spread","from":{"node":2,"field":0}}]}}
|
||||
@@ -0,0 +1,17 @@
|
||||
[
|
||||
{"op":"name","name":"m42_trend"},
|
||||
{"op":"doc","text":"fast/slow SMA crossover direction, clamped into the bias contract"},
|
||||
{"op":"source","role":"price","kind":"F64"},
|
||||
{"op":"add","type":"SMA","name":"fast","bind":{"length":{"I64":10}}},
|
||||
{"op":"add","type":"SMA","name":"slow","bind":{"length":{"I64":50}}},
|
||||
{"op":"add","type":"Sub","name":"spread"},
|
||||
{"op":"add","type":"Sign","name":"dir"},
|
||||
{"op":"add","type":"Bias","name":"bias","bind":{"scale":{"F64":1.0}}},
|
||||
{"op":"feed","role":"price","into":["fast.series","slow.series"]},
|
||||
{"op":"connect","from":"fast.value","to":"spread.lhs"},
|
||||
{"op":"connect","from":"slow.value","to":"spread.rhs"},
|
||||
{"op":"connect","from":"spread.value","to":"dir.value"},
|
||||
{"op":"connect","from":"dir.value","to":"bias.signal"},
|
||||
{"op":"expose","from":"bias.bias","as":"bias"},
|
||||
{"op":"tap","from":"spread.value","as":"spread"}
|
||||
]
|
||||
@@ -0,0 +1 @@
|
||||
{"format_version":1,"blueprint":{"name":"spine_lab_signal","doc":"fast/slow SMA difference clamped into a directional bias","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":4}}]}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":0.5}}]}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}}
|
||||
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"kind": "campaign",
|
||||
"name": "m42-spine-ger40-usdjpy",
|
||||
"description": "Walk-forward + bootstrap + cross-instrument floor for the m42 trend blueprint on two instruments.",
|
||||
"data": {
|
||||
"instruments": [
|
||||
"GER40",
|
||||
"USDJPY"
|
||||
],
|
||||
"windows": [
|
||||
{
|
||||
"from_ms": 1672531200000,
|
||||
"to_ms": 1735603200000
|
||||
}
|
||||
]
|
||||
},
|
||||
"strategies": [
|
||||
{
|
||||
"ref": {
|
||||
"content_id": "b657c7148655fe3774910614031a8dcae6be317f0bbe7de6cbabfeb5ccb1d7cf"
|
||||
},
|
||||
"axes": {
|
||||
"fast.length": {
|
||||
"kind": "I64",
|
||||
"values": [
|
||||
5,
|
||||
10,
|
||||
20
|
||||
]
|
||||
},
|
||||
"slow.length": {
|
||||
"kind": "I64",
|
||||
"values": [
|
||||
50,
|
||||
100
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"process": {
|
||||
"ref": {
|
||||
"content_id": "14e9ce32b1ea050c23e66df99b182e0de0c3634063c9a887407c98852b75fbae"
|
||||
}
|
||||
},
|
||||
"seed": 7,
|
||||
"presentation": {
|
||||
"persist_taps": [
|
||||
"r_equity",
|
||||
"net_r_equity"
|
||||
],
|
||||
"emit": [
|
||||
"family_table",
|
||||
"selection_report"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"format_version": 1,
|
||||
"kind": "process",
|
||||
"name": "m42-validation-spine",
|
||||
"description": "Grid to walk-forward, bootstrap the pooled OOS R, then floor across instruments.",
|
||||
"pipeline": [
|
||||
{
|
||||
"block": "std::grid"
|
||||
},
|
||||
{
|
||||
"block": "std::walk_forward",
|
||||
"in_sample_ms": 7776000000,
|
||||
"out_of_sample_ms": 2592000000,
|
||||
"step_ms": 2592000000,
|
||||
"mode": "rolling",
|
||||
"metric": "expectancy_r",
|
||||
"select": "argmax"
|
||||
},
|
||||
{
|
||||
"block": "std::monte_carlo",
|
||||
"resamples": 200,
|
||||
"block_len": 5
|
||||
},
|
||||
{
|
||||
"block": "std::generalize",
|
||||
"metric": "expectancy_r"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user