diff --git a/docs/authoring-guide.md b/docs/authoring-guide.md new file mode 100644 index 0000000..b410efc --- /dev/null +++ b/docs/authoring-guide.md @@ -0,0 +1,318 @@ +# Authoring guide: op-scripts, process documents, campaign documents + +`docs/project-layout.md` describes the shape of a project and the arc of a +research session; this document is the practical companion for the three +JSON artifact kinds you author headlessly along that arc: + +1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint) + through `aura graph build`; +2. a **process document** (role 5) — a named validation/eval methodology + through `aura process validate|introspect|register`; +3. a **campaign document** (role 6b) — experiment intent (instruments × + windows × strategy × axes × process) through `aura campaign + validate|introspect|register|run`. + +Each section below is a worked, verified example — every command shown was +run against this repo and the output is transcribed, not invented. The *why* +of this three-artifact split (closed-vocabulary data, never a logic DSL) +lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the +glossary; this document only teaches the *shape*. + +## 1. Op-scripts — building a strategy blueprint by hand + +An op-script is a JSON **array of ops**, replayed in order to construct a +node graph. `aura graph build` reads the op-script from **stdin** (there is +no file argument) and prints the canonical blueprint envelope to stdout: + +``` +$ aura graph build < smacross.json > blueprint.json +``` + +Nodes are referenced by an **identifier** (given by `add`, see below); ports +are dotted `.` on both sides of a wire. + +### The six ops + +| op | JSON shape | does | +|---|---|---| +| `source` | `{"op":"source","role":,"kind":}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). | +| `input` | `{"op":"input","role":}` | reserve an open root **input** role (kind inferred from the slots it feeds) — for a fragment meant to be wired by an *enclosing* graph. A standalone document built with `aura graph build` finalizes as a closed root, so an `input` role that is never bound refuses at the end: `finalize: root input role is unbound`. | +| `add` | `{"op":"add","type":,"name":?,"bind":{:}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`). `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `bind` sets zero or more of its params. | +| `feed` | `{"op":"feed","role":,"into":[, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). | +| `connect` | `{"op":"connect","from":,"to":}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). | +| `expose` | `{"op":"expose","from":,"as":}` | promote an interior output field to a boundary output under the alias `as` — the only op whose name key is a real *alias* (contrast `add`'s `name`, which is an identifier, not a rename). | + +Value forms are the typed-tag representations used everywhere in this +family of artifacts: + +- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` / + `{"Timestamp":}`; +- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` | + `"Bool"` | `"Timestamp"`. + +### Worked example: an SMA-crossover bias strategy + +```json +[ + {"op": "source", "role": "price", "kind": "F64"}, + {"op": "add", "type": "SMA", "name": "fast", "bind": {"length": {"I64": 2}}}, + {"op": "add", "type": "SMA", "name": "slow", "bind": {"length": {"I64": 4}}}, + {"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]}, + {"op": "add", "type": "Sub", "name": "sub"}, + {"op": "connect", "from": "fast.value", "to": "sub.lhs"}, + {"op": "connect", "from": "slow.value", "to": "sub.rhs"}, + {"op": "add", "type": "Bias", "name": "bias"}, + {"op": "connect", "from": "sub.value", "to": "bias.signal"}, + {"op": "expose", "from": "bias.bias", "as": "bias"} +] +``` + +(`bias.scale` is left unbound here on purpose — an open campaign axis, see +§3. This is the milestone fieldtest corpus's own example, verified below; +also on disk at `fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.) + +### Commands + +``` +$ aura graph build < smacross.json +{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...], + "input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}} +``` + +Introspection is build-free wherever possible: + +``` +$ aura graph introspect --vocabulary # one node type per line +$ aura graph introspect --node SMA # ports + params of one type +SMA + in series:F64 + out value:F64 + param length:I64 (bind {"I64": }) +$ aura graph introspect --unwired < partial.json # open slots of a partial op-script +sub.rhs:F64 +$ aura graph introspect --content-id smacross.json # SHA-256 of the canonical form +597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a +$ aura graph introspect --content-id smacross.json --identity-id # + debug-name-blind identity id, combinable +597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a +41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4 +$ aura graph introspect --params smacross.json # the raw param-space namespace (what a campaign's axes bind against) +fast.length:I64 +slow.length:I64 +bias.scale:F64 +``` + +`--content-id`, `--identity-id`, `--params`, and `graph register` all accept +**either** shape: the raw op-script array or an already-built `#155` +blueprint envelope (object) — shape-discriminated automatically, so you +never have to build first just to hash or register: + +``` +$ aura graph register smacross.json # inside a project (Aura.toml present) +registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json) +``` + +The printed content id is the address a campaign document's `strategies[].ref` +points at (§3). + +## 2. Process documents — a validation methodology (role 5) + +A process document is a named, versionable pipeline of **std stage blocks** +over a shared metric vocabulary. Discover the block vocabulary and each +block's typed slots headlessly: + +``` +$ aura process introspect --vocabulary +std::sweep evaluate the campaign's axes-space; reduce members to R metrics; select a winner +std::gate filter survivors by a conjunction of typed metric predicates +std::walk_forward rolling in-sample optimize + out-of-sample test +std::monte_carlo R-bootstrap over realised R (terminal annotator): ... +std::generalize cross-instrument worst-case floor (terminal annotator): ... +$ aura process introspect --block std::sweep +std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; select a winner + metric required, metric name (see metric_vocabulary) + select required, select rule: argmax | plateau:mean | plateau:worst + deflate optional, bool +``` + +### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize) + +```json +{ + "format_version": 1, + "kind": "process", + "name": "mra-full-v2-sweep-gate-wf-mc-generalize", + "description": "Full v2 anti-false-discovery pipeline.", + "pipeline": [ + { "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true }, + { "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] }, + { "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" }, + { "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 }, + { "block": "std::generalize", "metric": "expectancy_r" } + ] +} +``` + +(`description` is optional; everything else in the envelope — `format_version`, +`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired` +over a bare `{}` will tell you.) + +``` +$ aura process validate mra_2_process_full_v2.json +process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates +$ aura process register mra_2_process_full_v2.json # inside a project +registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json) +``` + +### The metric vocabulary — discoverable, not hand-copied + +`aura process introspect --metrics` (equivalently `aura campaign introspect +--metrics` — same roster) enumerates every metric name annotated with its +role, so you never have to guess which metrics a `select`/`gate`/`generalize` +field will accept: + +``` +$ aura process introspect --metrics +expectancy_r rankable | gate | generalize +win_rate gate +avg_win_r gate +avg_loss_r gate +profit_factor gate +max_r_drawdown gate +sqn rankable | gate | generalize +sqn_normalized rankable | gate | generalize +net_expectancy_r rankable | gate | generalize +n_trades gate +n_open_at_end gate +total_pips rankable | gate +max_drawdown rankable | gate +bias_sign_flips rankable | gate +deflated_score annotation +overfit_probability annotation +neighbourhood_score annotation +``` + +Read the three tags as three separate questions about one metric name: + +- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward` + `select` metric (the field a winner is chosen by)? This is the small + subset every strategy run always produces cheaply. +- **`gate`** — can this name be used in a `std::gate` predicate (a per-member + filter)? This roster is a superset of `rankable` — most emitted metrics + are filterable even when they make a poor ranking criterion. +- **`| generalize`** suffix — is this name usable as `std::generalize`'s + `metric` (needs an R-expectancy-shaped metric to floor across instruments)? + +That is why, for example, `profit_factor` shows up tagged only `gate`: every +member's metrics table carries it (so you can gate on it, e.g. "keep only +`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster, +so a `std::sweep` with `"select": "argmax"` cannot select on it, and +`std::generalize` cannot floor across instruments on it either. Attempting +either produces the same intrinsic refusal that names the metric and points +back at this verb (`aura process introspect --metrics`) rather than leaving +you to search the glossary. + +## 3. Campaign documents — experiment intent over instruments and windows + +A campaign document names the data (instruments × windows), one or more +strategies (by blueprint content/identity id) with their param axes, a +process reference, and what to persist/emit. + +``` +$ aura campaign introspect --unwired bare.json # bare.json contains just {} +open slot: format_version (required, must be 1) +open slot: kind (required, must be "campaign") +open slot: name (required, string) +open slot: data (required section: instruments + windows) +open slot: strategies (required, non-empty list of { ref, axes }) +open slot: process.ref (required, content id of a process document) +open slot: seed (required, non-negative integer) +open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit) +``` + +### Worked example: two instruments, one strategy, four axis points + +```json +{ + "format_version": 1, + "kind": "campaign", + "name": "mra-ger40-fra40-smacross-full-v2", + "seed": 42, + "data": { + "instruments": ["GER40", "FRA40"], + "windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ] + }, + "strategies": [ + { + "ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" }, + "axes": { + "fast.length": { "kind": "I64", "values": [2, 4] }, + "slow.length": { "kind": "I64", "values": [8, 16] }, + "bias.scale": { "kind": "F64", "values": [0.5] } + } + } + ], + "process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } }, + "presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] } +} +``` + +The `strategies[].ref.content_id` and `process.ref.content_id` are exactly +the ids printed by `aura graph register` / `aura process register` above — +a campaign never inlines a strategy or process body, only its content id. +Each axis name (`fast.length`, …) must name an open param of the referenced +blueprint (`aura graph introspect --params`, §1) and declare that param's +`ScalarKind`. + +### Validate — three tiers, honest degradation + +``` +$ aura campaign validate mra_3_campaign_full_v2.json # outside any project +campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s) +referential checks skipped (no Aura.toml found up from /home/…) +``` + +Inside a project, with the strategy blueprint and process both already +registered, the same command runs two further tiers: + +``` +$ aura campaign validate mra_3_campaign_full_v2.json # inside a project, refs registered +campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s) +campaign document valid (referential): all references resolve, axes are in the param space +campaign document valid (executable): pipeline shape and static guards pass +``` + +- **intrinsic** — the document's own shape is well-formed (always checked). +- **referential** — every `ref` resolves in the project's store and every + axis names a real, correctly-typed open param (needs a project). +- **executable** — the process pipeline's static guards pass against this + campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a + data-free preflight, so "valid" here means "runnable" (needs a project; + it does not fetch or touch market data). + +### Register and run + +``` +$ aura campaign register mra_3_campaign_full_v2.json +registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json) +$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd +{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}} +… +{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42, + "cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],"stages":[ + {"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]}, + {"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]}, + {"strategy":"597d719b…","instrument":"FRA40",...}], + "generalizations":[{"strategy_ordinal":0,"window_ordinal":0, + "generalization":{"selection_metric":"expectancy_r","n_instruments":2, + "worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]}, + "winners":[...]}], + "trace_name":"42edebd2-0"}} +``` + +`aura campaign run` is register-then-run sugar for a `.json` file, but the +canonical address is always the content id — running a bare file the first +time registers it implicitly. `aura campaign runs` lists stored +realizations; `aura campaign runs ` dumps the bare stored record(s) (not +the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps` +is non-empty, the run also persists the named taps under +`runs/traces//…`, chartable with `aura chart`. diff --git a/docs/project-layout.md b/docs/project-layout.md index b3bf221..b1a124f 100644 --- a/docs/project-layout.md +++ b/docs/project-layout.md @@ -90,6 +90,16 @@ logic. IONOS LLMs appear only as a *runtime data source* (e.g. a news-agent node emitting a bias), recorded before it enters a backtest, and only with your per-session consent. +Wiring *already-built* node types into a graph, and naming experiment intent +over such a graph, is a separate, closed-vocabulary **data** surface, not Rust: +an **op-script** (`aura graph build`) assembles a strategy blueprint from node +types already in the vocabulary, a **process document** (`aura process`) names +a validation/eval methodology, and a **campaign document** (`aura campaign`) +names instruments × windows × strategy × param axes × process. All three are +headless-authorable and introspectable without a build. See +[`docs/authoring-guide.md`](authoring-guide.md) for the worked grammar and +command sequences. + ## A day in the life 1. **You** (in Claude Code, inside `ger40-lab/`): "I suspect the 3rd 15m candle