Brummel/Aura
1
0
Fork 0
Code Issues 33 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: authoring guide for op-scripts, process & campaign documents

Browse Source 
Close two milestone-fieldtest doc-gaps (F6/F8) with one pass now that the
surfaces are stable. A newcomer could only learn the three headless JSON
artifact kinds from the fieldtest corpus: the op-script grammar was
undocumented (#208) and there was no worked process/campaign example nor a
pointer to the discoverable metric vocabulary (#197).

Add docs/authoring-guide.md — a worked, verified example per artifact kind
(op-script -> graph build; process document -> process validate/register;
campaign document -> campaign validate/register/run), every command run
against the binary and its output transcribed. Documents all six ops
(source/input/add/feed/connect/expose) with their JSON shapes, and teaches
the metric roster split (rankable / gate / | generalize) pointing at
'aura process introspect --metrics' as the source of truth rather than
hand-copying the rosters. A short paragraph in project-layout.md links it.

Docs only; no code or CLI change (the surfaces already suffice). Grammar
grounded on construction.rs; examples faithful to the milestone corpus.

closes #197
closes #208
This commit is contained in:
Brummel
2026-07-04 15:30:03 +02:00
parent 61c8828fad
commit e7c7bded3e
2 changed files with 328 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/authoring-guide.md
+318
View File
@@ -0,0 +1,318 @@
# Authoring guide: op-scripts, process documents, campaign documents
`docs/project-layout.md` describes the shape of a project and the arc of a
research session; this document is the practical companion for the three
JSON artifact kinds you author headlessly along that arc:
1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint)
through `aura graph build`;
2. a **process document** (role 5) — a named validation/eval methodology
through `aura process validate|introspect|register`;
3. a **campaign document** (role 6b) — experiment intent (instruments ×
windows × strategy × axes × process) through `aura campaign
validate|introspect|register|run`.
Each section below is a worked, verified example — every command shown was
run against this repo and the output is transcribed, not invented. The *why*
of this three-artifact split (closed-vocabulary data, never a logic DSL)
lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the
glossary; this document only teaches the *shape*.
## 1. Op-scripts — building a strategy blueprint by hand
An op-script is a JSON **array of ops**, replayed in order to construct a
node graph. `aura graph build` reads the op-script from **stdin** (there is
no file argument) and prints the canonical blueprint envelope to stdout:
```
$ aura graph build < smacross.json > blueprint.json
```
Nodes are referenced by an **identifier** (given by `add`, see below); ports
are dotted `<identifier>.<port>` on both sides of a wire.
### The six ops
| op | JSON shape | does |
|---|---|---|
| `source` | `{"op":"source","role":<str>,"kind":<ScalarKind>}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). |
| `input` | `{"op":"input","role":<str>}` | reserve an open root **input** role (kind inferred from the slots it feeds) — for a fragment meant to be wired by an *enclosing* graph. A standalone document built with `aura graph build` finalizes as a closed root, so an `input` role that is never bound refuses at the end: `finalize: root input role <name> is unbound`. |
| `add` | `{"op":"add","type":<TypeId>,"name":<str>?,"bind":{<param>:<Scalar>}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`). `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `bind` sets zero or more of its params. |
| `feed` | `{"op":"feed","role":<str>,"into":[<port>, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). |
| `connect` | `{"op":"connect","from":<port>,"to":<port>}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). |
| `expose` | `{"op":"expose","from":<port>,"as":<str>}` | promote an interior output field to a boundary output under the alias `as` — the only op whose name key is a real *alias* (contrast `add`'s `name`, which is an identifier, not a rename). |
Value forms are the typed-tag representations used everywhere in this
family of artifacts:
- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` /
`{"Timestamp":<i64 ms>}`;
- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` |
`"Bool"` | `"Timestamp"`.
### Worked example: an SMA-crossover bias strategy
```json
[
{"op": "source", "role": "price", "kind": "F64"},
{"op": "add", "type": "SMA", "name": "fast", "bind": {"length": {"I64": 2}}},
{"op": "add", "type": "SMA", "name": "slow", "bind": {"length": {"I64": 4}}},
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
{"op": "add", "type": "Sub", "name": "sub"},
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
{"op": "add", "type": "Bias", "name": "bias"},
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
{"op": "expose", "from": "bias.bias", "as": "bias"}
]
```
(`bias.scale` is left unbound here on purpose — an open campaign axis, see
§3. This is the milestone fieldtest corpus's own example, verified below;
also on disk at `fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.)
### Commands
```
$ aura graph build < smacross.json
{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...],
"input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}}
```
Introspection is build-free wherever possible:
```
$ aura graph introspect --vocabulary # one node type per line
$ aura graph introspect --node SMA # ports + params of one type
SMA
in series:F64
out value:F64
param length:I64 (bind {"I64": <v>})
$ aura graph introspect --unwired < partial.json # open slots of a partial op-script
sub.rhs:F64
$ aura graph introspect --content-id smacross.json # SHA-256 of the canonical form
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
$ aura graph introspect --content-id smacross.json --identity-id # + debug-name-blind identity id, combinable
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4
$ aura graph introspect --params smacross.json # the raw param-space namespace (what a campaign's axes bind against)
fast.length:I64
slow.length:I64
bias.scale:F64
```
`--content-id`, `--identity-id`, `--params`, and `graph register` all accept
**either** shape: the raw op-script array or an already-built `#155`
blueprint envelope (object) — shape-discriminated automatically, so you
never have to build first just to hash or register:
```
$ aura graph register smacross.json # inside a project (Aura.toml present)
registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json)
```
The printed content id is the address a campaign document's `strategies[].ref`
points at (§3).
## 2. Process documents — a validation methodology (role 5)
A process document is a named, versionable pipeline of **std stage blocks**
over a shared metric vocabulary. Discover the block vocabulary and each
block's typed slots headlessly:
```
$ aura process introspect --vocabulary
std::sweep evaluate the campaign's axes-space; reduce members to R metrics; select a winner
std::gate filter survivors by a conjunction of typed metric predicates
std::walk_forward rolling in-sample optimize + out-of-sample test
std::monte_carlo R-bootstrap over realised R (terminal annotator): ...
std::generalize cross-instrument worst-case floor (terminal annotator): ...
$ aura process introspect --block std::sweep
std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; select a winner
metric required, metric name (see metric_vocabulary)
select required, select rule: argmax | plateau:mean | plateau:worst
deflate optional, bool
```
### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize)
```json
{
"format_version": 1,
"kind": "process",
"name": "mra-full-v2-sweep-gate-wf-mc-generalize",
"description": "Full v2 anti-false-discovery pipeline.",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::generalize", "metric": "expectancy_r" }
]
}
```
(`description` is optional; everything else in the envelope — `format_version`,
`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired`
over a bare `{}` will tell you.)
```
$ aura process validate mra_2_process_full_v2.json
process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates
$ aura process register mra_2_process_full_v2.json # inside a project
registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json)
```
### The metric vocabulary — discoverable, not hand-copied
`aura process introspect --metrics` (equivalently `aura campaign introspect
--metrics` — same roster) enumerates every metric name annotated with its
role, so you never have to guess which metrics a `select`/`gate`/`generalize`
field will accept:
```
$ aura process introspect --metrics
expectancy_r rankable | gate | generalize
win_rate gate
avg_win_r gate
avg_loss_r gate
profit_factor gate
max_r_drawdown gate
sqn rankable | gate | generalize
sqn_normalized rankable | gate | generalize
net_expectancy_r rankable | gate | generalize
n_trades gate
n_open_at_end gate
total_pips rankable | gate
max_drawdown rankable | gate
bias_sign_flips rankable | gate
deflated_score annotation
overfit_probability annotation
neighbourhood_score annotation
```
Read the three tags as three separate questions about one metric name:
- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward`
`select` metric (the field a winner is chosen by)? This is the small
subset every strategy run always produces cheaply.
- **`gate`** — can this name be used in a `std::gate` predicate (a per-member
filter)? This roster is a superset of `rankable` — most emitted metrics
are filterable even when they make a poor ranking criterion.
- **`| generalize`** suffix — is this name usable as `std::generalize`'s
`metric` (needs an R-expectancy-shaped metric to floor across instruments)?
That is why, for example, `profit_factor` shows up tagged only `gate`: every
member's metrics table carries it (so you can gate on it, e.g. "keep only
`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster,
so a `std::sweep` with `"select": "argmax"` cannot select on it, and
`std::generalize` cannot floor across instruments on it either. Attempting
either produces the same intrinsic refusal that names the metric and points
back at this verb (`aura process introspect --metrics`) rather than leaving
you to search the glossary.
## 3. Campaign documents — experiment intent over instruments and windows
A campaign document names the data (instruments × windows), one or more
strategies (by blueprint content/identity id) with their param axes, a
process reference, and what to persist/emit.
```
$ aura campaign introspect --unwired bare.json # bare.json contains just {}
open slot: format_version (required, must be 1)
open slot: kind (required, must be "campaign")
open slot: name (required, string)
open slot: data (required section: instruments + windows)
open slot: strategies (required, non-empty list of { ref, axes })
open slot: process.ref (required, content id of a process document)
open slot: seed (required, non-negative integer)
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
```
### Worked example: two instruments, one strategy, four axis points
```json
{
"format_version": 1,
"kind": "campaign",
"name": "mra-ger40-fra40-smacross-full-v2",
"seed": 42,
"data": {
"instruments": ["GER40", "FRA40"],
"windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
},
"strategies": [
{
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
"axes": {
"fast.length": { "kind": "I64", "values": [2, 4] },
"slow.length": { "kind": "I64", "values": [8, 16] },
"bias.scale": { "kind": "F64", "values": [0.5] }
}
}
],
"process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } },
"presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] }
}
```
The `strategies[].ref.content_id` and `process.ref.content_id` are exactly
the ids printed by `aura graph register` / `aura process register` above —
a campaign never inlines a strategy or process body, only its content id.
Each axis name (`fast.length`, …) must name an open param of the referenced
blueprint (`aura graph introspect --params`, §1) and declare that param's
`ScalarKind`.
### Validate — three tiers, honest degradation
```
$ aura campaign validate mra_3_campaign_full_v2.json # outside any project
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s)
referential checks skipped (no Aura.toml found up from /home/…)
```
Inside a project, with the strategy blueprint and process both already
registered, the same command runs two further tiers:
```
$ aura campaign validate mra_3_campaign_full_v2.json # inside a project, refs registered
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s)
campaign document valid (referential): all references resolve, axes are in the param space
campaign document valid (executable): pipeline shape and static guards pass
```
- **intrinsic** — the document's own shape is well-formed (always checked).
- **referential** — every `ref` resolves in the project's store and every
axis names a real, correctly-typed open param (needs a project).
- **executable** — the process pipeline's static guards pass against this
campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a
data-free preflight, so "valid" here means "runnable" (needs a project;
it does not fetch or touch market data).
### Register and run
```
$ aura campaign register mra_3_campaign_full_v2.json
registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json)
$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd
{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}}
…
{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42,
"cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],"stages":[
{"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]},
{"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]},
{"strategy":"597d719b…","instrument":"FRA40",...}],
"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,
"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
"worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]},
"winners":[...]}],
"trace_name":"42edebd2-0"}}
```
`aura campaign run` is register-then-run sugar for a `.json` file, but the
canonical address is always the content id — running a bare file the first
time registers it implicitly. `aura campaign runs` lists stored
realizations; `aura campaign runs <id>` dumps the bare stored record(s) (not
the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps`
is non-empty, the run also persists the named taps under
`runs/traces/<trace_name>/…`, chartable with `aura chart`.
docs/project-layout.md
+10
View File
@@ -90,6 +90,16 @@ logic. IONOS LLMs appear only as a *runtime data source* (e.g. a news-agent node
emitting a bias), recorded before it enters a backtest, and only with your
per-session consent.
Wiring *already-built* node types into a graph, and naming experiment intent
over such a graph, is a separate, closed-vocabulary **data** surface, not Rust:
an **op-script** (`aura graph build`) assembles a strategy blueprint from node
types already in the vocabulary, a **process document** (`aura process`) names
a validation/eval methodology, and a **campaign document** (`aura campaign`)
names instruments × windows × strategy × param axes × process. All three are
headless-authorable and introspectable without a build. See
[`docs/authoring-guide.md`](authoring-guide.md) for the worked grammar and
command sequences.
## A day in the life
1. **You** (in Claude Code, inside `ger40-lab/`): "I suspect the 3rd 15m candle