Aura/docs/authoring-guide.md

# Authoring guide: op-scripts, process documents, campaign documents

`docs/project-layout.md` describes the shape of a project and the arc of a
research session; this document is the practical companion for the three
JSON artifact kinds you author headlessly along that arc:

1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint)
   through `aura graph build`;
2. a **process document** (role 5) — a named validation/eval methodology
   through `aura process validate|introspect|register`;
3. a **campaign document** (role 6b) — experiment intent (instruments ×
   windows × strategy × axes × process) through `aura campaign
   validate|introspect|register|run`.

Each section below is a worked, verified example — every command shown was
run against this repo and the output is transcribed, not invented. The *why*
of this three-artifact split (closed-vocabulary data, never a logic DSL)
lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the
glossary; this document only teaches the *shape*.

## 1. Op-scripts — building a strategy blueprint by hand

An op-script is a JSON **array of ops**, replayed in order to construct a
node graph. `aura graph build` reads the op-script from **stdin** (there is
no file argument) and prints the canonical blueprint envelope to stdout:

```
$ aura graph build < smacross.json > blueprint.json
```

Nodes are referenced by an **identifier** (given by `add`, see below); ports
are dotted `<identifier>.<port>` on both sides of a wire.

### The six ops

| op | JSON shape | does |
|---|---|---|
| `source` | `{"op":"source","role":<str>,"kind":<ScalarKind>}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). |
| `input` | `{"op":"input","role":<str>}` | reserve an open root **input** role (kind inferred from the slots it feeds) — for a fragment meant to be wired by an *enclosing* graph. A standalone document built with `aura graph build` finalizes as a closed root, so an `input` role that is never bound refuses at the end: `finalize: root input role <name> is unbound`. |
| `add` | `{"op":"add","type":<TypeId>,"name":<str>?,"bind":{<param>:<Scalar>}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`). `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `bind` sets zero or more of its params. |
| `feed` | `{"op":"feed","role":<str>,"into":[<port>, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). |
| `connect` | `{"op":"connect","from":<port>,"to":<port>}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). |
| `expose` | `{"op":"expose","from":<port>,"as":<str>}` | promote an interior output field to a boundary output under the alias `as` — the only op whose name key is a real *alias* (contrast `add`'s `name`, which is an identifier, not a rename). |

Value forms are the typed-tag representations used everywhere in this
family of artifacts:

- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` /
  `{"Timestamp":<i64 ms>}`;
- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` |
  `"Bool"` | `"Timestamp"`.

### Worked example: an SMA-crossover bias strategy

```json
[
  {"op": "source", "role": "price", "kind": "F64"},
  {"op": "add", "type": "SMA", "name": "fast", "bind": {"length": {"I64": 2}}},
  {"op": "add", "type": "SMA", "name": "slow", "bind": {"length": {"I64": 4}}},
  {"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
  {"op": "add", "type": "Sub", "name": "sub"},
  {"op": "connect", "from": "fast.value", "to": "sub.lhs"},
  {"op": "connect", "from": "slow.value", "to": "sub.rhs"},
  {"op": "add", "type": "Bias", "name": "bias"},
  {"op": "connect", "from": "sub.value", "to": "bias.signal"},
  {"op": "expose", "from": "bias.bias", "as": "bias"}
]
```

(`bias.scale` is left unbound here on purpose — an open campaign axis, see
§3. This is the milestone fieldtest corpus's own example, verified below;
also on disk at `fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.)

### Commands

```
$ aura graph build < smacross.json
{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...],
 "input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}}
```

Introspection is build-free wherever possible:

```
$ aura graph introspect --vocabulary        # one node type per line
$ aura graph introspect --node SMA          # ports + params of one type
SMA
  in   series:F64
  out  value:F64
  param length:I64  (bind {"I64": <v>})
$ aura graph introspect --unwired < partial.json   # open slots of a partial op-script
sub.rhs:F64
$ aura graph introspect --content-id smacross.json           # SHA-256 of the canonical form
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
$ aura graph introspect --content-id smacross.json --identity-id  # + debug-name-blind identity id, combinable
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4
$ aura graph introspect --params smacross.json   # the raw param-space namespace (what a campaign's axes bind against)
fast.length:I64
slow.length:I64
bias.scale:F64
```

`--content-id`, `--identity-id`, `--params`, and `graph register` all accept
**either** shape: the raw op-script array or an already-built `#155`
blueprint envelope (object) — shape-discriminated automatically, so you
never have to build first just to hash or register:

```
$ aura graph register smacross.json      # inside a project (Aura.toml present)
registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json)
```

The printed content id is the address a campaign document's `strategies[].ref`
points at (§3).

## 2. Process documents — a validation methodology (role 5)

A process document is a named, versionable pipeline of **std stage blocks**
over a shared metric vocabulary. Discover the block vocabulary and each
block's typed slots headlessly:

```
$ aura process introspect --vocabulary
std::sweep         evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
std::gate          filter survivors by a conjunction of typed metric predicates
std::walk_forward  rolling in-sample optimize + out-of-sample test
std::monte_carlo   R-bootstrap over realised R (terminal annotator): ...
std::generalize    cross-instrument worst-case floor (terminal annotator): ...
$ aura process introspect --block std::sweep
std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
  metric               optional, metric name (see metric_vocabulary)
  select               optional, select rule: argmax | plateau:mean | plateau:worst
  deflate              optional, bool
```

`metric` and `select` form one **selection group**: all-or-nothing (a
document naming one without the other is refused, "the selection group is
all-or-nothing"), `deflate` composes only when the group is present. Omit
the group entirely for a **selection-free sweep** — the family itself is
the result, no winner is chosen. A selection-free sweep is only legal as
the pipeline's *last* stage (the executor's preflight refuses one followed
by any other block, since a downstream stage would have no nominee to
consume):

```json
{
  "format_version": 1,
  "kind": "process",
  "name": "explore-only-sweep",
  "pipeline": [ { "block": "std::sweep" } ]
}
```

The executor records this stage's family (every member run) but no
`StageSelection` and no nominee — recording a winner here would fabricate a
selection intent the document never expressed.

### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize)

```json
{
  "format_version": 1,
  "kind": "process",
  "name": "mra-full-v2-sweep-gate-wf-mc-generalize",
  "description": "Full v2 anti-false-discovery pipeline.",
  "pipeline": [
    { "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
    { "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
    { "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
    { "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
    { "block": "std::generalize", "metric": "expectancy_r" }
  ]
}
```

(`description` is optional; everything else in the envelope — `format_version`,
`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired`
over a bare `{}` will tell you.)

```
$ aura process validate mra_2_process_full_v2.json
process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates
$ aura process register mra_2_process_full_v2.json    # inside a project
registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json)
```

### The metric vocabulary — discoverable, not hand-copied

`aura process introspect --metrics` (equivalently `aura campaign introspect
--metrics` — same roster) enumerates every metric name annotated with its
role, so you never have to guess which metrics a `select`/`gate`/`generalize`
field will accept:

```
$ aura process introspect --metrics
expectancy_r             rankable | gate | generalize
win_rate                 gate
avg_win_r                gate
avg_loss_r               gate
profit_factor            gate
max_r_drawdown           gate
sqn                      rankable | gate | generalize
sqn_normalized           rankable | gate | generalize
net_expectancy_r         rankable | gate | generalize
n_trades                 gate
n_open_at_end            gate
total_pips               rankable | gate
max_drawdown             rankable | gate
bias_sign_flips          rankable | gate
deflated_score           annotation
overfit_probability      annotation
neighbourhood_score      annotation
```

Read the three tags as three separate questions about one metric name:

- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward`
  `select` metric (the field a winner is chosen by)? This is the small
  subset every strategy run always produces cheaply.
- **`gate`** — can this name be used in a `std::gate` predicate (a per-member
  filter)? This roster is a superset of `rankable` — most emitted metrics
  are filterable even when they make a poor ranking criterion.
- **`| generalize`** suffix — is this name usable as `std::generalize`'s
  `metric` (needs an R-expectancy-shaped metric to floor across instruments)?

That is why, for example, `profit_factor` shows up tagged only `gate`: every
member's metrics table carries it (so you can gate on it, e.g. "keep only
`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster,
so a `std::sweep` with `"select": "argmax"` cannot select on it, and
`std::generalize` cannot floor across instruments on it either. Attempting
either produces the same intrinsic refusal that names the metric and points
back at this verb (`aura process introspect --metrics`) rather than leaving
you to search the glossary.

## 3. Campaign documents — experiment intent over instruments and windows

A campaign document names the data (instruments × windows), one or more
strategies (by blueprint content/identity id) with their param axes, a
process reference, and what to persist/emit.

```
$ aura campaign introspect --unwired bare.json   # bare.json contains just {}
open slot: format_version (required, must be 1)
open slot: kind (required, must be "campaign")
open slot: name (required, string)
open slot: data (required section: instruments + windows)
open slot: strategies (required, non-empty list of { ref, axes })
open slot: process.ref (required, content id of a process document)
open slot: seed (required, non-negative integer)
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
```

### Worked example: two instruments, one strategy, four axis points

```json
{
  "format_version": 1,
  "kind": "campaign",
  "name": "mra-ger40-fra40-smacross-full-v2",
  "seed": 42,
  "data": {
    "instruments": ["GER40", "FRA40"],
    "windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
  },
  "strategies": [
    {
      "ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
      "axes": {
        "fast.length": { "kind": "I64", "values": [2, 4] },
        "slow.length": { "kind": "I64", "values": [8, 16] },
        "bias.scale": { "kind": "F64", "values": [0.5] }
      }
    }
  ],
  "process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } },
  "presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] }
}
```

The `strategies[].ref.content_id` and `process.ref.content_id` are exactly
the ids printed by `aura graph register` / `aura process register` above —
a campaign never inlines a strategy or process body, only its content id.
Each axis name (`fast.length`, …) must name an open param of the referenced
blueprint (`aura graph introspect --params`, §1) and declare that param's
`ScalarKind`.

### Validate — three tiers, honest degradation

```
$ aura campaign validate mra_3_campaign_full_v2.json          # outside any project
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s)
referential checks skipped (no Aura.toml found up from /home/…)
```

Inside a project, with the strategy blueprint and process both already
registered, the same command runs two further tiers:

```
$ aura campaign validate mra_3_campaign_full_v2.json          # inside a project, refs registered
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 1 window(s)
campaign document valid (referential): all references resolve, axes are in the param space
campaign document valid (executable): pipeline shape and static guards pass
```

- **intrinsic** — the document's own shape is well-formed (always checked).
- **referential** — every `ref` resolves in the project's store and every
  axis names a real, correctly-typed open param (needs a project).
- **executable** — the process pipeline's static guards pass against this
  campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a
  data-free preflight, so "valid" here means "runnable" (needs a project;
  it does not fetch or touch market data).

### Register and run

```
$ aura campaign register mra_3_campaign_full_v2.json
registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json)
$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd
{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}}
…
{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42,
  "cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],"stages":[
    {"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]},
    {"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]},
   {"strategy":"597d719b…","instrument":"FRA40",...}],
  "generalizations":[{"strategy_ordinal":0,"window_ordinal":0,
    "generalization":{"selection_metric":"expectancy_r","n_instruments":2,
      "worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]},
    "winners":[...]}],
  "trace_name":"42edebd2-0"}}
```

`aura campaign run` is register-then-run sugar for a `.json` file, but the
canonical address is always the content id — running a bare file the first
time registers it implicitly. `aura campaign runs` lists stored
realizations; `aura campaign runs <id>` dumps the bare stored record(s) (not
the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps`
is non-empty, the run also persists the named taps under
`runs/traces/<trace_name>/…`, chartable with `aura chart`.