Files
Aura/docs/authoring-guide.md
T
claude b3b7115825 docs(ledger, glossary, guide): the one-namespace axis prose
The C24 axis-discovery paragraph loses its obsolete rationale (names
prefixed by the wrapping, discovery pinned to the sweep verb) in favour
of the one-raw-namespace statement; C18 records the ratified #246
bound-override coincidence contract (param_space OR bound_param_space,
no schema flag) as the document-side absorption #328 demanded; the
glossary sweep and use entries, the authoring-guide worked block, and
the README axis examples move to raw names. Refusal transcripts quote
the byte-actual binary output (review caught an invented aura: error:
prefix; the campaign-validate transcript shows the real fault_block
shape). Frozen corpora (fieldtests/, *.history.md) keep their
historical wrapped prose.

refs #328
2026-07-25 01:33:18 +02:00

747 lines
40 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Authoring guide: op-scripts, process documents, campaign documents
`docs/project-layout.md` describes the shape of a project and the arc of a
research session; this document is the practical companion for the three
JSON artifact kinds you author headlessly along that arc:
1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint)
through `aura graph build`;
2. a **process document** (role 5) — a named validation/eval methodology
through `aura process validate|introspect|register|show`;
3. a **campaign document** (role 6b) — experiment intent (instruments ×
windows × strategy × axes × process) through `aura campaign
validate|introspect|register|show|run|runs`.
Each section below is a worked, verified example — every command shown was
run against this repo and the output is transcribed, not invented. The *why*
of this three-artifact split (closed-vocabulary data, never a logic DSL)
lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the
glossary; this document only teaches the *shape*.
All three artifact kinds above assume the node **types** they reference
already exist. §0 covers the one piece of this arc that is Rust, not data:
adding a new node type in the first place.
## 0. Authoring a new node in Rust
An op-script's `add` op (§1) instantiates a node **type** from the closed
vocabulary — but that vocabulary itself is not data, it is compiled Rust.
Adding a new node type (a new indicator, combinator, or signal primitive) is
role-2 work: you write a `Node` implementation, describe it to the bootstrap
with a `PrimitiveBuilder` recipe, and roster the type id so the loader can
find it by name. This is the one seam in the whole arc where the answer is
"write Rust", not "write JSON" (C17/C20) — and it is a small, fixed shape,
the same shape every node in `aura-std` already follows.
### Where the code goes
- **The project's own signal doesn't need a crate by default.** `aura new`
scaffolds a **data-only** project (`docs/project-layout.md`, "A project
repo (two tiers)") — a strategy over the std vocabulary is a
blueprint/campaign document, not Rust: the scaffold ships **one** closed
`signal.json` starter that serves both verbs — `aura run` uses its bound
values as-is, `aura sweep --axis fast.length=2,4,8` overrides them
(bound = default, not fixed); `--list-axes` lists the open knobs and the
bound defaults alike, so every override target is discoverable.
- **A project-specific *native* node type** — the moment §0's three-part
pattern is actually needed — lives in an attached **node crate**:
`aura nodes new <name>` scaffolds a sibling cdylib crate (`Cargo.toml` +
`src/lib.rs`, registered under the project's own `<namespace>::` prefix)
and appends its path to the project's `Aura.toml [nodes]` section.
- **A block promoted to universal** — reused across projects and folded into
the engine itself — lives in `crates/aura-std/` (or its sibling domain
crates: `aura-market`, `aura-strategy`), unprefixed, and is rostered in
`crates/aura-vocabulary/src/lib.rs` instead of a node crate's
`vocabulary()` function. The pattern below is identical either way; only
the rostering call site differs (see "Rostering the type" below).
### The three-part pattern
Every node type — std or project-local — is three things:
1. **A `Node` implementation.** A plain struct holding whatever state the
node needs between cycles (often just its output cell), plus three
methods: `lookbacks()` (how many past values, per input, the node reads —
`vec![1]` for a node that only ever looks at the current cycle),
`eval(ctx) -> Option<&[Cell]>` (the per-cycle computation — return `None`
until every input the node needs has fired at least once; this warm-up
filter is what keeps a downstream consumer from ever observing a
fabricated value, C8), and `label()` (a short, human-readable debug
string — not the type id used for serialization, see below). The input
window `ctx.f64_in(i)` hands those past values back **financial-style:
index 0 is the newest cycle, index `k` is `k` cycles back** — so a
`lookbacks()` of `vec![n]` makes `w[0]..=w[n-1]` the last `n` values,
newest first, and a lookback-spanning node (momentum, ATR, RSI) reads its
span as `w[0]` vs `w[length]`, never the other way round.
2. **A `PrimitiveBuilder` recipe.** A `builder()` constructor that pairs a
`NodeSchema` (its input ports as `PortSpec`, its output fields as
`FieldSpec`, and its bindable params as `ParamSpec`) with a **build
closure** `|p| Box::new(Type::new(...))` that reads bound param values out
of `p` positionally (`p[0].f64()`, `p[1].i64()`, …, matching the `params`
order in the schema) and constructs the node. This one recipe is what lets
the bootstrap turn a blueprint's *serialized* param values into a live,
concrete node — a project or op-script never constructs a node directly.
3. **Rostering the type id.** The recipe is useless until something maps the
serialized type-id string (e.g. `"Scale"`, `"my_lab::ThirdCandle"`) back to
its `builder()`. This is a closed, compiled-in `match` — never a dynamic
registry (domain invariant 9) — so a node exists in the vocabulary only if
its type id is added to exactly one of these two match tables:
- **Project-side:** the `vocabulary()` / `type_ids()` pair the
`aura_core::aura_project!` macro wires up (every `aura new` scaffold
emits a starter pair — see the worked example below). Add one match arm
to `vocabulary()` and one entry to `type_ids()`'s slice.
- **Std-side** (only when promoting a node into the shipped std
vocabulary): one line in the `std_vocabulary_roster!` macro invocation in
[`crates/aura-vocabulary/src/lib.rs`](../crates/aura-vocabulary/src/lib.rs)
`"TypeId" => Type,` — which expands into both the resolver `match` and
the enumerable type-id list, so the two surfaces cannot drift apart. A
zero-arg `Type::builder()` rosters directly; an **arg-bearing** type
(structural, non-scalar construction — see "Session anchoring" below and
`docs/glossary.md`'s `construction arg` entry, #271) rosters the exact
same way — its `builder()` is itself zero-arg, returning a *pending*
recipe the `add` op's `args` configures before any `bind`.
An unrostered type fails safe either way: the loader refuses with a clean
`LoadError::UnknownNodeType` naming the missing id, and the type is simply
absent from `aura graph introspect --vocabulary` — never a silent partial
load.
### Worked example: `Scale`, a one-input, one-param node
This is the starter node `aura nodes new` writes into every freshly attached
node crate's `src/lib.rs` (`__NS__` is the node crate's namespace) —
copy-pasteable, and already exercised end to end by the scaffold's own tests:
```rust
use aura_core::{
Cell, Ctx, FieldSpec, Firing, Node, NodeSchema, ParamSpec, PortSpec, PrimitiveBuilder,
ScalarKind,
};
/// One-input scalar gain: emits `input * factor`. Emits `None` until its
/// input has a value (warm-up filter, C8).
pub struct Scale {
factor: f64,
out: [Cell; 1],
}
impl Scale {
pub fn new(factor: f64) -> Self {
Self { factor, out: [Cell::from_f64(0.0)] }
}
// Replace `doc` below when renaming this sample node — it must keep
// describing the node's own behaviour, not this scaffolding step.
pub fn builder() -> PrimitiveBuilder {
PrimitiveBuilder::new(
"my_lab::Scale",
NodeSchema {
inputs: vec![PortSpec {
kind: ScalarKind::F64,
firing: Firing::Any,
name: "value".into(),
}],
output: vec![FieldSpec { name: "value".into(), kind: ScalarKind::F64 }],
params: vec![ParamSpec { name: "factor".into(), kind: ScalarKind::F64 }],
// C29: every vocabulary entry ships a one-line meaning — the
// doc field is required (E0063 without it) and gated at load.
doc: "scalar gain: emits the input times the factor param",
},
|p| Box::new(Scale::new(p[0].f64())),
)
}
}
impl Node for Scale {
fn lookbacks(&self) -> Vec<usize> {
vec![1]
}
fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
let w = ctx.f64_in(0);
if w.is_empty() {
return None;
}
self.out[0] = Cell::from_f64(w[0] * self.factor);
Some(&self.out)
}
fn label(&self) -> String {
format!("my_lab::Scale({})", self.factor)
}
}
fn vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
match type_id {
"my_lab::Scale" => Some(Scale::builder()),
_ => None,
}
}
fn type_ids() -> &'static [&'static str] {
&["my_lab::Scale"]
}
aura_core::aura_project! {
namespace: "my_lab",
vocabulary: vocabulary,
type_ids: type_ids,
}
```
Reading this top to bottom against the three-part pattern: `Scale` is the
`Node` impl (one input, one param, a one-cycle lookback); `Scale::builder()`
is the `PrimitiveBuilder` recipe (`params: vec![ParamSpec { name: "factor",
... }]` declares the one bindable knob, and the build closure `|p|
Box::new(Scale::new(p[0].f64()))` reads it back positionally at bootstrap
time); and `vocabulary()` / `type_ids()` — wired up by `aura_project!` — are
the rostering. Once this compiles into the project's `cdylib`, `my_lab::Scale`
is a normal citizen of the vocabulary: `aura graph introspect --vocabulary`
lists it, `aura graph introspect --node my_lab::Scale` shows its port/param
shape exactly like a std node, and an op-script's `add` op can instantiate it
by that type id.
### Other worked examples, if you need a different arity
`aura-std` itself has several small, deliberately minimal nodes worth reading
alongside `Scale` for the shapes that recur most:
- [`crates/aura-std/src/const_node.rs`](../crates/aura-std/src/const_node.rs)
(`Const`) — another single-param node, but a *source*-shaped one: it needs
a driving input purely to be evaluated at all (a zero-input node never
fires in the total-push engine, C8), and ignores that input's actual value.
- [`crates/aura-std/src/div.rs`](../crates/aura-std/src/div.rs) (`Div`),
[`crates/aura-std/src/max.rs`](../crates/aura-std/src/max.rs) (`Max`),
[`crates/aura-std/src/min.rs`](../crates/aura-std/src/min.rs) (`Min`) —
two-input, paramless combinators (`inputs: vec![lhs, rhs]`, `params:
vec![]`, an eval that reads `ctx.f64_in(0)` and `ctx.f64_in(1)`).
- [`crates/aura-std/src/abs.rs`](../crates/aura-std/src/abs.rs) (`Abs`) — the
minimal case: one input, no params, `lookbacks() == vec![1]`.
## 1. Op-scripts — building a strategy blueprint by hand
An op-script is a JSON **array of ops**, replayed in order to construct a
node graph. `aura graph build` reads the op-script from **stdin** (there is
no file argument) and prints the canonical blueprint envelope to stdout:
```
$ aura graph build < smacross.json > blueprint.json
```
Nodes are referenced by an **identifier** (given by `add`, see below); ports
are dotted `<identifier>.<port>` on both sides of a wire.
### The ten ops
| op | JSON shape | does |
|---|---|---|
| `doc` | `{"op":"doc","text":<str>}` | declare the composite's one-line meaning (C29) — the op-script twin of the Rust builder's `.doc(...)`. Required before `aura graph register` accepts the product (the store refuses a doc-less composite); at most one per script (a second refuses: `a doc op may appear at most once`). The text is gated: empty or merely restating the composite's name refuses at register. |
| `source` | `{"op":"source","role":<str>,"kind":<ScalarKind>}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). |
| `input` | `{"op":"input","role":<str>}` | reserve an open root **input** role (kind inferred from the slots it feeds) — the formal parameter of an **open pattern**, a fragment meant to be wired by an *enclosing* graph. An open pattern builds and registers like any blueprint. Running it standalone is governed by the ordinary run gates: the harness binds input roles to archive columns **by name** (C26), so a pattern whose roles match the data runs as-is; what refuses, by name, is a role the harness cannot bind — or the run surface's other gates (a signal without a bias output or tap; free knobs). |
| `add` | `{"op":"add","type":<TypeId>,"name":<str>?,"args":{<arg>:<str>}?,"bind":{<param>:<Scalar>}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`) — see §0 below for how a type gets into that vocabulary in the first place. `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `args` (#271) configures an **arg-bearing** type's structural, non-scalar construction (`Session`'s timezone/open, `LinComb`/`CostSum`'s arity) — a closed table of string values, applied BEFORE `bind`; `aura graph introspect --node <T>` lists a type's declared `arg` rows. `bind` sets zero or more of the (now real) params. |
| `use` | `{"op":"use","ref":{"content_id":<id-or-prefix>}\|{"name":<label>},"name":<str>?,"bind":{<path>:<Scalar>}?}` | splice a **registered blueprint** into the graph as a nested composite under an instance identifier (`name`; default: the stored composite's own name). The ref resolves by content id (full or unique prefix) or by a register-time label; `graph build` echoes every resolution (`aura: note: use "…": … -> <id>`) so the exact id can be pinned. The instance's open input roles wire as `<instance>.<role>`, its outputs as `<instance>.<field>`; its open params surface path-qualified (`<instance>.<node>.<param>`), sweepable (ganging an instance's params is not yet supported — the `gang` op refuses a composite instance's member path). A doc-less stored source refuses at the op (C29). The emitted blueprint contains the subgraph *inline* — no reference survives into the artifact. |
| `feed` | `{"op":"feed","role":<str>,"into":[<port>, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). |
| `connect` | `{"op":"connect","from":<port>,"to":<port>}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). |
| `expose` | `{"op":"expose","from":<port>,"as":<str>}` | promote an interior output field to a boundary output under the alias `as` — a real *alias* (a terminal boundary name, not a referenceable identifier like `add`'s `name`). Together with `tap`, one of the two ops whose `as` key is a terminal name rather than an identifier. |
| `tap` | `{"op":"tap","from":<port>,"as":<str>}` | declare a **measurement tap** on an interior output field under the name `as` — the output-side twin of `expose` (a recorded observation point, not a boundary output; a `Composite.taps` entry, C27). A single `aura run` constructs a recorder at each declared tap and persists its per-cycle series as a `ColumnarTrace`; a sweep leaves it inert. Tap names are their own namespace and must be unique (a second `tap` under one name refuses: `duplicate tap name`). |
| `gang` | `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}` | Fuse two or more sibling params into ONE public knob: the member addresses leave the sweepable param space and `as` replaces them; the bound or swept value fans out to every member at bootstrap. Members must share one scalar kind and stay open (un-bound). |
Value forms are the typed-tag representations used everywhere in this
family of artifacts:
- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` /
`{"Timestamp":<i64 ms>}`;
- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` |
`"Bool"` | `"Timestamp"`.
### Worked example: an SMA-crossover bias strategy
```json
[
{"op": "source", "role": "price", "kind": "F64"},
{"op": "add", "type": "SMA", "name": "fast"},
{"op": "add", "type": "SMA", "name": "slow"},
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
{"op": "add", "type": "Sub", "name": "sub"},
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
{"op": "add", "type": "Bias", "name": "bias"},
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
{"op": "expose", "from": "bias.bias", "as": "bias"}
]
```
(`fast.length`, `slow.length`, and `bias.scale` are all left unbound here on
purpose — three open campaign axes, see §3. This is the milestone fieldtest
corpus's own example, verified below; byte-identical to the on-disk
`fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.)
A `bind` in an `add` op pins that param to a value and removes it from the
**open** param space (`--params`): a bound param is a **default** (#246) — a
run uses it as-is, while any campaign axis or `aura sweep --axis` naming it
re-opens it for that family and binds it per cell. `--list-axes` lists it
after the open knobs as `<name>:<KIND> default=<value>`. `bind` is for a
value the strategy carries by default; leave a param unbound, as all three
are here, to make binding it mandatory for every sweep.
A param therefore has three states: **open** (an axis every sweep MUST bind),
**bound** (a default any axis MAY override, #246), and **ganged** (open, but
fused with its siblings under ONE public knob declared by a `gang` op; the
member addresses are unbindable and only the gang's own single-segment name —
e.g. `channel_length` — appears in `--list-axes`).
### Worked example: declaring a measurement tap
A **declared tap** (C27) names an interior output wire as a recorded
observation point — the output-side twin of `expose`, but a *measurement*
rather than a boundary output. It is authored by the same `"node.field"`
addressing every other op uses; there is no raw node index. Extending the
crossover above to record the raw fastslow spread (the signal *before* the
bias scaling) under the name `spread`:
```json
[
{"op": "source", "role": "price", "kind": "F64"},
{"op": "add", "type": "SMA", "name": "fast"},
{"op": "add", "type": "SMA", "name": "slow"},
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
{"op": "add", "type": "Sub", "name": "sub"},
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
{"op": "add", "type": "Bias", "name": "bias"},
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
{"op": "tap", "from": "sub.value", "as": "spread"},
{"op": "expose", "from": "bias.bias", "as": "bias"}
]
```
The built blueprint carries a `taps` array naming the resolved wire
(`{"name":"spread","from":{"node":<sub's index>,"field":0}}` — the name is
addressed, the index is resolved *for* you). A single `aura run <blueprint>`
then constructs a recorder at that point and persists the `spread` series as a
`ColumnarTrace` in the run's trace store, chartable by its name. A tap is inert
in a sweep (no per-cell recorder) — it is a single-run observation surface.
(`sub.value` is tapped here purely to illustrate; any interior output field is
a legal tap source, and a producer needs no other consumer to be tapped.)
### Session anchoring: the `SessionFrankfurt` preset
To anchor logic to the trading session, the closed vocabulary ships a
`SessionFrankfurt` node — the Frankfurt cash open (09:00 `Europe/Berlin`)
baked in, DST-correct. Its schema:
- **input** `trigger: f64` — wired from a once-per-bar stream (e.g. a 15m
resampler's `close`); its *value is ignored*, it only clocks the node to fire
once per completed bar;
- **param** `period_minutes: i64` — the bar width to index by (conventionally
`15`, matching the resampler);
- **output** `bars_since_open: i64` — the count of completed `period_minutes`
bars since the local session open, in tz-aware local wall-clock time.
Because a resampler emits on exact `:00/:15/:30/:45` boundaries and `ctx.now()`
at emission is the *just-closed* bar's close instant, in-session bar closes
read exact positive multiples: the 09:0009:15 bar closes at **09:15 → 1**,
09:3009:45 at **09:45 → 3**, 10:0010:15 at **10:15 → 5**. Pre-open instants
give `<= 0` (non-actionable). To gate on "the third bar of the session",
compare `bars_since_open` against a constant (an `EqConst(==3)`); there is no
separate in-session bool — `bars_since_open` alone is the contract. DST is
handled by `chrono-tz`, so the same local minute reads the same index in
summer (CEST) and winter (CET).
The index is derived from the clock (`ctx.now()`), never counted from trigger
firings — so the trigger's cadence only sets how *often* the node emits, not
what it emits. Feeding a denser stream (e.g. a raw m1 `price`) is equally
valid: the node fires once per input tick and reports the same
`period_minutes`-bar index throughout that bar. The once-per-bar wiring above
is the convention when you want exactly one emission per completed bar; it is
not a correctness requirement.
Wire it like any node —
`{"op":"add","type":"SessionFrankfurt","bind":{"period_minutes":{"I64":15}}}`,
feed its `trigger` from a once-per-bar stream, and read `bars_since_open`.
**Any other timezone/open: `Session` + `args` (#271).** `SessionFrankfurt`
stays baked sugar for the one Frankfurt open; the canonical path for ANY IANA
timezone and local open time is the base `Session` type through the typed
construction-args channel — `args` configures it BEFORE `bind`:
```json
{"op":"add","type":"Session","name":"ny",
"args":{"tz":"America/New_York","open":"09:30"},
"bind":{"period_minutes":{"I64":15}}}
```
`tz` is `chrono_tz`'s own exact, case-sensitive IANA name (`"berlin"` refuses;
`"Europe/Berlin"` accepts); `open` is strict zero-padded local `HH:MM`
(`"9:30"` refuses; `"09:30"` accepts); a `Count` arg (`LinComb`/`CostSum`'s
arity) is a plain positive decimal — no sign, no leading zeros (`"03"`
refuses; `"3"` accepts). The accepted string IS the canonical form, no
normalization — and note the deliberate asymmetry: each kind's canonical
form is its domain's conventional notation, fixed-width for wall-clock
times, minimal for numbers. Giving `Session` no `args` at all refuses the `add`
op (`node <name> is missing required arg "tz"`) — an arg-bearing type never
silently enters the graph half-configured. `aura graph introspect --node
Session` lists both declared `arg` rows with their kind and hint text before
you write the JSON.
### Commands
```
$ aura graph build < smacross.json
{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...],
"input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}}
```
The built blueprint renders visually, too: `aura graph blueprint.json` emits an
interactive HTML DAG so a mis-wire is visible before any run (`aura graph` with
no file renders the built-in sample; a named-but-unreadable file is a usage
error). Introspection is build-free wherever possible:
```
$ aura graph introspect --vocabulary # one node type per line
$ aura graph introspect --node SMA # ports + params of one type
SMA
in series:F64
out value:F64
param length:I64 (bind {"I64": <v>})
$ aura graph introspect --unwired < partial.json # open slots of a partial op-script
sub.rhs:F64
$ aura graph introspect --content-id smacross.json # SHA-256 of the canonical form
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
$ aura graph introspect --content-id smacross.json --identity-id # + debug-name-blind identity id, combinable
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4
$ aura graph introspect --params smacross.json # the raw param-space namespace (what a campaign's axes bind against)
fast.length:I64
slow.length:I64
bias.scale:F64
```
These printed names are the **one** axis namespace (#328): op-script params, a
campaign document's `strategies[].axes` keys (§3), and `aura sweep <blueprint>
--list-axes` / `--axis` (glossary `sweep`) all speak the same raw
`<node>.<param>` form — `--params` and `--list-axes` are line-identical (open
params bare, bound params trailing `default=<value>`), and every discovered
name is verbatim legal as a document axis key, bound params included (#246's
re-open contract):
```
$ aura sweep smacross.json --axis fast.length=3:9:2 # synthetic: raw works
$ aura sweep smacross.json --axis bias.scale=0.25,0.5 --real ... # real: raw works
```
The older `<blueprint>.<node>.<param>` wrapped form (e.g. `graph.fast.length`,
what pre-#328 transcripts and `--list-axes` output used to print) is retired
from the surface; naming it refuses with a translation pointer to the raw
candidate, on both intake seams:
```
$ aura sweep smacross.json --axis graph.fast.length=...
aura: axis "graph.fast.length": axis names are raw node.param paths —
use "fast.length" (the wrapped --list-axes form was retired, #328)
$ aura campaign validate wrapped.json # a document quoting an old transcript
aura: campaign references do not resolve:
strategy 9f3a: axis "graph.fast.length" is not in the param space; axis
names are raw node.param paths — did you mean "fast.length"?
```
A ganged knob's raw address has one path segment less than a member address
would — it sits at the composite's own level, like a role name (e.g.
`channel_length`, not `channel_hi.length`).
`--content-id`, `--identity-id`, `--params`, and `graph register` all accept
**either** shape: the raw op-script array or an already-built `#155`
blueprint envelope (object) — shape-discriminated automatically, so you
never have to build first just to hash or register:
```
$ aura graph register smacross.json # inside a project (Aura.toml present)
registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json)
```
The printed content id is the address a campaign document's `strategies[].ref`
points at (§3).
`graph register <file> --name <label>` additionally records a **blueprint
label** — a legible, re-pointable handle for the `use` op's `{"name":…}`
ref form (re-registering under the same label repoints it; the echo names
the previous target). `aura graph introspect --registered` lists the
labels with their id prefix and root doc line — the discovery surface for
what `use` can reference.
## 2. Process documents — a validation methodology (role 5)
A process document is a named, versionable pipeline of **std stage blocks**
over a shared metric vocabulary. Discover the block vocabulary and each
block's typed slots headlessly:
```
$ aura process introspect --vocabulary
std::sweep evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
std::gate filter survivors by a conjunction of typed metric predicates
std::walk_forward rolling in-sample optimize + out-of-sample test
std::monte_carlo R-bootstrap over realised R (terminal annotator): ...
std::generalize cross-instrument worst-case floor (terminal annotator): ...
std::grid enumerate the axis grid as parameter points for the next stage (enumerate-only leading stage): ...
$ aura process introspect --block std::sweep
std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
metric optional, metric name (see metric_vocabulary)
select optional, select rule: argmax | plateau:mean | plateau:worst
deflate optional, bool
```
`metric` and `select` form one **selection group**: all-or-nothing (a
document naming one without the other is refused, "the selection group is
all-or-nothing"), `deflate` composes only when the group is present. Omit
the group entirely for a **selection-free sweep** — the family itself is
the result, no winner is chosen. A selection-free sweep is only legal as
the pipeline's *last* stage (the executor's preflight refuses one followed
by any other block, since a downstream stage would have no nominee to
consume):
```json
{
"format_version": 1,
"kind": "process",
"name": "explore-only-sweep",
"pipeline": [ { "block": "std::sweep" } ]
}
```
The executor records this stage's family (every member run) but no
`StageSelection` and no nominee — recording a winner here would fabricate a
selection intent the document never expressed.
### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize)
```json
{
"format_version": 1,
"kind": "process",
"name": "mra-full-v2-sweep-gate-wf-mc-generalize",
"description": "Full v2 anti-false-discovery pipeline.",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::generalize", "metric": "expectancy_r" }
]
}
```
(`description` is optional; everything else in the envelope — `format_version`,
`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired`
over a bare `{}` will tell you.)
```
$ aura process validate mra_2_process_full_v2.json
process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates
$ aura process register mra_2_process_full_v2.json # inside a project
registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json)
```
### The metric vocabulary — discoverable, not hand-copied
`aura process introspect --metrics` (equivalently `aura campaign introspect
--metrics` — same roster) enumerates every metric name annotated with its
role, so you never have to guess which metrics a `select`/`gate`/`generalize`
field will accept:
```
$ aura process introspect --metrics
expectancy_r rankable | gate | generalize
win_rate gate
avg_win_r gate
avg_loss_r gate
profit_factor gate
max_r_drawdown gate
sqn rankable | gate | generalize
sqn_normalized rankable | gate | generalize
net_expectancy_r rankable | gate | generalize
n_trades gate
n_open_at_end gate
total_pips rankable | gate
max_drawdown rankable | gate
bias_sign_flips rankable | gate
deflated_score annotation
overfit_probability annotation
neighbourhood_score annotation
```
Read the three tags as three separate questions about one metric name:
- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward`
`select` metric (the field a winner is chosen by)? This is the small
subset every strategy run always produces cheaply.
- **`gate`** — can this name be used in a `std::gate` predicate (a per-member
filter)? This roster is a superset of `rankable` — most emitted metrics
are filterable even when they make a poor ranking criterion.
- **`| generalize`** suffix — is this name usable as `std::generalize`'s
`metric` (needs an R-expectancy-shaped metric to floor across instruments)?
That is why, for example, `profit_factor` shows up tagged only `gate`: every
member's metrics table carries it (so you can gate on it, e.g. "keep only
`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster,
so a `std::sweep` with `"select": "argmax"` cannot select on it, and
`std::generalize` cannot floor across instruments on it either. Attempting
either produces the same intrinsic refusal that names the metric and points
back at this verb (`aura process introspect --metrics`) rather than leaving
you to search the glossary.
## 3. Campaign documents — experiment intent over instruments and windows
A campaign document names the data (instruments × windows), one or more
strategies (by blueprint content/identity id) with their param axes, a
process reference, and what to persist/emit.
```
$ aura campaign introspect --unwired bare.json # bare.json contains just {}
open slot: format_version (required, must be 1)
open slot: kind (required, must be "campaign")
open slot: name (required, string)
open slot: data (required section: instruments + windows)
open slot: risk (optional, list of stop regimes { vol: { length, k } } | { vol_tf: { period_minutes, length, k } }; absent = one default regime)
open slot: cost (optional, list of cost models { constant: { cost_per_trade } } | { vol_slippage: { slip_vol_mult } } | { carry: { carry_per_cycle } }; absent = zero cost, net = gross)
open slot: strategies (required, non-empty list of { ref, axes })
open slot: process.ref (required, content id of a process document)
open slot: seed (required, non-negative integer)
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
```
### Worked example: two instruments, one strategy, four axis points, two stop regimes
```json
{
"format_version": 1,
"kind": "campaign",
"name": "mra-ger40-fra40-smacross-full-v2",
"seed": 42,
"data": {
"instruments": ["GER40", "FRA40"],
"windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
},
"risk": [
{ "vol": { "length": 3, "k": 1.5 } },
{ "vol": { "length": 3, "k": 3.0 } }
],
"cost": [
{ "constant": { "cost_per_trade": 0.02 } },
{ "vol_slippage": { "slip_vol_mult": 0.1 } }
],
"strategies": [
{
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
"axes": {
"fast.length": { "kind": "I64", "values": [2, 4] },
"slow.length": { "kind": "I64", "values": [8, 16] },
"bias.scale": { "kind": "F64", "values": [0.5] }
}
}
],
"process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } },
"presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] }
}
```
The `strategies[].ref.content_id` and `process.ref.content_id` are exactly
the ids printed by `aura graph register` / `aura process register` above —
a campaign never inlines a strategy or process body, only its content id.
Each axis name (`fast.length`, …) must name an open param of the referenced
blueprint (`aura graph introspect --params`, §1) and declare that param's
`ScalarKind`.
The optional `risk` list is the campaign's structural risk axis: every cell
runs under every listed stop regime, so cells differ by execution discipline,
never by signal — the regime's stop defines the risk unit R. Absent or empty,
the matrix runs one implicit default regime (the same vol regime the
orchestration verbs bind when their stop flags are omitted).
The optional `cost` list is the campaign's cost model (#234): each entry
charges the trade stream in R — `constant` per closed trade, `vol_slippage`
proportional to local volatility per closed trade, `carry` per held cycle —
and the charges sum into the net curve, so every member reports net metrics
beside gross and the `net_r_equity` tap records the cost-dragged curve.
Absent or empty, the cost model is zero and net equals gross. The bound cost
knobs are stamped on each member's manifest (`cost[k].<knob>` params), so a
costed family reproduces bit-identically like any other.
### Validate — three tiers, honest degradation
```
$ aura campaign validate mra_3_campaign_full_v2.json # outside any project
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
referential checks skipped (no Aura.toml found up from /home/…)
```
Inside a project, with the strategy blueprint and process both already
registered, the same command runs two further tiers:
```
$ aura campaign validate mra_3_campaign_full_v2.json # inside a project, refs registered
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
campaign document valid (referential): all references resolve, axes are in the param space
campaign document valid (executable): pipeline shape and static guards pass
```
- **intrinsic** — the document's own shape is well-formed (always checked).
- **referential** — every `ref` resolves in the project's store and every
axis names a real, correctly-typed open param (needs a project).
- **executable** — the process pipeline's static guards pass against this
campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a
data-free preflight, so "valid" here means "runnable" (needs a project;
it does not fetch or touch market data).
### Register and run
```
$ aura campaign register mra_3_campaign_full_v2.json
registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json)
$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd
{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}}
{"family_id":"42edebd2-0-GER40-w0-s0-0-r1","report":{...}}
{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42,
"cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],
"regime":{"vol":{"length":3,"k":1.5}},"stages":[
{"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]},
{"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]},
{"strategy":"597d719b…","instrument":"GER40","regime":{"vol":{"length":3,"k":3.0}},"regime_ordinal":1,...},
{"strategy":"597d719b…","instrument":"FRA40",...},
{"strategy":"597d719b…","instrument":"FRA40","regime_ordinal":1,...}],
"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,
"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
"worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]},
"winners":[...]},
{"strategy_ordinal":0,"window_ordinal":0,"regime_ordinal":1,...}],
"trace_name":"42edebd2-0"}}
```
With two stop regimes every (instrument, window) cell runs twice — the four
cells above are the "4 cell(s)" the validate summary counted. The second
regime's family ids and trace dirs carry the `-r1` ordinal suffix (the
default/first regime stays unsuffixed), each cell record names its regime,
and generalization is keyed per regime — regimes are compared, never pooled.
`aura campaign run` is register-then-run sugar for a `.json` file, but the
canonical address is always the content id — running a bare file the first
time registers it implicitly. `aura campaign runs` lists stored
realizations; `aura campaign runs <id>` dumps the bare stored record(s) (not
the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps`
is non-empty, the run also persists the named taps under
`runs/traces/<trace_name>/…`, chartable with `aura chart`. The
`sweep`/`walkforward --trace` analog persists every member's taps as a
family charted the same way, by the handle the run prints (members keyed
`<cell>/<member>`) — also by the `--trace <NAME>` you chose, when that name
uniquely names one recorded run (`aura chart <NAME>` resolves it against the
stored campaign documents; a name reused across runs refuses rather than
guessing which one you mean).
Exit codes: a clean run exits 0; a usage error exits 2; a refusal before any
cell runs (invalid document, missing project, unresolvable strategy) exits 1;
a run that **completes with one or more failed cells** exits 3 — the run record
and every healthy cell persist, and the failed cells are named on stderr
(#272). A **gate-emptied cell** — a `std::gate` stage that filters out every
member — is a *successful* cell, not a failed one: the gate legitimately
answered "no survivors", so the run still exits 0, the cell's realization is
recorded as truncated at that stage, and an informational `aura:`-prefixed
note names the cell on stderr. Exit 3 is reserved for cells that could not be
evaluated (faults), never for an empty-but-valid result.