d7874e2569
Per-cycle fieldtest over the #246 surface, consumer-perspective, public interface only: the scaffold quickstart end to end (aura new -> run -> --list-axes -> override sweep), --list-axes on a partially-open blueprint, an override family reproduced bit-identically (3/3 members), and the error surface (wrong axis name, kind-mismatched value — both exit 2). Corpus under fieldtests/cycle-246-bound-override/. Dispositions: the one bug (F6) is fixed in this commit — authoring-guide §1 still asserted a bound param 'never shows up in --list-axes' and 'no aura sweep --axis can reopen it', contradicting the shipped #246 semantics and the C12 amendment; the three-states paragraph now reads open = must-bind, bound = overridable default. The friction finding (F5, KindMismatch/MissingKnob printed as raw Debug structs beside polished prose) is filed as #247. refs #246
606 lines
30 KiB
Markdown
606 lines
30 KiB
Markdown
# Authoring guide: op-scripts, process documents, campaign documents
|
||
|
||
`docs/project-layout.md` describes the shape of a project and the arc of a
|
||
research session; this document is the practical companion for the three
|
||
JSON artifact kinds you author headlessly along that arc:
|
||
|
||
1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint)
|
||
through `aura graph build`;
|
||
2. a **process document** (role 5) — a named validation/eval methodology
|
||
through `aura process validate|introspect|register`;
|
||
3. a **campaign document** (role 6b) — experiment intent (instruments ×
|
||
windows × strategy × axes × process) through `aura campaign
|
||
validate|introspect|register|run`.
|
||
|
||
Each section below is a worked, verified example — every command shown was
|
||
run against this repo and the output is transcribed, not invented. The *why*
|
||
of this three-artifact split (closed-vocabulary data, never a logic DSL)
|
||
lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the
|
||
glossary; this document only teaches the *shape*.
|
||
|
||
All three artifact kinds above assume the node **types** they reference
|
||
already exist. §0 covers the one piece of this arc that is Rust, not data:
|
||
adding a new node type in the first place.
|
||
|
||
## 0. Authoring a new node in Rust
|
||
|
||
An op-script's `add` op (§1) instantiates a node **type** from the closed
|
||
vocabulary — but that vocabulary itself is not data, it is compiled Rust.
|
||
Adding a new node type (a new indicator, combinator, or signal primitive) is
|
||
role-2 work: you write a `Node` implementation, describe it to the bootstrap
|
||
with a `PrimitiveBuilder` recipe, and roster the type id so the loader can
|
||
find it by name. This is the one seam in the whole arc where the answer is
|
||
"write Rust", not "write JSON" (C17/C20) — and it is a small, fixed shape,
|
||
the same shape every node in `aura-std` already follows.
|
||
|
||
### Where the code goes
|
||
|
||
- **The project's own signal doesn't need a crate by default.** `aura new`
|
||
scaffolds a **data-only** project (`docs/project-layout.md`, "A project
|
||
repo (two tiers)") — a strategy over the std vocabulary is a
|
||
blueprint/campaign document, not Rust: the scaffold ships **one** closed
|
||
`signal.json` starter that serves both verbs — `aura run` uses its bound
|
||
values as-is, `aura sweep --axis <bp>.fast.length=2,4,8` overrides them
|
||
(bound = default, not fixed); `--list-axes` lists the open knobs and the
|
||
bound defaults alike, so every override target is discoverable.
|
||
- **A project-specific *native* node type** — the moment §0's three-part
|
||
pattern is actually needed — lives in an attached **node crate**:
|
||
`aura nodes new <name>` scaffolds a sibling cdylib crate (`Cargo.toml` +
|
||
`src/lib.rs`, registered under the project's own `<namespace>::` prefix)
|
||
and appends its path to the project's `Aura.toml [nodes]` section.
|
||
- **A block promoted to universal** — reused across projects and folded into
|
||
the engine itself — lives in `crates/aura-std/src/`, unprefixed, and is
|
||
rostered in `crates/aura-std/src/vocabulary.rs` instead of a node crate's
|
||
`vocabulary()` function. The pattern below is identical either way; only
|
||
the rostering call site differs (see "Rostering the type" below).
|
||
|
||
### The three-part pattern
|
||
|
||
Every node type — std or project-local — is three things:
|
||
|
||
1. **A `Node` implementation.** A plain struct holding whatever state the
|
||
node needs between cycles (often just its output cell), plus three
|
||
methods: `lookbacks()` (how many past values, per input, the node reads —
|
||
`vec![1]` for a node that only ever looks at the current cycle),
|
||
`eval(ctx) -> Option<&[Cell]>` (the per-cycle computation — return `None`
|
||
until every input the node needs has fired at least once; this warm-up
|
||
filter is what keeps a downstream consumer from ever observing a
|
||
fabricated value, C8), and `label()` (a short, human-readable debug
|
||
string — not the type id used for serialization, see below). The input
|
||
window `ctx.f64_in(i)` hands those past values back **financial-style:
|
||
index 0 is the newest cycle, index `k` is `k` cycles back** — so a
|
||
`lookbacks()` of `vec![n]` makes `w[0]..=w[n-1]` the last `n` values,
|
||
newest first, and a lookback-spanning node (momentum, ATR, RSI) reads its
|
||
span as `w[0]` vs `w[length]`, never the other way round.
|
||
2. **A `PrimitiveBuilder` recipe.** A `builder()` constructor that pairs a
|
||
`NodeSchema` (its input ports as `PortSpec`, its output fields as
|
||
`FieldSpec`, and its bindable params as `ParamSpec`) with a **build
|
||
closure** `|p| Box::new(Type::new(...))` that reads bound param values out
|
||
of `p` positionally (`p[0].f64()`, `p[1].i64()`, …, matching the `params`
|
||
order in the schema) and constructs the node. This one recipe is what lets
|
||
the bootstrap turn a blueprint's *serialized* param values into a live,
|
||
concrete node — a project or op-script never constructs a node directly.
|
||
3. **Rostering the type id.** The recipe is useless until something maps the
|
||
serialized type-id string (e.g. `"Scale"`, `"my_lab::ThirdCandle"`) back to
|
||
its `builder()`. This is a closed, compiled-in `match` — never a dynamic
|
||
registry (domain invariant 9) — so a node exists in the vocabulary only if
|
||
its type id is added to exactly one of these two match tables:
|
||
- **Project-side:** the `vocabulary()` / `type_ids()` pair the
|
||
`aura_core::aura_project!` macro wires up (every `aura new` scaffold
|
||
emits a starter pair — see the worked example below). Add one match arm
|
||
to `vocabulary()` and one entry to `type_ids()`'s slice.
|
||
- **Std-side** (only when promoting a node into `aura-std` itself): one
|
||
line in the `std_vocabulary_roster!` macro invocation in
|
||
[`crates/aura-std/src/vocabulary.rs`](../crates/aura-std/src/vocabulary.rs)
|
||
— `"TypeId" => Type,` — which expands into both the resolver `match` and
|
||
the enumerable type-id list, so the two surfaces cannot drift apart.
|
||
An unrostered type fails safe either way: the loader refuses with a clean
|
||
`LoadError::UnknownNodeType` naming the missing id, and the type is simply
|
||
absent from `aura graph introspect --vocabulary` — never a silent partial
|
||
load.
|
||
|
||
### Worked example: `Scale`, a one-input, one-param node
|
||
|
||
This is the starter node `aura nodes new` writes into every freshly attached
|
||
node crate's `src/lib.rs` (`__NS__` is the node crate's namespace) —
|
||
copy-pasteable, and already exercised end to end by the scaffold's own tests:
|
||
|
||
```rust
|
||
use aura_core::{
|
||
Cell, Ctx, FieldSpec, Firing, Node, NodeSchema, ParamSpec, PortSpec, PrimitiveBuilder,
|
||
ScalarKind,
|
||
};
|
||
|
||
/// One-input scalar gain: emits `input * factor`. Emits `None` until its
|
||
/// input has a value (warm-up filter, C8).
|
||
pub struct Scale {
|
||
factor: f64,
|
||
out: [Cell; 1],
|
||
}
|
||
|
||
impl Scale {
|
||
pub fn new(factor: f64) -> Self {
|
||
Self { factor, out: [Cell::from_f64(0.0)] }
|
||
}
|
||
|
||
pub fn builder() -> PrimitiveBuilder {
|
||
PrimitiveBuilder::new(
|
||
"my_lab::Scale",
|
||
NodeSchema {
|
||
inputs: vec![PortSpec {
|
||
kind: ScalarKind::F64,
|
||
firing: Firing::Any,
|
||
name: "value".into(),
|
||
}],
|
||
output: vec![FieldSpec { name: "value".into(), kind: ScalarKind::F64 }],
|
||
params: vec![ParamSpec { name: "factor".into(), kind: ScalarKind::F64 }],
|
||
},
|
||
|p| Box::new(Scale::new(p[0].f64())),
|
||
)
|
||
}
|
||
}
|
||
|
||
impl Node for Scale {
|
||
fn lookbacks(&self) -> Vec<usize> {
|
||
vec![1]
|
||
}
|
||
fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
|
||
let w = ctx.f64_in(0);
|
||
if w.is_empty() {
|
||
return None;
|
||
}
|
||
self.out[0] = Cell::from_f64(w[0] * self.factor);
|
||
Some(&self.out)
|
||
}
|
||
fn label(&self) -> String {
|
||
format!("my_lab::Scale({})", self.factor)
|
||
}
|
||
}
|
||
|
||
fn vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
|
||
match type_id {
|
||
"my_lab::Scale" => Some(Scale::builder()),
|
||
_ => None,
|
||
}
|
||
}
|
||
|
||
fn type_ids() -> &'static [&'static str] {
|
||
&["my_lab::Scale"]
|
||
}
|
||
|
||
aura_core::aura_project! {
|
||
namespace: "my_lab",
|
||
vocabulary: vocabulary,
|
||
type_ids: type_ids,
|
||
}
|
||
```
|
||
|
||
Reading this top to bottom against the three-part pattern: `Scale` is the
|
||
`Node` impl (one input, one param, a one-cycle lookback); `Scale::builder()`
|
||
is the `PrimitiveBuilder` recipe (`params: vec![ParamSpec { name: "factor",
|
||
... }]` declares the one bindable knob, and the build closure `|p|
|
||
Box::new(Scale::new(p[0].f64()))` reads it back positionally at bootstrap
|
||
time); and `vocabulary()` / `type_ids()` — wired up by `aura_project!` — are
|
||
the rostering. Once this compiles into the project's `cdylib`, `my_lab::Scale`
|
||
is a normal citizen of the vocabulary: `aura graph introspect --vocabulary`
|
||
lists it, `aura graph introspect --node my_lab::Scale` shows its port/param
|
||
shape exactly like a std node, and an op-script's `add` op can instantiate it
|
||
by that type id.
|
||
|
||
### Other worked examples, if you need a different arity
|
||
|
||
`aura-std` itself has several small, deliberately minimal nodes worth reading
|
||
alongside `Scale` for the shapes that recur most:
|
||
|
||
- [`crates/aura-std/src/const_node.rs`](../crates/aura-std/src/const_node.rs)
|
||
(`Const`) — another single-param node, but a *source*-shaped one: it needs
|
||
a driving input purely to be evaluated at all (a zero-input node never
|
||
fires in the total-push engine, C8), and ignores that input's actual value.
|
||
- [`crates/aura-std/src/div.rs`](../crates/aura-std/src/div.rs) (`Div`),
|
||
[`crates/aura-std/src/max.rs`](../crates/aura-std/src/max.rs) (`Max`),
|
||
[`crates/aura-std/src/min.rs`](../crates/aura-std/src/min.rs) (`Min`) —
|
||
two-input, paramless combinators (`inputs: vec![lhs, rhs]`, `params:
|
||
vec![]`, an eval that reads `ctx.f64_in(0)` and `ctx.f64_in(1)`).
|
||
- [`crates/aura-std/src/abs.rs`](../crates/aura-std/src/abs.rs) (`Abs`) — the
|
||
minimal case: one input, no params, `lookbacks() == vec![1]`.
|
||
|
||
## 1. Op-scripts — building a strategy blueprint by hand
|
||
|
||
An op-script is a JSON **array of ops**, replayed in order to construct a
|
||
node graph. `aura graph build` reads the op-script from **stdin** (there is
|
||
no file argument) and prints the canonical blueprint envelope to stdout:
|
||
|
||
```
|
||
$ aura graph build < smacross.json > blueprint.json
|
||
```
|
||
|
||
Nodes are referenced by an **identifier** (given by `add`, see below); ports
|
||
are dotted `<identifier>.<port>` on both sides of a wire.
|
||
|
||
### The seven ops
|
||
|
||
| op | JSON shape | does |
|
||
|---|---|---|
|
||
| `source` | `{"op":"source","role":<str>,"kind":<ScalarKind>}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). |
|
||
| `input` | `{"op":"input","role":<str>}` | reserve an open root **input** role (kind inferred from the slots it feeds) — for a fragment meant to be wired by an *enclosing* graph. A standalone document built with `aura graph build` finalizes as a closed root, so an `input` role that is never bound refuses at the end: `finalize: root input role <name> is unbound`. |
|
||
| `add` | `{"op":"add","type":<TypeId>,"name":<str>?,"bind":{<param>:<Scalar>}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`) — see §0 below for how a type gets into that vocabulary in the first place. `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `bind` sets zero or more of its params. |
|
||
| `feed` | `{"op":"feed","role":<str>,"into":[<port>, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). |
|
||
| `connect` | `{"op":"connect","from":<port>,"to":<port>}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). |
|
||
| `expose` | `{"op":"expose","from":<port>,"as":<str>}` | promote an interior output field to a boundary output under the alias `as` — the only op whose name key is a real *alias* (contrast `add`'s `name`, which is an identifier, not a rename). |
|
||
| `gang` | `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}` | Fuse two or more sibling params into ONE public knob: the member addresses leave the sweepable param space and `as` replaces them; the bound or swept value fans out to every member at bootstrap. Members must share one scalar kind and stay open (un-bound). |
|
||
|
||
Value forms are the typed-tag representations used everywhere in this
|
||
family of artifacts:
|
||
|
||
- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` /
|
||
`{"Timestamp":<i64 ms>}`;
|
||
- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` |
|
||
`"Bool"` | `"Timestamp"`.
|
||
|
||
### Worked example: an SMA-crossover bias strategy
|
||
|
||
```json
|
||
[
|
||
{"op": "source", "role": "price", "kind": "F64"},
|
||
{"op": "add", "type": "SMA", "name": "fast"},
|
||
{"op": "add", "type": "SMA", "name": "slow"},
|
||
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
|
||
{"op": "add", "type": "Sub", "name": "sub"},
|
||
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
|
||
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
|
||
{"op": "add", "type": "Bias", "name": "bias"},
|
||
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
|
||
{"op": "expose", "from": "bias.bias", "as": "bias"}
|
||
]
|
||
```
|
||
|
||
(`fast.length`, `slow.length`, and `bias.scale` are all left unbound here on
|
||
purpose — three open campaign axes, see §3. This is the milestone fieldtest
|
||
corpus's own example, verified below; byte-identical to the on-disk
|
||
`fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.)
|
||
|
||
A `bind` in an `add` op pins that param to a value and removes it from the
|
||
**open** param space (`--params`): a bound param is a **default** (#246) — a
|
||
run uses it as-is, while any campaign axis or `aura sweep --axis` naming it
|
||
re-opens it for that family and binds it per cell. `--list-axes` lists it
|
||
after the open knobs as `<name>:<KIND> default=<value>`. `bind` is for a
|
||
value the strategy carries by default; leave a param unbound, as all three
|
||
are here, to make binding it mandatory for every sweep.
|
||
|
||
A param therefore has three states: **open** (an axis every sweep MUST bind),
|
||
**bound** (a default any axis MAY override, #246), and **ganged** (open, but
|
||
fused with its siblings under ONE public knob declared by a `gang` op; the
|
||
member addresses are unbindable and only the gang's own single-segment name —
|
||
wrapped like any knob, e.g. `graph.channel_length` — appears in `--list-axes`).
|
||
|
||
### Commands
|
||
|
||
```
|
||
$ aura graph build < smacross.json
|
||
{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...],
|
||
"input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}}
|
||
```
|
||
|
||
The built blueprint renders visually, too: `aura graph blueprint.json` emits an
|
||
interactive HTML DAG so a mis-wire is visible before any run (`aura graph` with
|
||
no file renders the built-in sample; a named-but-unreadable file is a usage
|
||
error). Introspection is build-free wherever possible:
|
||
|
||
```
|
||
$ aura graph introspect --vocabulary # one node type per line
|
||
$ aura graph introspect --node SMA # ports + params of one type
|
||
SMA
|
||
in series:F64
|
||
out value:F64
|
||
param length:I64 (bind {"I64": <v>})
|
||
$ aura graph introspect --unwired < partial.json # open slots of a partial op-script
|
||
sub.rhs:F64
|
||
$ aura graph introspect --content-id smacross.json # SHA-256 of the canonical form
|
||
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
|
||
$ aura graph introspect --content-id smacross.json --identity-id # + debug-name-blind identity id, combinable
|
||
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
|
||
41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4
|
||
$ aura graph introspect --params smacross.json # the raw param-space namespace (what a campaign's axes bind against)
|
||
fast.length:I64
|
||
slow.length:I64
|
||
bias.scale:F64
|
||
```
|
||
|
||
These printed names are the **raw param-space namespace**: op-script params
|
||
and a campaign document's `strategies[].axes` keys (§3) share this one raw
|
||
form. There is a third, *wrapped* surface: the dissolved `aura sweep
|
||
<blueprint> --axis` CLI (glossary `sweep`) accepts only the names `aura
|
||
sweep <blueprint> --list-axes` prints — the raw name prefixed with the
|
||
root-composite instance name, `graph.<param>` — never the raw form; the
|
||
campaign document the sweep sugar generates still stores the raw form
|
||
(#210):
|
||
|
||
```
|
||
raw (--params, campaign axes): fast.length
|
||
wrapped (--list-axes, --axis): graph.fast.length
|
||
```
|
||
|
||
A ganged knob's raw address has one path segment less than a member address
|
||
would — it sits at the composite's own level, like a role name (e.g.
|
||
`channel_length`, not `channel_hi.length`) — and wraps identically (`graph.channel_length`).
|
||
|
||
`--content-id`, `--identity-id`, `--params`, and `graph register` all accept
|
||
**either** shape: the raw op-script array or an already-built `#155`
|
||
blueprint envelope (object) — shape-discriminated automatically, so you
|
||
never have to build first just to hash or register:
|
||
|
||
```
|
||
$ aura graph register smacross.json # inside a project (Aura.toml present)
|
||
registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json)
|
||
```
|
||
|
||
The printed content id is the address a campaign document's `strategies[].ref`
|
||
points at (§3).
|
||
|
||
## 2. Process documents — a validation methodology (role 5)
|
||
|
||
A process document is a named, versionable pipeline of **std stage blocks**
|
||
over a shared metric vocabulary. Discover the block vocabulary and each
|
||
block's typed slots headlessly:
|
||
|
||
```
|
||
$ aura process introspect --vocabulary
|
||
std::sweep evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
|
||
std::gate filter survivors by a conjunction of typed metric predicates
|
||
std::walk_forward rolling in-sample optimize + out-of-sample test
|
||
std::monte_carlo R-bootstrap over realised R (terminal annotator): ...
|
||
std::generalize cross-instrument worst-case floor (terminal annotator): ...
|
||
$ aura process introspect --block std::sweep
|
||
std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
|
||
metric optional, metric name (see metric_vocabulary)
|
||
select optional, select rule: argmax | plateau:mean | plateau:worst
|
||
deflate optional, bool
|
||
```
|
||
|
||
`metric` and `select` form one **selection group**: all-or-nothing (a
|
||
document naming one without the other is refused, "the selection group is
|
||
all-or-nothing"), `deflate` composes only when the group is present. Omit
|
||
the group entirely for a **selection-free sweep** — the family itself is
|
||
the result, no winner is chosen. A selection-free sweep is only legal as
|
||
the pipeline's *last* stage (the executor's preflight refuses one followed
|
||
by any other block, since a downstream stage would have no nominee to
|
||
consume):
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "process",
|
||
"name": "explore-only-sweep",
|
||
"pipeline": [ { "block": "std::sweep" } ]
|
||
}
|
||
```
|
||
|
||
The executor records this stage's family (every member run) but no
|
||
`StageSelection` and no nominee — recording a winner here would fabricate a
|
||
selection intent the document never expressed.
|
||
|
||
### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize)
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "process",
|
||
"name": "mra-full-v2-sweep-gate-wf-mc-generalize",
|
||
"description": "Full v2 anti-false-discovery pipeline.",
|
||
"pipeline": [
|
||
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
|
||
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
|
||
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
|
||
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
|
||
{ "block": "std::generalize", "metric": "expectancy_r" }
|
||
]
|
||
}
|
||
```
|
||
|
||
(`description` is optional; everything else in the envelope — `format_version`,
|
||
`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired`
|
||
over a bare `{}` will tell you.)
|
||
|
||
```
|
||
$ aura process validate mra_2_process_full_v2.json
|
||
process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates
|
||
$ aura process register mra_2_process_full_v2.json # inside a project
|
||
registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json)
|
||
```
|
||
|
||
### The metric vocabulary — discoverable, not hand-copied
|
||
|
||
`aura process introspect --metrics` (equivalently `aura campaign introspect
|
||
--metrics` — same roster) enumerates every metric name annotated with its
|
||
role, so you never have to guess which metrics a `select`/`gate`/`generalize`
|
||
field will accept:
|
||
|
||
```
|
||
$ aura process introspect --metrics
|
||
expectancy_r rankable | gate | generalize
|
||
win_rate gate
|
||
avg_win_r gate
|
||
avg_loss_r gate
|
||
profit_factor gate
|
||
max_r_drawdown gate
|
||
sqn rankable | gate | generalize
|
||
sqn_normalized rankable | gate | generalize
|
||
net_expectancy_r rankable | gate | generalize
|
||
n_trades gate
|
||
n_open_at_end gate
|
||
total_pips rankable | gate
|
||
max_drawdown rankable | gate
|
||
bias_sign_flips rankable | gate
|
||
deflated_score annotation
|
||
overfit_probability annotation
|
||
neighbourhood_score annotation
|
||
```
|
||
|
||
Read the three tags as three separate questions about one metric name:
|
||
|
||
- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward`
|
||
`select` metric (the field a winner is chosen by)? This is the small
|
||
subset every strategy run always produces cheaply.
|
||
- **`gate`** — can this name be used in a `std::gate` predicate (a per-member
|
||
filter)? This roster is a superset of `rankable` — most emitted metrics
|
||
are filterable even when they make a poor ranking criterion.
|
||
- **`| generalize`** suffix — is this name usable as `std::generalize`'s
|
||
`metric` (needs an R-expectancy-shaped metric to floor across instruments)?
|
||
|
||
That is why, for example, `profit_factor` shows up tagged only `gate`: every
|
||
member's metrics table carries it (so you can gate on it, e.g. "keep only
|
||
`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster,
|
||
so a `std::sweep` with `"select": "argmax"` cannot select on it, and
|
||
`std::generalize` cannot floor across instruments on it either. Attempting
|
||
either produces the same intrinsic refusal that names the metric and points
|
||
back at this verb (`aura process introspect --metrics`) rather than leaving
|
||
you to search the glossary.
|
||
|
||
## 3. Campaign documents — experiment intent over instruments and windows
|
||
|
||
A campaign document names the data (instruments × windows), one or more
|
||
strategies (by blueprint content/identity id) with their param axes, a
|
||
process reference, and what to persist/emit.
|
||
|
||
```
|
||
$ aura campaign introspect --unwired bare.json # bare.json contains just {}
|
||
open slot: format_version (required, must be 1)
|
||
open slot: kind (required, must be "campaign")
|
||
open slot: name (required, string)
|
||
open slot: data (required section: instruments + windows)
|
||
open slot: risk (optional, list of stop regimes { vol: { length, k } }; absent = one default regime)
|
||
open slot: cost (optional, list of cost models { constant: { cost_per_trade } } | { vol_slippage: { slip_vol_mult } } | { carry: { carry_per_cycle } }; absent = zero cost, net = gross)
|
||
open slot: strategies (required, non-empty list of { ref, axes })
|
||
open slot: process.ref (required, content id of a process document)
|
||
open slot: seed (required, non-negative integer)
|
||
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
|
||
```
|
||
|
||
### Worked example: two instruments, one strategy, four axis points, two stop regimes
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "campaign",
|
||
"name": "mra-ger40-fra40-smacross-full-v2",
|
||
"seed": 42,
|
||
"data": {
|
||
"instruments": ["GER40", "FRA40"],
|
||
"windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
|
||
},
|
||
"risk": [
|
||
{ "vol": { "length": 3, "k": 1.5 } },
|
||
{ "vol": { "length": 3, "k": 3.0 } }
|
||
],
|
||
"cost": [
|
||
{ "constant": { "cost_per_trade": 0.02 } },
|
||
{ "vol_slippage": { "slip_vol_mult": 0.1 } }
|
||
],
|
||
"strategies": [
|
||
{
|
||
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
|
||
"axes": {
|
||
"fast.length": { "kind": "I64", "values": [2, 4] },
|
||
"slow.length": { "kind": "I64", "values": [8, 16] },
|
||
"bias.scale": { "kind": "F64", "values": [0.5] }
|
||
}
|
||
}
|
||
],
|
||
"process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } },
|
||
"presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] }
|
||
}
|
||
```
|
||
|
||
The `strategies[].ref.content_id` and `process.ref.content_id` are exactly
|
||
the ids printed by `aura graph register` / `aura process register` above —
|
||
a campaign never inlines a strategy or process body, only its content id.
|
||
Each axis name (`fast.length`, …) must name an open param of the referenced
|
||
blueprint (`aura graph introspect --params`, §1) and declare that param's
|
||
`ScalarKind`.
|
||
|
||
The optional `risk` list is the campaign's structural risk axis: every cell
|
||
runs under every listed stop regime, so cells differ by execution discipline,
|
||
never by signal — the regime's stop defines the risk unit R. Absent or empty,
|
||
the matrix runs one implicit default regime (the same vol regime the
|
||
orchestration verbs bind when their stop flags are omitted).
|
||
|
||
The optional `cost` list is the campaign's cost model (#234): each entry
|
||
charges the trade stream in R — `constant` per closed trade, `vol_slippage`
|
||
proportional to local volatility per closed trade, `carry` per held cycle —
|
||
and the charges sum into the net curve, so every member reports net metrics
|
||
beside gross and the `net_r_equity` tap records the cost-dragged curve.
|
||
Absent or empty, the cost model is zero and net equals gross. The bound cost
|
||
knobs are stamped on each member's manifest (`cost[k].<knob>` params), so a
|
||
costed family reproduces bit-identically like any other.
|
||
|
||
### Validate — three tiers, honest degradation
|
||
|
||
```
|
||
$ aura campaign validate mra_3_campaign_full_v2.json # outside any project
|
||
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
|
||
referential checks skipped (no Aura.toml found up from /home/…)
|
||
```
|
||
|
||
Inside a project, with the strategy blueprint and process both already
|
||
registered, the same command runs two further tiers:
|
||
|
||
```
|
||
$ aura campaign validate mra_3_campaign_full_v2.json # inside a project, refs registered
|
||
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
|
||
campaign document valid (referential): all references resolve, axes are in the param space
|
||
campaign document valid (executable): pipeline shape and static guards pass
|
||
```
|
||
|
||
- **intrinsic** — the document's own shape is well-formed (always checked).
|
||
- **referential** — every `ref` resolves in the project's store and every
|
||
axis names a real, correctly-typed open param (needs a project).
|
||
- **executable** — the process pipeline's static guards pass against this
|
||
campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a
|
||
data-free preflight, so "valid" here means "runnable" (needs a project;
|
||
it does not fetch or touch market data).
|
||
|
||
### Register and run
|
||
|
||
```
|
||
$ aura campaign register mra_3_campaign_full_v2.json
|
||
registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json)
|
||
$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd
|
||
{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}}
|
||
{"family_id":"42edebd2-0-GER40-w0-s0-0-r1","report":{...}}
|
||
…
|
||
{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42,
|
||
"cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],
|
||
"regime":{"vol":{"length":3,"k":1.5}},"stages":[
|
||
{"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]},
|
||
{"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]},
|
||
{"strategy":"597d719b…","instrument":"GER40","regime":{"vol":{"length":3,"k":3.0}},"regime_ordinal":1,...},
|
||
{"strategy":"597d719b…","instrument":"FRA40",...},
|
||
{"strategy":"597d719b…","instrument":"FRA40","regime_ordinal":1,...}],
|
||
"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,
|
||
"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
|
||
"worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]},
|
||
"winners":[...]},
|
||
{"strategy_ordinal":0,"window_ordinal":0,"regime_ordinal":1,...}],
|
||
"trace_name":"42edebd2-0"}}
|
||
```
|
||
|
||
With two stop regimes every (instrument, window) cell runs twice — the four
|
||
cells above are the "4 cell(s)" the validate summary counted. The second
|
||
regime's family ids and trace dirs carry the `-r1` ordinal suffix (the
|
||
default/first regime stays unsuffixed), each cell record names its regime,
|
||
and generalization is keyed per regime — regimes are compared, never pooled.
|
||
|
||
`aura campaign run` is register-then-run sugar for a `.json` file, but the
|
||
canonical address is always the content id — running a bare file the first
|
||
time registers it implicitly. `aura campaign runs` lists stored
|
||
realizations; `aura campaign runs <id>` dumps the bare stored record(s) (not
|
||
the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps`
|
||
is non-empty, the run also persists the named taps under
|
||
`runs/traces/<trace_name>/…`, chartable with `aura chart`. The
|
||
`sweep`/`walkforward --trace` analog persists every member's taps as a
|
||
family charted the same way, by the handle the run prints (members keyed
|
||
`<cell>/<member>`) — also by the `--trace <NAME>` you chose, when that name
|
||
uniquely names one recorded run (`aura chart <NAME>` resolves it against the
|
||
stored campaign documents; a name reused across runs refuses rather than
|
||
guessing which one you mean).
|