10570b75f8
The authoring guide's ops table grows the name row (ten -> eleven) plus a paragraph on what the graph default costs a multi-strategy project (indistinguishable store entries, one shared trace directory, colliding use-splice instance defaults). C24's op-script grammar records the eleventh verb and the two-seam shape gate with its C29 read-back exemption; the glossary's blueprint-label entry now separates the render name (op-settable, keys trace runs and instance defaults) from the registry label (register --name, explicit, orthogonal). refs #331
757 lines
41 KiB
Markdown
757 lines
41 KiB
Markdown
# Authoring guide: op-scripts, process documents, campaign documents
|
||
|
||
`docs/project-layout.md` describes the shape of a project and the arc of a
|
||
research session; this document is the practical companion for the three
|
||
JSON artifact kinds you author headlessly along that arc:
|
||
|
||
1. an **op-script** (role 6a) — builds a node graph (a strategy blueprint)
|
||
through `aura graph build`;
|
||
2. a **process document** (role 5) — a named validation/eval methodology
|
||
through `aura process validate|introspect|register|show`;
|
||
3. a **campaign document** (role 6b) — experiment intent (instruments ×
|
||
windows × strategy × axes × process) through `aura campaign
|
||
validate|introspect|register|show|run|runs`.
|
||
|
||
Each section below is a worked, verified example — every command shown was
|
||
run against this repo and the output is transcribed, not invented. The *why*
|
||
of this three-artifact split (closed-vocabulary data, never a logic DSL)
|
||
lives in the design ledger (`docs/design/INDEX.md`, C20/C25) and the
|
||
glossary; this document only teaches the *shape*.
|
||
|
||
All three artifact kinds above assume the node **types** they reference
|
||
already exist. §0 covers the one piece of this arc that is Rust, not data:
|
||
adding a new node type in the first place.
|
||
|
||
## 0. Authoring a new node in Rust
|
||
|
||
An op-script's `add` op (§1) instantiates a node **type** from the closed
|
||
vocabulary — but that vocabulary itself is not data, it is compiled Rust.
|
||
Adding a new node type (a new indicator, combinator, or signal primitive) is
|
||
role-2 work: you write a `Node` implementation, describe it to the bootstrap
|
||
with a `PrimitiveBuilder` recipe, and roster the type id so the loader can
|
||
find it by name. This is the one seam in the whole arc where the answer is
|
||
"write Rust", not "write JSON" (C17/C20) — and it is a small, fixed shape,
|
||
the same shape every node in `aura-std` already follows.
|
||
|
||
### Where the code goes
|
||
|
||
- **The project's own signal doesn't need a crate by default.** `aura new`
|
||
scaffolds a **data-only** project (`docs/project-layout.md`, "A project
|
||
repo (two tiers)") — a strategy over the std vocabulary is a
|
||
blueprint/campaign document, not Rust: the scaffold ships **one** closed
|
||
`signal.json` starter that serves both verbs — `aura run` uses its bound
|
||
values as-is, `aura sweep --axis fast.length=2,4,8` overrides them
|
||
(bound = default, not fixed); `--list-axes` lists the open knobs and the
|
||
bound defaults alike, so every override target is discoverable.
|
||
- **A project-specific *native* node type** — the moment §0's three-part
|
||
pattern is actually needed — lives in an attached **node crate**:
|
||
`aura nodes new <name>` scaffolds a sibling cdylib crate (`Cargo.toml` +
|
||
`src/lib.rs`, registered under the project's own `<namespace>::` prefix)
|
||
and appends its path to the project's `Aura.toml [nodes]` section.
|
||
- **A block promoted to universal** — reused across projects and folded into
|
||
the engine itself — lives in `crates/aura-std/` (or its sibling domain
|
||
crates: `aura-market`, `aura-strategy`), unprefixed, and is rostered in
|
||
`crates/aura-vocabulary/src/lib.rs` instead of a node crate's
|
||
`vocabulary()` function. The pattern below is identical either way; only
|
||
the rostering call site differs (see "Rostering the type" below).
|
||
|
||
### The three-part pattern
|
||
|
||
Every node type — std or project-local — is three things:
|
||
|
||
1. **A `Node` implementation.** A plain struct holding whatever state the
|
||
node needs between cycles (often just its output cell), plus three
|
||
methods: `lookbacks()` (how many past values, per input, the node reads —
|
||
`vec![1]` for a node that only ever looks at the current cycle),
|
||
`eval(ctx) -> Option<&[Cell]>` (the per-cycle computation — return `None`
|
||
until every input the node needs has fired at least once; this warm-up
|
||
filter is what keeps a downstream consumer from ever observing a
|
||
fabricated value, C8), and `label()` (a short, human-readable debug
|
||
string — not the type id used for serialization, see below). The input
|
||
window `ctx.f64_in(i)` hands those past values back **financial-style:
|
||
index 0 is the newest cycle, index `k` is `k` cycles back** — so a
|
||
`lookbacks()` of `vec![n]` makes `w[0]..=w[n-1]` the last `n` values,
|
||
newest first, and a lookback-spanning node (momentum, ATR, RSI) reads its
|
||
span as `w[0]` vs `w[length]`, never the other way round.
|
||
2. **A `PrimitiveBuilder` recipe.** A `builder()` constructor that pairs a
|
||
`NodeSchema` (its input ports as `PortSpec`, its output fields as
|
||
`FieldSpec`, and its bindable params as `ParamSpec`) with a **build
|
||
closure** `|p| Box::new(Type::new(...))` that reads bound param values out
|
||
of `p` positionally (`p[0].f64()`, `p[1].i64()`, …, matching the `params`
|
||
order in the schema) and constructs the node. This one recipe is what lets
|
||
the bootstrap turn a blueprint's *serialized* param values into a live,
|
||
concrete node — a project or op-script never constructs a node directly.
|
||
3. **Rostering the type id.** The recipe is useless until something maps the
|
||
serialized type-id string (e.g. `"Scale"`, `"my_lab::ThirdCandle"`) back to
|
||
its `builder()`. This is a closed, compiled-in `match` — never a dynamic
|
||
registry (domain invariant 9) — so a node exists in the vocabulary only if
|
||
its type id is added to exactly one of these two match tables:
|
||
- **Project-side:** the `vocabulary()` / `type_ids()` pair the
|
||
`aura_core::aura_project!` macro wires up (every `aura new` scaffold
|
||
emits a starter pair — see the worked example below). Add one match arm
|
||
to `vocabulary()` and one entry to `type_ids()`'s slice.
|
||
- **Std-side** (only when promoting a node into the shipped std
|
||
vocabulary): one line in the `std_vocabulary_roster!` macro invocation in
|
||
[`crates/aura-vocabulary/src/lib.rs`](../crates/aura-vocabulary/src/lib.rs)
|
||
— `"TypeId" => Type,` — which expands into both the resolver `match` and
|
||
the enumerable type-id list, so the two surfaces cannot drift apart. A
|
||
zero-arg `Type::builder()` rosters directly; an **arg-bearing** type
|
||
(structural, non-scalar construction — see "Session anchoring" below and
|
||
`docs/glossary.md`'s `construction arg` entry, #271) rosters the exact
|
||
same way — its `builder()` is itself zero-arg, returning a *pending*
|
||
recipe the `add` op's `args` configures before any `bind`.
|
||
An unrostered type fails safe either way: the loader refuses with a clean
|
||
`LoadError::UnknownNodeType` naming the missing id, and the type is simply
|
||
absent from `aura graph introspect --vocabulary` — never a silent partial
|
||
load.
|
||
|
||
### Worked example: `Scale`, a one-input, one-param node
|
||
|
||
This is the starter node `aura nodes new` writes into every freshly attached
|
||
node crate's `src/lib.rs` (`__NS__` is the node crate's namespace) —
|
||
copy-pasteable, and already exercised end to end by the scaffold's own tests:
|
||
|
||
```rust
|
||
use aura_core::{
|
||
Cell, Ctx, FieldSpec, Firing, Node, NodeSchema, ParamSpec, PortSpec, PrimitiveBuilder,
|
||
ScalarKind,
|
||
};
|
||
|
||
/// One-input scalar gain: emits `input * factor`. Emits `None` until its
|
||
/// input has a value (warm-up filter, C8).
|
||
pub struct Scale {
|
||
factor: f64,
|
||
out: [Cell; 1],
|
||
}
|
||
|
||
impl Scale {
|
||
pub fn new(factor: f64) -> Self {
|
||
Self { factor, out: [Cell::from_f64(0.0)] }
|
||
}
|
||
// Replace `doc` below when renaming this sample node — it must keep
|
||
// describing the node's own behaviour, not this scaffolding step.
|
||
pub fn builder() -> PrimitiveBuilder {
|
||
PrimitiveBuilder::new(
|
||
"my_lab::Scale",
|
||
NodeSchema {
|
||
inputs: vec![PortSpec {
|
||
kind: ScalarKind::F64,
|
||
firing: Firing::Any,
|
||
name: "value".into(),
|
||
}],
|
||
output: vec![FieldSpec { name: "value".into(), kind: ScalarKind::F64 }],
|
||
params: vec![ParamSpec { name: "factor".into(), kind: ScalarKind::F64 }],
|
||
// C29: every vocabulary entry ships a one-line meaning — the
|
||
// doc field is required (E0063 without it) and gated at load.
|
||
doc: "scalar gain: emits the input times the factor param",
|
||
},
|
||
|p| Box::new(Scale::new(p[0].f64())),
|
||
)
|
||
}
|
||
}
|
||
|
||
impl Node for Scale {
|
||
fn lookbacks(&self) -> Vec<usize> {
|
||
vec![1]
|
||
}
|
||
fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
|
||
let w = ctx.f64_in(0);
|
||
if w.is_empty() {
|
||
return None;
|
||
}
|
||
self.out[0] = Cell::from_f64(w[0] * self.factor);
|
||
Some(&self.out)
|
||
}
|
||
fn label(&self) -> String {
|
||
format!("my_lab::Scale({})", self.factor)
|
||
}
|
||
}
|
||
|
||
fn vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
|
||
match type_id {
|
||
"my_lab::Scale" => Some(Scale::builder()),
|
||
_ => None,
|
||
}
|
||
}
|
||
|
||
fn type_ids() -> &'static [&'static str] {
|
||
&["my_lab::Scale"]
|
||
}
|
||
|
||
aura_core::aura_project! {
|
||
namespace: "my_lab",
|
||
vocabulary: vocabulary,
|
||
type_ids: type_ids,
|
||
}
|
||
```
|
||
|
||
Reading this top to bottom against the three-part pattern: `Scale` is the
|
||
`Node` impl (one input, one param, a one-cycle lookback); `Scale::builder()`
|
||
is the `PrimitiveBuilder` recipe (`params: vec![ParamSpec { name: "factor",
|
||
... }]` declares the one bindable knob, and the build closure `|p|
|
||
Box::new(Scale::new(p[0].f64()))` reads it back positionally at bootstrap
|
||
time); and `vocabulary()` / `type_ids()` — wired up by `aura_project!` — are
|
||
the rostering. Once this compiles into the project's `cdylib`, `my_lab::Scale`
|
||
is a normal citizen of the vocabulary: `aura graph introspect --vocabulary`
|
||
lists it, `aura graph introspect --node my_lab::Scale` shows its port/param
|
||
shape exactly like a std node, and an op-script's `add` op can instantiate it
|
||
by that type id.
|
||
|
||
### Other worked examples, if you need a different arity
|
||
|
||
`aura-std` itself has several small, deliberately minimal nodes worth reading
|
||
alongside `Scale` for the shapes that recur most:
|
||
|
||
- [`crates/aura-std/src/const_node.rs`](../crates/aura-std/src/const_node.rs)
|
||
(`Const`) — another single-param node, but a *source*-shaped one: it needs
|
||
a driving input purely to be evaluated at all (a zero-input node never
|
||
fires in the total-push engine, C8), and ignores that input's actual value.
|
||
- [`crates/aura-std/src/div.rs`](../crates/aura-std/src/div.rs) (`Div`),
|
||
[`crates/aura-std/src/max.rs`](../crates/aura-std/src/max.rs) (`Max`),
|
||
[`crates/aura-std/src/min.rs`](../crates/aura-std/src/min.rs) (`Min`) —
|
||
two-input, paramless combinators (`inputs: vec![lhs, rhs]`, `params:
|
||
vec![]`, an eval that reads `ctx.f64_in(0)` and `ctx.f64_in(1)`).
|
||
- [`crates/aura-std/src/abs.rs`](../crates/aura-std/src/abs.rs) (`Abs`) — the
|
||
minimal case: one input, no params, `lookbacks() == vec![1]`.
|
||
|
||
## 1. Op-scripts — building a strategy blueprint by hand
|
||
|
||
An op-script is a JSON **array of ops**, replayed in order to construct a
|
||
node graph. `aura graph build` reads the op-script from **stdin** (there is
|
||
no file argument) and prints the canonical blueprint envelope to stdout:
|
||
|
||
```
|
||
$ aura graph build < smacross.json > blueprint.json
|
||
```
|
||
|
||
Nodes are referenced by an **identifier** (given by `add`, see below); ports
|
||
are dotted `<identifier>.<port>` on both sides of a wire.
|
||
|
||
### The eleven ops
|
||
|
||
| op | JSON shape | does |
|
||
|---|---|---|
|
||
| `name` | `{"op":"name","name":<str>}` | set the composite's **render name**, script-level and at-most-once (a second `name` op refuses: `a script names its blueprint at most once`). The name must be a single path segment (non-empty, no `/`, `\`, `.` or `..`) since it keys a trace directory (`traces/<name>/`) at run time. Omitting the op keeps the CLI's own default, `"graph"`. |
|
||
| `doc` | `{"op":"doc","text":<str>}` | declare the composite's one-line meaning (C29) — the op-script twin of the Rust builder's `.doc(...)`. Required before `aura graph register` accepts the product (the store refuses a doc-less composite); at most one per script (a second refuses: `a doc op may appear at most once`). The text is gated: empty or merely restating the composite's name refuses at register. |
|
||
| `source` | `{"op":"source","role":<str>,"kind":<ScalarKind>}` | reserve a bound root **source** role of `kind` — a real input the harness feeds (e.g. `"price"`). |
|
||
| `input` | `{"op":"input","role":<str>}` | reserve an open root **input** role (kind inferred from the slots it feeds) — the formal parameter of an **open pattern**, a fragment meant to be wired by an *enclosing* graph. An open pattern builds and registers like any blueprint. Running it standalone is governed by the ordinary run gates: the harness binds input roles to archive columns **by name** (C26), so a pattern whose roles match the data runs as-is; what refuses, by name, is a role the harness cannot bind — or the run surface's other gates (a signal without a bias output or tap; free knobs). |
|
||
| `add` | `{"op":"add","type":<TypeId>,"name":<str>?,"args":{<arg>:<str>}?,"bind":{<param>:<Scalar>}?}` | instantiate a node of a type in the closed vocabulary (`aura graph introspect --vocabulary`) — see §0 below for how a type gets into that vocabulary in the first place. `name` becomes the node's identifier for later ops (default: the type's own lowercase label — two unnamed nodes of the same type then collide). `args` (#271) configures an **arg-bearing** type's structural, non-scalar construction (`Session`'s timezone/open, `LinComb`/`CostSum`'s arity) — a closed table of string values, applied BEFORE `bind`; `aura graph introspect --node <T>` lists a type's declared `arg` rows. `bind` sets zero or more of the (now real) params. |
|
||
| `use` | `{"op":"use","ref":{"content_id":<id-or-prefix>}\|{"name":<label>},"name":<str>?,"bind":{<path>:<Scalar>}?}` | splice a **registered blueprint** into the graph as a nested composite under an instance identifier (`name`; default: the stored composite's own name). The ref resolves by content id (full or unique prefix) or by a register-time label; `graph build` echoes every resolution (`aura: note: use "…": … -> <id>`) so the exact id can be pinned. The instance's open input roles wire as `<instance>.<role>`, its outputs as `<instance>.<field>`; its open params surface path-qualified (`<instance>.<node>.<param>`), sweepable (ganging an instance's params is not yet supported — the `gang` op refuses a composite instance's member path). A doc-less stored source refuses at the op (C29). The emitted blueprint contains the subgraph *inline* — no reference survives into the artifact. |
|
||
| `feed` | `{"op":"feed","role":<str>,"into":[<port>, …]}` | fan a previously-declared role into one or more interior input slots, all-or-nothing (a failing target leaves none of the batch wired). |
|
||
| `connect` | `{"op":"connect","from":<port>,"to":<port>}` | wire one interior output field to one interior input slot. A `connect` that would close a dataflow cycle is rejected immediately — the only legal feedback path is an explicit delay/state node (domain invariant 5). |
|
||
| `expose` | `{"op":"expose","from":<port>,"as":<str>}` | promote an interior output field to a boundary output under the alias `as` — a real *alias* (a terminal boundary name, not a referenceable identifier like `add`'s `name`). Together with `tap`, one of the two ops whose `as` key is a terminal name rather than an identifier. |
|
||
| `tap` | `{"op":"tap","from":<port>,"as":<str>}` | declare a **measurement tap** on an interior output field under the name `as` — the output-side twin of `expose` (a recorded observation point, not a boundary output; a `Composite.taps` entry, C27). A single `aura run` constructs a recorder at each declared tap and persists its per-cycle series as a `ColumnarTrace`; a sweep leaves it inert. Tap names are their own namespace and must be unique (a second `tap` under one name refuses: `duplicate tap name`). |
|
||
| `gang` | `{"op":"gang","as":"channel_length","into":["channel_hi.length","channel_lo.length"]}` | Fuse two or more sibling params into ONE public knob: the member addresses leave the sweepable param space and `as` replaces them; the bound or swept value fans out to every member at bootstrap. Members must share one scalar kind and stay open (un-bound). |
|
||
|
||
Omitting `name` is fine for a single strategy, but it has three live
|
||
consequences once a project has more than one: every unnamed store document
|
||
is indistinguishable by name (`"graph"` again), every unnamed strategy's
|
||
default tap recording lands in the same shared `traces/graph/` directory
|
||
(later runs overwrite earlier ones), and two `use`-splices of unnamed
|
||
blueprints collide on the same default instance identifier (`graph`) inside
|
||
the same composing script. A one-line `name` op dissolves all three at once.
|
||
|
||
Value forms are the typed-tag representations used everywhere in this
|
||
family of artifacts:
|
||
|
||
- a bind value is `{"I64":2}` / `{"F64":0.5}` / `{"Bool":true}` /
|
||
`{"Timestamp":<i64 ms>}`;
|
||
- a `kind` field is the capitalized `ScalarKind` name: `"I64"` | `"F64"` |
|
||
`"Bool"` | `"Timestamp"`.
|
||
|
||
### Worked example: an SMA-crossover bias strategy
|
||
|
||
```json
|
||
[
|
||
{"op": "source", "role": "price", "kind": "F64"},
|
||
{"op": "add", "type": "SMA", "name": "fast"},
|
||
{"op": "add", "type": "SMA", "name": "slow"},
|
||
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
|
||
{"op": "add", "type": "Sub", "name": "sub"},
|
||
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
|
||
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
|
||
{"op": "add", "type": "Bias", "name": "bias"},
|
||
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
|
||
{"op": "expose", "from": "bias.bias", "as": "bias"}
|
||
]
|
||
```
|
||
|
||
(`fast.length`, `slow.length`, and `bias.scale` are all left unbound here on
|
||
purpose — three open campaign axes, see §3. This is the milestone fieldtest
|
||
corpus's own example, verified below; byte-identical to the on-disk
|
||
`fieldtests/milestone-research-artifacts/mra_1_strategy_smacross.json`.)
|
||
|
||
A `bind` in an `add` op pins that param to a value and removes it from the
|
||
**open** param space (`--params`): a bound param is a **default** (#246) — a
|
||
run uses it as-is, while any campaign axis or `aura sweep --axis` naming it
|
||
re-opens it for that family and binds it per cell. `--list-axes` lists it
|
||
after the open knobs as `<name>:<KIND> default=<value>`. `bind` is for a
|
||
value the strategy carries by default; leave a param unbound, as all three
|
||
are here, to make binding it mandatory for every sweep.
|
||
|
||
A param therefore has three states: **open** (an axis every sweep MUST bind),
|
||
**bound** (a default any axis MAY override, #246), and **ganged** (open, but
|
||
fused with its siblings under ONE public knob declared by a `gang` op; the
|
||
member addresses are unbindable and only the gang's own single-segment name —
|
||
e.g. `channel_length` — appears in `--list-axes`).
|
||
|
||
### Worked example: declaring a measurement tap
|
||
|
||
A **declared tap** (C27) names an interior output wire as a recorded
|
||
observation point — the output-side twin of `expose`, but a *measurement*
|
||
rather than a boundary output. It is authored by the same `"node.field"`
|
||
addressing every other op uses; there is no raw node index. Extending the
|
||
crossover above to record the raw fast−slow spread (the signal *before* the
|
||
bias scaling) under the name `spread`:
|
||
|
||
```json
|
||
[
|
||
{"op": "source", "role": "price", "kind": "F64"},
|
||
{"op": "add", "type": "SMA", "name": "fast"},
|
||
{"op": "add", "type": "SMA", "name": "slow"},
|
||
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
|
||
{"op": "add", "type": "Sub", "name": "sub"},
|
||
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
|
||
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
|
||
{"op": "add", "type": "Bias", "name": "bias"},
|
||
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
|
||
{"op": "tap", "from": "sub.value", "as": "spread"},
|
||
{"op": "expose", "from": "bias.bias", "as": "bias"}
|
||
]
|
||
```
|
||
|
||
The built blueprint carries a `taps` array naming the resolved wire
|
||
(`{"name":"spread","from":{"node":<sub's index>,"field":0}}` — the name is
|
||
addressed, the index is resolved *for* you). A single `aura run <blueprint>`
|
||
then constructs a recorder at that point and persists the `spread` series as a
|
||
`ColumnarTrace` in the run's trace store, chartable by its name. A tap is inert
|
||
in a sweep (no per-cell recorder) — it is a single-run observation surface.
|
||
(`sub.value` is tapped here purely to illustrate; any interior output field is
|
||
a legal tap source, and a producer needs no other consumer to be tapped.)
|
||
|
||
### Session anchoring: the `SessionFrankfurt` preset
|
||
|
||
To anchor logic to the trading session, the closed vocabulary ships a
|
||
`SessionFrankfurt` node — the Frankfurt cash open (09:00 `Europe/Berlin`)
|
||
baked in, DST-correct. Its schema:
|
||
|
||
- **input** `trigger: f64` — wired from a once-per-bar stream (e.g. a 15m
|
||
resampler's `close`); its *value is ignored*, it only clocks the node to fire
|
||
once per completed bar;
|
||
- **param** `period_minutes: i64` — the bar width to index by (conventionally
|
||
`15`, matching the resampler);
|
||
- **output** `bars_since_open: i64` — the count of completed `period_minutes`
|
||
bars since the local session open, in tz-aware local wall-clock time.
|
||
|
||
Because a resampler emits on exact `:00/:15/:30/:45` boundaries and `ctx.now()`
|
||
at emission is the *just-closed* bar's close instant, in-session bar closes
|
||
read exact positive multiples: the 09:00–09:15 bar closes at **09:15 → 1**,
|
||
09:30–09:45 at **09:45 → 3**, 10:00–10:15 at **10:15 → 5**. Pre-open instants
|
||
give `<= 0` (non-actionable). To gate on "the third bar of the session",
|
||
compare `bars_since_open` against a constant (an `EqConst(==3)`); there is no
|
||
separate in-session bool — `bars_since_open` alone is the contract. DST is
|
||
handled by `chrono-tz`, so the same local minute reads the same index in
|
||
summer (CEST) and winter (CET).
|
||
|
||
The index is derived from the clock (`ctx.now()`), never counted from trigger
|
||
firings — so the trigger's cadence only sets how *often* the node emits, not
|
||
what it emits. Feeding a denser stream (e.g. a raw m1 `price`) is equally
|
||
valid: the node fires once per input tick and reports the same
|
||
`period_minutes`-bar index throughout that bar. The once-per-bar wiring above
|
||
is the convention when you want exactly one emission per completed bar; it is
|
||
not a correctness requirement.
|
||
|
||
Wire it like any node —
|
||
`{"op":"add","type":"SessionFrankfurt","bind":{"period_minutes":{"I64":15}}}`,
|
||
feed its `trigger` from a once-per-bar stream, and read `bars_since_open`.
|
||
|
||
**Any other timezone/open: `Session` + `args` (#271).** `SessionFrankfurt`
|
||
stays baked sugar for the one Frankfurt open; the canonical path for ANY IANA
|
||
timezone and local open time is the base `Session` type through the typed
|
||
construction-args channel — `args` configures it BEFORE `bind`:
|
||
|
||
```json
|
||
{"op":"add","type":"Session","name":"ny",
|
||
"args":{"tz":"America/New_York","open":"09:30"},
|
||
"bind":{"period_minutes":{"I64":15}}}
|
||
```
|
||
|
||
`tz` is `chrono_tz`'s own exact, case-sensitive IANA name (`"berlin"` refuses;
|
||
`"Europe/Berlin"` accepts); `open` is strict zero-padded local `HH:MM`
|
||
(`"9:30"` refuses; `"09:30"` accepts); a `Count` arg (`LinComb`/`CostSum`'s
|
||
arity) is a plain positive decimal — no sign, no leading zeros (`"03"`
|
||
refuses; `"3"` accepts). The accepted string IS the canonical form, no
|
||
normalization — and note the deliberate asymmetry: each kind's canonical
|
||
form is its domain's conventional notation, fixed-width for wall-clock
|
||
times, minimal for numbers. Giving `Session` no `args` at all refuses the `add`
|
||
op (`node <name> is missing required arg "tz"`) — an arg-bearing type never
|
||
silently enters the graph half-configured. `aura graph introspect --node
|
||
Session` lists both declared `arg` rows with their kind and hint text before
|
||
you write the JSON.
|
||
|
||
### Commands
|
||
|
||
```
|
||
$ aura graph build < smacross.json
|
||
{"format_version":1,"blueprint":{"name":"graph","nodes":[...],"edges":[...],
|
||
"input_roles":[{"name":"price",...,"source":"F64"}],"output":[...]}}
|
||
```
|
||
|
||
The built blueprint renders visually, too: `aura graph blueprint.json` emits an
|
||
interactive HTML DAG so a mis-wire is visible before any run (`aura graph` with
|
||
no file renders the built-in sample; a named-but-unreadable file is a usage
|
||
error). Introspection is build-free wherever possible:
|
||
|
||
```
|
||
$ aura graph introspect --vocabulary # one node type per line
|
||
$ aura graph introspect --node SMA # ports + params of one type
|
||
SMA
|
||
in series:F64
|
||
out value:F64
|
||
param length:I64 (bind {"I64": <v>})
|
||
$ aura graph introspect --unwired < partial.json # open slots of a partial op-script
|
||
sub.rhs:F64
|
||
$ aura graph introspect --content-id smacross.json # SHA-256 of the canonical form
|
||
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
|
||
$ aura graph introspect --content-id smacross.json --identity-id # + debug-name-blind identity id, combinable
|
||
597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a
|
||
41bab46ce78356eeab2d2a4e03daaf2117eb970a1c3ef880264553bf662453a4
|
||
$ aura graph introspect --params smacross.json # the raw param-space namespace (what a campaign's axes bind against)
|
||
fast.length:I64
|
||
slow.length:I64
|
||
bias.scale:F64
|
||
```
|
||
|
||
These printed names are the **one** axis namespace (#328): op-script params, a
|
||
campaign document's `strategies[].axes` keys (§3), and `aura sweep <blueprint>
|
||
--list-axes` / `--axis` (glossary `sweep`) all speak the same raw
|
||
`<node>.<param>` form — `--params` and `--list-axes` are line-identical (open
|
||
params bare, bound params trailing `default=<value>`), and every discovered
|
||
name is verbatim legal as a document axis key, bound params included (#246's
|
||
re-open contract):
|
||
|
||
```
|
||
$ aura sweep smacross.json --axis fast.length=3:9:2 # synthetic: raw works
|
||
$ aura sweep smacross.json --axis bias.scale=0.25,0.5 --real ... # real: raw works
|
||
```
|
||
|
||
The older `<blueprint>.<node>.<param>` wrapped form (e.g. `graph.fast.length`,
|
||
what pre-#328 transcripts and `--list-axes` output used to print) is retired
|
||
from the surface; naming it refuses with a translation pointer to the raw
|
||
candidate, on both intake seams:
|
||
|
||
```
|
||
$ aura sweep smacross.json --axis graph.fast.length=...
|
||
aura: axis "graph.fast.length": axis names are raw node.param paths —
|
||
use "fast.length" (the wrapped --list-axes form was retired, #328)
|
||
|
||
$ aura campaign validate wrapped.json # a document quoting an old transcript
|
||
aura: campaign references do not resolve:
|
||
strategy f5fdf729d0333f0286c19d57143fb7933137f27a6128e72489c508096c7a2240:
|
||
axis "graph.fast.length" is not in the param space; axis names are raw
|
||
node.param paths — did you mean "fast.length"?
|
||
```
|
||
|
||
A ganged knob's raw address has one path segment less than a member address
|
||
would — it sits at the composite's own level, like a role name (e.g.
|
||
`channel_length`, not `channel_hi.length`).
|
||
|
||
`--content-id`, `--identity-id`, `--params`, and `graph register` all accept
|
||
**either** shape: the raw op-script array or an already-built `#155`
|
||
blueprint envelope (object) — shape-discriminated automatically, so you
|
||
never have to build first just to hash or register:
|
||
|
||
```
|
||
$ aura graph register smacross.json # inside a project (Aura.toml present)
|
||
registered blueprint 597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a (…/runs/blueprints/597d…json)
|
||
```
|
||
|
||
The printed content id is the address a campaign document's `strategies[].ref`
|
||
points at (§3).
|
||
|
||
`graph register <file> --name <label>` additionally records a **blueprint
|
||
label** — a legible, re-pointable handle for the `use` op's `{"name":…}`
|
||
ref form (re-registering under the same label repoints it; the echo names
|
||
the previous target). `aura graph introspect --registered` lists the
|
||
labels with their id prefix and root doc line — the discovery surface for
|
||
what `use` can reference.
|
||
|
||
## 2. Process documents — a validation methodology (role 5)
|
||
|
||
A process document is a named, versionable pipeline of **std stage blocks**
|
||
over a shared metric vocabulary. Discover the block vocabulary and each
|
||
block's typed slots headlessly:
|
||
|
||
```
|
||
$ aura process introspect --vocabulary
|
||
std::sweep evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
|
||
std::gate filter survivors by a conjunction of typed metric predicates
|
||
std::walk_forward rolling in-sample optimize + out-of-sample test
|
||
std::monte_carlo R-bootstrap over realised R (terminal annotator): ...
|
||
std::generalize cross-instrument worst-case floor (terminal annotator): ...
|
||
std::grid enumerate the axis grid as parameter points for the next stage (enumerate-only leading stage): ...
|
||
$ aura process introspect --block std::sweep
|
||
std::sweep — evaluate the campaign's axes-space; reduce members to R metrics; optionally select a winner (the selection group metric+select is all-or-nothing; omitted = selection-free, terminal-stage-only)
|
||
metric optional, metric name (see metric_vocabulary)
|
||
select optional, select rule: argmax | plateau:mean | plateau:worst
|
||
deflate optional, bool
|
||
```
|
||
|
||
`metric` and `select` form one **selection group**: all-or-nothing (a
|
||
document naming one without the other is refused, "the selection group is
|
||
all-or-nothing"), `deflate` composes only when the group is present. Omit
|
||
the group entirely for a **selection-free sweep** — the family itself is
|
||
the result, no winner is chosen. A selection-free sweep is only legal as
|
||
the pipeline's *last* stage (the executor's preflight refuses one followed
|
||
by any other block, since a downstream stage would have no nominee to
|
||
consume):
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "process",
|
||
"name": "explore-only-sweep",
|
||
"pipeline": [ { "block": "std::sweep" } ]
|
||
}
|
||
```
|
||
|
||
The executor records this stage's family (every member run) but no
|
||
`StageSelection` and no nominee — recording a winner here would fabricate a
|
||
selection intent the document never expressed.
|
||
|
||
### Worked example: full v2 pipeline (sweep → gate → walk-forward → Monte-Carlo → generalize)
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "process",
|
||
"name": "mra-full-v2-sweep-gate-wf-mc-generalize",
|
||
"description": "Full v2 anti-false-discovery pipeline.",
|
||
"pipeline": [
|
||
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
|
||
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
|
||
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
|
||
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
|
||
{ "block": "std::generalize", "metric": "expectancy_r" }
|
||
]
|
||
}
|
||
```
|
||
|
||
(`description` is optional; everything else in the envelope — `format_version`,
|
||
`kind`, `name`, `pipeline` — is required, as `aura process introspect --unwired`
|
||
over a bare `{}` will tell you.)
|
||
|
||
```
|
||
$ aura process validate mra_2_process_full_v2.json
|
||
process document valid (intrinsic): 5 pipeline blocks, 1 gate predicates
|
||
$ aura process register mra_2_process_full_v2.json # inside a project
|
||
registered process cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9 (…/runs/processes/cd91…json)
|
||
```
|
||
|
||
### The metric vocabulary — discoverable, not hand-copied
|
||
|
||
`aura process introspect --metrics` (equivalently `aura campaign introspect
|
||
--metrics` — same roster) enumerates every metric name annotated with its
|
||
role, so you never have to guess which metrics a `select`/`gate`/`generalize`
|
||
field will accept:
|
||
|
||
```
|
||
$ aura process introspect --metrics
|
||
expectancy_r rankable | gate | generalize
|
||
win_rate gate
|
||
avg_win_r gate
|
||
avg_loss_r gate
|
||
profit_factor gate
|
||
max_r_drawdown gate
|
||
sqn rankable | gate | generalize
|
||
sqn_normalized rankable | gate | generalize
|
||
net_expectancy_r rankable | gate | generalize
|
||
n_trades gate
|
||
n_open_at_end gate
|
||
total_pips rankable | gate
|
||
max_drawdown rankable | gate
|
||
bias_sign_flips rankable | gate
|
||
deflated_score annotation
|
||
overfit_probability annotation
|
||
neighbourhood_score annotation
|
||
```
|
||
|
||
Read the three tags as three separate questions about one metric name:
|
||
|
||
- **`rankable`** — can this name be used as a `std::sweep` / `std::walk_forward`
|
||
`select` metric (the field a winner is chosen by)? This is the small
|
||
subset every strategy run always produces cheaply.
|
||
- **`gate`** — can this name be used in a `std::gate` predicate (a per-member
|
||
filter)? This roster is a superset of `rankable` — most emitted metrics
|
||
are filterable even when they make a poor ranking criterion.
|
||
- **`| generalize`** suffix — is this name usable as `std::generalize`'s
|
||
`metric` (needs an R-expectancy-shaped metric to floor across instruments)?
|
||
|
||
That is why, for example, `profit_factor` shows up tagged only `gate`: every
|
||
member's metrics table carries it (so you can gate on it, e.g. "keep only
|
||
`profit_factor > 1.0`"), but it is not in the small ranking-eligible roster,
|
||
so a `std::sweep` with `"select": "argmax"` cannot select on it, and
|
||
`std::generalize` cannot floor across instruments on it either. Attempting
|
||
either produces the same intrinsic refusal that names the metric and points
|
||
back at this verb (`aura process introspect --metrics`) rather than leaving
|
||
you to search the glossary.
|
||
|
||
## 3. Campaign documents — experiment intent over instruments and windows
|
||
|
||
A campaign document names the data (instruments × windows), one or more
|
||
strategies (by blueprint content/identity id) with their param axes, a
|
||
process reference, and what to persist/emit.
|
||
|
||
```
|
||
$ aura campaign introspect --unwired bare.json # bare.json contains just {}
|
||
open slot: format_version (required, must be 1)
|
||
open slot: kind (required, must be "campaign")
|
||
open slot: name (required, string)
|
||
open slot: data (required section: instruments + windows)
|
||
open slot: risk (optional, list of stop regimes { vol: { length, k } } | { vol_tf: { period_minutes, length, k } }; absent = one default regime)
|
||
open slot: cost (optional, list of cost models { constant: { cost_per_trade } } | { vol_slippage: { slip_vol_mult } } | { carry: { carry_per_cycle } }; absent = zero cost, net = gross)
|
||
open slot: strategies (required, non-empty list of { ref, axes })
|
||
open slot: process.ref (required, content id of a process document)
|
||
open slot: seed (required, non-negative integer)
|
||
open slot: presentation (required section: persist_taps (equity | exposure | r_equity | net_r_equity) + emit)
|
||
```
|
||
|
||
### Worked example: two instruments, one strategy, four axis points, two stop regimes
|
||
|
||
```json
|
||
{
|
||
"format_version": 1,
|
||
"kind": "campaign",
|
||
"name": "mra-ger40-fra40-smacross-full-v2",
|
||
"seed": 42,
|
||
"data": {
|
||
"instruments": ["GER40", "FRA40"],
|
||
"windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
|
||
},
|
||
"risk": [
|
||
{ "vol": { "length": 3, "k": 1.5 } },
|
||
{ "vol": { "length": 3, "k": 3.0 } }
|
||
],
|
||
"cost": [
|
||
{ "constant": { "cost_per_trade": 0.02 } },
|
||
{ "vol_slippage": { "slip_vol_mult": 0.1 } }
|
||
],
|
||
"strategies": [
|
||
{
|
||
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
|
||
"axes": {
|
||
"fast.length": { "kind": "I64", "values": [2, 4] },
|
||
"slow.length": { "kind": "I64", "values": [8, 16] },
|
||
"bias.scale": { "kind": "F64", "values": [0.5] }
|
||
}
|
||
}
|
||
],
|
||
"process": { "ref": { "content_id": "cd91270ca61ad42f56939231a7803a1d6d7aaa2b70bf79cde9da9284683b86b9" } },
|
||
"presentation": { "persist_taps": ["equity", "r_equity"], "emit": ["family_table", "selection_report"] }
|
||
}
|
||
```
|
||
|
||
The `strategies[].ref.content_id` and `process.ref.content_id` are exactly
|
||
the ids printed by `aura graph register` / `aura process register` above —
|
||
a campaign never inlines a strategy or process body, only its content id.
|
||
Each axis name (`fast.length`, …) must name an open param of the referenced
|
||
blueprint (`aura graph introspect --params`, §1) and declare that param's
|
||
`ScalarKind`.
|
||
|
||
The optional `risk` list is the campaign's structural risk axis: every cell
|
||
runs under every listed stop regime, so cells differ by execution discipline,
|
||
never by signal — the regime's stop defines the risk unit R. Absent or empty,
|
||
the matrix runs one implicit default regime (the same vol regime the
|
||
orchestration verbs bind when their stop flags are omitted).
|
||
|
||
The optional `cost` list is the campaign's cost model (#234): each entry
|
||
charges the trade stream in R — `constant` per closed trade, `vol_slippage`
|
||
proportional to local volatility per closed trade, `carry` per held cycle —
|
||
and the charges sum into the net curve, so every member reports net metrics
|
||
beside gross and the `net_r_equity` tap records the cost-dragged curve.
|
||
Absent or empty, the cost model is zero and net equals gross. The bound cost
|
||
knobs are stamped on each member's manifest (`cost[k].<knob>` params), so a
|
||
costed family reproduces bit-identically like any other.
|
||
|
||
### Validate — three tiers, honest degradation
|
||
|
||
```
|
||
$ aura campaign validate mra_3_campaign_full_v2.json # outside any project
|
||
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
|
||
referential checks skipped (no Aura.toml found up from /home/…)
|
||
```
|
||
|
||
Inside a project, with the strategy blueprint and process both already
|
||
registered, the same command runs two further tiers:
|
||
|
||
```
|
||
$ aura campaign validate mra_3_campaign_full_v2.json # inside a project, refs registered
|
||
campaign document valid (intrinsic): 1 strategy(ies), 3 axes (4 points), 2 instrument(s), 1 window(s), 2 regime(s) — 4 cell(s)
|
||
campaign document valid (referential): all references resolve, axes are in the param space
|
||
campaign document valid (executable): pipeline shape and static guards pass
|
||
```
|
||
|
||
- **intrinsic** — the document's own shape is well-formed (always checked).
|
||
- **referential** — every `ref` resolves in the project's store and every
|
||
axis names a real, correctly-typed open param (needs a project).
|
||
- **executable** — the process pipeline's static guards pass against this
|
||
campaign's shape (e.g. `std::generalize` needs ≥ 2 instruments) — a
|
||
data-free preflight, so "valid" here means "runnable" (needs a project;
|
||
it does not fetch or touch market data).
|
||
|
||
### Register and run
|
||
|
||
```
|
||
$ aura campaign register mra_3_campaign_full_v2.json
|
||
registered campaign 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd (…/runs/campaigns/42ed…json)
|
||
$ aura campaign run 42edebd2159de708009ba21e1ed4aea2cffabc373bf761c5765f79c190b677bd
|
||
{"family_id":"42edebd2-0-GER40-w0-s0-0","report":{...}}
|
||
{"family_id":"42edebd2-0-GER40-w0-s0-0-r1","report":{...}}
|
||
…
|
||
{"campaign_run":{"campaign":"42edebd2…","process":"cd9127…","run":0,"seed":42,
|
||
"cells":[{"strategy":"597d719b…","instrument":"GER40","window_ms":[...],
|
||
"regime":{"vol":{"length":3,"k":1.5}},"stages":[
|
||
{"block":"std::sweep",...},{"block":"std::gate","survivor_ordinals":[0,1,2]},
|
||
{"block":"std::walk_forward",...},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{...}}}]},
|
||
{"strategy":"597d719b…","instrument":"GER40","regime":{"vol":{"length":3,"k":3.0}},"regime_ordinal":1,...},
|
||
{"strategy":"597d719b…","instrument":"FRA40",...},
|
||
{"strategy":"597d719b…","instrument":"FRA40","regime_ordinal":1,...}],
|
||
"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,
|
||
"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
|
||
"worst_case":0.0436…,"sign_agreement":2,"per_instrument":[["GER40",0.0436…],["FRA40",0.0484…]]},
|
||
"winners":[...]},
|
||
{"strategy_ordinal":0,"window_ordinal":0,"regime_ordinal":1,...}],
|
||
"trace_name":"42edebd2-0"}}
|
||
```
|
||
|
||
With two stop regimes every (instrument, window) cell runs twice — the four
|
||
cells above are the "4 cell(s)" the validate summary counted. The second
|
||
regime's family ids and trace dirs carry the `-r1` ordinal suffix (the
|
||
default/first regime stays unsuffixed), each cell record names its regime,
|
||
and generalization is keyed per regime — regimes are compared, never pooled.
|
||
|
||
`aura campaign run` is register-then-run sugar for a `.json` file, but the
|
||
canonical address is always the content id — running a bare file the first
|
||
time registers it implicitly. `aura campaign runs` lists stored
|
||
realizations; `aura campaign runs <id>` dumps the bare stored record(s) (not
|
||
the `{"campaign_run": …}` emit wrapper above). If `presentation.persist_taps`
|
||
is non-empty, the run also persists the named taps under
|
||
`runs/traces/<trace_name>/…`, chartable with `aura chart`. The
|
||
`sweep`/`walkforward --trace` analog persists every member's taps as a
|
||
family charted the same way, by the handle the run prints (members keyed
|
||
`<cell>/<member>`) — also by the `--trace <NAME>` you chose, when that name
|
||
uniquely names one recorded run (`aura chart <NAME>` resolves it against the
|
||
stored campaign documents; a name reused across runs refuses rather than
|
||
guessing which one you mean).
|
||
|
||
Exit codes: a clean run exits 0; a usage error exits 2; a refusal before any
|
||
cell runs (invalid document, missing project, unresolvable strategy) exits 1;
|
||
a run that **completes with one or more failed cells** exits 3 — the run record
|
||
and every healthy cell persist, and the failed cells are named on stderr
|
||
(#272). A **gate-emptied cell** — a `std::gate` stage that filters out every
|
||
member — is a *successful* cell, not a failed one: the gate legitimately
|
||
answered "no survivors", so the run still exits 0, the cell's realization is
|
||
recorded as truncated at that stage, and an informational `aura:`-prefixed
|
||
note names the cell on stderr. Exit 3 is reserved for cells that could not be
|
||
evaluated (faults), never for an empty-but-valid result.
|