fieldtest: cycle-0108 annotator-stages corpus (GREEN)

Verdict: honest, recorded, reproducible inference on top of the 0107
loop — pooled_oos provably pools the wf OOS windows (n_trades 948+895
= 1843), per_survivor flags a razor-thin gate-passer at
prob_le_zero 0.525 (the inference layer earning its keep), the
two-instrument std::generalize records worst_case/sign_agreement over
last-wf-window nominees, byte-identical across three runs modulo the
run counter, and all eight boundary probes refuse loudly at the right
tier. 0 bugs.

Triage: F1-F5 carry-on; F6 (campaign validate blesses
executor-unrunnable shapes — the third tier invisible) and F11 (no
CLI read-back for recorded realizations) filed as issues; F7/F9/F10
doc-tightens follow this commit inline; F8 (generalize R-metric set
undiscoverable + misleading refusal prose for R-denominated-but-
unranked metrics) filed with the --metrics tag half. The fieldtest
spec rides in docs/specs/ as next-cycle planner input per convention.

refs #200
This commit is contained in:
2026-07-04 01:52:09 +02:00
parent ae2fac3212
commit 6138894b07
32 changed files with 998 additions and 0 deletions
@@ -0,0 +1,198 @@
# Fieldtest — cycle 0108 — annotator stages execute (#200) — 2026-07-04
**Role:** downstream consumer (methodology / campaign designer), public
interface only (ledger C18 0107/0108 realization paragraphs + C1, glossary,
CLI `--help` + introspection verbs, the cycle-0107 fieldtest corpus as shape
hints). No `crates/` source read. Binary built from HEAD
(`cargo build --workspace`, `target/debug/aura`, commit
`ae2fac3212f931bf5a3b277ba36fbe4010bf7b9d`). Real data: scaffolded a fresh
`demo_project/` (`aura new` — nested `.git` correctly skipped, 0107 orthogonal
fix confirmed), `cargo build` for the dylib, ran the executor over the local
Pepperstone archive (`/mnt/tickdata/Pepperstone`, GER40 + FRA40, 2024-09).
## Fixtures (this directory)
- `c0108_strategy_smacross.json` / `_built.json` — the open (sweepable) SMA-cross
strategy op-script + its `graph build` envelope (content id `597d71…`, 3 open
params `fast.length` / `slow.length` / `bias.scale`)
- `c0108_process_full_pooled.json` — sweep(deflate) → gate → walk_forward →
monte_carlo (the PooledOos path)
- `c0108_process_mc_persurvivor.json` — sweep → gate → monte_carlo (no wf → the
PerSurvivor path)
- `c0108_process_full_generalize.json` — sweep → gate → wf → mc → generalize
(the full v2 suffix)
- `c0108_campaign_full_pooled.json` / `_mc_persurvivor.json` — GER40 2024-09
- `c0108_campaign_2instr_generalize.json` — GER40 + FRA40, the end-to-end
generalize
- `c0108_probe_{gate_after_mc,wf_after_mc,dup_mc,gen_not_last,zero_resamples,`
`zero_blocklen,nonr_generalize}.json` — boundary probes + their
`c0108_probe_campaign_*.json` runners; `c0108_probe_campaign_singleinstr_`
`generalize.json`, `c0108_probe_generalize_maxrdrawdown.json`
- `c0108_realization_campaign_runs.jsonl` — the recorded realization (copied out
of the project-gitignored `runs/` store): 3 pooled runs, 1 per-survivor, 1
two-instrument-with-generalizations, 3 sweep→generalize metric probes
## Findings
### F1 [working] The full annotator suffix closes — pooled_oos over the wf OOS series
`aura campaign run c0108_campaign_full_pooled.json` executes
`sweep → gate → walk_forward → monte_carlo` over GER40 2024-09 (exit 0). The
`std::monte_carlo` stage records exactly the ledger's PooledOos semantics:
`"bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,…},
"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}`.
`n_trades:1843` is exactly the two wf OOS windows pooled in roll order
(948 + 895 — the two walk_forward OOS member counts). One `r_bootstrap` over the
whole pooled series, seeded from the doc `seed=42`. Intent → execution →
recorded, honest inference. → carry-on.
### F2 [working] No-wf campaign records one bootstrap per survivor; the honesty seam bites
`aura campaign run c0108_campaign_mc_persurvivor.json` (sweep → gate → mc, no wf)
records `per_survivor` as `[[ordinal, bootstrap], …]` — one entry per gate
survivor `[0,1,2]`, each an `r_bootstrap` over that member's full in-memory R
series (`n_trades` 3903 / 2581 / 3811, matching the sweep members). Survivor 1
(fast=2/slow=16) passed the `expectancy_r > 0` gate at a razor-thin
`expectancy_r = 0.0019`, yet its bootstrap shows `prob_le_zero: 0.525` — the mc
annotator correctly flags a gate-passing member as coin-flip noise. This is the
inference layer earning its keep. → carry-on.
### F3 [working] Determinism is byte-identical across re-runs (modulo the run counter)
Three runs of `c0108_campaign_full_pooled.json` (run 0/1/2) produce
byte-identical `campaign_run` lines after normalizing only the per-campaign run
counter (the `"run"` field + the family-id trailing run index). Every bootstrap
percentile, `prob_le_zero`, gate survivor set, and selection reproduces exactly.
C1's "pure function of doc + stores + data" holds through the annotator stages.
→ carry-on.
### F4 [working] Campaign-scope generalize closes end-to-end over two instruments
`aura campaign run c0108_campaign_2instr_generalize.json`
(sweep → gate → wf → mc → generalize, GER40 + FRA40, exit 0) runs both cells then
records a campaign-scope `generalizations` entry keyed (strategy 0 × window 0):
`"generalization":{"selection_metric":"expectancy_r","n_instruments":2,
"worst_case":0.04367535583327283,"sign_agreement":2,
"per_instrument":[["GER40",0.0437…],["FRA40",0.0484…]]}`.
The nominees are each cell's last wf-OOS winner (fast=4/slow=8), not the
sweep winner (fast=2/slow=8) — matching the ledger. `worst_case` is the weaker
instrument (GER40); divergent winners would surface via `winners[]` params,
never averaged. The serde-default sparse widening is visible in the store: the
`generalizations` key appears only on this two-instrument line; the four
single-cell lines omit it entirely. → carry-on.
### F5 [working] The annotator tier boundary is honest and precise on both sides
Every boundary probe refuses cleanly with per-pair prose and the right exit code:
- gate after mc — intrinsic (`process validate`) — `stage 2: a gate cannot follow a terminal stage (monte_carlo/generalize)` — exit 1
- wf after mc — executor (`campaign run`; validate passes) — `process pipeline is not executable: stage 2: std::walk_forward cannot follow std::monte_carlo` — exit 1
- dup monte_carlo — executor — `stage 2: std::monte_carlo may appear at most once` — exit 1
- generalize not last — executor — `stage 2: std::monte_carlo cannot follow std::generalize` — exit 1
- zero resamples — executor — `process stage 1: monte_carlo resamples must be > 0` — exit 1
- zero block_len — executor — `process stage 1: monte_carlo block_len must be > 0` — exit 1
- non-R generalize metric — executor — `std::generalize metric "total_pips" is not an R metric (gross/net R families generalize; pip metrics do not)` — exit 1
- single-instrument generalize — executor — `campaign has 1 instrument(s); std::generalize needs at least 2` — exit 1
The intrinsic tier catches only gate-after-terminal; the executor preflight
catches the shape/param/metric faults — exactly the "test-pinned on both sides"
boundary the ledger describes. → carry-on.
### F6 [friction] `campaign validate` blesses process shapes the executor then refuses
`wf after mc` is a pure process-shape fault — no data or instrument count needed
to detect it — yet `aura campaign validate c0108_probe_campaign_wf_after_mc.json`
returns exit 0 (`valid (intrinsic) … valid (referential): all references
resolve`), and only `campaign run` refuses it. Same for dup-mc and
generalize-not-last. The author sees a green validate, authors on, and is
stopped only at run time. The three-tier model (intrinsic / referential /
executor-preflight) is invisible from the validate surface. (Kin to 0107 F7:
"valid" ≠ "runnable".) Repro: `campaign validate` then `campaign run` on any of
the three shape probes.
→ plan: run the executor shape/param preflight at `campaign validate` time (it is
a pure process property), or name the third tier in the validate output.
### F7 [doc-gap] The pooled-vs-per-survivor duality is invisible to the author surface
`aura process introspect --block std::monte_carlo` says only
`R-bootstrap over realised R (terminal annotator)` + `resamples` / `block_len`.
Nothing in `--block`, `--vocabulary`, `--metrics`, or the glossary tells the
author that the stage's output shape and semantics flip on whether a
walk_forward precedes it (one pooled-OOS bootstrap vs one bootstrap per
survivor over in-sample series). The distinction is documented only in the
design ledger (C18 0108) — not on the surface a headless campaign author reads.
A user cannot predict from introspection what `std::monte_carlo` will produce.
→ tighten docs: fold the position-dependent semantics into the `--block`
description (and/or a glossary `stage bootstrap` entry with the two output
shapes).
### F8 [doc-gap] The generalize R-metric set is unpredictable, and the refusal prose misleads
`std::generalize` requires an "R metric", but no introspection surface says which
of the 17 metrics qualify. `--metrics` tags are `rankable | gate | annotation`
none marks R / generalizable. Probing the boundary:
- accepted: `expectancy_r`, `net_expectancy_r`, `sqn`, `sqn_normalized`
- refused: `total_pips`, `max_drawdown` (both rankable!), `win_rate`, and
`max_r_drawdown` — the last is R-denominated (its name literally contains
`_r`, it is a drawdown in R) yet is refused with
`… is not an R metric (gross/net R families generalize; pip metrics do not)`.
Calling `max_r_drawdown` a "pip metric" is wrong; the real rule is "the
rankable gross/net-R expectancy family", which no surface states.
So the author must guess the metric, and the one honest hint (the refusal) both
under-describes the accepted set and mis-describes why an R metric was refused.
Repro: `aura campaign run c0108_probe_campaign_nonr_generalize.json` and the
`c0108_probe_generalize_maxrdrawdown.json` process.
→ tighten: add a `generalize`/R applicability tag to `--metrics`; correct the
refusal prose for R-denominated-but-unranked metrics.
### F9 [doc-gap] The mc slot hint says "non-negative" but the executor refuses 0
`--block std::monte_carlo` labels both `resamples` and `block_len`
`required, non-negative integer (u32)`. Non-negative includes 0, but 0 is
refused at run (`… must be > 0`). The hint describes the u32 type, not the
valid range, so it silently contradicts the guard the executor enforces.
Repro: `aura campaign run c0108_probe_campaign_zero_resamples.json`.
→ tighten the slot hint to "positive integer" (or "≥ 1").
### F10 [doc-gap] The recorded realization's output schema is undocumented
The realization a consumer reads — `prob_le_zero`, `per_survivor`, `pooled_oos`,
`e_r` percentiles, `worst_case`, `sign_agreement`, `n_instruments` — appears in
no public prose (grep of glossary + INDEX finds none of the wire-format field
names; only the internal enum names `PooledOos` / `PerSurvivor` are in the
ledger). A user seeing `prob_le_zero: 0.274` has no public definition (it is
P[bootstrap E[R] ≤ 0]). Also undocumented: the stdout emit wraps each record as
`{"campaign_run": …}` while the persisted `campaign_runs.jsonl` line is the
bare record — a consumer parsing the store must know to drop the wrapper.
→ tighten docs: define the campaign-run realization output fields (a glossary
entry or a `campaign introspect --realization` schema dump).
### F11 [friction] The annotator realizations are write-only — no CLI read-back
`aura runs` exposes only `families` / `family`; there is no verb to read back a
recorded `campaign_run`. The bootstrap distributions and generalization scores
the annotator stages compute are visible only in the run's own stdout; once the
terminal scrolls, the only access is hand-parsing `runs/campaign_runs.jsonl`
(which is also project-gitignored). Every cell × stage × per-survivor bootstrap
is concentrated into one JSONL line, which grows unboundedly with campaign size
and is unreadable raw. → plan: an `aura campaign runs` / `run <id>` read-back
verb, at least for the realized bootstrap / generalization records.
## Verdict
Yes — the annotator stages deliver honest, recorded, reproducible inference on
top of the 0107 loop. The v2 suffix `sweep [gate]* [wf]? [mc]? [generalize]?`
executes end to end over real data: `std::monte_carlo` bootstraps the stage's
incoming R-evidence with the exact position-dependent semantics the ledger
promises (PooledOos over the wf OOS series — `n_trades` provably the pooled OOS
count; PerSurvivor over each gate survivor's in-sample series), and
`std::generalize` runs at campaign scope over per-cell nominees, scoring the
worst-case R floor and exposing divergent winners rather than averaging them.
The inference is honest (a gate-passing survivor is flagged
`prob_le_zero=0.525`; a single-instrument generalize refuses rather than
fabricating a floor), recorded (the sparse serde widening lands cleanly — the
`generalizations` key appears only where earned), and reproducible
(byte-identical across three runs modulo the run counter, C1 intact). The
annotator tier boundary is a genuine strength: eight distinct
shape/param/metric/instrument faults each refuse loudly, precisely, and at the
right tier with exit 1.
Nothing here is a bug. What remains is discoverability, concentrated at the
author's first contact with the two new stages: the mc pooled-vs-per-survivor
duality (F7), the generalize R-metric set + misleading refusal prose (F8), the
"non-negative" slot hint that contradicts the `>0` guard (F9), the entirely
undocumented realization output schema (F10), a validate tier that greenlights
executor-unrunnable shapes (F6), and the absence of any read-back for the
realizations the stages so carefully record (F11). The engine is honest; the
map to it is thin.
Counts: working 5 (F1F5), friction 2 (F6, F11), doc-gap 4 (F7F10),
spec_gap 0, bugs 0.
@@ -0,0 +1,22 @@
{
"format_version": 1,
"kind": "campaign",
"name": "ger40-fra40-smacross-full-suffix",
"seed": 42,
"data": {
"instruments": ["GER40", "FRA40"],
"windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ]
},
"strategies": [
{
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
"axes": {
"fast.length": { "kind": "I64", "values": [2, 4] },
"slow.length": { "kind": "I64", "values": [8, 16] },
"bias.scale": { "kind": "F64", "values": [0.5] }
}
}
],
"process": { "ref": { "content_id": "f73db6e6b918fdcde51b9572435af9612fa213679a0c92fe194c46bcc98a2d62" } },
"presentation": { "persist_taps": [], "emit": ["family_table", "selection_report"] }
}
@@ -0,0 +1,19 @@
{
"format_version": 1,
"kind": "campaign",
"name": "ger40-smacross-wf-mc",
"seed": 42,
"data": { "instruments": ["GER40"], "windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ] },
"strategies": [
{
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
"axes": {
"fast.length": { "kind": "I64", "values": [2, 4] },
"slow.length": { "kind": "I64", "values": [8, 16] },
"bias.scale": { "kind": "F64", "values": [0.5] }
}
}
],
"process": { "ref": { "content_id": "c94d0a59c347275c1df85fd6d08993774b61c629dc1c5595498d73100ed462c7" } },
"presentation": { "persist_taps": [], "emit": ["family_table", "selection_report"] }
}
@@ -0,0 +1,19 @@
{
"format_version": 1,
"kind": "campaign",
"name": "ger40-smacross-mc-persurvivor",
"seed": 42,
"data": { "instruments": ["GER40"], "windows": [ { "from_ms": 1725148800000, "to_ms": 1727740800000 } ] },
"strategies": [
{
"ref": { "content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a" },
"axes": {
"fast.length": { "kind": "I64", "values": [2, 4] },
"slow.length": { "kind": "I64", "values": [8, 16] },
"bias.scale": { "kind": "F64", "values": [0.5] }
}
}
],
"process": { "ref": { "content_id": "26a206a82f2e654df8fea920c199f26039cc915bef413f54d446b160ce72321e" } },
"presentation": { "persist_taps": [], "emit": ["family_table", "selection_report"] }
}
@@ -0,0 +1,57 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-dup-mc",
"seed": 42,
"data": {
"instruments": [
"GER40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "d969be348c2280a031bca585d408c77e94ff69aac534a73366b0fb4dd3a60594"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,58 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-gen-not-last",
"seed": 42,
"data": {
"instruments": [
"GER40",
"FRA40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "775bca753bcee755767d96ff05371b1b3669d3191bf52507fe7bb9d16bf2fe36"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,58 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-nonr-generalize",
"seed": 42,
"data": {
"instruments": [
"GER40",
"FRA40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "9ae08fb2f5eef3f12c8918b88fb171b2a2715c021f212fa7a42dcac7fc2e7551"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,57 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-singleinstr-generalize",
"seed": 42,
"data": {
"instruments": [
"GER40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "f73db6e6b918fdcde51b9572435af9612fa213679a0c92fe194c46bcc98a2d62"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,57 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-wf-after-mc",
"seed": 42,
"data": {
"instruments": [
"GER40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "847d5f487101827b9de029b8be45f8f31bd6be7a795a42b30d54112117beac2e"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,57 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-zero-blocklen",
"seed": 42,
"data": {
"instruments": [
"GER40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "528ac3d80c1183a625dfb75959eac9fe8b4d0b8d594072a53c77d56bee7c47e2"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,57 @@
{
"format_version": 1,
"kind": "campaign",
"name": "probe-zero-resamples",
"seed": 42,
"data": {
"instruments": [
"GER40"
],
"windows": [
{
"from_ms": 1725148800000,
"to_ms": 1727740800000
}
]
},
"strategies": [
{
"ref": {
"content_id": "597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a"
},
"axes": {
"fast.length": {
"kind": "I64",
"values": [
2,
4
]
},
"slow.length": {
"kind": "I64",
"values": [
8,
16
]
},
"bias.scale": {
"kind": "F64",
"values": [
0.5
]
}
}
}
],
"process": {
"ref": {
"content_id": "90fa4f42f67937967e67f071c87b4d29e28562908fcb98536cc200134b3c998c"
}
},
"presentation": {
"persist_taps": [],
"emit": [
"family_table"
]
}
}
@@ -0,0 +1,6 @@
{ "format_version": 1, "kind": "process", "name": "probe-dup-mc",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::monte_carlo", "resamples": 500, "block_len": 3 }
] }
@@ -0,0 +1,6 @@
{ "format_version": 1, "kind": "process", "name": "probe-gate-after-mc",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] }
] }
@@ -0,0 +1,6 @@
{ "format_version": 1, "kind": "process", "name": "probe-gen-not-last",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::generalize", "metric": "expectancy_r" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 }
] }
@@ -0,0 +1,3 @@
{ "format_version": 1, "kind": "process", "name": "genprobe-max_r_drawdown",
"pipeline": [ { "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::generalize", "metric": "max_r_drawdown" } ] }
@@ -0,0 +1,5 @@
{ "format_version": 1, "kind": "process", "name": "probe-nonr-generalize",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::generalize", "metric": "total_pips" }
] }
@@ -0,0 +1,6 @@
{ "format_version": 1, "kind": "process", "name": "probe-wf-after-mc",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" }
] }
@@ -0,0 +1,5 @@
{ "format_version": 1, "kind": "process", "name": "probe-zero-blocklen",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 0 }
] }
@@ -0,0 +1,5 @@
{ "format_version": 1, "kind": "process", "name": "probe-zero-resamples",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 0, "block_len": 5 }
] }
@@ -0,0 +1,13 @@
{
"format_version": 1,
"kind": "process",
"name": "sweep-gate-wf-mc-generalize",
"description": "The full annotator suffix: sweep, gate, rolling walk-forward, pooled-OOS R-bootstrap, then a campaign-scope cross-instrument generalize on expectancy_r.",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 },
{ "block": "std::generalize", "metric": "expectancy_r" }
]
}
@@ -0,0 +1,12 @@
{
"format_version": 1,
"kind": "process",
"name": "robust-sweep-gate-wf-mc",
"description": "Sweep with deflation, gate on positive expectancy, rolling walk-forward, then a moving-block R-bootstrap of the pooled OOS series.",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax", "deflate": true },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::walk_forward", "in_sample_ms": 1209600000, "out_of_sample_ms": 604800000, "step_ms": 604800000, "mode": "rolling", "metric": "sqn", "select": "argmax" },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 }
]
}
@@ -0,0 +1,11 @@
{
"format_version": 1,
"kind": "process",
"name": "sweep-gate-mc-persurvivor",
"description": "Sweep, gate on positive expectancy, then a moving-block R-bootstrap of each surviving member's fresh in-memory R series (no walk-forward, so per-survivor).",
"pipeline": [
{ "block": "std::sweep", "metric": "sqn", "select": "argmax" },
{ "block": "std::gate", "all": [ { "metric": "expectancy_r", "cmp": "gt", "value": 0.0 } ] },
{ "block": "std::monte_carlo", "resamples": 1000, "block_len": 5 }
]
}
@@ -0,0 +1,8 @@
{"campaign":"4d14d5ec431b7767b85b5be04a5165f5e6127c6486a9249b306c7f2aeebf14e0","process":"c94d0a59c347275c1df85fd6d08993774b61c629dc1c5595498d73100ed462c7","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"4d14d5ec-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax","deflated_score":-1.0290009962190139,"overfit_probability":0.3876123876123876,"n_resamples":1000,"block_len":5,"seed":42}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::walk_forward","family_id":"4d14d5ec-0-GER40-w0-s2-0"},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,"p5":-0.045358717986233116,"p25":-0.003867370198210889,"p50":0.025041122400765753,"p75":0.054149001087649175,"p95":0.09609753891677572},"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}}]}]}
{"campaign":"4b558e12a78fc4234cb5f9d475205ce56ce742f7a9aacaec3770f1d4a679f6e4","process":"26a206a82f2e654df8fea920c199f26039cc915bef413f54d446b160ce72321e","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"4b558e12-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax"}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::monte_carlo","bootstrap":{"per_survivor":[[0,{"e_r":{"mean":0.025298818957544374,"p5":-0.01849982068009793,"p25":0.005707492926717921,"p50":0.024186660903793872,"p75":0.045175657505028846,"p95":0.07270312095687298},"prob_le_zero":0.184,"n_trades":3903,"block_len":5,"n_resamples":1000}],[1,{"e_r":{"mean":-0.0007863451183027098,"p5":-0.06191237500077215,"p25":-0.02635431542922357,"p50":-0.0015498863526125085,"p75":0.024015596809324934,"p95":0.06066777913809223},"prob_le_zero":0.525,"n_trades":2581,"block_len":5,"n_resamples":1000}],[2,{"e_r":{"mean":0.025873482678292634,"p5":-0.02258807583447356,"p25":0.004121717360938905,"p50":0.024475583833216418,"p75":0.045850761312665606,"p95":0.07739775879325204},"prob_le_zero":0.21,"n_trades":3811,"block_len":5,"n_resamples":1000}]]}}]}]}
{"campaign":"4d14d5ec431b7767b85b5be04a5165f5e6127c6486a9249b306c7f2aeebf14e0","process":"c94d0a59c347275c1df85fd6d08993774b61c629dc1c5595498d73100ed462c7","run":1,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"4d14d5ec-0-GER40-w0-s0-1","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax","deflated_score":-1.0290009962190139,"overfit_probability":0.3876123876123876,"n_resamples":1000,"block_len":5,"seed":42}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::walk_forward","family_id":"4d14d5ec-0-GER40-w0-s2-1"},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,"p5":-0.045358717986233116,"p25":-0.003867370198210889,"p50":0.025041122400765753,"p75":0.054149001087649175,"p95":0.09609753891677572},"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}}]}]}
{"campaign":"4d14d5ec431b7767b85b5be04a5165f5e6127c6486a9249b306c7f2aeebf14e0","process":"c94d0a59c347275c1df85fd6d08993774b61c629dc1c5595498d73100ed462c7","run":2,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"4d14d5ec-0-GER40-w0-s0-2","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax","deflated_score":-1.0290009962190139,"overfit_probability":0.3876123876123876,"n_resamples":1000,"block_len":5,"seed":42}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::walk_forward","family_id":"4d14d5ec-0-GER40-w0-s2-2"},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,"p5":-0.045358717986233116,"p25":-0.003867370198210889,"p50":0.025041122400765753,"p75":0.054149001087649175,"p95":0.09609753891677572},"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}}]}]}
{"campaign":"18ab89606044e981ac30b091973b4d5b0b2cc0b3bf1c6825742ad9a7ae6b4a32","process":"f73db6e6b918fdcde51b9572435af9612fa213679a0c92fe194c46bcc98a2d62","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"18ab8960-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax","deflated_score":-1.0290009962190139,"overfit_probability":0.3876123876123876,"n_resamples":1000,"block_len":5,"seed":42}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::walk_forward","family_id":"18ab8960-0-GER40-w0-s2-0"},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{"mean":0.02572811864069487,"p5":-0.045358717986233116,"p25":-0.003867370198210889,"p50":0.025041122400765753,"p75":0.054149001087649175,"p95":0.09609753891677572},"prob_le_zero":0.274,"n_trades":1843,"block_len":5,"n_resamples":1000}}}]},{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"FRA40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"18ab8960-0-FRA40-w0-s0-0","selection":{"winner_ordinal":2,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":2.1027529408673753,"mode":"Argmax","deflated_score":-0.25026485410828503,"overfit_probability":0.08491508491508491,"n_resamples":1000,"block_len":5,"seed":42}}},{"block":"std::gate","survivor_ordinals":[0,1,2]},{"block":"std::walk_forward","family_id":"18ab8960-0-FRA40-w0-s2-0"},{"block":"std::monte_carlo","bootstrap":{"pooled_oos":{"e_r":{"mean":0.013305753279586063,"p5":-0.09686234697635485,"p25":-0.02935938855824511,"p50":0.014375417056209604,"p75":0.0572151928623397,"p95":0.11740134416304716},"prob_le_zero":0.412,"n_trades":1272,"block_len":5,"n_resamples":1000}}}]}],"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,"generalization":{"selection_metric":"expectancy_r","n_instruments":2,"worst_case":0.04367535583327283,"sign_agreement":2,"per_instrument":[["GER40",0.04367535583327283],["FRA40",0.048403786424396446]]},"winners":[["GER40",[["graph.fast.length",{"I64":4}],["graph.slow.length",{"I64":8}],["graph.bias.scale",{"F64":0.5}]]],["FRA40",[["graph.fast.length",{"I64":4}],["graph.slow.length",{"I64":8}],["graph.bias.scale",{"F64":0.5}]]]]}]}
{"campaign":"f385116e32fa23775e59ca9f853a3c42a70ba61b38aa004ac02c105b30f21a2b","process":"1263772be43db2771a55d3c0b345bdacfa701f6358c3fc72122f48361783a539","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"f385116e-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax"}}}]},{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"FRA40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"f385116e-0-FRA40-w0-s0-0","selection":{"winner_ordinal":2,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":2.1027529408673753,"mode":"Argmax"}}}]}],"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,"generalization":{"selection_metric":"net_expectancy_r","n_instruments":2,"worst_case":0.028321444460569407,"sign_agreement":2,"per_instrument":[["GER40",0.028321444460569407],["FRA40",0.09120457579140544]]},"winners":[["GER40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]]],["FRA40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]]]]}]}
{"campaign":"b0bc058aa20675e05599e3b333e07be18eec84e3d23e58cd97cae644f4802fb7","process":"845ab7392b6be7b6650e05282e729d3c42c61c5b1f1a536bac261bf59e3317ce","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"b0bc058a-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax"}}}]},{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"FRA40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"b0bc058a-0-FRA40-w0-s0-0","selection":{"winner_ordinal":2,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":2.1027529408673753,"mode":"Argmax"}}}]}],"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,"generalization":{"selection_metric":"sqn","n_instruments":2,"worst_case":1.0540214890316064,"sign_agreement":2,"per_instrument":[["GER40",1.0540214890316064],["FRA40",2.1027529408673753]]},"winners":[["GER40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]]],["FRA40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]]]]}]}
{"campaign":"033d8cd0cab382c2cc2393877af919f050dcf7aed3033664fed7bc2ed756cd51","process":"236fb6363106e993b3b9c634987a765c482a2c4c76da78dbdb6f711ac65e1708","run":0,"seed":42,"cells":[{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"GER40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"033d8cd0-0-GER40-w0-s0-0","selection":{"winner_ordinal":0,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":1.0540214890316064,"mode":"Argmax"}}}]},{"strategy":"597d719b7ac607158cda3e68cd497387620397a5e93087e23da512876dafba9a","instrument":"FRA40","window_ms":[1725148800000,1727740800000],"stages":[{"block":"std::sweep","family_id":"033d8cd0-0-FRA40-w0-s0-0","selection":{"winner_ordinal":2,"params":[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]],"selection":{"selection_metric":"sqn","n_trials":4,"raw_winner_metric":2.1027529408673753,"mode":"Argmax"}}}]}],"generalizations":[{"strategy_ordinal":0,"window_ordinal":0,"generalization":{"selection_metric":"sqn_normalized","n_instruments":2,"worst_case":0.16871363772287948,"sign_agreement":2,"per_instrument":[["GER40",0.16871363772287948],["FRA40",0.4182978458755474]]},"winners":[["GER40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":2}],["slow.length",{"I64":8}]]],["FRA40",[["bias.scale",{"F64":0.5}],["fast.length",{"I64":4}],["slow.length",{"I64":8}]]]]}]}
@@ -0,0 +1,12 @@
[
{"op": "source", "role": "price", "kind": "F64"},
{"op": "add", "type": "SMA", "name": "fast"},
{"op": "add", "type": "SMA", "name": "slow"},
{"op": "feed", "role": "price", "into": ["fast.series", "slow.series"]},
{"op": "add", "type": "Sub", "name": "sub"},
{"op": "connect", "from": "fast.value", "to": "sub.lhs"},
{"op": "connect", "from": "slow.value", "to": "sub.rhs"},
{"op": "add", "type": "Bias", "name": "bias"},
{"op": "connect", "from": "sub.value", "to": "bias.signal"},
{"op": "expose", "from": "bias.bias", "as": "bias"}
]
@@ -0,0 +1 @@
{"format_version":1,"blueprint":{"name":"graph","nodes":[{"primitive":{"type":"SMA","name":"fast"}},{"primitive":{"type":"SMA","name":"slow"}},{"primitive":{"type":"Sub","name":"sub"}},{"primitive":{"type":"Bias","name":"bias"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":3,"field":0,"name":"bias"}]}}
@@ -0,0 +1,3 @@
/target
Cargo.lock
/runs
@@ -0,0 +1,3 @@
# Static project context only (C17); paths only.
[paths]
runs = "runs"
@@ -0,0 +1,10 @@
# demo_project — an aura research project
This crate is an aura project: a cdylib of node/strategy blueprints the
`aura` host loads during research (see the engine's docs/project-layout.md).
- Build: `cargo build` (the next `aura` invocation loads the fresh dylib)
- Run: `aura run blueprints/signal.json` (from anywhere inside this dir)
- Nodes live in `src/`; each is registered in `vocabulary()`/`type_ids()`
under the `demo_project::` namespace prefix.
- Topology is data (`blueprints/*.json`), node logic is Rust.
@@ -0,0 +1,14 @@
[package]
name = "demo_project"
version = "0.1.0"
edition = "2024"
publish = false
[lib]
crate-type = ["cdylib"]
[dependencies]
aura-core = { path = "/home/brummel/dev/aura/crates/aura-core" }
# Standalone workspace root: never a member of an enclosing workspace.
[workspace]
@@ -0,0 +1 @@
{"format_version":1,"blueprint":{"name":"demo_project_signal","nodes":[{"primitive":{"type":"SMA","name":"fast","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":2}}]}},{"primitive":{"type":"SMA","name":"slow","bound":[{"pos":0,"name":"length","kind":"I64","value":{"I64":4}}]}},{"primitive":{"type":"Sub"}},{"primitive":{"type":"Bias","name":"bias","bound":[{"pos":0,"name":"scale","kind":"F64","value":{"F64":0.5}}]}},{"primitive":{"type":"demo_project::Identity"}}],"edges":[{"from":0,"to":2,"slot":0,"from_field":0},{"from":1,"to":2,"slot":1,"from_field":0},{"from":2,"to":3,"slot":0,"from_field":0},{"from":3,"to":4,"slot":0,"from_field":0}],"input_roles":[{"name":"price","targets":[{"node":0,"slot":0},{"node":1,"slot":0}],"source":"F64"}],"output":[{"node":4,"field":0,"name":"bias"}]}}
@@ -0,0 +1,76 @@
//! demo_project — an aura research project: node logic + the exported vocabulary.
//!
//! Every node type this crate provides is registered in `vocabulary()` /
//! `type_ids()` under the `demo_project::` namespace prefix; the `aura` host loads
//! this cdylib per invocation and merges it with the std vocabulary.
use aura_core::{
Cell, Ctx, FieldSpec, Firing, Node, NodeSchema, PortSpec, PrimitiveBuilder,
ScalarKind,
};
/// One-input f64 pass-through. Emits `None` until its input has a value.
pub struct Identity {
out: [Cell; 1],
}
impl Identity {
pub fn new() -> Self {
Self { out: [Cell::from_f64(0.0)] }
}
pub fn builder() -> PrimitiveBuilder {
PrimitiveBuilder::new(
"demo_project::Identity",
NodeSchema {
inputs: vec![PortSpec {
kind: ScalarKind::F64,
firing: Firing::Any,
name: "value".into(),
}],
output: vec![FieldSpec { name: "value".into(), kind: ScalarKind::F64 }],
params: vec![],
},
|_| Box::new(Identity::new()),
)
}
}
impl Default for Identity {
fn default() -> Self {
Self::new()
}
}
impl Node for Identity {
fn lookbacks(&self) -> Vec<usize> {
vec![1]
}
fn eval(&mut self, ctx: Ctx<'_>) -> Option<&[Cell]> {
let w = ctx.f64_in(0);
if w.is_empty() {
return None;
}
self.out[0] = Cell::from_f64(w[0]);
Some(&self.out)
}
fn label(&self) -> String {
"demo_project::Identity".to_string()
}
}
fn vocabulary(type_id: &str) -> Option<PrimitiveBuilder> {
match type_id {
"demo_project::Identity" => Some(Identity::builder()),
_ => None,
}
}
fn type_ids() -> &'static [&'static str] {
&["demo_project::Identity"]
}
aura_core::aura_project! {
namespace: "demo_project",
vocabulary: vocabulary,
type_ids: type_ids,
}