From 8378241a04091d1f694df1805c2b0c40ee20d9c8 Mon Sep 17 00:00:00 2001 From: Brummel Date: Fri, 3 Jul 2026 12:48:31 +0200 Subject: [PATCH] spec: 0106 research-artifact vocabularies (boss-signed) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Boss-signed via grounding-check PASS (fresh-context agent): all eleven load-bearing current-behaviour assumptions ratified by named green tests (shared content-id primitive + canonical golden, blueprint-store round-trip, introspect exactly-one-of guard, exit-code split, prose-fault seam, typed Scalar bind cells, RMetrics/RunMetrics field pins, FamilySelection annotations, SelectionMode/--select parity, Aura.toml discovery, blueprint_from_json + param_space). Cycle 0106 of milestone "Research artifacts — process & campaign documents": two closed-vocabulary JSON document types (process = named validation methodology wrapping shipped primitives; campaign = persisted experiment intent), intrinsic + referential validation, op-script-precedent introspection contract, content-addressed stores beside blueprints, new leaf crate aura-research. No executor this cycle. refs #189 (design basis #188) --- .../0106-research-artifact-vocabularies.md | 365 ++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 docs/specs/0106-research-artifact-vocabularies.md diff --git a/docs/specs/0106-research-artifact-vocabularies.md b/docs/specs/0106-research-artifact-vocabularies.md new file mode 100644 index 0000000..e5163a7 --- /dev/null +++ b/docs/specs/0106-research-artifact-vocabularies.md @@ -0,0 +1,365 @@ +# Research-artifact vocabularies: process & campaign documents — Design Spec + +**Date:** 2026-07-03 +**Status:** Draft — awaiting sign-off (boss run: grounding-check gate) +**Authors:** orchestrator + Claude +**Cycle:** 0106 — milestone "Research artifacts — process & campaign documents"; +work item #189 (design basis: #188 body + both addenda + fork-triage + +resolution comments; spec-bearing fork resolutions: the cycle-0106 +reconciliation comment on #189) + +## Goal + +Give roles 5 and 6b (#188) their artifacts: two new **closed-vocabulary, +typed, serialized document types** — the **process document** (a named, +versionable validation/eval methodology) and the **campaign document** +(persisted experiment intent: data windows × strategy references × param +axes × process reference × data-level presentation) — with **parsing, +two-tier validation, the op-script-precedent introspection contract, and +content-addressing**. **No executor in this cycle**: the deliverable is that +methodology and experiment intent stop being smeared over CLI flags and +shell history and become nameable, diffable, content-addressed documents an +LLM or human can author and validate headless, without a Rust compile and +without executing a run. + +Power ceiling is the ratified P1 tier (#188 addendum 2026-07-03 12:08): +bounded axes over declared finite sets and structured gates between stages. +Ladders (the third ratified P1 family) are **within the ceiling but not in +the v1 vocabulary** — they describe adaptive retry, which has no host +semantics until an executor exists; they land with the executor cycle. The +v1 process vocabulary is a thin named layer over **shipped** primitives +only (fork-triage decided item 5): the R-metrics reduction, the +argmax/plateau selection rules, deflation annotation, the R-bootstrap +Monte-Carlo, and cross-instrument generalization. + +## Architecture + +One new leaf crate **`crates/aura-research`** (workspace member; deps: +`aura-core`, `serde`, `serde_json`, `sha2`) owns: + +- the two document types and their block/construct types (serde derives + directly on them — these types ARE the wire format; the engine is not + involved), +- parsing (`format_version` envelope, deny-unknown-fields), +- **intrinsic validation** (store-free): vocabulary membership, typed + slots, P1 constraints, +- the **introspection contract**: enumerate vocabulary, describe one + block's typed slots, list a partial document's open slots, +- **canonical serialization + content id** — and the canonical-bytes + SHA-256 helper moves here as the library-hosted single primitive; + `aura-cli`'s existing `content_id` (crates/aura-cli/src/main.rs:2587, + shared today by `topology_hash` at main.rs:2595 and + `graph introspect --content-id`/`--identity-id`) delegates to it, + byte-identically. + +**`crates/aura-registry`** gains the content-addressed stores for both +document types beside the existing blueprint store +(`put_blueprint`/`get_blueprint`, crates/aura-registry/src/lib.rs:119/127): +`processes/.json`, `campaigns/.json`, plus the +**referential validation** tier (registry depends on engine and can load +blueprints; aura-research stays a leaf and never sees a store). + +**`crates/aura-cli`** gains two role-addressed verb families — `aura +process …` (role 5) and `aura campaign …` (role 6b) — each with `validate +` and `introspect` mirroring the shipped `graph introspect` flag +idiom (`--vocabulary` / `--block ` / `--unwired ` / +`--content-id `, exactly-one-of enforced as in +crates/aura-cli/src/graph_construct.rs:199), and `register ` to put a +valid document into the project's store. Faults are presented as +house-style prose at the binary seam (the #162 convention; error types +stay Display-free and by-identifier); usage errors exit 2, runtime +refusals exit 1 (the #175 split). + +## Concrete code shapes + +### The worked author example (role 5 + role 6b, no Rust, no run) + +A methodology author writes `wf-deflated-screen.process.json`: + +```json +{ + "format_version": 1, + "kind": "process", + "name": "wf-deflated-screen", + "description": "Sweep, keep only deflated-positive candidates, then walk-forward.", + "pipeline": [ + { "block": "std::sweep", + "metric": "net_expectancy_r", + "select": "plateau:worst", + "deflate": true }, + { "block": "std::gate", + "all": [ + { "metric": "net_expectancy_r", "cmp": "gt", "value": 0.0 }, + { "metric": "overfit_probability", "cmp": "lt", "value": 0.1 } + ] }, + { "block": "std::walk_forward", + "folds": 4, + "in_sample_bars": 4000, + "out_of_sample_bars": 1000, + "metric": "net_expectancy_r", + "select": "argmax" } + ] +} +``` + +```console +$ aura process validate wf-deflated-screen.process.json +process document valid (intrinsic): 3 pipeline blocks, 2 gate predicates +$ aura process introspect --content-id wf-deflated-screen.process.json +content:4e2d… +$ aura process register wf-deflated-screen.process.json # inside a project +registered process content:4e2d… (runs/processes/4e2d….json) +``` + +A campaign author writes `ger40-momentum-screen.campaign.json`, referencing +the process and a strategy **by id only** (reference-only slot, #188 +resolution comment): + +```json +{ + "format_version": 1, + "kind": "campaign", + "name": "ger40-momentum-screen", + "data": { + "instruments": ["GER40"], + "windows": [ { "from_ms": 1704067200000, "to_ms": 1719792000000 } ] + }, + "strategies": [ + { "ref": { "content_id": "9f3a…" }, + "axes": { "fast": [8, 12, 16], "slow": [21, 34] } } + ], + "process": { "ref": { "content_id": "4e2d…" } }, + "seed": 1, + "presentation": { + "persist_taps": ["net_r_equity"], + "emit": ["family_table", "selection_report"] + } +} +``` + +```console +$ aura campaign validate ger40-momentum-screen.campaign.json +campaign document valid (intrinsic): 1 strategy, 2 axes (6 points), process ref, 1 window +campaign document valid (referential): strategy 9f3a… found, process 4e2d… found, axes ⊆ param space +$ aura campaign introspect --vocabulary +std::data campaign section: instruments + windows +std::strategy campaign section: strategy ref (content or identity id) + param axes +… +$ aura campaign introspect --unwired draft.campaign.json +open slot: process.ref (required, one of: content_id, identity_id) +open slot: strategies[0].axes.slow (axis declared empty — a P1 axis is a non-empty finite set) +``` + +Authoring feedback loops (`--unwired`, precise refusals) replace the +compile cycle; the Blockly litmus test is checkable because +`--vocabulary`/`--block` enumerate every block and its typed slots. + +### Document types (aura-research, primary shapes) + +```rust +// crates/aura-research/src/lib.rs (new crate) +#[derive(Serialize, Deserialize, ...)] +pub struct ProcessDoc { + pub format_version: u32, // envelope, must be 1 + pub kind: DocKind, // tag: Process + pub name: String, + #[serde(default, skip_serializing_if = "...")] + pub description: Option, + pub pipeline: Vec, +} + +#[derive(Serialize, Deserialize, ...)] +#[serde(tag = "block")] +pub enum StageBlock { + #[serde(rename = "std::sweep")] + Sweep { metric: MetricId, select: SelectRule, #[serde(default)] deflate: bool }, + #[serde(rename = "std::gate")] + Gate { all: Vec }, // non-empty conjunction + #[serde(rename = "std::walk_forward")] + WalkForward { folds: u32, in_sample_bars: u64, out_of_sample_bars: u64, + metric: MetricId, select: SelectRule }, + #[serde(rename = "std::monte_carlo")] + MonteCarlo { resamples: u32, block_len: u32 }, // terminal annotator in v1 + #[serde(rename = "std::generalize")] + Generalize { metric: MetricId }, // terminal annotator in v1 +} + +#[derive(Serialize, Deserialize, ...)] +pub struct Predicate { pub metric: MetricId, pub cmp: Cmp, pub value: f64 } +pub enum Cmp { Gt, Ge, Lt, Le } // closed comparator set; no eq on floats + +// MetricId: newtype over the closed declared-metric vocabulary — the scalar +// field names of shipped RMetrics/RunMetrics (crates/aura-analysis/src/lib.rs:58/:86) +// plus the selection annotations deflated_score / overfit_probability / +// neighbourhood_score (FamilySelection, crates/aura-analysis/src/lib.rs:32). +// SelectRule: Argmax | PlateauMean | PlateauWorst — mirrors SelectionMode +// (crates/aura-analysis/src/lib.rs:17) and the CLI --select strings. +``` + +```rust +#[derive(Serialize, Deserialize, ...)] +pub struct CampaignDoc { + pub format_version: u32, + pub kind: DocKind, // tag: Campaign + pub name: String, + #[serde(default, ...)] pub description: Option, + pub data: DataSection, // instruments: Vec, windows: Vec + pub strategies: Vec, // ref + axes + pub process: ProcessRef, // { ref: DocRef } — reference-only, never inline + pub seed: u64, + pub presentation: Presentation, // persist_taps: Vec, emit: Vec +} + +#[derive(Serialize, Deserialize, ...)] +pub enum DocRef { ContentId(String), IdentityId(String) } // one-of, typed slots +pub struct StrategyEntry { pub r#ref: DocRef, pub axes: BTreeMap> } +pub enum EmitKind { FamilyTable, SelectionReport } // closed v1 set +``` + +Axis cells are typed `aura_core::Scalar` values — the same typed-cell +pattern the op-script binds use (`OpDoc::Add { bind: BTreeMap }`, crates/aura-cli/src/graph_construct.rs:37). + +### Validation, canonical form, introspection (secondary shapes) + +```rust +// intrinsic tier (aura-research; store-free) +pub fn validate_process(doc: &ProcessDoc) -> Vec; +pub fn validate_campaign(doc: &CampaignDoc) -> Vec; +// DocFault: by-identifier, Display-free (CLI phrases it) — e.g. +// UnknownMetric{stage,metric}, EmptyAxis{strategy,axis}, EmptyPipeline, +// GateAfterTerminalStage{stage}, BadFormatVersion(u32), EmptyConjunction{stage}, +// NoWindow, NoStrategy, UnknownEmitKind(String) … + +// canonical form + ids (aura-research) +pub fn process_to_json(doc: &ProcessDoc) -> String; // canonical: omit-defaults, +pub fn campaign_to_json(doc: &CampaignDoc) -> String; // struct field order, no trailing \n +pub fn content_id_of(canonical_bytes: &str) -> String; // SHA-256 hex — THE primitive; + // aura-cli::content_id delegates here + +// introspection contract (aura-research) +pub fn process_vocabulary() -> Vec; // id + one-line doc +pub fn campaign_vocabulary() -> Vec; +pub fn describe_block(id: &str) -> Option; // typed slots: name/kind/required/enum values +pub fn open_slots_process(text: &str) -> Result, DocError>; // partial-doc probe +pub fn open_slots_campaign(text: &str) -> Result, DocError>; +``` + +```rust +// referential tier (aura-registry; needs stores + engine) +pub fn put_process(dir: &Path, canonical_json: &str) -> Result; // returns content id +pub fn get_process(dir: &Path, content_id: &str) -> Result; +pub fn put_campaign(…) / get_campaign(…); // same pattern as put_blueprint/get_blueprint +pub fn validate_campaign_refs(doc: &CampaignDoc, roots: &StoreRoots) + -> Vec; +// RefFault: ProcessNotFound(id), StrategyNotFound(id), IdentityAmbiguityEmpty(id), +// AxisNotInParamSpace{strategy_ref, axis} — axes ⊆ the referenced blueprint's +// param_space (blueprint loaded from the store via blueprint_from_json). +``` + +Intrinsic P1 constraints enforced (the ratified ceiling made structural): +every axis is a non-empty finite value list; every gate is a non-empty +conjunction of typed `{metric, cmp, value}` triples whose metrics are in +the declared vocabulary; gates may not follow `std::monte_carlo` / +`std::generalize` (terminal annotators in v1 — no shipped aggregation +metric exists for their outputs to gate on; widening this is executor-cycle +work); no construct carries variables, recursion, or unbounded iteration — +such shapes are unrepresentable, not merely refused. + +### CLI wiring (secondary) + +`Command::Process(ProcessCmd)` / `Command::Campaign(CampaignCmd)` in +crates/aura-cli/src/main.rs beside `Command::Graph`; a new +`crates/aura-cli/src/research_docs.rs` module (graph_construct.rs +analogue) holds dispatch + fault presentation. `validate` runs intrinsic +always, referential when a project env is discovered (Aura.toml discovery, +crates/aura-cli/src/project.rs:169) and reports each tier on its own line; +outside a project the referential line is +`referential checks skipped (no Aura.toml found up from )`. +`register` refuses invalid documents; `--content-id` refuses +intrinsically-invalid documents (an invalid document has no canonical +form). + +## Components + +- `crates/aura-research` (new): document types, intrinsic validation, + canonical form + content id (the moved primitive), introspection + contract, `DocFault`/`DocError`. +- `crates/aura-registry`: `processes/`/`campaigns/` stores (blueprint-store + pattern), referential tier (`validate_campaign_refs`), `RefFault`. +- `crates/aura-cli`: `process`/`campaign` verb families, presentation of + faults, `content_id` delegation to `aura_research::content_id_of`. +- `Cargo.toml` (workspace): new member. + +## Data flow + +Author writes JSON text → `parse` (envelope + deny-unknown-fields) → +intrinsic `validate_*` (vocabulary membership, typed slots, P1 +constraints) → canonical `*_to_json` → `content_id_of` → `register` puts +canonical bytes into the project store keyed by content id → a campaign's +`DocRef`s resolve against those stores (referential tier), and its axes +resolve against the referenced blueprint's `param_space` loaded from the +blueprint store. Nothing executes; the registry stays the World's memory +(C18), now holding intent documents beside blueprints and run records. + +## Error handling + +Three by-identifier fault layers, all Display-free with CLI-side prose +(the #162 seam convention, pinned like +`graph_build_renders_a_bind_kind_mismatch_as_prose`): + +- `DocError` — parse-level: not JSON, bad envelope (`format_version` ≠ 1), + unknown `kind`, unknown fields, malformed ref. +- `DocFault` — intrinsic-validation findings (list, not first-fault): + unknown metric/block/emit kind, empty axis/pipeline/conjunction, gate + after a terminal stage, missing sections. +- `RefFault` — referential findings: unresolved process/strategy id, + identity id resolving to no stored blueprint, axis not in the referenced + strategy's param space. + +Exit codes: clap usage errors 2; validation refusals and runtime faults 1 +(the #175 split, already pinned at the binary seam). + +## Testing strategy + +- **aura-research unit tests:** parse/serialize round-trips for both + types; canonical-form golden pins (byte-exact, the blueprint-golden + pattern); content-id stability goldens; one refusal test per `DocFault` + variant (incl. every P1 constraint); vocabulary/describe/open-slots + shape tests (every block's slots enumerable — the Blockly litmus pin); + a test that `serde` rejects an inline process body in the campaign's + `process` slot (reference-only is structural). +- **aura-registry tests:** store round-trip for both types (hash-keyed + like the blueprint-store tests); `validate_campaign_refs` positive + + one per `RefFault` (fixtures: a stored blueprint with a known + `param_space`). +- **aura-cli seam tests:** binary-level validate (valid, intrinsic + refusal, referential refusal inside a scaffolded project, outside-project + skip line); introspect `--vocabulary`/`--block`/`--unwired`/ + `--content-id` outputs; prose (no Debug leak) pinned; exit codes pinned; + `content_id` delegation equivalence (`topology_hash`/`--content-id` + bytes unchanged — guarded by the existing golden + `signal_serializes_to_canonical_golden` and id tests staying green + untouched). +- Full gates: `cargo test --workspace`, `cargo clippy --workspace + --all-targets -- -D warnings`, `cargo doc --workspace --no-deps`. + +## Acceptance criteria + +1. The worked example above runs verbatim (modulo hash hex): both + documents author as text, validate headless with per-tier report lines, + yield stable content ids, and register into a project's store — no + Rust compile, no run executed. +2. `aura process introspect` / `aura campaign introspect` expose the full + introspection contract (vocabulary, block schema, open slots of a + partial document, content id) with the exactly-one-of flag guard. +3. Every invalid shape in the fault tables refuses with house-style prose + naming the offending element; no Rust `Debug` form reaches the user; + exit codes follow the usage-2/runtime-1 split. +4. The declared-metric vocabulary, select rules, and stage blocks are + exactly the shipped-primitive wrappers named in this spec — nothing + invented; ladders and any executor semantics are absent. +5. The pre-existing suite stays green unchanged; blueprint content/identity + ids and `topology_hash` are byte-identical to before (the content-id + primitive moved, not changed). +6. Workspace gates green: test, clippy `-D warnings`, doc build.