spec: 0096 discover a loaded blueprint's sweepable axes (boss-signed)
Add `aura sweep <blueprint.json> --list-axes`: a read-only query sub-mode that lists a loaded blueprint's open sweepable knobs (one `<name>:<kind>` per line, param_space() order), then exits. Closes the discovery gap #169 surfaced — today the only way to learn the axis names is to guess one and read a MissingKnob/UnknownKnob error, and that error names only the rejected knob, never the valid set. Prerequisite for #173 (IS-refit walk-forward must know which axes to re-fit per in-sample window). Derived surface decision (recorded on #169): the query lands on the SWEEP verb, not `graph introspect`. The axis names `--axis` accepts are produced by the sweep's own harness-wrapping — the loaded signal (name "stage1_signal") is nested via wrap_stage1r as a BlueprintNode::Composite, so collect_params prefixes the names (stage1_signal.fast.length), NOT the raw param_space (fast.length; empirically: the bare name → UnknownKnob). Only the sweep verb, which owns the wrapping, can print the names the user will actually pass; a shared blueprint_axis_probe single-sources the probe for both the sweep terminal and --list-axes, so listed == swept by construction (and stays correct across #159's harness retirement). Boss-signed on grounding-check PASS (independent, fresh-context; one BLOCK raised + fixed: the malformed-sweep .expect panic-path is now pinned by a this-cycle E2E over unknown_node.json, not an untested current-behaviour claim). Engine/registry untouched (invariants 1/8/9). refs #169
This commit is contained in:
@@ -0,0 +1,234 @@
|
||||
# Discover a loaded blueprint's sweepable axis names — Design Spec
|
||||
|
||||
**Date:** 2026-07-01
|
||||
**Status:** Draft — awaiting user spec review
|
||||
**Authors:** orchestrator + Claude
|
||||
|
||||
## Goal
|
||||
|
||||
Give a data-level blueprint author a way to **list a loaded blueprint's
|
||||
sweepable axis names before sweeping it** — `aura sweep <blueprint.json>
|
||||
--list-axes` prints one `<name>:<kind>` line per open knob, exactly as the
|
||||
`--axis` flag names them, then exits. Today the only way to learn those names
|
||||
is to guess an axis and read a `MissingKnob`/`UnknownKnob` error — and that
|
||||
error names only the *rejected* knob, never the valid set, so there is no
|
||||
discovery path at all (#169, surfaced by the cycle-0093 architect drift
|
||||
review). This is the prerequisite for #173 (IS-refit walk-forward), which
|
||||
must know which axes to re-fit per in-sample window.
|
||||
|
||||
## Architecture
|
||||
|
||||
The load-bearing fact (grounded against the current tree, recorded as a
|
||||
derived decision on #169): the axis names `--axis` accepts are **produced by
|
||||
the sweep's own harness-wrapping, not by the raw blueprint**. A
|
||||
loaded-blueprint sweep resolves axes against
|
||||
`wrap_stage1r(reload(doc)).param_space()` (`crates/aura-cli/src/main.rs`,
|
||||
`blueprint_sweep_family`): the loaded signal composite (`name ==
|
||||
"stage1_signal"`) is nested as a `BlueprintNode::Composite`, and
|
||||
`collect_params` (`crates/aura-engine/src/blueprint.rs`) prefixes a nested
|
||||
composite's params with that composite's own `name`. So the swept names are
|
||||
**prefixed** — `stage1_signal.fast.length`, not the raw `fast.length` the
|
||||
blueprint's own `param_space()` yields.
|
||||
|
||||
Consequence for the surface: **only the sweep verb can honestly list the
|
||||
names**, because only it owns the wrapping. So `--list-axes` is a query
|
||||
sub-mode on `aura sweep`, computing the *same* probe the sweep resolves
|
||||
against, via a single shared helper `blueprint_axis_probe`. `graph
|
||||
introspect --axes` over the raw blueprint would print `fast.length` (names
|
||||
the sweep rejects); teaching `graph introspect` to apply `wrap_stage1r`
|
||||
would couple the general read-only-query surface to one research harness.
|
||||
Correctness beats query-surface cohesion. Single-sourcing the probe also
|
||||
keeps `--list-axes` correct by construction when the hard-wired stage1-r
|
||||
wrapping is retired (#159): the listed names track whatever the sweep
|
||||
actually resolves, with no second source of truth.
|
||||
|
||||
Scope: `crates/aura-cli` only. No `aura-engine` / `aura-registry` change —
|
||||
pure read-only CLI introspection. Domain invariants 1 (determinism), 8
|
||||
(frozen engine), 9 (registry domain-free) untouched.
|
||||
|
||||
## Concrete code shapes
|
||||
|
||||
### Worked user program (the acceptance evidence)
|
||||
|
||||
An author holds a blueprint file and wants to sweep it, but does not know the
|
||||
axis names:
|
||||
|
||||
```console
|
||||
$ aura sweep crates/aura-cli/tests/fixtures/stage1_signal_open.json --list-axes
|
||||
stage1_signal.fast.length:I64
|
||||
stage1_signal.slow.length:I64
|
||||
|
||||
$ # now they can sweep, using exactly those names:
|
||||
$ aura sweep crates/aura-cli/tests/fixtures/stage1_signal_open.json \
|
||||
--axis stage1_signal.fast.length=2,3 --axis stage1_signal.slow.length=4,6
|
||||
# ... runs a 2x2 sweep family ...
|
||||
```
|
||||
|
||||
A **closed** blueprint (every knob bound) has no open axes — the listing is
|
||||
empty and exit is 0, a valid "nothing to sweep here" answer, not an error:
|
||||
|
||||
```console
|
||||
$ aura sweep crates/aura-cli/tests/fixtures/stage1_signal.json --list-axes
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
`--list-axes` is a query, mutually exclusive with an actual sweep:
|
||||
|
||||
```console
|
||||
$ aura sweep …open.json --list-axes --axis stage1_signal.fast.length=2,3
|
||||
aura: --list-axes lists axes and takes no other flags
|
||||
$ echo $?
|
||||
2
|
||||
```
|
||||
|
||||
The two printed names are byte-for-byte what `--axis` binds — empirically, on
|
||||
the current tree, `--axis stage1_signal.fast.length=2,3` passes name
|
||||
resolution while `--axis fast.length=2,3` returns `UnknownKnob("fast.length")`.
|
||||
|
||||
### Implementation shape (secondary)
|
||||
|
||||
**New shared probe helper** — single source for the wrapped axis namespace,
|
||||
used by BOTH the sweep terminal and `--list-axes`:
|
||||
|
||||
```rust
|
||||
/// The exact wrapped probe the loaded-blueprint sweep resolves its axes
|
||||
/// against: the loaded signal wrapped in the stage1-r scaffolding (stop
|
||||
/// bound, reduce, no cost), taps discarded. `param_space()` on it is the
|
||||
/// axis namespace `--axis` binds; `.axis()` consumes it to seed a sweep.
|
||||
/// The reload is infallible under the SAME dispatch-boundary contract
|
||||
/// `blueprint_sweep_family` already relies on (main.rs ~3175-3178): the doc
|
||||
/// is `blueprint_from_json`-validated at the `["sweep", ..]` boundary
|
||||
/// (main.rs 4108-4110) before this runs, so a malformed doc has already
|
||||
/// exited 2 and never reaches the `.expect`. This cycle's malformed-sweep
|
||||
/// E2E pins that guard on the `--list-axes` path (see Testing strategy).
|
||||
fn blueprint_axis_probe(doc: &str) -> Composite {
|
||||
let signal = blueprint_from_json(doc, &|t| std_vocabulary(t))
|
||||
.expect("doc parse-validated at the dispatch boundary; reload is infallible");
|
||||
let (tx_eq, _) = mpsc::channel();
|
||||
let (tx_ex, _) = mpsc::channel();
|
||||
let (tx_r, _) = mpsc::channel();
|
||||
let (tx_req, _) = mpsc::channel();
|
||||
wrap_stage1r(signal, tx_eq, tx_ex, tx_r, tx_req, false, true, None)
|
||||
}
|
||||
```
|
||||
|
||||
`blueprint_sweep_family` (main.rs ~3186-3191) replaces its inline channel +
|
||||
`wrap_stage1r` probe construction with `let probe = blueprint_axis_probe(doc);`
|
||||
(it still consumes `probe` via `.axis()`, so the helper returns the `Composite`,
|
||||
not just the space). The listing path takes only the space:
|
||||
|
||||
```rust
|
||||
/// `aura sweep <blueprint.json> --list-axes`: one `<name>:<kind>` line per
|
||||
/// open sweepable knob, in `param_space()` order; a closed blueprint prints
|
||||
/// nothing. Names are exactly what `--axis` binds (same probe as the sweep).
|
||||
fn list_blueprint_axes(doc: &str) {
|
||||
for p in blueprint_axis_probe(doc).param_space() {
|
||||
println!("{}:{:?}", p.name, p.kind); // ScalarKind Debug -> I64/F64/Bool/Timestamp
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Arg grammar** — `SweepBlueprintArgs` gains a `list_axes: bool`;
|
||||
`parse_sweep_blueprint_args` recognizes `--list-axes` and enforces that it
|
||||
appears **alone** (no `--axis`/`--name`/`--trace`/`--real`):
|
||||
|
||||
```rust
|
||||
struct SweepBlueprintArgs { axes: Vec<(String, Vec<Scalar>)>, name: String,
|
||||
persist: bool, data: DataChoice, list_axes: bool }
|
||||
// parse: on "--list-axes" set list_axes = true; if list_axes && any other
|
||||
// flag was seen -> Err("--list-axes lists axes and takes no other flags").
|
||||
```
|
||||
|
||||
**Dispatch** — the existing `["sweep", rest @ ..]` blueprint branch
|
||||
(main.rs ~4112-4118), after the boundary read + `blueprint_from_json`
|
||||
validation that already reject a malformed file:
|
||||
|
||||
```rust
|
||||
let SweepBlueprintArgs { axes, name, persist, data, list_axes } =
|
||||
parse_sweep_blueprint_args(&rest[1..]).unwrap_or_else(|msg| { eprintln!("aura: {msg}"); exit(2) });
|
||||
if list_axes {
|
||||
list_blueprint_axes(&doc);
|
||||
return;
|
||||
}
|
||||
run_blueprint_sweep(&doc, &axes, &name, persist, DataSource::from_choice(data));
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
- `blueprint_axis_probe(doc) -> Composite` (new, main.rs) — the single-sourced
|
||||
wrapped probe; `blueprint_sweep_family` refactored to call it.
|
||||
- `list_blueprint_axes(doc)` (new, main.rs) — prints `name:kind` per open knob.
|
||||
- `SweepBlueprintArgs.list_axes` + `parse_sweep_blueprint_args` (modified) —
|
||||
parse `--list-axes`, enforce standalone.
|
||||
- `["sweep", ..]` blueprint dispatch (modified) — short-circuit to the listing.
|
||||
|
||||
## Data flow
|
||||
|
||||
`aura sweep <bp.json> --list-axes` → boundary read + `blueprint_from_json`
|
||||
validate (existing) → `parse_sweep_blueprint_args` sets `list_axes` →
|
||||
`list_blueprint_axes(doc)` → `blueprint_axis_probe(doc)` wraps the reloaded
|
||||
signal in stage1-r scaffolding (stop bound, reduce, no cost) →
|
||||
`.param_space()` → print each `name:kind` in order → exit 0.
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Malformed / unreadable blueprint** → the `["sweep", ..]` dispatch reads the
|
||||
`.json` path and runs `blueprint_from_json` validation *before* arg parsing
|
||||
(main.rs 4098-4111), so a malformed doc exits 2 with a named `aura: <path>:
|
||||
<e>` (empirically today: `unknown_node.json` → `aura: …: UnknownNodeType("Nope")`,
|
||||
exit 2, no panic) *before* `list_blueprint_axes` — hence `blueprint_axis_probe`'s
|
||||
`.expect` is unreachable on this path. That guard is currently pinned only on the
|
||||
`run` verb (`aura_run_rejects_an_unknown_node_blueprint`), **not** the `sweep`
|
||||
verb; **this cycle delivers the E2E that pins it on the `sweep --list-axes` path**
|
||||
(reusing `unknown_node.json`), closing the gap the new `.expect` would otherwise
|
||||
leave untested. No new error path is added — the fix is the missing *test*.
|
||||
- **`--list-axes` combined with any other flag** → `parse_sweep_blueprint_args`
|
||||
returns `Err("--list-axes lists axes and takes no other flags")`, rendered
|
||||
`aura: <msg>` + exit 2.
|
||||
- **Closed blueprint (0 open knobs)** → empty stdout, exit 0. Not an error.
|
||||
- No panics on any user-reachable input; the probe reload is infallible only
|
||||
because the doc was validated at the boundary (same contract as the sweep).
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Unit — probe names** (`cargo test -p aura-cli --bin aura
|
||||
blueprint_axis_probe`): load `stage1_signal_open.json`, assert
|
||||
`blueprint_axis_probe(doc).param_space()` names == `["stage1_signal.fast.length",
|
||||
"stage1_signal.slow.length"]` with kinds `[I64, I64]`; load `stage1_signal.json`,
|
||||
assert the space is empty. This pins the prefixed-name fact (grounded by the
|
||||
existing `param_space_is_flat_path_qualified_and_slot_disambiguated` /
|
||||
`param_space_reflects_only_open_knobs` /
|
||||
`param_space_empty_for_paramless_and_empty_blueprints` in
|
||||
`crates/aura-engine/src/blueprint.rs`).
|
||||
- **Unit — arg grammar** (`cargo test -p aura-cli --bin aura
|
||||
parse_sweep_blueprint_args`): `["--list-axes"]` → `list_axes: true`;
|
||||
`["--list-axes","--axis","stage1_signal.fast.length=2,3"]` → `Err`.
|
||||
- **E2E** (`cargo test -p aura-cli --test cli_run <filter>`, where the existing
|
||||
blueprint-sweep E2Es live): `aura sweep <open fixture> --list-axes` → stdout
|
||||
exactly the two lines, exit 0; `aura sweep <closed fixture> --list-axes` →
|
||||
empty stdout, exit 0; `aura sweep <open fixture> --list-axes --axis
|
||||
stage1_signal.fast.length=2,3` → exit 2 + `--list-axes lists axes` on stderr.
|
||||
- **E2E — malformed-blueprint safety** (same `--test cli_run`): `aura sweep
|
||||
crates/aura-cli/tests/fixtures/unknown_node.json --list-axes` → exit 2 + a named
|
||||
`aura: …: UnknownNodeType(…)` on stderr + **no panic** (`stderr` free of
|
||||
`panicked`). This pins the dispatch-boundary guard on the `sweep --list-axes`
|
||||
path, so `blueprint_axis_probe`'s `.expect` is provably unreachable — the gap
|
||||
the grounding-check flagged (the guard is otherwise pinned only on the `run`
|
||||
verb).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. `aura sweep <blueprint.json> --list-axes` prints one `<name>:<kind>` line per
|
||||
open sweepable knob, in `param_space()` order, exit 0 — and each printed name
|
||||
is byte-for-byte accepted by `--axis` on the same verb (`stage1_signal.fast.length`,
|
||||
not `fast.length`).
|
||||
2. A closed blueprint prints nothing and exits 0.
|
||||
3. `--list-axes` combined with any other flag is a usage error (exit 2).
|
||||
4. A malformed/unreadable blueprint on the `--list-axes` path is rejected with a
|
||||
named `aura:` error and exit 2, never a panic — pinned by a new `sweep
|
||||
--list-axes` E2E over `unknown_node.json` (closing the gap that the guard was
|
||||
pinned only on the `run` verb).
|
||||
5. No `aura-engine` / `aura-registry` change; invariants 1, 8, 9 preserved.
|
||||
6. `cargo build --workspace`, `cargo test --workspace`, and `cargo clippy
|
||||
--workspace --all-targets -- -D warnings` are green.
|
||||
Reference in New Issue
Block a user