Brummel
780d823067
Milestone-close validation of the World/C21 blueprint-data family surface, run
from the PUBLIC interface only (design ledger + op-script grammar; blueprints
authored as a consumer via `aura graph build`, no crates/ source read). Binary
built from 4710fcc.
The milestone's FUNCTIONAL PROMISE HOLDS (3 working findings): a consumer-authored
blueprint round-trips the whole verb family (graph build → list-axes → sweep →
rank → reproduce; walkforward → reproduce; mc → reproduce); all three FamilyKinds
(Sweep 4/4, MonteCarlo 5/5, WalkForward 3/3) reproduce bit-identically through the
one shared content-addressed store (the C18 core promise); list-axes→sweep→rank
composes and self-corrects.
Findings surfaced for follow-on (NOT fixed here — filed to the tracker): 2 bugs
(aura run on an open blueprint PANICS exit 101 instead of a clean exit-2 like mc;
closed→walkforward --axis prints the rejection twice), 2 friction (sweep-side
closed error terser than mc's exemplary message; mc reproduce lines omit the
seed), 2 spec-gaps (every open knob is mandatory on sweep but the docs frame
list-axes as a menu; the whole blueprint-data verb family is invisible from
`--help` / no README — empirically confirms #159). Detail per finding in the spec.
fieldtests/world-c21-milestone/ carries the op-scripts, authored blueprints,
captured outputs, and run_all.sh (regenerates the gitignored runs/ telemetry).
refs #170
50 lines
2.9 KiB
Bash
Executable File
50 lines
2.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Milestone fieldtest — World/C21: run + orchestrate harness FAMILIES from
|
|
# blueprint-data. Driven entirely from the public surface (design ledger C24/C18/
|
|
# C21 + glossary + the documented op-script grammar). No source was read.
|
|
#
|
|
# Build first: cargo build --workspace (binary: target/debug/aura)
|
|
# Then: ./run_all.sh
|
|
set -u
|
|
BIN="${AURA_BIN:-../../target/debug/aura}"
|
|
|
|
echo "===================================================================="
|
|
echo "Scenario 1 — discover -> sweep -> list -> rank (OPEN blueprint)"
|
|
echo "===================================================================="
|
|
$BIN graph build < w1_open_crossover.ops.json > w1_open_crossover.bp.json
|
|
$BIN sweep w1_open_crossover.bp.json --list-axes # discover the 3 open knobs
|
|
# NOTE: every open knob is MANDATORY; a subset -> MissingKnob (see w4_4f).
|
|
$BIN sweep w1_open_crossover.bp.json \
|
|
--axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \
|
|
--name w1_cross_sweep
|
|
$BIN runs families
|
|
$BIN runs family w1_cross_sweep-0 rank sqn_normalized
|
|
$BIN reproduce w1_cross_sweep-0 # 4/4 bit-identical
|
|
|
|
echo "===================================================================="
|
|
echo "Scenario 2 — walk-forward -> reproduce (CAPSTONE, OPEN blueprint)"
|
|
echo "===================================================================="
|
|
$BIN walkforward w1_open_crossover.bp.json \
|
|
--axis graph.fast.length=2,3 --axis graph.slow.length=5,10 --axis graph.bias.scale=0.5 \
|
|
--select argmax --name w2_wf
|
|
$BIN reproduce w2_wf-0 # 3/3 bit-identical
|
|
|
|
echo "===================================================================="
|
|
echo "Scenario 3 — monte-carlo -> reproduce (CLOSED blueprint)"
|
|
echo "===================================================================="
|
|
$BIN graph build < w3_closed_crossover.ops.json > w3_closed_crossover.bp.json
|
|
$BIN sweep w3_closed_crossover.bp.json --list-axes # empty: fully bound
|
|
$BIN mc w3_closed_crossover.bp.json --seeds 5 --name w3_mc
|
|
$BIN reproduce w3_mc-0 # 5/5 bit-identical
|
|
|
|
echo "===================================================================="
|
|
echo "Scenario 4 — cross-verb coherence + error actionability"
|
|
echo "===================================================================="
|
|
$BIN sweep w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4a closed->sweep
|
|
$BIN mc w1_open_crossover.bp.json --seeds 3 # 4b open->mc (exemplary err)
|
|
$BIN walkforward w3_closed_crossover.bp.json --axis graph.fast.length=2,3 # 4c (dup diagnostic)
|
|
$BIN sweep w1_open_crossover.bp.json --axis graph.nonesuch.length=2 \
|
|
--axis graph.slow.length=5 --axis graph.bias.scale=0.5 # 4d unknown axis
|
|
$BIN reproduce no-such-family-99 # 4e unknown family
|
|
$BIN run w1_open_crossover.bp.json # 4 BUG: PANICS (exit 101)
|