Enforce the shell boundary structurally and evaluate a text-operable successor to the one-shot CLI #295
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Motivation
The
auraCLI has grown in roughly a dozen accretion waves over 2026-06-03..2026-07-20 (324 commits touchingcrates/aura-cli) and has repeatedly proven limiting in practice. A read-only structural survey (2026-07-20) examined two questions: (1) is the CLI surface actually separated from the engine logic, and (2) is a one-shot, flag-driven CLI structurally sufficient as the control surface for what the design ledger names the product — the World as a program, the meta-level that dynamically constructs and orchestrates families of harnesses (docs/design/INDEX.md, entry C21).Provenance: read-only survey of the workspace, the design ledger, the git history of
crates/aura-cli, and the first real downstream research project (aura-quadriga, a separate repository); every "exists only in the binary" assertion below was checked against the public exports of all library crates before being recorded as fact (16 of 16 such assertions survived that check).This entry records the full findings so they are not lost, and derives two work items (§ Consequences).
Findings
The separation is largely real
LOC split of
crates/aura-cli(13,312 lines total; bucket sizes are estimates from a full read): ~2,100 tests, ~2,050 clap mechanics, ~1,850 presentation (stdout/JSON/chart assembly), ~3,150 orchestration glue, ~1,600 genuine domain logic. The heavy machinery is genuinely library code: sweep enumeration (aura_engine::sweep,GridSpace/ListSpace/RandomSpace), walk-forward rolling (aura_engine::walk_forward,WindowRoller,param_stability), MC bootstrap (aura_backtest::monte_carlo,r_bootstrap), selection/deflation (aura_registry::optimize/optimize_plateau/optimize_deflated/rank_by), the worst-case-R floor (aura_registry::generalization), and the campaign executor itself (aura_campaign::execute: preflight, instrument-chunked parallel cell loop, stage sequencing, per-cell fault isolation, registry writes). The verb dissolution (#210 — the four research verbs rebuilt as thin sugar over generated campaign documents) held:crates/aura-cli/src/verb_sugar.rsis pure translation argv → generated campaign/process documents → the one executor.One load-bearing hole: the member-run recipe exists only in the binary
The canonical single-member harness assembly — strategy → RiskExecutor(stop) → cost graph → SimBroker → R-records → RunReport — exists only inside the binary (
wrap_r/run_signal_r/run_blueprint_member,crates/aura-cli/src/main.rs:1497-2110), together with input-role/data binding (crates/aura-cli/src/binding.rs) and the blueprint bound-param reopen glue (main.rs:2123,main.rs:2260). Consequence:aura_campaign::executeis a public library with exactly one mandatory plug (theMemberRunnertrait) — and the only implementation of that plug is private to the binary (CliMemberRunner,crates/aura-cli/src/campaign_run.rs:308-407). A downstream World program can reach the entire family/validation machinery but must first re-author the per-member backtest harness (ascrates/aura-ingest/examples/ger40_breakout_*.rsin fact do by hand).Library reachability of the nine core operations from a downstream crate (without shelling out to the binary):
aura_registry::generalizationis a pure public fnmain.rs:1884,main.rs:1935)main.rs:1233-1381)MemberRunnerimpl CLI-privaterender.rs+ assetsinclude_str!'d into the binary; aura-cli declares no[lib]targetCLI-only domain logic (each verified to have no public library equivalent)
information_coefficient(main.rs:1884-1927) — forward-return construction, exact-timestamp alignment, Pearson correlation, one-sided permutation null; onlypearson_corr/permute/SplitMix64are library primitives in aura-analysis.stop_rule_from_params(main.rs:1162-1174),cost_specs_from_params(main.rs:1184-1213),point_from_params(main.rs:1132-1150), and their forward half, the manifest stop/cost stamping (main.rs:2062-2090).fit_wf_ms_sizes/wf_ms_sizes(main.rs:3418-3450) — walk-forward roller sizing, ratio-preserving fit to short windows.main.rs:2566-2631).intersect_shared_window(main.rs:3769-3780) — shared-window reduction for generalize.data_coverage_report(main.rs:4224-4258) andwindow_coverage/interior_gap_months(crates/aura-cli/src/campaign_run.rs:409-464).campaign_run.rs:790-1157).bind_axeset al.,campaign_run.rs:191-288) —MemberRunnerhands params as(name, Scalar)and delegates binding to the consumer, so the binding convention itself is consumer-side and currently binary-only.campaign_run.rs:55-109).crates/aura-cli/src/project.rs:297-398), authoring-loop staleness heuristic (project.rs:435-492), hand-rolled civil-calendar math (project.rs:400-433).decimate,main.rs:219-291) and the entire self-contained HTML assembly (crates/aura-cli/src/render.rs).#[cfg(test)]-only builders that regenerate the shipped example blueprints (r_breakout_signal/r_meanrev_signal/r_channel_signal,main.rs:2783-2943).Near-duplication:
walkforward_summary_json_from_reports(main.rs:933-978) reimplements the per-axis stability reduction inline instead of calling the publicaura_engine::param_stabilitythat its live-result twin (walkforward_summary_json,main.rs:898) uses.The ledger already assigns the CLI a smaller role than it plays
Target picture per
docs/design/INDEX.md: C14 (headless core, two faces — the engine is a UI-agnostic library; the CLI is one face on it); C28 (internal stratification — the shell imports everything and is imported by nothing); C25 (role model — every artifact class has a canonical, complete, text-serialized form, executable headless; experiment intent lives in content-addressed documents, not flags; the #188 diagnosis: intent must not evaporate into shell history); C21/C24 (the World is a program owning topology as serializable data). The dependency direction of C28 holds today; the content rule does not: the binary is currently the only place that defines what a standard aura backtest is.Residual flag surface:
enum Command(main.rs:3021) has 15 top-level verbs; the run-ish quintet Run/Sweep/Walkforward/Mc/Generalize carries 48 positional+flag fields combined, dominated by campaign-document vocabulary in argv form (--axis,--from/--to,--stop-length/--stop-k,--select,--block-len,--resamples,--seeds) rather than genuine CLI shape (--list-axesand the name-vs-trace persistence choice are the small true-CLI residue).Downstream evidence: the surface strains where it carries intent
Observed 2026-07-20 in the aura-quadriga project tree (separate repository, consuming this engine):
--version), #272 (per-cell fault isolation).The erosion mechanism is still active
The newest feature — the measurement run verb + Information Coefficient (
aura measure ic, landed 2026-07-18..20, commita9d36dd) — was born entirely CLI-private: both the measurement-run orchestration and the IC metric live only inmain.rs, although the aura-analysis crate had just been split out for exactly this metric family. Absent structural counter-pressure, new capability lands in the binary by default.Consequences — two work items
Guiding statement: the CLI does not need to become more powerful — it needs to become less necessary.
1. Make the shell boundary structural, not prose
The boundary must be enforced by actual crate structure so the erosion cannot recur silently; a prose convention demonstrably does not hold (see the measurement/IC finding above). Direction, as options — the enforcement mechanism is itself a design decision:
aura_campaign::executeships a defaultMemberRunnerand the binary shrinks to argv → document → executor → presentation.Rationale: the member-run recipe is the definition of what an aura backtest is, and a definition belongs in the library that the product (the World program, C21) can consume — not in one of two faces.
2. Evaluate a text-operable control surface beyond the one-shot CLI
The World's ambition — generate, mutate, structurally search harness families (C24) — exceeds a one-shot process by construction; the ledger already notes the CLI is load-and-hold one-shot only because the process is (C13) and defers the long-running host (C22). Needed: a control surface stronger than argv that remains text-first and therefore LLM-operable (C25: canonical text artifacts, headless execution). Candidate directions to evaluate (unranked, undecided):
These are not mutually exclusive; the load-bearing fork is the layering — which surface is canonical and which are projections. Constraint from C25 in every variant: each artifact class keeps a canonical, complete, text-serialized form; a surface is a projection over that form, never a second home for intent.
Design triage for this cycle (2026-07-20)
A structured multi-stance design review (minimalist / power-user-composability / maintainer / skeptic / off-ledger-wildcard lenses) populated the load-bearing forks this issue leaves open. Each fork below is decided where the positions converged on a cited shared ground; the one genuine direction fork is recorded open. All crate-graph claims cited here were re-verified against the working tree before recording.
Decided — member-run recipe home (item 1)
The recipe becomes library code in two halves, because the crate graph forbids a single home:
aura-compositesdepends onaura-backtest(verified incrates/aura-composites/Cargo.toml), so the harness-assembly half — which wiresrisk_executor/cost_graphfromaura-compositesand opens data viaaura-ingest— cannot live inaura-backtestwithout a dependency cycle.RunReport) lands inaura-backtest— the destination the design ledger already earmarks (C28 Status, phased-realization item 3: "the per-run scaffold as a library"). All four grounded lenses converged on this citation.wrap_r/run_signal_r/run_blueprint_member), input binding (the C26resolve_bindingmodule), the C1-load-bearing param-to-config translator pairs, the axis-to-param binding convention (bind_axes), the risk-regime-to-builder mapping — lands in a new assembly crate occupying a new, named C28 position: it imports the ladder rungs,aura-composites,aura-ingest, and the process column (aura-campaign,aura-registry); it implements theMemberRunnerseam with a public default implementation; it is imported by the shell and by downstream World programs. TheMemberRunnertrait stays inaura-campaign— that crate's deliberate exclusion of harness/data-binding dependencies (itsCargo.tomlheader comment) remains intact, since the column never imports the assembly crate; consumers pass the runner in. Grounds: C28 rung-5 text + phase-3 line;crates/aura-campaign/Cargo.tomlheader comment; the existing production edgeaura-campaign -> aura-backtest(#291/#292).crates/aura-campaign/src/lib.rs:64), and the hand-authoredcrates/aura-ingest/examples/ger40_breakout_*.rsdemonstrate the re-author cost a monolithic default would leave in place; a downstream variant recipe (different cost wiring, swapped stop logic) must be recombinable from parts.crates/aura-cli/src/main.rs:1884-1927) move to a newaura-measurementcrate (C28 rung 3, which the ledger notes has no crate yet). The open duplicate-timestamp-semantics question is not resolved by the move — the code moves as-is (refs #294).Decided — enforcement (item 1)
Crate moves alone demonstrably do not stop erosion: the IC landed CLI-private days after
aura-analysiswas split out for exactly this metric family (this issue, "The erosion mechanism is still active"). Enforcement is therefore both the crate graph and the existing structural test extended: thec28_layeringtable (seven rows today, verified) grows to the full workspace — includingaura-composites,aura-campaign,aura-registry,aura-research,aura-ingest, the new crates, and a shell row — plus a shell-content check: the bin crate declares no[lib]target and its module set is a closed allow-list of argv/dispatch/presentation modules. Grounds: C28 Why ("Enforcement is the crate graph itself"); the repeatedly-extendedc28_layeringprecedent; this issue's erosion finding.Decided — rendering (item 1, deferred)
Rendering/presentation stays in the shell this cycle; its extraction into a library crate is deferred to the cycle that builds the C22 web face, where its real second consumer fixes the library boundary. The shell-content rule bans domain logic, not presentation, so the boundary stays enforceable meanwhile. Grounds: C22 realization note (rendering-in-the-shell is the ratified split today); C28 Forbids (the split is warranted by a second consumer); the open-threads list (local server undesigned). One lens dissented — extract now, citing this issue's own bundling of rendering under item 1 — and the deferral is decided on the substantive ground that a library boundary cut against zero real consumers is a rewrite risk, not a saving. The eventual consumers are tracked at #72 and #150.
Decided — canonical layering (item 2)
The text artifact vocabulary (blueprints, process/campaign documents, registry records) is the canonical layer; every control surface — the one-shot CLI's executor verbs, any future long-running host, any MCP face, a World program — is a projection/executor over those artifacts, never a second home for intent. This confirms rather than invents: C25 already states it as an invariant ("every artifact class has a canonical, complete, text-serialized form; ... visual surfaces are stateless projections"). All four grounded lenses converged on the same C25 citation. A ledger amendment recording the control-surface consequence ships with this cycle.
Decided — cycle scope
This cycle ships: the two-half extraction (including the measurement seeding), the extended structural test, and the C28/C25 ledger amendments (including marking phase 3's outcome and updating the C28 import-rule prose — see the noted concerns). Recorded honestly: the extraction closes a ratified structural debt (C28's own phase plan) and un-blocks the future World program; it was not a demonstrated downstream blocker — the one real downstream project's observed friction was document-vocabulary gaps, never Rust-level reachability (this issue, "Downstream evidence"). OUT of scope: any new operational surface, the document-first flag migration, rendering extraction, and #294.
Open direction fork (item 2 — the owner's call)
Which projection is built first once the substrate lands:
--axis,--from/--to,--stop-length/--stop-k,--select,--block-len,--resamples,--seeds) fully into document vocabulary; the CLI reduces to a small executor verb set.Three lenses leaned (d) on demand-driven-precedent grounds; one leaned (a) on the ground that the verb dissolution (#210) already made the quintet thin sugar over generated documents, so (a) completes an existing line and is a prerequisite either way — a host or MCP face would wrap the same completed vocabulary. No source settles which surface the owner wants to drive the system through next, so the ranking is a genuine direction fork. Status: the substrate work proceeds; the ranking awaits the owner's decision.
Noted concerns (carried into the spec)
aura-campaign -> aura-backtestproduction dependency, and the assembly-crate position is new prose entirely. A silent divergence would reproduce the erosion class this cycle exists to close.Spec auto-signed (2026-07-20)
The cycle's spec — structural shell boundary: the member-run recipe, input binding, the param-to-config translators, and the IC computation move out of the binary into library crates; a shipped public default
MemberRunner; enforcement via the extended structural layering test over the full workspace; ledger amendments (C28 assembly position, C25/C14 control-surface consequence, C26 realization note) — was signed autonomously. The signature is an independent fresh-context grounding check returning PASS: all ten load-bearing assumptions about current codebase behaviour were ratified against currently-green tests. No human signed; the owner was notified and retains a veto.Derived decisions folded into the spec at production time (rationale inline):
aura-runner(it implements theMemberRunnerseam; the semantically apt alternative "harness" would overload an established glossary term naming the closed root graph), the measurement-rung crateaura-measurement(the rung's own ledger name).Cycle outcome (2026-07-21)
The shell-boundary cycle shipped on the branch
worktree-issue-295-shell-boundary, unmerged, awaiting review — six commits above the anchor2cf4574:5c2ac98— the extraction:aura-runner(assembly position: binding, translators, harness assembly, family builders, reproduce, project loading,DefaultMemberRunner),aura-measurement(the IC),aura-backtest::scaffold; the shell reduced to argv/translation/presentation; worked consumer example.170c6c8— enforcement + records: full-workspacec28_layering(completeness assertion, shell-content check), ledger amendments (C28 assembly position + corrected import rule + phase-3 done + provenance note; C25/C14 control-surface projections; C26 realization), library-only E2E with a C1 determinism pin.5006766— cycle-close audit: architect drift resolved in-commit (C28 status contradiction; #147 registry-dispatch tests relocated to the measurement crate; the externaldata-serverentry points recorded and guard-pinned; shell-prose buckets completed).9df217d— fieldtest corpus: four downstream-consumer examples, library-only link graph verified; the cycle's acceptance claim substantiated (all four axes compiled first-try from public docs alone).78e68e6+4ed6455— the fieldtest's one bug, RED-first: family-id enumeration vs reproduce keying (#298, fixed with ambiguity refusal + handle precedence; a C18 record-reality rider corrects the stale exit-code prose).Verification at head: the full workspace suite green (1477 passed, 0 failed), clippy
-D warningsclean; behaviour byte-identity pinned throughout by the untouched shell E2E suites.Filed forward during the run: #296 (library-surface stability discipline, idea), #297 (finish the
RunnerErrorconversion in the assembly crate), #299 (reproduce-API friction from the fieldtest). The fieldtest report survives in the working tree as the next planning cycle's reference input.Status: work complete and gated; the branch awaits review and the ratified merge. The one open design residue remains the control-surface direction fork recorded in the triage comment on this issue (document-first completion / long-running host / MCP face / wait-for-demand) — the owner's call, blocking nothing on the branch.
Merge record
The cycle branch was ratified and fast-forward merged: main
2cf4574→4ed6455(6 commits), pushed to origin. #298 auto-closed via the fix commit. Worktree and run branch removed; the git-ignored fieldtest spec (docs/specs/fieldtest-shell-boundary.md) was carried into the main checkout as the next planner input.Remaining open item on this issue: the F5 control-surface fork (which text-operable surface follows the one-shot CLI) — under discussion with the user.
Direction fork resolved (owner decision)
The control-surface direction fork recorded in the design triage (issues/295#issuecomment-3979) is decided: the owner chose (a) document-first completion — migrate the run quintet's residual flag vocabulary fully into document vocabulary, reducing the CLI to a small executor verb set — as the next surface step (owner decision, 2026-07-21). The grounds carried with the choice: it completes the #210 verb-dissolution line, and any later host or MCP face would wrap the same completed vocabulary, so the step is a prerequisite of every successor surface. The host-vs-MCP ranking beyond it stays demand-driven — deliberately unranked until a concrete consumer forces it.
Follow-up cycle: #300. Both work items of this issue are resolved — the shell boundary shipped structurally (merged at
4ed6455), and the successor-surface evaluation ends in the decided direction. Closing.Post-close field evidence for the open direction fork (item 2 — which projection is built first): the 2026-07-22 external field test, an LLM agent driving the release binary through a full research day with no source or docs access (triage in #314).
aura campaign validate/register/run/introspectinvoked 27/20/16/13 times over the day) — two independent field sources converging on documents as the place where intent settles.The ranking remains the owner's call; this comment adds field weight, not a decision.