Brummel
4fac529de0
Milestone-scope fieldtest (the functional leg of the Walking-skeleton
milestone-close gate), from the public interface only. Three end-to-end
scenarios, all green:
- the newcomer CLI path: `aura run` -> single-line {manifest,metrics} JSON,
exit 0, byte-identical twice (C1), non-degenerate trace (C22), bad-args ->
exit 2;
- the real-data thread: real AAPL.US 2006-08 M1 bars -> load_m1_window ->
close_stream -> SMA-cross -> Exposure -> SimBroker -> summarize -> RunReport,
deterministic AND populated;
- epoch-ns coherence: C3 normalization holds ingest -> run -> sink.
Roll-up: the milestone DELIVERS its promise end to end (ingest -> one signal ->
exposure -> sim-optimal broker -> pip-equity metric -> structured RunReport,
deterministic, populated) on BOTH the synthetic and real-data paths. 0 bugs.
Headline concern (verified by the orchestrator): issue #19's premise is WRONG at
the shipped SMA(2)/SMA(4) demo config — 21 AAPL.US bars warm the cross and
produce a populated trace (total_pips=-1.5, max_drawdown=65.5, 6 sign flips),
not all-zero. The degeneracy #19 saw was deep-SMA/param-dependent, not intrinsic
to the symbol. #19 to be re-scoped. Other findings: `aura run <junk>` exit 0
(already #16), `aura --help` lands on the exit-2 error path (friction), SimBroker
two-f64-slot order unchecked at bootstrap (documented footgun) — both filed.
57 lines
2.4 KiB
Bash
Executable File
57 lines
2.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Milestone fieldtest — scenario 1: THE NEWCOMER CLI PATH
|
|
#
|
|
# A downstream user with nothing but the public docs (aura's rustdoc module
|
|
# blurb: "aura run bootstraps a built-in sample signal-quality harness ...
|
|
# prints the run's metrics + manifest as canonical JSON to stdout") invokes the
|
|
# real binary and checks the milestone promise end to end:
|
|
# - a single-line JSON RunReport with the documented {manifest, metrics} shape
|
|
# - exit 0
|
|
# - byte-identical across two runs (C1 determinism)
|
|
# - a NON-degenerate populated trace (C22 "newcomer sees a populated trace")
|
|
# - the bad-args / usage contract
|
|
#
|
|
# Run from the repo root. Builds from HEAD first (never a stale artifact).
|
|
set -u
|
|
cd "$(git rev-parse --show-toplevel)"
|
|
cargo build -q -p aura-cli || { echo "BUILD FAILED"; exit 1; }
|
|
BIN=target/debug/aura
|
|
|
|
fail=0
|
|
|
|
A=$("$BIN" run); ea=$?
|
|
B=$("$BIN" run); eb=$?
|
|
|
|
echo "report: $A"
|
|
[ "$ea" -eq 0 ] && [ "$eb" -eq 0 ] || { echo "FAIL: run exit $ea/$eb (want 0)"; fail=1; }
|
|
[ "$A" = "$B" ] || { echo "FAIL: two runs differ (C1)"; fail=1; }
|
|
# single line: no embedded newline
|
|
[ "$(printf '%s' "$A" | wc -l)" -eq 0 ] || { echo "FAIL: not single-line"; fail=1; }
|
|
# documented shape
|
|
case "$A" in
|
|
*'"manifest"'*'"metrics"'*'"total_pips"'*'"max_drawdown"'*'"exposure_sign_flips"'*) ;;
|
|
*) echo "FAIL: missing documented {manifest,metrics} keys"; fail=1;;
|
|
esac
|
|
# NON-degenerate: at least one of the metrics is non-zero (a populated trace).
|
|
# The synthetic sample stream rises then reverses -> 1 sign flip, real drawdown.
|
|
case "$A" in
|
|
*'"exposure_sign_flips":0'*) echo "FAIL: degenerate trace (0 sign flips)"; fail=1;;
|
|
esac
|
|
|
|
# bad-args contract: no subcommand and unknown subcommand -> exit 2, usage on
|
|
# stderr, empty stdout.
|
|
"$BIN" >/tmp/ms1_o 2>/tmp/ms1_e; e=$?
|
|
[ "$e" -eq 2 ] || { echo "FAIL: no-arg exit $e (want 2)"; fail=1; }
|
|
[ -s /tmp/ms1_o ] && { echo "FAIL: no-arg wrote stdout"; fail=1; }
|
|
grep -q usage /tmp/ms1_e || { echo "FAIL: no usage on stderr"; fail=1; }
|
|
|
|
"$BIN" frobnicate >/tmp/ms1_o 2>/tmp/ms1_e; e=$?
|
|
[ "$e" -eq 2 ] || { echo "FAIL: bad-subcmd exit $e (want 2)"; fail=1; }
|
|
|
|
# OBSERVED ANOMALY (recorded, not asserted): `aura run extra junk` -> exit 0,
|
|
# trailing args silently ignored. Documented as a spec_gap in the spec.
|
|
"$BIN" run extra junk >/tmp/ms1_o 2>/dev/null; echo "run+junk exit: $? (observed lenient)"
|
|
|
|
[ "$fail" -eq 0 ] && echo "SCENARIO 1: PASS" || echo "SCENARIO 1: FAIL"
|
|
exit "$fail"
|