Files
Aura/fieldtests/milestone-cost-model-graph/FINDINGS.md
T
Brummel dff99c93e4 fieldtest(milestone): cost-model graph (#148) — capability delivered green
Milestone-close gate for "Cost-model graph (in R)" (#148). The fieldtester exercised
the milestone capability end-to-end from the public CLI only (no implementation
source), on the stage1-r harness, against ledger C10 + the glossary.

Roll-up: friction_found — 0 capability bugs; the milestone promise is empirically
delivered:
- bare gross-R run is clean (net == gross, no cost forced);
- per-trade cost: gross pinned, net drops strictly/monotonically, net_r_equity tap
  persists;
- the capstone per-held-cycle carry BLEEDS the net-R curve continuously over a hold
  (the gross-net gap grows 0.367 -> 0.734 -> 1.101 -> 1.468 across one hold, vs the
  per-trade run's flat 0.367) — the holding-cost shape change, observable end-to-end;
- composition is exact-additive (the composed drag equals the sum of the single-cost
  drags to the last ULP) and `aura chart` renders the real net_r_equity series.

Evidence under fieldtests/milestone-cost-model-graph/ (run_scenarios.sh +
captured_outputs.txt + traces/ + the charted bleed + FINDINGS.md).

Non-blocking findings filed forward (none gates the close):
- cost-flag CLI ergonomics — units hint, negative-rate diagnostic, non-R-harness
  no-op + a C10 spec_gap -> #153;
- docs/project-layout.md describes the retired `aura backtest --broker` path and omits
  the cost flags -> folded into #151.

refs #148
2026-06-29 02:35:24 +02:00

12 KiB
Raw Blame History

Fieldtest — milestone #148 "Cost-model graph (in R)" — 2026-06-29

Status: Draft — awaiting orchestrator triage Author: fieldtester (dispatched by fieldtest skill, milestone-close gate)

Scope

Milestone #148 promises a composable cost-model graph in R: a strategy's gross-R signal is turned into a net-R curve by composing cost nodes, each a labelled stress parameter, on the stage1-r harness. The capability must span both charge modes — per-trade costs (deduct at close) and per-held-cycle costs (accrue over the hold) — so a research user can measure how much friction costs their edge and see whether a holding cost changes the shape of the equity curve. Shipped across cycles 00810085: ConstantCost, the net_r_equity tap + summarize_r cost fold, VolSlippageCost + CostSum, the CostNode trait, the cost_graph composite-builder, and per-held-cycle accrual (CarryCost + ChargeMode). Public surface exercised: the aura CLI (run --harness stage1-r with --cost-per-trade / --slip-vol-mult / --carry-per-cycle, --trace, chart --tap), against the design ledger C10 and the glossary. Binary exercised: target/debug/aura freshly built from HEAD (cargo build --workspace, commit 84ec2ab).

The milestone promise is delivered: all four end-to-end scenarios behave as the contract states, including the capstone holding-cost shape change and exact cost composition. No capability bug. The findings are doc/diagnostic friction at the public surface and one undefined-behaviour spec_gap; none blocks the close.

Examples

All under fieldtests/milestone-cost-model-graph/. run_scenarios.sh reproduces every command; captured_outputs.txt is its recorded output; traces/ holds the r_equity / net_r_equity series the runs persisted (the runs/ store is gitignored); chart_carry_net_r_equity.html is the charted capstone bleed.

run_scenarios.sh §1 — bare gross-R, no cost model

  • aura run --harness stage1-r with no cost flag.
  • Fits scope: the zero-cost floor — C10's "a bare gross-R run with no cost model attached is valid", the additive-on baseline every cost composes onto.
  • Outcome: ran; expectancy_r = net_expectancy_r = 1.2710005136982836 exactly (net == gross when no cost is attached). Clean.

run_scenarios.sh §2 — flat per-trade cost (ConstantCost, deduct at close)

  • aura run --harness stage1-r --cost-per-trade {0.0001,0.001,0.01}, plus a --trace pertrade_cost run.
  • Fits scope: the per-trade charge mode (cycle 0081), "how much does friction cost my edge?".
  • Outcome: ran; gross expectancy_r stays pinned at 1.2710005136982836 while net_expectancy_r drops strictly and monotonically with cost (1.24020.9632-1.8069); the net_r_equity tap persists under --trace. Matched expected.

run_scenarios.sh §3 — per-held-cycle carry (CarryCost, accrue over hold) — capstone

  • aura run --harness stage1-r --carry-per-cycle 0.001 --trace carry_cost.
  • Fits scope: the milestone capstone (cycle 0085) — the per-held-cycle charge mode; "does a holding cost change the SHAPE of the curve?".
  • Outcome: ran; net_expectancy_r = -0.2679604926492904. The net_r_equity gap (gross net) grows continuously over a single hold — ts4..ts7: 0.367 → 0.734 → 1.101 → 1.468 — versus the per-trade run's flat 0.367 over the same hold. The holding cost bleeds the curve continuously rather than stepping at close. Matched expected exactly.

run_scenarios.sh §4 — compose all three costs + chart

  • aura run --harness stage1-r --cost-per-trade 0.001 --slip-vol-mult 0.5 --carry-per-cycle 0.001 --trace composed_cost, each single cost alone for contrast, then aura chart composed/carry --tap net_r_equity.
  • Fits scope: the cost graph composes (cycles 0082/0084) — N cost nodes sum into one net-R curve via CostSum/cost_graph.
  • Outcome: ran; composed net_expectancy_r = -1.2927895081922123, strictly below every single cost alone (0.5540 slip, 0.9632 per-trade, -0.2680 carry). The composed net-R drag (gross composed = 2.5637900218904959) equals the exact sum of the three single-cost drags (0.30779220 + 0.71703681 + 1.53896101 = 2.56379002, agreeing to the last ULP). The chart HTML bakes the real net_r_equity points array in (uPlot), so the bleed renders — not a shell. Matched expected.

Findings

[working] Gross-only run is clean: net == gross with no cost

  • run_scenarios.sh §1.
  • net_expectancy_r is byte-equal to expectancy_r (1.2710005136982836) and no net_r_equity is forced; the cost layer is genuinely optional and additive (C10 "the zero-cost baseline is the default-simple floor").
  • Why working: the optional-cost contract holds exactly; the win to protect is that attaching no cost changes nothing.
  • Recommended action: carry-on.

[working] Per-trade cost: gross pinned, net drops monotonically, tap persists

  • run_scenarios.sh §2.
  • Across three magnitudes gross expectancy_r never moves; net_expectancy_r is strictly below it and monotone decreasing; runs/traces/pertrade_cost/ contains a net_r_equity.json sibling of r_equity.json.
  • Why working: the per-trade charge mode + the gross/net split + the named tap all behave as C10 / the glossary describe, first try.
  • Recommended action: carry-on.

[working] Capstone — holding cost bleeds the curve continuously over the hold

  • run_scenarios.sh §3 (+ traces/carry_cost_net_r_equity.json).
  • The grossnet gap during the cycles-4..7 hold grows linearly (0.367, 0.734, 1.101, 1.468), where the per-trade run's gap over the same hold is flat (0.367, 0.367, 0.367, 0.367). The per-held-cycle accrual makes the net-R equity curve bleed continuously across a hold instead of stepping at close — the milestone's headline "shape change", directly observable in the persisted series.
  • Why working: this is the milestone capstone (cycle 0085, approach B) and it is empirically true end-to-end from the CLI, not merely asserted in a unit test.
  • Recommended action: carry-on.

[working] Composition is exact-additive and the chart renders the bleed

  • run_scenarios.sh §4 (+ chart_carry_net_r_equity.html, traces/composed_cost_*.json).
  • Three heterogeneous cost factors (per-trade + state-dependent slippage + per-held-cycle carry) sum into one net-R curve; the composed drag equals the exact sum of the single-cost drags (to the last ULP) and composed net is strictly below every single. aura chart emits a self-contained HTML page with the actual net_r_equity points array embedded.
  • Why working: CostSum / cost_graph "one home for cost, no double-count" (C10) is exact, the flags compose as the usage string implies, and the headline artifact charts as a real result render (matching the "render results, not a shell" rule).
  • Recommended action: carry-on.

[friction] Negative cost-rate guard prints the bare usage string, names no cause

  • aura run --harness stage1-r --carry-per-cycle -0.001 (and --cost-per-trade -0.5).
  • Verbatim: aura: usage: aura run [--harness <sma|macd|stage1-r>] ... then exit=2. The guard correctly refuses a negative rate (capability-safe, per the commit-noted "negative-rate guard"), but the diagnostic is the entire 1-line usage dump — it never says why (e.g. "cost rate must be ≥ 0"), and the usage string itself never states the non-negativity constraint, so a user who fat-fingers a sign cannot tell a rejected value from a mistyped flag name.
  • Why friction: the task is refused correctly but the surface gives an uninformative diagnostic that names no cause.
  • Recommended action: plan (a tidy iteration: emit cost rate must be non-negative, got <v> instead of the generic usage dump).

[friction] Cost flags silently no-op on a non-R harness (sma / macd)

  • aura run --harness sma --cost-per-trade 0.001exit=0, output byte-identical to the no-cost sma run ("total_pips":-0.13000000000001358, no r block, no net_*, no warning).
  • Why friction: a cost flag attached to a harness that has no R-evaluator / executor chain is silently swallowed; a user who believes they measured a cost-adjusted result got an unmodified one with no signal that the flag did nothing. (Distinct from the spec_gap below: this entry is the missing-feedback foot-gun, regardless of which behaviour is chosen.)
  • Recommended action: plan (warn-on-ignored-cost-flag, or reject the combination).

[spec_gap] Cost-flag-on-non-R-harness behaviour is undefined by the ledger

  • Same repro as the friction above (--cost-per-trade on --harness sma).
  • C10 / the glossary define cost only against the R chain (gross R → net R) and say a bare gross-R run with no cost is valid; neither constrains what a cost flag should do on a harness (sma / macd) that produces no R at all. The project picked the "silently accept and ignore" reading; "reject: cost requires an R-evaluator harness" was equally plausible. I did not have to guess to complete the milestone scenarios (they all use stage1-r), but a downstream user mixing flags hits an unspecified corner.
  • Why spec_gap: the ledger is silent on the case and the project chose one of two equally-plausible readings.
  • Recommended action: ratify (pick error-vs-noop and add a one-line C10 note; pairs with the friction fix above).

[friction] Cost-flag units / scale are undiscoverable at the CLI

  • All scenarios. The usage string offers only [--cost-per-trade <f64>] [--slip-vol-mult <f64>] [--carry-per-cycle <f64>] — no unit, no per-close-vs-per-held-cycle note, no scale guidance. I had to trial 0.0001 / 0.001 / 0.01 to find a magnitude that moved net-R sensibly, because the R-normalizer (the stop distance |entry stop|, the denominator of cost-in-R = cost_per_trade / |entry stop|) is not surfaced anywhere a CLI user can see, and --carry-per-cycle's unit (price units, per engine cycle — not per day / not an overnight swap) is invisible. The semantics are in C10 and the glossary "cost model" entry, so the capability is discoverable in the ledger — just not at the point of use.
  • Why friction: the surface forced a trial-and-error dance to choose a sensible stress-parameter value that a one-line units hint would have removed.
  • Recommended action: plan / docwriter (a per-flag units note in the usage string: "price units, charged in R as cost/|entrystop|; --carry-per-cycle is per engine cycle").

[friction] The public "how aura is used" doc describes the retired pre-reframe path

  • docs/project-layout.md (the only public usage doc) — its "day in the life" still says aura backtest … --broker pepperstone yields a "realistic currency curve" alongside an "exposure stream" from a "realistic broker". All three — aura backtest, the --broker realistic broker, the exposure framing — are retired by the 2026-06-28 C10 reframe; the actual surface is aura run --harness stage1-r --cost-per-trade … producing net-R, which the doc never mentions. A downstream user reading it to learn how to charge cost is pointed at a path that no longer exists and never told about the cost flags.
  • Why friction: a public-surface document that actively misleads about the shipped capability (doc rot, not a ledger gap — C10 itself is current).
  • Recommended action: plan / docwriter (refresh project-layout.md to the post-reframe aura run + cost-flag surface).

Recommendation summary

Finding Class Action
Gross-only run clean (net == gross) working carry-on
Per-trade: gross pinned, net monotone, tap persists working carry-on
Capstone: holding cost bleeds over the hold working carry-on
Composition exact-additive + chart renders working carry-on
Negative-rate guard prints bare usage, no cause friction plan
Cost flags silently no-op on sma/macd friction plan
Cost-flag-on-non-R-harness undefined by ledger spec_gap ratify
Cost-flag units/scale undiscoverable at CLI friction plan / docwriter
project-layout.md describes the retired path friction plan / docwriter

Milestone-close roll-up: friction_found. The capability is delivered end-to-end (both charge modes, composition, the holding-cost shape change); no capability bug blocks the close. All findings are doc / diagnostic friction or an undefined-corner spec_gap — non-gating for the milestone close.