dff99c93e4
Milestone-close gate for "Cost-model graph (in R)" (#148). The fieldtester exercised the milestone capability end-to-end from the public CLI only (no implementation source), on the stage1-r harness, against ledger C10 + the glossary. Roll-up: friction_found — 0 capability bugs; the milestone promise is empirically delivered: - bare gross-R run is clean (net == gross, no cost forced); - per-trade cost: gross pinned, net drops strictly/monotonically, net_r_equity tap persists; - the capstone per-held-cycle carry BLEEDS the net-R curve continuously over a hold (the gross-net gap grows 0.367 -> 0.734 -> 1.101 -> 1.468 across one hold, vs the per-trade run's flat 0.367) — the holding-cost shape change, observable end-to-end; - composition is exact-additive (the composed drag equals the sum of the single-cost drags to the last ULP) and `aura chart` renders the real net_r_equity series. Evidence under fieldtests/milestone-cost-model-graph/ (run_scenarios.sh + captured_outputs.txt + traces/ + the charted bleed + FINDINGS.md). Non-blocking findings filed forward (none gates the close): - cost-flag CLI ergonomics — units hint, negative-rate diagnostic, non-R-harness no-op + a C10 spec_gap -> #153; - docs/project-layout.md describes the retired `aura backtest --broker` path and omits the cost flags -> folded into #151. refs #148
211 lines
12 KiB
Markdown
211 lines
12 KiB
Markdown
# Fieldtest — milestone #148 "Cost-model graph (in R)" — 2026-06-29
|
||
|
||
**Status:** Draft — awaiting orchestrator triage
|
||
**Author:** fieldtester (dispatched by fieldtest skill, milestone-close gate)
|
||
|
||
## Scope
|
||
|
||
Milestone #148 promises a composable **cost-model graph in R**: a strategy's
|
||
gross-R signal is turned into a **net-R** curve by composing cost nodes, each a
|
||
labelled stress parameter, on the `stage1-r` harness. The capability must span
|
||
**both** charge modes — per-trade costs (deduct at close) and per-held-cycle
|
||
costs (accrue over the hold) — so a research user can measure how much friction
|
||
costs their edge and see whether a holding cost changes the **shape** of the
|
||
equity curve. Shipped across cycles 0081–0085: `ConstantCost`, the `net_r_equity`
|
||
tap + `summarize_r` cost fold, `VolSlippageCost` + `CostSum`, the `CostNode`
|
||
trait, the `cost_graph` composite-builder, and per-held-cycle accrual
|
||
(`CarryCost` + `ChargeMode`). Public surface exercised: the `aura` CLI
|
||
(`run --harness stage1-r` with `--cost-per-trade` / `--slip-vol-mult` /
|
||
`--carry-per-cycle`, `--trace`, `chart --tap`), against the design ledger C10 and
|
||
the glossary. Binary exercised: `target/debug/aura` freshly built from HEAD
|
||
(`cargo build --workspace`, commit 84ec2ab).
|
||
|
||
The milestone promise is **delivered**: all four end-to-end scenarios behave as
|
||
the contract states, including the capstone holding-cost shape change and exact
|
||
cost composition. No capability bug. The findings are doc/diagnostic friction at
|
||
the public surface and one undefined-behaviour spec_gap; none blocks the close.
|
||
|
||
## Examples
|
||
|
||
All under `fieldtests/milestone-cost-model-graph/`. `run_scenarios.sh` reproduces
|
||
every command; `captured_outputs.txt` is its recorded output; `traces/` holds the
|
||
`r_equity` / `net_r_equity` series the runs persisted (the `runs/` store is
|
||
gitignored); `chart_carry_net_r_equity.html` is the charted capstone bleed.
|
||
|
||
### run_scenarios.sh §1 — bare gross-R, no cost model
|
||
- `aura run --harness stage1-r` with no cost flag.
|
||
- Fits scope: the zero-cost floor — C10's "a bare gross-R run with no cost model
|
||
attached is valid", the additive-on baseline every cost composes onto.
|
||
- Outcome: ran; `expectancy_r` = `net_expectancy_r` = `1.2710005136982836`
|
||
exactly (net == gross when no cost is attached). Clean.
|
||
|
||
### run_scenarios.sh §2 — flat per-trade cost (ConstantCost, deduct at close)
|
||
- `aura run --harness stage1-r --cost-per-trade {0.0001,0.001,0.01}`, plus a
|
||
`--trace pertrade_cost` run.
|
||
- Fits scope: the per-trade charge mode (cycle 0081), "how much does friction
|
||
cost my edge?".
|
||
- Outcome: ran; gross `expectancy_r` stays pinned at `1.2710005136982836`
|
||
while `net_expectancy_r` drops strictly and monotonically with cost
|
||
(`1.2402` → `0.9632` → `-1.8069`); the `net_r_equity` tap persists under
|
||
`--trace`. Matched expected.
|
||
|
||
### run_scenarios.sh §3 — per-held-cycle carry (CarryCost, accrue over hold) — capstone
|
||
- `aura run --harness stage1-r --carry-per-cycle 0.001 --trace carry_cost`.
|
||
- Fits scope: the milestone capstone (cycle 0085) — the per-held-cycle charge
|
||
mode; "does a holding cost change the SHAPE of the curve?".
|
||
- Outcome: ran; `net_expectancy_r` = `-0.2679604926492904`. The `net_r_equity`
|
||
gap (gross − net) **grows continuously over a single hold** — ts4..ts7:
|
||
`0.367 → 0.734 → 1.101 → 1.468` — versus the per-trade run's **flat** `0.367`
|
||
over the same hold. The holding cost bleeds the curve continuously rather than
|
||
stepping at close. Matched expected exactly.
|
||
|
||
### run_scenarios.sh §4 — compose all three costs + chart
|
||
- `aura run --harness stage1-r --cost-per-trade 0.001 --slip-vol-mult 0.5
|
||
--carry-per-cycle 0.001 --trace composed_cost`, each single cost alone for
|
||
contrast, then `aura chart composed/carry --tap net_r_equity`.
|
||
- Fits scope: the cost graph **composes** (cycles 0082/0084) — N cost nodes sum
|
||
into one net-R curve via `CostSum`/`cost_graph`.
|
||
- Outcome: ran; composed `net_expectancy_r` = `-1.2927895081922123`, strictly
|
||
below every single cost alone (`0.5540` slip, `0.9632` per-trade, `-0.2680`
|
||
carry). The composed net-R drag (`gross − composed = 2.5637900218904959`)
|
||
equals the **exact** sum of the three single-cost drags
|
||
(`0.30779220 + 0.71703681 + 1.53896101 = 2.56379002`, agreeing to the last
|
||
ULP). The chart HTML bakes the real `net_r_equity` points array in (uPlot), so
|
||
the bleed renders — not a shell. Matched expected.
|
||
|
||
## Findings
|
||
|
||
### [working] Gross-only run is clean: net == gross with no cost
|
||
- run_scenarios.sh §1.
|
||
- `net_expectancy_r` is byte-equal to `expectancy_r` (`1.2710005136982836`) and
|
||
no `net_r_equity` is forced; the cost layer is genuinely optional and additive
|
||
(C10 "the zero-cost baseline is the default-simple floor").
|
||
- Why working: the optional-cost contract holds exactly; the win to protect is
|
||
that attaching no cost changes nothing.
|
||
- Recommended action: carry-on.
|
||
|
||
### [working] Per-trade cost: gross pinned, net drops monotonically, tap persists
|
||
- run_scenarios.sh §2.
|
||
- Across three magnitudes gross `expectancy_r` never moves; `net_expectancy_r`
|
||
is strictly below it and monotone decreasing; `runs/traces/pertrade_cost/`
|
||
contains a `net_r_equity.json` sibling of `r_equity.json`.
|
||
- Why working: the per-trade charge mode + the gross/net split + the named tap
|
||
all behave as C10 / the glossary describe, first try.
|
||
- Recommended action: carry-on.
|
||
|
||
### [working] Capstone — holding cost bleeds the curve continuously over the hold
|
||
- run_scenarios.sh §3 (+ `traces/carry_cost_net_r_equity.json`).
|
||
- The gross−net gap during the cycles-4..7 hold grows linearly
|
||
(`0.367, 0.734, 1.101, 1.468`), where the per-trade run's gap over the same
|
||
hold is flat (`0.367, 0.367, 0.367, 0.367`). The per-held-cycle accrual makes
|
||
the net-R equity curve bleed continuously across a hold instead of stepping at
|
||
close — the milestone's headline "shape change", directly observable in the
|
||
persisted series.
|
||
- Why working: this is the milestone capstone (cycle 0085, approach B) and it is
|
||
empirically true end-to-end from the CLI, not merely asserted in a unit test.
|
||
- Recommended action: carry-on.
|
||
|
||
### [working] Composition is exact-additive and the chart renders the bleed
|
||
- run_scenarios.sh §4 (+ `chart_carry_net_r_equity.html`,
|
||
`traces/composed_cost_*.json`).
|
||
- Three heterogeneous cost factors (per-trade + state-dependent slippage +
|
||
per-held-cycle carry) sum into one net-R curve; the composed drag equals the
|
||
exact sum of the single-cost drags (to the last ULP) and composed net is
|
||
strictly below every single. `aura chart` emits a self-contained HTML page with
|
||
the actual `net_r_equity` points array embedded.
|
||
- Why working: `CostSum` / `cost_graph` "one home for cost, no double-count"
|
||
(C10) is exact, the flags compose as the usage string implies, and the headline
|
||
artifact charts as a real result render (matching the "render results, not a
|
||
shell" rule).
|
||
- Recommended action: carry-on.
|
||
|
||
### [friction] Negative cost-rate guard prints the bare usage string, names no cause
|
||
- `aura run --harness stage1-r --carry-per-cycle -0.001` (and `--cost-per-trade
|
||
-0.5`).
|
||
- Verbatim: `aura: usage: aura run [--harness <sma|macd|stage1-r>] ...` then
|
||
`exit=2`. The guard correctly refuses a negative rate (capability-safe, per the
|
||
commit-noted "negative-rate guard"), but the diagnostic is the entire 1-line
|
||
usage dump — it never says *why* (e.g. "cost rate must be ≥ 0"), and the usage
|
||
string itself never states the non-negativity constraint, so a user who
|
||
fat-fingers a sign cannot tell a rejected value from a mistyped flag name.
|
||
- Why friction: the task is refused correctly but the surface gives an
|
||
uninformative diagnostic that names no cause.
|
||
- Recommended action: plan (a tidy iteration: emit `cost rate must be
|
||
non-negative, got <v>` instead of the generic usage dump).
|
||
|
||
### [friction] Cost flags silently no-op on a non-R harness (sma / macd)
|
||
- `aura run --harness sma --cost-per-trade 0.001` → `exit=0`, output
|
||
byte-identical to the no-cost `sma` run (`"total_pips":-0.13000000000001358`,
|
||
no `r` block, no `net_*`, no warning).
|
||
- Why friction: a cost flag attached to a harness that has no R-evaluator /
|
||
executor chain is silently swallowed; a user who believes they measured a
|
||
cost-adjusted result got an unmodified one with no signal that the flag did
|
||
nothing. (Distinct from the spec_gap below: this entry is the missing-feedback
|
||
foot-gun, regardless of which behaviour is chosen.)
|
||
- Recommended action: plan (warn-on-ignored-cost-flag, or reject the
|
||
combination).
|
||
|
||
### [spec_gap] Cost-flag-on-non-R-harness behaviour is undefined by the ledger
|
||
- Same repro as the friction above (`--cost-per-trade` on `--harness sma`).
|
||
- C10 / the glossary define cost only against the R chain (gross R → net R) and
|
||
say a bare gross-R run with no cost is valid; neither constrains what a cost
|
||
flag should do on a harness (`sma` / `macd`) that produces *no* R at all. The
|
||
project picked the "silently accept and ignore" reading; "reject: cost requires
|
||
an R-evaluator harness" was equally plausible. I did not have to guess to
|
||
complete the milestone scenarios (they all use `stage1-r`), but a downstream
|
||
user mixing flags hits an unspecified corner.
|
||
- Why spec_gap: the ledger is silent on the case and the project chose one of two
|
||
equally-plausible readings.
|
||
- Recommended action: ratify (pick error-vs-noop and add a one-line C10 note;
|
||
pairs with the friction fix above).
|
||
|
||
### [friction] Cost-flag units / scale are undiscoverable at the CLI
|
||
- All scenarios. The usage string offers only `[--cost-per-trade <f64>]
|
||
[--slip-vol-mult <f64>] [--carry-per-cycle <f64>]` — no unit, no
|
||
per-close-vs-per-held-cycle note, no scale guidance. I had to trial
|
||
`0.0001 / 0.001 / 0.01` to find a magnitude that moved net-R sensibly, because
|
||
the R-normalizer (the stop distance `|entry − stop|`, the denominator of
|
||
`cost-in-R = cost_per_trade / |entry − stop|`) is not surfaced anywhere a CLI
|
||
user can see, and `--carry-per-cycle`'s unit (price units, **per engine cycle**
|
||
— not per day / not an overnight swap) is invisible. The semantics *are* in
|
||
C10 and the glossary "cost model" entry, so the capability is discoverable in
|
||
the ledger — just not at the point of use.
|
||
- Why friction: the surface forced a trial-and-error dance to choose a sensible
|
||
stress-parameter value that a one-line units hint would have removed.
|
||
- Recommended action: plan / docwriter (a per-flag units note in the usage
|
||
string: "price units, charged in R as cost/|entry−stop|; --carry-per-cycle is
|
||
per engine cycle").
|
||
|
||
### [friction] The public "how aura is used" doc describes the retired pre-reframe path
|
||
- `docs/project-layout.md` (the only public usage doc) — its "day in the life"
|
||
still says `aura backtest … --broker pepperstone` yields a "realistic currency
|
||
curve" alongside an "exposure stream" from a "realistic broker". All three —
|
||
`aura backtest`, the `--broker` realistic broker, the exposure framing — are
|
||
**retired** by the 2026-06-28 C10 reframe; the actual surface is
|
||
`aura run --harness stage1-r --cost-per-trade …` producing net-R, which the doc
|
||
never mentions. A downstream user reading it to learn how to charge cost is
|
||
pointed at a path that no longer exists and never told about the cost flags.
|
||
- Why friction: a public-surface document that actively misleads about the
|
||
shipped capability (doc rot, not a ledger gap — C10 itself is current).
|
||
- Recommended action: plan / docwriter (refresh `project-layout.md` to the
|
||
post-reframe `aura run` + cost-flag surface).
|
||
|
||
## Recommendation summary
|
||
|
||
| Finding | Class | Action |
|
||
|---|---|---|
|
||
| Gross-only run clean (net == gross) | working | carry-on |
|
||
| Per-trade: gross pinned, net monotone, tap persists | working | carry-on |
|
||
| Capstone: holding cost bleeds over the hold | working | carry-on |
|
||
| Composition exact-additive + chart renders | working | carry-on |
|
||
| Negative-rate guard prints bare usage, no cause | friction | plan |
|
||
| Cost flags silently no-op on sma/macd | friction | plan |
|
||
| Cost-flag-on-non-R-harness undefined by ledger | spec_gap | ratify |
|
||
| Cost-flag units/scale undiscoverable at CLI | friction | plan / docwriter |
|
||
| project-layout.md describes the retired path | friction | plan / docwriter |
|
||
|
||
**Milestone-close roll-up:** `friction_found`. The capability is delivered
|
||
end-to-end (both charge modes, composition, the holding-cost shape change); no
|
||
capability bug blocks the close. All findings are doc / diagnostic friction or an
|
||
undefined-corner spec_gap — non-gating for the milestone close.
|