Brummel/Aura
1
0
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity

Render remaining catalogue formulas as KaTeX math (sections A,B,D-J + master index) 

Roll out the section-C treatment across the whole Metric Catalogue: the
formula column of every table and the formula tokens of the compact master
index are now KaTeX inline math. Thresholds, [L]/[C]/[CORR] markers,
computed-from cells and prose are untouched; descriptive pseudo-expressions
were deliberately left as code. Abs-bars use \lvert/\rvert to keep the
markdown tables intact.

82 formula edits applied, each adversarially verified for mathematical
equivalence to the original and parse-validated against KaTeX; all 140 math
spans confirmed rendering through Gitea's gfm pipeline.
Brummel
2026-06-15 17:25:53 +02:00
parent 5b4dc96ecd
commit 86e5511be7
1 changed files with 82 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Metric-Catalogue.md
+82 -82
@@ -23,7 +23,7 @@ Robustness is read from the *geometry* of that surface, not the single best cell
| metric | formula | computed from | threshold |
|---|---|---|---|
| Plateau-to-peak ratio | `R = mean_{p∈Nbhd(p*)} f(p) / f(p*)` | metric array over a k-step neighbourhood of the argmax | `R ≳ 0.8–0.9` robust plateau; `< ~0.7` lucky spike **[C]** |
| Plateau-to-peak ratio | $R = \operatorname{mean}_{p\in\mathrm{Nbhd}(p^*)} f(p) / f(p^*)$ | metric array over a k-step neighbourhood of the argmax | `R ≳ 0.8–0.9` robust plateau; `< ~0.7` lucky spike **[C]** |
| Parameter sensitivity | `max(f over ±1-step) − min(...)` (or std) | perturb each axis one grid step around `p*` | small spread = robust; edge should survive ±10–20% param change **[C]** |
| Surface smoothness | `mean|f(adjacent) − f|` (mean abs gradient) | adjacent-cell differences | lower = smoother; global-best `> ~20–30%` above neighbours = spike |
@@ -39,10 +39,10 @@ significance of the win (see §J).
| metric (as objective) | formula | computed from | threshold |
|---|---|---|---|
| Sharpe | `sqrt(P)·mean(r)/std(r)` | `r` | §C Sharpe |
| Calmar / CAR-over-MDD | `mean(r)·P / MaxDD` | `r`, `E` | §D Calmar |
| Ulcer Performance Index | `(R_p − R_f) / UlcerIndex` | `r`, `E` | more overfit-resistant than Calmar |
| Profit factor | `gross_profit / gross_loss` | trade list | §E |
| Sharpe | $\sqrt{P}\cdot\operatorname{mean}(r)/\operatorname{std}(r)$ | `r` | §C Sharpe |
| Calmar / CAR-over-MDD | $\operatorname{mean}(r)\cdot P / \mathrm{MaxDD}$ | `r`, `E` | §D Calmar |
| Ulcer Performance Index | $(R_p - R_f) / \mathrm{UlcerIndex}$ | `r`, `E` | more overfit-resistant than Calmar |
| Profit factor | $\mathrm{gross\_profit} / \mathrm{gross\_loss}$ | trade list | §E |
**Caveat:** ratios with small denominators (low-variance / few-trade cells)
explode → spurious winners; enforce a minimum trade count / minimum exposure.
@@ -81,17 +81,17 @@ trade list required.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Max drawdown | `max_t(HWM−E)` (level) or `/HWM` (%) | `E` | stat-arb/MN `<10–15%`, HF `<15–20%`, L/S unacceptable `>25–30%`, `>40%` very high **[C; no universal cutoff]** |
| Recovery asymmetry | DD fraction `d` needs gain `d/(1−d)` | a DD depth | exact: −20%→+25%, −50%→+100% **[L, algebraic]** |
| Average drawdown | `(1/n)Σ|DD|` (=Pain Index) OR episode-mean of trough depths | `E`, episode segmentation | comparative; `AvgDD/MaxDD≈1` → worst typical |
| Max drawdown | $\max_t(\mathrm{HWM}-E)$ (level) or $/\mathrm{HWM}$ (%) | `E` | stat-arb/MN `<10–15%`, HF `<15–20%`, L/S unacceptable `>25–30%`, `>40%` very high **[C; no universal cutoff]** |
| Recovery asymmetry | DD fraction `d` needs gain $d/(1-d)$ | a DD depth | exact: −20%→+25%, −50%→+100% **[L, algebraic]** |
| Average drawdown | $(1/n)\sum\lvert\mathrm{DD}\rvert$ (=Pain Index) OR episode-mean of trough depths | `E`, episode segmentation | comparative; `AvgDD/MaxDD≈1` → worst typical |
| Duration / time-to-recovery / time-underwater | per-episode index arithmetic; total UW = #(DD<0)/n | `E` + time index | heuristic recovery ≈ `MDD/per-period-return`; flag censored open DD **[C]** |
| Longest drawdown | `max_k(t_recover − t_peak)` | episode segmentation | shorter better; often a *different* episode than max-depth |
| Ulcer Index | `sqrt((Σ_i D'_i²)/n)`, `D'_i = 100·(E_i−max_so_far)/max_so_far` | `E` | `=0` no DD; `> ~10` severe; comparative **[C]** |
| Ulcer Perf Index / Martin | `(R_p−R_f)/UlcerIndex` | `r`, `E` | higher better; no band; more overfit-robust objective **[L]** |
| Pain Index | `(Σ_i |D'_i|)/n`, n = *total* obs | `E` | comparative; `< MaxDD`. **[CORR:** it EMBEDS duration — a *remedy*, NOT a metric that "ignores time"; Bacon 2008 p.89; n=total obs else it collapses to Average Drawdown**]** |
| Pain Ratio | `(R_p−R_f)/PainIndex` | `r`, `E` | higher better; Bacon 2008 p.91 **[CORR page]** |
| Recovery Factor | `NetProfit/|MaxDrawdown|` | final `E` / MaxDD | poor `<1`, good `2–5`, excellent `≥5` **[formula L; thresholds C]** |
| Expected MaxDD (Magdon-Ismail) | Brownian: linear (μ<0), `~sqrt(T)` (μ=0, exact `sqrt(π/2)·σ·sqrt(T)`), log (μ>0) | μ,σ from `r`; classify regime | not graded — a yardstick; observed ≫ E[MaxDD] = regime break / fat tails **[L asymptotics]** |
| Longest drawdown | $\max_k(t_{\mathrm{recover}}-t_{\mathrm{peak}})$ | episode segmentation | shorter better; often a *different* episode than max-depth |
| Ulcer Index | $\sqrt{(\sum_i {D'_i}^2)/n}$, $D'_i=100\cdot(E_i-\mathrm{max\_so\_far})/\mathrm{max\_so\_far}$ | `E` | `=0` no DD; `> ~10` severe; comparative **[C]** |
| Ulcer Perf Index / Martin | $(R_p-R_f)/\mathrm{UlcerIndex}$ | `r`, `E` | higher better; no band; more overfit-robust objective **[L]** |
| Pain Index | $(\sum_i\lvert D'_i\rvert)/n$, n = *total* obs | `E` | comparative; `< MaxDD`. **[CORR:** it EMBEDS duration — a *remedy*, NOT a metric that "ignores time"; Bacon 2008 p.89; n=total obs else it collapses to Average Drawdown**]** |
| Pain Ratio | $(R_p-R_f)/\mathrm{PainIndex}$ | `r`, `E` | higher better; Bacon 2008 p.91 **[CORR page]** |
| Recovery Factor | $\mathrm{NetProfit}/\lvert\mathrm{MaxDrawdown}\rvert$ | final `E` / MaxDD | poor `<1`, good `2–5`, excellent `≥5` **[formula L; thresholds C]** |
| Expected MaxDD (Magdon-Ismail) | Brownian: linear (μ<0), $\sim\sqrt{T}$ (μ=0, exact $\sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T}$), log (μ>0) | μ,σ from `r`; classify regime | not graded — a yardstick; observed ≫ E[MaxDD] = regime break / fat tails **[L asymptotics]** |
**Load-bearing point:** MaxDD is a downward-biased, sample-length-dependent point
estimate — the cure is Monte-Carlo (report the *distribution* of MaxDD, not one
@@ -106,17 +106,17 @@ its close; P&L = `Σ r_t` over the span; direction = sign of the held position.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Win rate p | `wins/total_trades` | trades | not gradable alone; vs breakeven `1/(1+R)` |
| **Profit factor** | `gross_profit/gross_loss` | trades | `<1` losing, `1.0–1.2` breakeven, `1.2–1.5` realistic, `1.5–2.0` good, `2.0–3.0` exceptional-but-scrutinize, `>3` overfit flag. **[CORR:** swing target HIGHER than day-trade; ~200+ trades needed; live degrades PF ~10–20%**]** |
| Expectancy | `(p·AvgWin) − ((1−p)·AvgLoss)` | trades | only hard threshold is sign `>0` |
| Payoff ratio R | `AvgWin/AvgLoss` | trades | good = exceeds breakeven `(1−p)/p` |
| **PF↔win-rate identity** | `PF=(p·R)/(1−p)`, inverse `p=PF/(PF+R)` | p, R | exact identity (holds iff R = ratio of *averages*) **[L]** |
| Breakeven win rate | `1/(1+R)` | R | 50%@1:1, 33.3%@1:2, 25%@1:3, 16.7%@1:5 **[L]** |
| SQN (Van Tharp) | `sqrt(N)·mean(R-mult)/std(R-mult)`, N often capped 100 | per-trade R-multiples | `<1.6` below, `1.6–2.0` avg, `2.0–2.5` good, `2.5–3.0` excellent, `>7` overfit **[C]** |
| Win rate p | $\mathrm{wins}/\mathrm{total\_trades}$ | trades | not gradable alone; vs breakeven `1/(1+R)` |
| **Profit factor** | $\mathrm{gross\_profit}/\mathrm{gross\_loss}$ | trades | `<1` losing, `1.0–1.2` breakeven, `1.2–1.5` realistic, `1.5–2.0` good, `2.0–3.0` exceptional-but-scrutinize, `>3` overfit flag. **[CORR:** swing target HIGHER than day-trade; ~200+ trades needed; live degrades PF ~10–20%**]** |
| Expectancy | $(p\cdot\mathrm{AvgWin})-((1-p)\cdot\mathrm{AvgLoss})$ | trades | only hard threshold is sign `>0` |
| Payoff ratio R | $\mathrm{AvgWin}/\mathrm{AvgLoss}$ | trades | good = exceeds breakeven `(1−p)/p` |
| **PF↔win-rate identity** | $\mathrm{PF}=(p\cdot R)/(1-p)$, inverse $p=\mathrm{PF}/(\mathrm{PF}+R)$ | p, R | exact identity (holds iff R = ratio of *averages*) **[L]** |
| Breakeven win rate | $1/(1+R)$ | R | 50%@1:1, 33.3%@1:2, 25%@1:3, 16.7%@1:5 **[L]** |
| SQN (Van Tharp) | $\sqrt{N}\cdot\operatorname{mean}(\text{R-mult})/\operatorname{std}(\text{R-mult})$, N often capped 100 | per-trade R-multiples | `<1.6` below, `1.6–2.0` avg, `2.0–2.5` good, `2.5–3.0` excellent, `>7` overfit **[C]** |
| MAE / MFE | max adverse / favorable excursion = `−min`/`max(running unreal. PnL)` | per-bar running P&L since entry | no cutoff; winners cluster at small MAE; large MFE-vs-realized gap → exiting early |
| Trade efficiency | exit `ActualGain/MFE`; range `(Exit−Entry)/(High−Low)` | per-trade MFE/MAE/range | 0–100%; `<50%` exit eff poor **[C]** |
| Turnover | `Σ|Δposition|` (or SEC `min(buys,sells)/avg_assets`) | position series | style-dependent; net-of-cost expectancy must stay `>0` |
| R-multiple / expectancy-in-R | per trade `PnL/1R`; mean `p·R−(1−p)` | trades + defined 1R | sign `>0`; ~`+0.5R` strong **[C]** — needs a defined risk/stop |
| Trade efficiency | exit $\mathrm{ActualGain}/\mathrm{MFE}$; range $(\text{Exit}-\text{Entry})/(\text{High}-\text{Low})$ | per-trade MFE/MAE/range | 0–100%; `<50%` exit eff poor **[C]** |
| Turnover | $\sum\lvert\Delta\mathrm{position}\rvert$ (or SEC $\min(\mathrm{buys},\mathrm{sells})/\mathrm{avg\_assets}$) | position series | style-dependent; net-of-cost expectancy must stay `>0` |
| R-multiple / expectancy-in-R | per trade $\mathrm{PnL}/1R$; mean $p\cdot R-(1-p)$ | trades + defined 1R | sign `>0`; ~`+0.5R` strong **[C]** — needs a defined risk/stop |
## F. Tail, distribution & risk-of-ruin
@@ -126,30 +126,30 @@ empirical / Monte-Carlo estimates.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Historical VaR_α | `−(1−α)` empirical quantile of r (5th @95%, 1st @99%) | sort `r`, index `floor((1−α)N)` | Basel traffic-light @250d/99%: GREEN 0–4, YELLOW 5–9, RED 10+ exceptions **[L]** |
| Parametric VaR | `−(μ+σ·Φ⁻¹(1−α))·W`; z = 1.645@95, 2.326@99, 1.282@90 | μ,σ of `r` | same grading; **iid-normal trap** **[L multipliers]** |
| Historical VaR_α | $-(1-\alpha)$ empirical quantile of r (5th @95%, 1st @99%) | sort `r`, index `floor((1−α)N)` | Basel traffic-light @250d/99%: GREEN 0–4, YELLOW 5–9, RED 10+ exceptions **[L]** |
| Parametric VaR | $-(\mu+\sigma\cdot\Phi^{-1}(1-\alpha))\cdot W$; z = 1.645@95, 2.326@99, 1.282@90 | μ,σ of `r` | same grading; **iid-normal trap** **[L multipliers]** |
| Historical ES/CVaR | `mean of worst (1−α)N returns` | sorted `r`, `k=ceil((1−α)N)` | coherent (VaR is not); FRTB uses 97.5% ES ≈ 99% VaR under normality **[L]** |
| Parametric ES | `μ+σ·φ(Φ⁻¹(α))/(1−α)`; mult ~2.063@95, 2.665@99 | μ,σ | worse than VaR for fat tails **[L]** |
| Tail ratio | `q95(r)/|q05(r)|` | percentiles of `r` | `>1` favourable, `<1` dangerous **[L 95/5]** |
| Skewness | `(1/N)Σ(r−μ)³/σ³` | `r` | sign-driven; persistent negative = red flag even with high Sharpe |
| Excess kurtosis | `(1/N)Σ(r−μ)⁴/σ⁴ − 3` | `r` | `0`=normal; `>0` leptokurtic (typical, dangerous); daily 3–10+ **[L]** |
| Downside deviation | `sqrt((1/N)Σ[min(0,r−T)]²)`, divisor total N | `r`, T | feeds Sortino **[L, divisor=total N]** |
| Risk of ruin (closed) | `((1−A)/(1+A))^U`, `A=p·R−q`, `U=capital/risk-per-trade` | trades p, R | `<1–5%`, desks `<1%`, pros `<0.1%`; `>5–10%` reject **[C]** |
| Parametric ES | $\mu+\sigma\cdot\varphi(\Phi^{-1}(\alpha))/(1-\alpha)$; mult ~2.063@95, 2.665@99 | μ,σ | worse than VaR for fat tails **[L]** |
| Tail ratio | $q95(r)/\lvert q05(r)\rvert$ | percentiles of `r` | `>1` favourable, `<1` dangerous **[L 95/5]** |
| Skewness | $(1/N)\sum(r-\mu)^3/\sigma^3$ | `r` | sign-driven; persistent negative = red flag even with high Sharpe |
| Excess kurtosis | $(1/N)\sum(r-\mu)^4/\sigma^4 - 3$ | `r` | `0`=normal; `>0` leptokurtic (typical, dangerous); daily 3–10+ **[L]** |
| Downside deviation | $\sqrt{(1/N)\sum[\min(0,r-T)]^2}$, divisor total N | `r`, T | feeds Sortino **[L, divisor=total N]** |
| Risk of ruin (closed) | $((1-A)/(1+A))^U$, $A=p\cdot R-q$, $U=\text{capital}/\text{risk-per-trade}$ | trades p, R | `<1–5%`, desks `<1%`, pros `<0.1%`; `>5–10%` reject **[C]** |
| Risk of ruin (empirical) | `fraction of N Monte-Carlo paths breaching the ruin threshold` | multi-run dist. | **preferred**; sidesteps iid-normal; default R=50% capital |
| Kelly / optimal-f | `f*=(b·p−q)/b`; `f*=(μ−r)/σ²` | `r`/odds | deploy `0 < f ≤ ~0.5·f*`; above full Kelly strictly dominated **[L]** |
| Gain-to-Pain | `Σ(r)/|Σ(neg r)|` | `r` | (monthly) ~1 acc, ~2 outstanding, 3 excellent, ~4 world-class, `<1` poor **[C; state the basis]** |
| Common-Sense ratio | `TailRatio · GainToPain` | the two above | `>1` robust, `<1` fragile **[L breakpoint]** |
| Kelly / optimal-f | $f^*=(b\cdot p-q)/b$; $f^*=(\mu-r)/\sigma^2$ | `r`/odds | deploy `0 < f ≤ ~0.5·f*`; above full Kelly strictly dominated **[L]** |
| Gain-to-Pain | $\sum(r)/\lvert\sum(\text{neg } r)\rvert$ | `r` | (monthly) ~1 acc, ~2 outstanding, 3 excellent, ~4 world-class, `<1` poor **[C; state the basis]** |
| Common-Sense ratio | $\mathrm{TailRatio}\cdot\mathrm{GainToPain}$ | the two above | `>1` robust, `<1` fragile **[L breakpoint]** |
## G. Walk-forward analysis
| metric | formula | computed from | threshold |
|---|---|---|---|
| WFE (return) | `(OOS_profit/OOS_bars)/(IS_profit/IS_bars)` | per-fold IS-optimal/OOS profit, bar counts | `≥0.5` robust, `~1.0` ideal, `>0.7` excellent, `0.5–0.7` good, `0.3–0.5` mediocre, `<0.3` poor; `>1.0` is a *warning* (under-fit IS / lucky OOS), not superiority |
| WFE (Sharpe) | `OOS_Sharpe/IS_Sharpe` | per-fold Sharpe over `r` | `>0.5` robust, `~1.0` ideal, `<0.3` red flag |
| IS-OOS gap | `IS_Sharpe − OOS_Sharpe` per fold, averaged | per-fold Sharpes | `>0.5`/cycle = overfitting; healthy mean `<0.3` |
| WFE (return) | $(\mathrm{OOS\_profit}/\mathrm{OOS\_bars})/(\mathrm{IS\_profit}/\mathrm{IS\_bars})$ | per-fold IS-optimal/OOS profit, bar counts | `≥0.5` robust, `~1.0` ideal, `>0.7` excellent, `0.5–0.7` good, `0.3–0.5` mediocre, `<0.3` poor; `>1.0` is a *warning* (under-fit IS / lucky OOS), not superiority |
| WFE (Sharpe) | $\mathrm{OOS\_Sharpe}/\mathrm{IS\_Sharpe}$ | per-fold Sharpe over `r` | `>0.5` robust, `~1.0` ideal, `<0.3` red flag |
| IS-OOS gap | $\mathrm{IS\_Sharpe}-\mathrm{OOS\_Sharpe}$ per fold, averaged | per-fold Sharpes | `>0.5`/cycle = overfitting; healthy mean `<0.3` |
| Stitched OOS curve | chain each fold's OOS segment, re-based | family of OOS runs | net profitable; max DD `<40%` (TradeStation) |
| Parameter stability | dispersion of argmax params across folds; plateau check | per-fold winners | qualitative — broad plateau, clustered |
| Fraction profitable OOS windows | `#(OOS>0)/#windows` | per-window OOS P&L | majority `>50%`; worst-window OOS Sharpe `>0` |
| Fraction profitable OOS windows | $\#(\mathrm{OOS}>0)/\#\text{windows}$ | per-window OOS P&L | majority `>50%`; worst-window OOS Sharpe `>0` |
| Pardo verdict | net-profitable AND WFE `≥50%` AND maxDD `<40%`; filter %Prof `≥40`, PF `≥1.5` | the above | composite pass/fail **[software-default convention]** |
**Sizing:** IS:OOS 2×–5×; `≥6` windows (10+ preferred); non-overlapping OOS for a
@@ -166,9 +166,9 @@ schedule.
| Equity percentile fan | per-horizon {5,25,50,75,95} pct | N curves | descriptive; viability: 5th-pct ending equity `>0` |
| Stress (95th-pct) MaxDD | 95th pct of N MaxDD | per-run MaxDD | `<25%` good, `>35%` red flag; size capital off this **[C]** |
| Empirical VaR/ES | 5th-pct (1st @99) final P&L; ES = mean beyond | per-run terminal | 5th-pct ending equity `>0` |
| Probability of ruin | `ruin_count/N` (MaxDD>R, default 50%) | per-run MaxDD | `<1%` excellent, `1–5%` acceptable, `>5%` not recommended **[C]** |
| Prob. of a losing period | `#(period<0)/N` | per-run period returns | `>~30–40%` hard to hold |
| **MCPT p-value** | `(z+1)/(N+1)`, z=#perm `≥` real | permute bar-to-bar log returns, keep first price, re-run | `<0.05` min, `<0.01` strict, `>0.10` red flag, `~0.5` discard **[L: +1 in numerator and denominator]** |
| Probability of ruin | $\mathrm{ruin\_count}/N$ (MaxDD>R, default 50%) | per-run MaxDD | `<1%` excellent, `1–5%` acceptable, `>5%` not recommended **[C]** |
| Prob. of a losing period | $\#(\text{period}<0)/N$ | per-run period returns | `>~30–40%` hard to hold |
| **MCPT p-value** | $(z+1)/(N+1)$, z=#perm `≥` real | permute bar-to-bar log returns, keep first price, re-run | `<0.05` min, `<0.01` strict, `>0.10` red flag, `~0.5` discard **[L: +1 in numerator and denominator]** |
| White RC / Hansen SPA | bootstrap max-stat over L rules, recentered | per-rule rel-perf + benchmark | `p<0.05`; SPA more powerful than the conservative RC |
**Conventions:** N = 1,000 min, 5,000 recommended, 10,000+ for tail stability.
@@ -183,11 +183,11 @@ time-ordering edges.
|---|---|---|---|---|
| Stress/scenario | conditional worst-case | per-scenario MaxDD, profit, recover-time | strategy over crisis windows / shocked synthetic data | fail if stress DD > risk limit; excellent if net-positive through crises **[C; Fed CCAR]** |
| Regime | per-state metrics + fragility verdict | profit/Sharpe/DD masked by regime label | a regime-label series (vol/trend classifier or HMM) | edge only in one regime + negative elsewhere = fragile **[qualitative]** |
| Cost/slippage | net-vs-cost curve + break-even | `c* where net=0 ≈ gross_profit/n_trades`; `d(net Sharpe)/d(cost)` | a cost-modelling backtest, cost swept | reject if edge survives only at zero cost; break-even ≳2× real cost **[C]** |
| Capacity/impact | AUM ceiling + turnover | `impact ≈ Y·σ·sqrt(Q/V)`, Y~O(1); capacity = AUM where impact = gross alpha | order volumes + ADV | capacity ≫ target = scalable; per-day order `< ~5–10%` ADV **[C]** |
| Benchmark-relative | alpha/beta/IR + significance | `beta=Cov/Var`; Jensen alpha = regression intercept; `IR=alpha/TE` | `r` regressed on a market series `R_m` | alpha positive + significant (`|t|>2`); IR 0.4/0.6/1.0 **[beta/CAPM L; IR C]** |
| Null/sanity | "beat random?" | MCPT `p=(#perm≥real)/N`; random-entry: real in right tail | permuted input + a random-entry null strategy | `p<0.05` min, `≤0.01` strict, `>0.10` red flag, N`≥1000` **[L]** |
| Live-vs-backtest | overfitting/decay verdict | haircut `=(BT_Sharpe−live_Sharpe)/BT_Sharpe` | live series vs backtest prediction | `<30%` acceptable, `30–50%` typical, `>50–70%` largely overfit **[C; verify primary source]** |
| Cost/slippage | net-vs-cost curve + break-even | $c^*\text{ where net}=0\approx\mathrm{gross\_profit}/n_\text{trades}$; $d(\text{net Sharpe})/d(\text{cost})$ | a cost-modelling backtest, cost swept | reject if edge survives only at zero cost; break-even ≳2× real cost **[C]** |
| Capacity/impact | AUM ceiling + turnover | $\text{impact}\approx Y\cdot\sigma\cdot\sqrt{Q/V}$, Y~O(1); capacity = AUM where impact = gross alpha | order volumes + ADV | capacity ≫ target = scalable; per-day order `< ~5–10%` ADV **[C]** |
| Benchmark-relative | alpha/beta/IR + significance | $\beta=\operatorname{Cov}/\operatorname{Var}$; Jensen alpha = regression intercept; $\mathrm{IR}=\text{alpha}/\mathrm{TE}$ | `r` regressed on a market series `R_m` | alpha positive + significant (`|t|>2`); IR 0.4/0.6/1.0 **[beta/CAPM L; IR C]** |
| Null/sanity | "beat random?" | MCPT $p=(\#\text{perm}\ge\text{real})/N$; random-entry: real in right tail | permuted input + a random-entry null strategy | `p<0.05` min, `≤0.01` strict, `>0.10` red flag, N`≥1000` **[L]** |
| Live-vs-backtest | overfitting/decay verdict | haircut $=(\mathrm{BT\_Sharpe}-\mathrm{live\_Sharpe})/\mathrm{BT\_Sharpe}$ | live series vs backtest prediction | `<30%` acceptable, `30–50%` typical, `>50–70%` largely overfit **[C; verify primary source]** |
## J. Overfitting, multiple-testing & CPCV
@@ -196,14 +196,14 @@ kept the best" as the statistical problem it is.
| metric | formula | computed from | threshold |
|---|---|---|---|
| PSR(SR*) | `Φ[(SR_hat−SR*)·sqrt(T−1)/sqrt(1−g3·SR_hat+((g4−1)/4)·SR_hat²)]` | per-period `SR_hat`, T, g3=skew, g4=kurtosis (full, normal=3) of `r` | `>0.95` significant **[L]** |
| MinTRL | `1+(1−g3·SR_hat+((g4−1)/4)·SR_hat²)·(Z_α/(SR_hat−SR*))²` | as PSR + target α | need T `≥` MinTRL **[L]** |
| E[max SR_N] | `sqrt(V)·((1−γ)Φ⁻¹(1−1/N)+γΦ⁻¹(1−1/(Ne)))`, γ≈0.5772 | cross-trial Sharpe variance V, effective N | the moving hurdle; grows with N **[L]** |
| **DSR** | PSR at `SR*=SR_0=E[max SR_N]` | winner SR_hat/T/g3/g4 + V across trials + effective N | `>0.95` survives deflation **[L; ex: N=100,V=½,T=1250,g3=−3,g4=10 → DSR~0.90<0.95 REJECT]** |
| MinBTL | `≤ 2·ln(N)/E[max]²` years | N, target SR | 5y + `>~45` independent trials → guaranteed overfit **[C/L]** |
| **PBO (CSCV)** | matrix `(T×M)`; S even blocks; `C(S,S/2)` combos; `ω=(rank−0.5)/M`; `λ=ln(ω/(1−ω))`; `PBO=P[λ≤0]` | per-trial `r` series | `<0.5` sanity, `~0.5` alarm, `<0.1` excellent **[L]** |
| Harvey-Liu-Zhu haircut | `t=SR·sqrt(T)`; adjust p (Bonferroni `min(M·p,1)`/Holm/BHY); `HSR=SR·(adj_t/orig_t)` | SR, T, M | required new-factor t `~3.0` (not 2.0) |
| CPCV paths | splits `=C(N,k)`; paths `φ=(k/N)C(N,k)` | grouped purged+embargoed windows | N=5–10 groups, k=2; purge: drop train obs whose label span overlaps test; embargo h~1% of T (forward only) **[L]** |
| PSR(SR*) | $\Phi[(\mathrm{SR}_{hat}-\mathrm{SR}^*)\cdot\sqrt{T-1}/\sqrt{1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2}]$ | per-period `SR_hat`, T, g3=skew, g4=kurtosis (full, normal=3) of `r` | `>0.95` significant **[L]** |
| MinTRL | $1+(1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2)\cdot(Z_\alpha/(\mathrm{SR}_{hat}-\mathrm{SR}^*))^2$ | as PSR + target α | need T `≥` MinTRL **[L]** |
| E[max SR_N] | $\sqrt{V}\cdot((1-\gamma)\Phi^{-1}(1-1/N)+\gamma\Phi^{-1}(1-1/(Ne)))$, $\gamma\approx0.5772$ | cross-trial Sharpe variance V, effective N | the moving hurdle; grows with N **[L]** |
| **DSR** | PSR at $\mathrm{SR}^*=\mathrm{SR}_0=E[\max \mathrm{SR}_N]$ | winner SR_hat/T/g3/g4 + V across trials + effective N | `>0.95` survives deflation **[L; ex: N=100,V=½,T=1250,g3=−3,g4=10 → DSR~0.90<0.95 REJECT]** |
| MinBTL | $\le 2\cdot\ln(N)/E[\max]^2$ years | N, target SR | 5y + `>~45` independent trials → guaranteed overfit **[C/L]** |
| **PBO (CSCV)** | matrix $(T\times M)$; S even blocks; $C(S,S/2)$ combos; $\omega=(\mathrm{rank}-0.5)/M$; $\lambda=\ln(\omega/(1-\omega))$; $\mathrm{PBO}=P[\lambda\le0]$ | per-trial `r` series | `<0.5` sanity, `~0.5` alarm, `<0.1` excellent **[L]** |
| Harvey-Liu-Zhu haircut | $t=\mathrm{SR}\cdot\sqrt{T}$; adjust p (Bonferroni $\min(M\cdot p,1)$ / Holm / BHY); $\mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t})$ | SR, T, M | required new-factor t `~3.0` (not 2.0) |
| CPCV paths | splits $=C(N,k)$; paths $\varphi=(k/N)C(N,k)$ | grouped purged+embargoed windows | N=5–10 groups, k=2; purge: drop train obs whose label span overlaps test; embargo h~1% of T (forward only) **[L]** |
---
@@ -212,37 +212,37 @@ kept the best" as the statistical problem it is.
Inputs: **r** = per-period returns; **E** = equity curve; **trades** = closed
trades; **b** = benchmark; **multi** = multi-run distribution.
**Return** — Total `sum r` [C] · CAGR `mean(r)·P` [C] · Volatility `std(r)·sqrt(P)` [C].
**Return** — Total $\sum r$ [C] · CAGR $\operatorname{mean}(r)\cdot P$ [C] · Volatility $\operatorname{std}(r)\cdot\sqrt{P}$ [C].
**Risk-adjusted** — Sharpe `sqrt(P)·(mean(r)−r_f)/std(r−r_f)` [CORR denom] · Sortino
`…/sqrt(Σ min(0,r−MAR)²/N)`, good `>1` [CORR] · Calmar `mean(r)·P/|MaxDD|`,
`<1`/`1–3`/`>3` [CORR] · MAR `CAGR_inc/|MaxDD_inc|` [CORR window+numerator] · Sterling
`CompoundROR/|avgAnnMaxDD−10%|` [CORR] · Burke `(r_P−r_F)/sqrt(ΣD²)` [L] · Omega
`Σmax(0,r−θ)/Σmax(0,θ−r)` [L] · Info-Ratio `mean(r−b)/std(r−b)·sqrt(P)` [L/C] · UPI
`(R_p−R_f)/UlcerIndex` [L] · Pain-Ratio `(R_p−R_f)/PainIndex` [CORR page] ·
Recovery-Factor `NetProfit/|MaxDD|` [L/C] · Gain-to-Pain `Σr/|Σ neg r|` [C] ·
Common-Sense `TailRatio·GainToPain` [L] · SQN `sqrt(N)·mean(R)/std(R)` [C].
**Risk-adjusted** — Sharpe $\sqrt{P}\cdot(\operatorname{mean}(r)-r_f)/\operatorname{std}(r-r_f)$ [CORR denom] · Sortino
$\ldots/\sqrt{\sum\min(0,r-\mathrm{MAR})^2/N}$, good `>1` [CORR] · Calmar $\operatorname{mean}(r)\cdot P/\lvert\mathrm{MaxDD}\rvert$,
`<1`/`1–3`/`>3` [CORR] · MAR $\mathrm{CAGR}_{\text{inc}}/\lvert\mathrm{MaxDD}_{\text{inc}}\rvert$ [CORR window+numerator] · Sterling
$\mathrm{CompoundROR}/\lvert\mathrm{avgAnnMaxDD}-10\%\rvert$ [CORR] · Burke $(r_P-r_F)/\sqrt{\sum D^2}$ [L] · Omega
$\sum\max(0,r-\theta)/\sum\max(0,\theta-r)$ [L] · Info-Ratio $\operatorname{mean}(r-b)/\operatorname{std}(r-b)\cdot\sqrt{P}$ [L/C] · UPI
$(R_p-R_f)/\mathrm{UlcerIndex}$ [L] · Pain-Ratio $(R_p-R_f)/\mathrm{PainIndex}$ [CORR page] ·
Recovery-Factor $\mathrm{NetProfit}/\lvert\mathrm{MaxDD}\rvert$ [L/C] · Gain-to-Pain $\sum r/\lvert\sum\text{neg }r\rvert$ [C] ·
Common-Sense $\mathrm{TailRatio}\cdot\mathrm{GainToPain}$ [L] · SQN $\sqrt{N}\cdot\operatorname{mean}(R)/\operatorname{std}(R)$ [C].
**Drawdown** — MaxDD `max(HWM−E)` [C] · Recovery-asymmetry `d/(1−d)` [L] · AvgDD
`mean|DD|` [C] · Ulcer `sqrt(ΣD'²/n)` [C] · Pain `Σ|D'|/n` (n=total) [CORR embeds
duration] · Expected-MaxDD μ=0 `sqrt(π/2)·σ·sqrt(T)` [L].
**Drawdown** — MaxDD $\max(\mathrm{HWM}-E)$ [C] · Recovery-asymmetry $d/(1-d)$ [L] · AvgDD
$\operatorname{mean}\lvert\mathrm{DD}\rvert$ [C] · Ulcer $\sqrt{\sum D^{\prime 2}/n}$ [C] · Pain $\sum\lvert D^\prime\rvert/n$ (n=total) [CORR embeds
duration] · Expected-MaxDD μ=0 $\sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T}$ [L].
**Trade-level** — Win-rate `wins/total` [C] · Profit-Factor `gross_profit/gross_loss`,
`<1`/`1.5–2`/`>3 overfit` [CORR swing>day] · Expectancy `p·AvgWin−(1−p)·AvgLoss`,
sign>0 [L] · Payoff `AvgWin/AvgLoss` [L] · PF-identity `PF=(p·R)/(1−p)` [L] ·
Breakeven-win `1/(1+R)` [L] · MAE/MFE [C] · Trade-eff `ActualGain/MFE` [C] · Turnover
`Σ|Δposition|` [C].
**Trade-level** — Win-rate $\mathrm{wins}/\mathrm{total}$ [C] · Profit-Factor $\mathrm{gross\_profit}/\mathrm{gross\_loss}$,
`<1`/`1.5–2`/`>3 overfit` [CORR swing>day] · Expectancy $p\cdot\mathrm{AvgWin}-(1-p)\cdot\mathrm{AvgLoss}$,
sign>0 [L] · Payoff $\mathrm{AvgWin}/\mathrm{AvgLoss}$ [L] · PF-identity $\mathrm{PF}=(p\cdot R)/(1-p)$ [L] ·
Breakeven-win $1/(1+R)$ [L] · MAE/MFE [C] · Trade-eff $\mathrm{ActualGain}/\mathrm{MFE}$ [C] · Turnover
$\sum\lvert\Delta\mathrm{position}\rvert$ [C].
**Tail/distribution** — Hist-VaR `−q_{1−α}(r)`, Basel traffic-light [L] · Param-VaR
`−(μ+σ·z)`, z 1.645/2.326 [L] · Hist-ES `mean(worst (1−α)N)` [L] · Tail-Ratio
`q95/|q05|` [L] · Skew `mean((r−μ)³)/σ³` [C] · Excess-kurtosis `mean((r−μ)⁴)/σ⁴−3` [L]
· Downside-dev `sqrt(Σ min(0,r−T)²/N)` [L] · Risk-of-ruin `((1−A)/(1+A))^U` or
empirical, `<1%` pro [C] · Kelly `(μ−r)/σ²`, `≤0.5·f*` [L].
**Tail/distribution** — Hist-VaR $-q_{1-\alpha}(r)$, Basel traffic-light [L] · Param-VaR
$-(\mu+\sigma\cdot z)$, z 1.645/2.326 [L] · Hist-ES $\operatorname{mean}(\text{worst }(1-\alpha)N)$ [L] · Tail-Ratio
$q95/\lvert q05\rvert$ [L] · Skew $\operatorname{mean}((r-\mu)^3)/\sigma^3$ [C] · Excess-kurtosis $\operatorname{mean}((r-\mu)^4)/\sigma^4-3$ [L]
· Downside-dev $\sqrt{\sum\min(0,r-T)^2/N}$ [L] · Risk-of-ruin $((1-A)/(1+A))^U$ or
empirical, `<1%` pro [C] · Kelly $(\mu-r)/\sigma^2$, `≤0.5·f*` [L].
**Overfitting** — PSR `Φ[(SR−SR*)sqrt(T−1)/sqrt(1−g3·SR+((g4−1)/4)SR²)]`, `>0.95` [L]
· MinTRL [L] · DSR = PSR at `SR*=E[maxSR_N]`, `>0.95` [L] · MinBTL `≤2ln(N)/E[max]²`
[C/L] · PBO `P[ln(ω/(1−ω))≤0]`, `<0.5` sanity / `<0.1` excellent [L] · HLZ-haircut
`HSR=SR·(adj_t/orig_t)`, t`~3.0` [C/L] · CPCV `φ=(k/N)C(N,k)`, embargo h~1% [L].
**Overfitting** — PSR $\Phi[(\mathrm{SR}-\mathrm{SR}^*)\sqrt{T-1}/\sqrt{1-g_3\cdot\mathrm{SR}+((g_4-1)/4)\mathrm{SR}^2}]$, `>0.95` [L]
· MinTRL [L] · DSR = PSR at $\mathrm{SR}^*=\operatorname{E}[\max \mathrm{SR}_N]$, `>0.95` [L] · MinBTL $\le 2\ln(N)/\operatorname{E}[\max]^2$
[C/L] · PBO $P[\ln(\omega/(1-\omega))\le 0]$, `<0.5` sanity / `<0.1` excellent [L] · HLZ-haircut
$\mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t})$, t`~3.0` [C/L] · CPCV $\varphi=(k/N)C(N,k)$, embargo h~1% [L].
---