diff --git a/Metric-Catalogue.md b/Metric-Catalogue.md index b683ed4..0a5891a 100644 --- a/Metric-Catalogue.md +++ b/Metric-Catalogue.md @@ -23,7 +23,7 @@ Robustness is read from the *geometry* of that surface, not the single best cell | metric | formula | computed from | threshold | |---|---|---|---| -| Plateau-to-peak ratio | `R = mean_{p∈Nbhd(p*)} f(p) / f(p*)` | metric array over a k-step neighbourhood of the argmax | `R ≳ 0.8–0.9` robust plateau; `< ~0.7` lucky spike **[C]** | +| Plateau-to-peak ratio | $R = \operatorname{mean}_{p\in\mathrm{Nbhd}(p^*)} f(p) / f(p^*)$ | metric array over a k-step neighbourhood of the argmax | `R ≳ 0.8–0.9` robust plateau; `< ~0.7` lucky spike **[C]** | | Parameter sensitivity | `max(f over ±1-step) − min(...)` (or std) | perturb each axis one grid step around `p*` | small spread = robust; edge should survive ±10–20% param change **[C]** | | Surface smoothness | `mean|f(adjacent) − f|` (mean abs gradient) | adjacent-cell differences | lower = smoother; global-best `> ~20–30%` above neighbours = spike | @@ -39,10 +39,10 @@ significance of the win (see §J). | metric (as objective) | formula | computed from | threshold | |---|---|---|---| -| Sharpe | `sqrt(P)·mean(r)/std(r)` | `r` | §C Sharpe | -| Calmar / CAR-over-MDD | `mean(r)·P / MaxDD` | `r`, `E` | §D Calmar | -| Ulcer Performance Index | `(R_p − R_f) / UlcerIndex` | `r`, `E` | more overfit-resistant than Calmar | -| Profit factor | `gross_profit / gross_loss` | trade list | §E | +| Sharpe | $\sqrt{P}\cdot\operatorname{mean}(r)/\operatorname{std}(r)$ | `r` | §C Sharpe | +| Calmar / CAR-over-MDD | $\operatorname{mean}(r)\cdot P / \mathrm{MaxDD}$ | `r`, `E` | §D Calmar | +| Ulcer Performance Index | $(R_p - R_f) / \mathrm{UlcerIndex}$ | `r`, `E` | more overfit-resistant than Calmar | +| Profit factor | $\mathrm{gross\_profit} / \mathrm{gross\_loss}$ | trade list | §E | **Caveat:** ratios with small denominators (low-variance / few-trade cells) explode → spurious winners; enforce a minimum trade count / minimum exposure. @@ -81,17 +81,17 @@ trade list required. | metric | formula | computed from | threshold | |---|---|---|---| -| Max drawdown | `max_t(HWM−E)` (level) or `/HWM` (%) | `E` | stat-arb/MN `<10–15%`, HF `<15–20%`, L/S unacceptable `>25–30%`, `>40%` very high **[C; no universal cutoff]** | -| Recovery asymmetry | DD fraction `d` needs gain `d/(1−d)` | a DD depth | exact: −20%→+25%, −50%→+100% **[L, algebraic]** | -| Average drawdown | `(1/n)Σ|DD|` (=Pain Index) OR episode-mean of trough depths | `E`, episode segmentation | comparative; `AvgDD/MaxDD≈1` → worst typical | +| Max drawdown | $\max_t(\mathrm{HWM}-E)$ (level) or $/\mathrm{HWM}$ (%) | `E` | stat-arb/MN `<10–15%`, HF `<15–20%`, L/S unacceptable `>25–30%`, `>40%` very high **[C; no universal cutoff]** | +| Recovery asymmetry | DD fraction `d` needs gain $d/(1-d)$ | a DD depth | exact: −20%→+25%, −50%→+100% **[L, algebraic]** | +| Average drawdown | $(1/n)\sum\lvert\mathrm{DD}\rvert$ (=Pain Index) OR episode-mean of trough depths | `E`, episode segmentation | comparative; `AvgDD/MaxDD≈1` → worst typical | | Duration / time-to-recovery / time-underwater | per-episode index arithmetic; total UW = #(DD<0)/n | `E` + time index | heuristic recovery ≈ `MDD/per-period-return`; flag censored open DD **[C]** | -| Longest drawdown | `max_k(t_recover − t_peak)` | episode segmentation | shorter better; often a *different* episode than max-depth | -| Ulcer Index | `sqrt((Σ_i D'_i²)/n)`, `D'_i = 100·(E_i−max_so_far)/max_so_far` | `E` | `=0` no DD; `> ~10` severe; comparative **[C]** | -| Ulcer Perf Index / Martin | `(R_p−R_f)/UlcerIndex` | `r`, `E` | higher better; no band; more overfit-robust objective **[L]** | -| Pain Index | `(Σ_i |D'_i|)/n`, n = *total* obs | `E` | comparative; `< MaxDD`. **[CORR:** it EMBEDS duration — a *remedy*, NOT a metric that "ignores time"; Bacon 2008 p.89; n=total obs else it collapses to Average Drawdown**]** | -| Pain Ratio | `(R_p−R_f)/PainIndex` | `r`, `E` | higher better; Bacon 2008 p.91 **[CORR page]** | -| Recovery Factor | `NetProfit/|MaxDrawdown|` | final `E` / MaxDD | poor `<1`, good `2–5`, excellent `≥5` **[formula L; thresholds C]** | -| Expected MaxDD (Magdon-Ismail) | Brownian: linear (μ<0), `~sqrt(T)` (μ=0, exact `sqrt(π/2)·σ·sqrt(T)`), log (μ>0) | μ,σ from `r`; classify regime | not graded — a yardstick; observed ≫ E[MaxDD] = regime break / fat tails **[L asymptotics]** | +| Longest drawdown | $\max_k(t_{\mathrm{recover}}-t_{\mathrm{peak}})$ | episode segmentation | shorter better; often a *different* episode than max-depth | +| Ulcer Index | $\sqrt{(\sum_i {D'_i}^2)/n}$, $D'_i=100\cdot(E_i-\mathrm{max\_so\_far})/\mathrm{max\_so\_far}$ | `E` | `=0` no DD; `> ~10` severe; comparative **[C]** | +| Ulcer Perf Index / Martin | $(R_p-R_f)/\mathrm{UlcerIndex}$ | `r`, `E` | higher better; no band; more overfit-robust objective **[L]** | +| Pain Index | $(\sum_i\lvert D'_i\rvert)/n$, n = *total* obs | `E` | comparative; `< MaxDD`. **[CORR:** it EMBEDS duration — a *remedy*, NOT a metric that "ignores time"; Bacon 2008 p.89; n=total obs else it collapses to Average Drawdown**]** | +| Pain Ratio | $(R_p-R_f)/\mathrm{PainIndex}$ | `r`, `E` | higher better; Bacon 2008 p.91 **[CORR page]** | +| Recovery Factor | $\mathrm{NetProfit}/\lvert\mathrm{MaxDrawdown}\rvert$ | final `E` / MaxDD | poor `<1`, good `2–5`, excellent `≥5` **[formula L; thresholds C]** | +| Expected MaxDD (Magdon-Ismail) | Brownian: linear (μ<0), $\sim\sqrt{T}$ (μ=0, exact $\sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T}$), log (μ>0) | μ,σ from `r`; classify regime | not graded — a yardstick; observed ≫ E[MaxDD] = regime break / fat tails **[L asymptotics]** | **Load-bearing point:** MaxDD is a downward-biased, sample-length-dependent point estimate — the cure is Monte-Carlo (report the *distribution* of MaxDD, not one @@ -106,17 +106,17 @@ its close; P&L = `Σ r_t` over the span; direction = sign of the held position. | metric | formula | computed from | threshold | |---|---|---|---| -| Win rate p | `wins/total_trades` | trades | not gradable alone; vs breakeven `1/(1+R)` | -| **Profit factor** | `gross_profit/gross_loss` | trades | `<1` losing, `1.0–1.2` breakeven, `1.2–1.5` realistic, `1.5–2.0` good, `2.0–3.0` exceptional-but-scrutinize, `>3` overfit flag. **[CORR:** swing target HIGHER than day-trade; ~200+ trades needed; live degrades PF ~10–20%**]** | -| Expectancy | `(p·AvgWin) − ((1−p)·AvgLoss)` | trades | only hard threshold is sign `>0` | -| Payoff ratio R | `AvgWin/AvgLoss` | trades | good = exceeds breakeven `(1−p)/p` | -| **PF↔win-rate identity** | `PF=(p·R)/(1−p)`, inverse `p=PF/(PF+R)` | p, R | exact identity (holds iff R = ratio of *averages*) **[L]** | -| Breakeven win rate | `1/(1+R)` | R | 50%@1:1, 33.3%@1:2, 25%@1:3, 16.7%@1:5 **[L]** | -| SQN (Van Tharp) | `sqrt(N)·mean(R-mult)/std(R-mult)`, N often capped 100 | per-trade R-multiples | `<1.6` below, `1.6–2.0` avg, `2.0–2.5` good, `2.5–3.0` excellent, `>7` overfit **[C]** | +| Win rate p | $\mathrm{wins}/\mathrm{total\_trades}$ | trades | not gradable alone; vs breakeven `1/(1+R)` | +| **Profit factor** | $\mathrm{gross\_profit}/\mathrm{gross\_loss}$ | trades | `<1` losing, `1.0–1.2` breakeven, `1.2–1.5` realistic, `1.5–2.0` good, `2.0–3.0` exceptional-but-scrutinize, `>3` overfit flag. **[CORR:** swing target HIGHER than day-trade; ~200+ trades needed; live degrades PF ~10–20%**]** | +| Expectancy | $(p\cdot\mathrm{AvgWin})-((1-p)\cdot\mathrm{AvgLoss})$ | trades | only hard threshold is sign `>0` | +| Payoff ratio R | $\mathrm{AvgWin}/\mathrm{AvgLoss}$ | trades | good = exceeds breakeven `(1−p)/p` | +| **PF↔win-rate identity** | $\mathrm{PF}=(p\cdot R)/(1-p)$, inverse $p=\mathrm{PF}/(\mathrm{PF}+R)$ | p, R | exact identity (holds iff R = ratio of *averages*) **[L]** | +| Breakeven win rate | $1/(1+R)$ | R | 50%@1:1, 33.3%@1:2, 25%@1:3, 16.7%@1:5 **[L]** | +| SQN (Van Tharp) | $\sqrt{N}\cdot\operatorname{mean}(\text{R-mult})/\operatorname{std}(\text{R-mult})$, N often capped 100 | per-trade R-multiples | `<1.6` below, `1.6–2.0` avg, `2.0–2.5` good, `2.5–3.0` excellent, `>7` overfit **[C]** | | MAE / MFE | max adverse / favorable excursion = `−min`/`max(running unreal. PnL)` | per-bar running P&L since entry | no cutoff; winners cluster at small MAE; large MFE-vs-realized gap → exiting early | -| Trade efficiency | exit `ActualGain/MFE`; range `(Exit−Entry)/(High−Low)` | per-trade MFE/MAE/range | 0–100%; `<50%` exit eff poor **[C]** | -| Turnover | `Σ|Δposition|` (or SEC `min(buys,sells)/avg_assets`) | position series | style-dependent; net-of-cost expectancy must stay `>0` | -| R-multiple / expectancy-in-R | per trade `PnL/1R`; mean `p·R−(1−p)` | trades + defined 1R | sign `>0`; ~`+0.5R` strong **[C]** — needs a defined risk/stop | +| Trade efficiency | exit $\mathrm{ActualGain}/\mathrm{MFE}$; range $(\text{Exit}-\text{Entry})/(\text{High}-\text{Low})$ | per-trade MFE/MAE/range | 0–100%; `<50%` exit eff poor **[C]** | +| Turnover | $\sum\lvert\Delta\mathrm{position}\rvert$ (or SEC $\min(\mathrm{buys},\mathrm{sells})/\mathrm{avg\_assets}$) | position series | style-dependent; net-of-cost expectancy must stay `>0` | +| R-multiple / expectancy-in-R | per trade $\mathrm{PnL}/1R$; mean $p\cdot R-(1-p)$ | trades + defined 1R | sign `>0`; ~`+0.5R` strong **[C]** — needs a defined risk/stop | ## F. Tail, distribution & risk-of-ruin @@ -126,30 +126,30 @@ empirical / Monte-Carlo estimates. | metric | formula | computed from | threshold | |---|---|---|---| -| Historical VaR_α | `−(1−α)` empirical quantile of r (5th @95%, 1st @99%) | sort `r`, index `floor((1−α)N)` | Basel traffic-light @250d/99%: GREEN 0–4, YELLOW 5–9, RED 10+ exceptions **[L]** | -| Parametric VaR | `−(μ+σ·Φ⁻¹(1−α))·W`; z = 1.645@95, 2.326@99, 1.282@90 | μ,σ of `r` | same grading; **iid-normal trap** **[L multipliers]** | +| Historical VaR_α | $-(1-\alpha)$ empirical quantile of r (5th @95%, 1st @99%) | sort `r`, index `floor((1−α)N)` | Basel traffic-light @250d/99%: GREEN 0–4, YELLOW 5–9, RED 10+ exceptions **[L]** | +| Parametric VaR | $-(\mu+\sigma\cdot\Phi^{-1}(1-\alpha))\cdot W$; z = 1.645@95, 2.326@99, 1.282@90 | μ,σ of `r` | same grading; **iid-normal trap** **[L multipliers]** | | Historical ES/CVaR | `mean of worst (1−α)N returns` | sorted `r`, `k=ceil((1−α)N)` | coherent (VaR is not); FRTB uses 97.5% ES ≈ 99% VaR under normality **[L]** | -| Parametric ES | `μ+σ·φ(Φ⁻¹(α))/(1−α)`; mult ~2.063@95, 2.665@99 | μ,σ | worse than VaR for fat tails **[L]** | -| Tail ratio | `q95(r)/|q05(r)|` | percentiles of `r` | `>1` favourable, `<1` dangerous **[L 95/5]** | -| Skewness | `(1/N)Σ(r−μ)³/σ³` | `r` | sign-driven; persistent negative = red flag even with high Sharpe | -| Excess kurtosis | `(1/N)Σ(r−μ)⁴/σ⁴ − 3` | `r` | `0`=normal; `>0` leptokurtic (typical, dangerous); daily 3–10+ **[L]** | -| Downside deviation | `sqrt((1/N)Σ[min(0,r−T)]²)`, divisor total N | `r`, T | feeds Sortino **[L, divisor=total N]** | -| Risk of ruin (closed) | `((1−A)/(1+A))^U`, `A=p·R−q`, `U=capital/risk-per-trade` | trades p, R | `<1–5%`, desks `<1%`, pros `<0.1%`; `>5–10%` reject **[C]** | +| Parametric ES | $\mu+\sigma\cdot\varphi(\Phi^{-1}(\alpha))/(1-\alpha)$; mult ~2.063@95, 2.665@99 | μ,σ | worse than VaR for fat tails **[L]** | +| Tail ratio | $q95(r)/\lvert q05(r)\rvert$ | percentiles of `r` | `>1` favourable, `<1` dangerous **[L 95/5]** | +| Skewness | $(1/N)\sum(r-\mu)^3/\sigma^3$ | `r` | sign-driven; persistent negative = red flag even with high Sharpe | +| Excess kurtosis | $(1/N)\sum(r-\mu)^4/\sigma^4 - 3$ | `r` | `0`=normal; `>0` leptokurtic (typical, dangerous); daily 3–10+ **[L]** | +| Downside deviation | $\sqrt{(1/N)\sum[\min(0,r-T)]^2}$, divisor total N | `r`, T | feeds Sortino **[L, divisor=total N]** | +| Risk of ruin (closed) | $((1-A)/(1+A))^U$, $A=p\cdot R-q$, $U=\text{capital}/\text{risk-per-trade}$ | trades p, R | `<1–5%`, desks `<1%`, pros `<0.1%`; `>5–10%` reject **[C]** | | Risk of ruin (empirical) | `fraction of N Monte-Carlo paths breaching the ruin threshold` | multi-run dist. | **preferred**; sidesteps iid-normal; default R=50% capital | -| Kelly / optimal-f | `f*=(b·p−q)/b`; `f*=(μ−r)/σ²` | `r`/odds | deploy `0 < f ≤ ~0.5·f*`; above full Kelly strictly dominated **[L]** | -| Gain-to-Pain | `Σ(r)/|Σ(neg r)|` | `r` | (monthly) ~1 acc, ~2 outstanding, 3 excellent, ~4 world-class, `<1` poor **[C; state the basis]** | -| Common-Sense ratio | `TailRatio · GainToPain` | the two above | `>1` robust, `<1` fragile **[L breakpoint]** | +| Kelly / optimal-f | $f^*=(b\cdot p-q)/b$; $f^*=(\mu-r)/\sigma^2$ | `r`/odds | deploy `0 < f ≤ ~0.5·f*`; above full Kelly strictly dominated **[L]** | +| Gain-to-Pain | $\sum(r)/\lvert\sum(\text{neg } r)\rvert$ | `r` | (monthly) ~1 acc, ~2 outstanding, 3 excellent, ~4 world-class, `<1` poor **[C; state the basis]** | +| Common-Sense ratio | $\mathrm{TailRatio}\cdot\mathrm{GainToPain}$ | the two above | `>1` robust, `<1` fragile **[L breakpoint]** | ## G. Walk-forward analysis | metric | formula | computed from | threshold | |---|---|---|---| -| WFE (return) | `(OOS_profit/OOS_bars)/(IS_profit/IS_bars)` | per-fold IS-optimal/OOS profit, bar counts | `≥0.5` robust, `~1.0` ideal, `>0.7` excellent, `0.5–0.7` good, `0.3–0.5` mediocre, `<0.3` poor; `>1.0` is a *warning* (under-fit IS / lucky OOS), not superiority | -| WFE (Sharpe) | `OOS_Sharpe/IS_Sharpe` | per-fold Sharpe over `r` | `>0.5` robust, `~1.0` ideal, `<0.3` red flag | -| IS-OOS gap | `IS_Sharpe − OOS_Sharpe` per fold, averaged | per-fold Sharpes | `>0.5`/cycle = overfitting; healthy mean `<0.3` | +| WFE (return) | $(\mathrm{OOS\_profit}/\mathrm{OOS\_bars})/(\mathrm{IS\_profit}/\mathrm{IS\_bars})$ | per-fold IS-optimal/OOS profit, bar counts | `≥0.5` robust, `~1.0` ideal, `>0.7` excellent, `0.5–0.7` good, `0.3–0.5` mediocre, `<0.3` poor; `>1.0` is a *warning* (under-fit IS / lucky OOS), not superiority | +| WFE (Sharpe) | $\mathrm{OOS\_Sharpe}/\mathrm{IS\_Sharpe}$ | per-fold Sharpe over `r` | `>0.5` robust, `~1.0` ideal, `<0.3` red flag | +| IS-OOS gap | $\mathrm{IS\_Sharpe}-\mathrm{OOS\_Sharpe}$ per fold, averaged | per-fold Sharpes | `>0.5`/cycle = overfitting; healthy mean `<0.3` | | Stitched OOS curve | chain each fold's OOS segment, re-based | family of OOS runs | net profitable; max DD `<40%` (TradeStation) | | Parameter stability | dispersion of argmax params across folds; plateau check | per-fold winners | qualitative — broad plateau, clustered | -| Fraction profitable OOS windows | `#(OOS>0)/#windows` | per-window OOS P&L | majority `>50%`; worst-window OOS Sharpe `>0` | +| Fraction profitable OOS windows | $\#(\mathrm{OOS}>0)/\#\text{windows}$ | per-window OOS P&L | majority `>50%`; worst-window OOS Sharpe `>0` | | Pardo verdict | net-profitable AND WFE `≥50%` AND maxDD `<40%`; filter %Prof `≥40`, PF `≥1.5` | the above | composite pass/fail **[software-default convention]** | **Sizing:** IS:OOS 2×–5×; `≥6` windows (10+ preferred); non-overlapping OOS for a @@ -166,9 +166,9 @@ schedule. | Equity percentile fan | per-horizon {5,25,50,75,95} pct | N curves | descriptive; viability: 5th-pct ending equity `>0` | | Stress (95th-pct) MaxDD | 95th pct of N MaxDD | per-run MaxDD | `<25%` good, `>35%` red flag; size capital off this **[C]** | | Empirical VaR/ES | 5th-pct (1st @99) final P&L; ES = mean beyond | per-run terminal | 5th-pct ending equity `>0` | -| Probability of ruin | `ruin_count/N` (MaxDD>R, default 50%) | per-run MaxDD | `<1%` excellent, `1–5%` acceptable, `>5%` not recommended **[C]** | -| Prob. of a losing period | `#(period<0)/N` | per-run period returns | `>~30–40%` hard to hold | -| **MCPT p-value** | `(z+1)/(N+1)`, z=#perm `≥` real | permute bar-to-bar log returns, keep first price, re-run | `<0.05` min, `<0.01` strict, `>0.10` red flag, `~0.5` discard **[L: +1 in numerator and denominator]** | +| Probability of ruin | $\mathrm{ruin\_count}/N$ (MaxDD>R, default 50%) | per-run MaxDD | `<1%` excellent, `1–5%` acceptable, `>5%` not recommended **[C]** | +| Prob. of a losing period | $\#(\text{period}<0)/N$ | per-run period returns | `>~30–40%` hard to hold | +| **MCPT p-value** | $(z+1)/(N+1)$, z=#perm `≥` real | permute bar-to-bar log returns, keep first price, re-run | `<0.05` min, `<0.01` strict, `>0.10` red flag, `~0.5` discard **[L: +1 in numerator and denominator]** | | White RC / Hansen SPA | bootstrap max-stat over L rules, recentered | per-rule rel-perf + benchmark | `p<0.05`; SPA more powerful than the conservative RC | **Conventions:** N = 1,000 min, 5,000 recommended, 10,000+ for tail stability. @@ -183,11 +183,11 @@ time-ordering edges. |---|---|---|---|---| | Stress/scenario | conditional worst-case | per-scenario MaxDD, profit, recover-time | strategy over crisis windows / shocked synthetic data | fail if stress DD > risk limit; excellent if net-positive through crises **[C; Fed CCAR]** | | Regime | per-state metrics + fragility verdict | profit/Sharpe/DD masked by regime label | a regime-label series (vol/trend classifier or HMM) | edge only in one regime + negative elsewhere = fragile **[qualitative]** | -| Cost/slippage | net-vs-cost curve + break-even | `c* where net=0 ≈ gross_profit/n_trades`; `d(net Sharpe)/d(cost)` | a cost-modelling backtest, cost swept | reject if edge survives only at zero cost; break-even ≳2× real cost **[C]** | -| Capacity/impact | AUM ceiling + turnover | `impact ≈ Y·σ·sqrt(Q/V)`, Y~O(1); capacity = AUM where impact = gross alpha | order volumes + ADV | capacity ≫ target = scalable; per-day order `< ~5–10%` ADV **[C]** | -| Benchmark-relative | alpha/beta/IR + significance | `beta=Cov/Var`; Jensen alpha = regression intercept; `IR=alpha/TE` | `r` regressed on a market series `R_m` | alpha positive + significant (`|t|>2`); IR 0.4/0.6/1.0 **[beta/CAPM L; IR C]** | -| Null/sanity | "beat random?" | MCPT `p=(#perm≥real)/N`; random-entry: real in right tail | permuted input + a random-entry null strategy | `p<0.05` min, `≤0.01` strict, `>0.10` red flag, N`≥1000` **[L]** | -| Live-vs-backtest | overfitting/decay verdict | haircut `=(BT_Sharpe−live_Sharpe)/BT_Sharpe` | live series vs backtest prediction | `<30%` acceptable, `30–50%` typical, `>50–70%` largely overfit **[C; verify primary source]** | +| Cost/slippage | net-vs-cost curve + break-even | $c^*\text{ where net}=0\approx\mathrm{gross\_profit}/n_\text{trades}$; $d(\text{net Sharpe})/d(\text{cost})$ | a cost-modelling backtest, cost swept | reject if edge survives only at zero cost; break-even ≳2× real cost **[C]** | +| Capacity/impact | AUM ceiling + turnover | $\text{impact}\approx Y\cdot\sigma\cdot\sqrt{Q/V}$, Y~O(1); capacity = AUM where impact = gross alpha | order volumes + ADV | capacity ≫ target = scalable; per-day order `< ~5–10%` ADV **[C]** | +| Benchmark-relative | alpha/beta/IR + significance | $\beta=\operatorname{Cov}/\operatorname{Var}$; Jensen alpha = regression intercept; $\mathrm{IR}=\text{alpha}/\mathrm{TE}$ | `r` regressed on a market series `R_m` | alpha positive + significant (`|t|>2`); IR 0.4/0.6/1.0 **[beta/CAPM L; IR C]** | +| Null/sanity | "beat random?" | MCPT $p=(\#\text{perm}\ge\text{real})/N$; random-entry: real in right tail | permuted input + a random-entry null strategy | `p<0.05` min, `≤0.01` strict, `>0.10` red flag, N`≥1000` **[L]** | +| Live-vs-backtest | overfitting/decay verdict | haircut $=(\mathrm{BT\_Sharpe}-\mathrm{live\_Sharpe})/\mathrm{BT\_Sharpe}$ | live series vs backtest prediction | `<30%` acceptable, `30–50%` typical, `>50–70%` largely overfit **[C; verify primary source]** | ## J. Overfitting, multiple-testing & CPCV @@ -196,14 +196,14 @@ kept the best" as the statistical problem it is. | metric | formula | computed from | threshold | |---|---|---|---| -| PSR(SR*) | `Φ[(SR_hat−SR*)·sqrt(T−1)/sqrt(1−g3·SR_hat+((g4−1)/4)·SR_hat²)]` | per-period `SR_hat`, T, g3=skew, g4=kurtosis (full, normal=3) of `r` | `>0.95` significant **[L]** | -| MinTRL | `1+(1−g3·SR_hat+((g4−1)/4)·SR_hat²)·(Z_α/(SR_hat−SR*))²` | as PSR + target α | need T `≥` MinTRL **[L]** | -| E[max SR_N] | `sqrt(V)·((1−γ)Φ⁻¹(1−1/N)+γΦ⁻¹(1−1/(Ne)))`, γ≈0.5772 | cross-trial Sharpe variance V, effective N | the moving hurdle; grows with N **[L]** | -| **DSR** | PSR at `SR*=SR_0=E[max SR_N]` | winner SR_hat/T/g3/g4 + V across trials + effective N | `>0.95` survives deflation **[L; ex: N=100,V=½,T=1250,g3=−3,g4=10 → DSR~0.90<0.95 REJECT]** | -| MinBTL | `≤ 2·ln(N)/E[max]²` years | N, target SR | 5y + `>~45` independent trials → guaranteed overfit **[C/L]** | -| **PBO (CSCV)** | matrix `(T×M)`; S even blocks; `C(S,S/2)` combos; `ω=(rank−0.5)/M`; `λ=ln(ω/(1−ω))`; `PBO=P[λ≤0]` | per-trial `r` series | `<0.5` sanity, `~0.5` alarm, `<0.1` excellent **[L]** | -| Harvey-Liu-Zhu haircut | `t=SR·sqrt(T)`; adjust p (Bonferroni `min(M·p,1)`/Holm/BHY); `HSR=SR·(adj_t/orig_t)` | SR, T, M | required new-factor t `~3.0` (not 2.0) | -| CPCV paths | splits `=C(N,k)`; paths `φ=(k/N)C(N,k)` | grouped purged+embargoed windows | N=5–10 groups, k=2; purge: drop train obs whose label span overlaps test; embargo h~1% of T (forward only) **[L]** | +| PSR(SR*) | $\Phi[(\mathrm{SR}_{hat}-\mathrm{SR}^*)\cdot\sqrt{T-1}/\sqrt{1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2}]$ | per-period `SR_hat`, T, g3=skew, g4=kurtosis (full, normal=3) of `r` | `>0.95` significant **[L]** | +| MinTRL | $1+(1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2)\cdot(Z_\alpha/(\mathrm{SR}_{hat}-\mathrm{SR}^*))^2$ | as PSR + target α | need T `≥` MinTRL **[L]** | +| E[max SR_N] | $\sqrt{V}\cdot((1-\gamma)\Phi^{-1}(1-1/N)+\gamma\Phi^{-1}(1-1/(Ne)))$, $\gamma\approx0.5772$ | cross-trial Sharpe variance V, effective N | the moving hurdle; grows with N **[L]** | +| **DSR** | PSR at $\mathrm{SR}^*=\mathrm{SR}_0=E[\max \mathrm{SR}_N]$ | winner SR_hat/T/g3/g4 + V across trials + effective N | `>0.95` survives deflation **[L; ex: N=100,V=½,T=1250,g3=−3,g4=10 → DSR~0.90<0.95 REJECT]** | +| MinBTL | $\le 2\cdot\ln(N)/E[\max]^2$ years | N, target SR | 5y + `>~45` independent trials → guaranteed overfit **[C/L]** | +| **PBO (CSCV)** | matrix $(T\times M)$; S even blocks; $C(S,S/2)$ combos; $\omega=(\mathrm{rank}-0.5)/M$; $\lambda=\ln(\omega/(1-\omega))$; $\mathrm{PBO}=P[\lambda\le0]$ | per-trial `r` series | `<0.5` sanity, `~0.5` alarm, `<0.1` excellent **[L]** | +| Harvey-Liu-Zhu haircut | $t=\mathrm{SR}\cdot\sqrt{T}$; adjust p (Bonferroni $\min(M\cdot p,1)$ / Holm / BHY); $\mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t})$ | SR, T, M | required new-factor t `~3.0` (not 2.0) | +| CPCV paths | splits $=C(N,k)$; paths $\varphi=(k/N)C(N,k)$ | grouped purged+embargoed windows | N=5–10 groups, k=2; purge: drop train obs whose label span overlaps test; embargo h~1% of T (forward only) **[L]** | --- @@ -212,37 +212,37 @@ kept the best" as the statistical problem it is. Inputs: **r** = per-period returns; **E** = equity curve; **trades** = closed trades; **b** = benchmark; **multi** = multi-run distribution. -**Return** — Total `sum r` [C] · CAGR `mean(r)·P` [C] · Volatility `std(r)·sqrt(P)` [C]. +**Return** — Total $\sum r$ [C] · CAGR $\operatorname{mean}(r)\cdot P$ [C] · Volatility $\operatorname{std}(r)\cdot\sqrt{P}$ [C]. -**Risk-adjusted** — Sharpe `sqrt(P)·(mean(r)−r_f)/std(r−r_f)` [CORR denom] · Sortino -`…/sqrt(Σ min(0,r−MAR)²/N)`, good `>1` [CORR] · Calmar `mean(r)·P/|MaxDD|`, -`<1`/`1–3`/`>3` [CORR] · MAR `CAGR_inc/|MaxDD_inc|` [CORR window+numerator] · Sterling -`CompoundROR/|avgAnnMaxDD−10%|` [CORR] · Burke `(r_P−r_F)/sqrt(ΣD²)` [L] · Omega -`Σmax(0,r−θ)/Σmax(0,θ−r)` [L] · Info-Ratio `mean(r−b)/std(r−b)·sqrt(P)` [L/C] · UPI -`(R_p−R_f)/UlcerIndex` [L] · Pain-Ratio `(R_p−R_f)/PainIndex` [CORR page] · -Recovery-Factor `NetProfit/|MaxDD|` [L/C] · Gain-to-Pain `Σr/|Σ neg r|` [C] · -Common-Sense `TailRatio·GainToPain` [L] · SQN `sqrt(N)·mean(R)/std(R)` [C]. +**Risk-adjusted** — Sharpe $\sqrt{P}\cdot(\operatorname{mean}(r)-r_f)/\operatorname{std}(r-r_f)$ [CORR denom] · Sortino +$\ldots/\sqrt{\sum\min(0,r-\mathrm{MAR})^2/N}$, good `>1` [CORR] · Calmar $\operatorname{mean}(r)\cdot P/\lvert\mathrm{MaxDD}\rvert$, +`<1`/`1–3`/`>3` [CORR] · MAR $\mathrm{CAGR}_{\text{inc}}/\lvert\mathrm{MaxDD}_{\text{inc}}\rvert$ [CORR window+numerator] · Sterling +$\mathrm{CompoundROR}/\lvert\mathrm{avgAnnMaxDD}-10\%\rvert$ [CORR] · Burke $(r_P-r_F)/\sqrt{\sum D^2}$ [L] · Omega +$\sum\max(0,r-\theta)/\sum\max(0,\theta-r)$ [L] · Info-Ratio $\operatorname{mean}(r-b)/\operatorname{std}(r-b)\cdot\sqrt{P}$ [L/C] · UPI +$(R_p-R_f)/\mathrm{UlcerIndex}$ [L] · Pain-Ratio $(R_p-R_f)/\mathrm{PainIndex}$ [CORR page] · +Recovery-Factor $\mathrm{NetProfit}/\lvert\mathrm{MaxDD}\rvert$ [L/C] · Gain-to-Pain $\sum r/\lvert\sum\text{neg }r\rvert$ [C] · +Common-Sense $\mathrm{TailRatio}\cdot\mathrm{GainToPain}$ [L] · SQN $\sqrt{N}\cdot\operatorname{mean}(R)/\operatorname{std}(R)$ [C]. -**Drawdown** — MaxDD `max(HWM−E)` [C] · Recovery-asymmetry `d/(1−d)` [L] · AvgDD -`mean|DD|` [C] · Ulcer `sqrt(ΣD'²/n)` [C] · Pain `Σ|D'|/n` (n=total) [CORR embeds -duration] · Expected-MaxDD μ=0 `sqrt(π/2)·σ·sqrt(T)` [L]. +**Drawdown** — MaxDD $\max(\mathrm{HWM}-E)$ [C] · Recovery-asymmetry $d/(1-d)$ [L] · AvgDD +$\operatorname{mean}\lvert\mathrm{DD}\rvert$ [C] · Ulcer $\sqrt{\sum D^{\prime 2}/n}$ [C] · Pain $\sum\lvert D^\prime\rvert/n$ (n=total) [CORR embeds +duration] · Expected-MaxDD μ=0 $\sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T}$ [L]. -**Trade-level** — Win-rate `wins/total` [C] · Profit-Factor `gross_profit/gross_loss`, -`<1`/`1.5–2`/`>3 overfit` [CORR swing>day] · Expectancy `p·AvgWin−(1−p)·AvgLoss`, -sign>0 [L] · Payoff `AvgWin/AvgLoss` [L] · PF-identity `PF=(p·R)/(1−p)` [L] · -Breakeven-win `1/(1+R)` [L] · MAE/MFE [C] · Trade-eff `ActualGain/MFE` [C] · Turnover -`Σ|Δposition|` [C]. +**Trade-level** — Win-rate $\mathrm{wins}/\mathrm{total}$ [C] · Profit-Factor $\mathrm{gross\_profit}/\mathrm{gross\_loss}$, +`<1`/`1.5–2`/`>3 overfit` [CORR swing>day] · Expectancy $p\cdot\mathrm{AvgWin}-(1-p)\cdot\mathrm{AvgLoss}$, +sign>0 [L] · Payoff $\mathrm{AvgWin}/\mathrm{AvgLoss}$ [L] · PF-identity $\mathrm{PF}=(p\cdot R)/(1-p)$ [L] · +Breakeven-win $1/(1+R)$ [L] · MAE/MFE [C] · Trade-eff $\mathrm{ActualGain}/\mathrm{MFE}$ [C] · Turnover +$\sum\lvert\Delta\mathrm{position}\rvert$ [C]. -**Tail/distribution** — Hist-VaR `−q_{1−α}(r)`, Basel traffic-light [L] · Param-VaR -`−(μ+σ·z)`, z 1.645/2.326 [L] · Hist-ES `mean(worst (1−α)N)` [L] · Tail-Ratio -`q95/|q05|` [L] · Skew `mean((r−μ)³)/σ³` [C] · Excess-kurtosis `mean((r−μ)⁴)/σ⁴−3` [L] -· Downside-dev `sqrt(Σ min(0,r−T)²/N)` [L] · Risk-of-ruin `((1−A)/(1+A))^U` or -empirical, `<1%` pro [C] · Kelly `(μ−r)/σ²`, `≤0.5·f*` [L]. +**Tail/distribution** — Hist-VaR $-q_{1-\alpha}(r)$, Basel traffic-light [L] · Param-VaR +$-(\mu+\sigma\cdot z)$, z 1.645/2.326 [L] · Hist-ES $\operatorname{mean}(\text{worst }(1-\alpha)N)$ [L] · Tail-Ratio +$q95/\lvert q05\rvert$ [L] · Skew $\operatorname{mean}((r-\mu)^3)/\sigma^3$ [C] · Excess-kurtosis $\operatorname{mean}((r-\mu)^4)/\sigma^4-3$ [L] +· Downside-dev $\sqrt{\sum\min(0,r-T)^2/N}$ [L] · Risk-of-ruin $((1-A)/(1+A))^U$ or +empirical, `<1%` pro [C] · Kelly $(\mu-r)/\sigma^2$, `≤0.5·f*` [L]. -**Overfitting** — PSR `Φ[(SR−SR*)sqrt(T−1)/sqrt(1−g3·SR+((g4−1)/4)SR²)]`, `>0.95` [L] -· MinTRL [L] · DSR = PSR at `SR*=E[maxSR_N]`, `>0.95` [L] · MinBTL `≤2ln(N)/E[max]²` -[C/L] · PBO `P[ln(ω/(1−ω))≤0]`, `<0.5` sanity / `<0.1` excellent [L] · HLZ-haircut -`HSR=SR·(adj_t/orig_t)`, t`~3.0` [C/L] · CPCV `φ=(k/N)C(N,k)`, embargo h~1% [L]. +**Overfitting** — PSR $\Phi[(\mathrm{SR}-\mathrm{SR}^*)\sqrt{T-1}/\sqrt{1-g_3\cdot\mathrm{SR}+((g_4-1)/4)\mathrm{SR}^2}]$, `>0.95` [L] +· MinTRL [L] · DSR = PSR at $\mathrm{SR}^*=\operatorname{E}[\max \mathrm{SR}_N]$, `>0.95` [L] · MinBTL $\le 2\ln(N)/\operatorname{E}[\max]^2$ +[C/L] · PBO $P[\ln(\omega/(1-\omega))\le 0]$, `<0.5` sanity / `<0.1` excellent [L] · HLZ-haircut +$\mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t})$, t`~3.0` [C/L] · CPCV $\varphi=(k/N)C(N,k)$, embargo h~1% [L]. ---