Table of Contents
- Metric Catalogue
- A. Parameter sweep: the metric surface
- B. Optimization: the selected configuration
- C. Core performance & risk-adjusted ratios
- D. Drawdown & recovery (equity-curve functionals)
- E. Trade-level & efficiency metrics
- F. Tail, distribution & risk-of-ruin
- G. Walk-forward analysis
- H. Monte-Carlo
- I. Stress/scenario, regime, cost/capacity, benchmark-relative, sanity, live-vs-backtest
- J. Overfitting, multiple-testing & CPCV
- Master catalogue (compact index)
- References
Metric Catalogue
Formula-level reference for trading-strategy analysis. For each analysis tool and metric family: exact formula, what input it is computed from, and the conventional benchmark threshold. Narrative on Strategy Analysis & Validation; source-checking on Metric Verification Log.
Conventions. Per-period return r (r_t = E_t − E_{t-1}) = first
difference of the cumulative equity curve E. Trade list = the set of
closed trades. When returns are expressed as additive P&L increments,
total return = sum(r_t) and CAGR-style numerators use mean(r)·P. P =
periods/year from the data cadence. The risk-free rate r_f is often taken as
0 for frictionless P&L accounting. Markers: [L] law/exact, [C]
convention, [CORR] verification-corrected.
A. Parameter sweep: the metric surface
A sweep produces one metric vector per parameter cell, laid on the parameter grid. Robustness is read from the geometry of that surface, not the single best cell.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Plateau-to-peak ratio | R = \operatorname{mean}_{p\in\mathrm{Nbhd}(p^*)} f(p) / f(p^*) |
metric array over a k-step neighbourhood of the argmax | R ≳ 0.8–0.9 robust plateau; < ~0.7 lucky spike [C] |
| Parameter sensitivity | max(f over ±1-step) − min(...) (or std) |
perturb each axis one grid step around p* |
small spread = robust; edge should survive ±10–20% param change [C] |
| Surface smoothness | `mean | f(adjacent) − f | ` (mean abs gradient) |
Reading rule (load-bearing): pick a parameter value from the centre of the widest plateau, never the global peak.
B. Optimization: the selected configuration
Selection of the single best parameter set under an objective. The result should carry the winner's full scorecard, the objective it won under, and the deflated significance of the win (see §J). Objective choice (load-bearing): do not argmax raw net profit.
| metric (as objective) | formula | computed from | threshold |
|---|---|---|---|
| Sharpe | \sqrt{P}\cdot\operatorname{mean}(r)/\operatorname{std}(r) |
r |
§C Sharpe |
| Calmar / CAR-over-MDD | \operatorname{mean}(r)\cdot P / \mathrm{MaxDD} |
r, E |
§D Calmar |
| Ulcer Performance Index | (R_p - R_f) / \mathrm{UlcerIndex} |
r, E |
more overfit-resistant than Calmar |
| Profit factor | \mathrm{gross\_profit} / \mathrm{gross\_loss} |
trade list | §E |
Caveat: ratios with small denominators (low-variance / few-trade cells) explode → spurious winners; enforce a minimum trade count / minimum exposure. Optimizing any objective hard enough still overfits; optimization shifts the problem to the multiple-testing layer (§J), it does not remove it.
C. Core performance & risk-adjusted ratios
The single-run scorecard. All are pure functions of the per-period return series.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Total return | \sum_t r_t |
r |
none in isolation |
| CAGR / annualized | pip/additive: \operatorname{mean}(r)\cdot P; geometric: (\prod(1+r))^{P/T}-1 |
r, P |
asset-class relative |
| Volatility (ann.) | \operatorname{std}(r)\cdot\sqrt{P}, sample std 1/(T-1) |
r, P |
magnitude; sqrt(P) iid-only |
| Sharpe | \sqrt{P}\cdot\frac{\operatorname{mean}(r)-r_f}{\operatorname{std}(r-r_f)} |
r, r_f, P |
<0 bad, 0–1 acceptable, 1–2 good, 2–3 very good, >3 excellent [C]; sustained >2 short-sample = overfit flag. [CORR: denominator = std of the excess return std(r−r_f); equals std(r) only when r_f constant**]** |
| Sortino | \sqrt{P}\cdot\frac{\operatorname{mean}(r)-\mathrm{MAR}}{DD}, DD=\sqrt{\sum\min(0,r-\mathrm{MAR})^2/N} |
r, MAR (default 0), P |
<0 losing, 0–1 sub-optimal (one band), 1–2 good, 2–3 very good, >3 excellent; "good" = >1. [CORR: no 0.5 split; "good" bar >1 not >2; divisor total N**]** |
| Calmar | \operatorname{mean}(r)\cdot P/\lvert\mathrm{MaxDD}\rvert; strict = trailing-36mo monthly |
r, E |
<1 poor, 1–3 acceptable/good, >3 excellent; HF ~1.5–3.0. [CORR: >2 alone is NOT excellent (reserve for >3); 0.5–1.0 is "weak"] |
| MAR ratio | \mathrm{CAGR}(\text{inception})/\lvert\mathrm{MaxDD}(\text{inception})\rvert |
full E |
~>0.5 acc, >1 good [C]; differs from Calmar in window AND numerator (avg-annual vs CAGR) [CORR] |
| Sterling | \mathrm{CompoundROR}/\lvert\mathrm{avgAnnualMaxDD}-10\%\rvert |
annual DD episodes | no band; lower than Calmar. [CORR: subtract 10% inside the abs; 10% ≈ 1981 T-bill rate**]** |
| Burke | (r_P-r_F)/\sqrt{\sum_t D_t^2}; modified divides \sum D_t^2 by n |
DD-episode list | no band; higher better [L] |
| Omega(θ) | \sum\max(0,r-\theta)/\sum\max(0,\theta-r) |
r, θ |
=1 breakeven (θ=mean), >1 preferred [L] |
| Information ratio | \operatorname{mean}(r-b)/\operatorname{std}(r-b), ann. \times\sqrt{P} |
r, benchmark b |
0.4–0.6 good, 0.61–1.0 very good, ≥1.0 exceptional [formula L; thresholds C] |
| Treynor | (E(R_p)-R_f)/\beta, \beta=\operatorname{Cov}(R_p,R_m)/\operatorname{Var}(R_m) |
r, market R_m |
no abs threshold; ranking only [L] |
Annualization rule [L as convention]: mean ×P, std ×sqrt(P), ratio
×sqrt(P); P=252 daily (trading days, not 365), 52 weekly, 12 monthly. Valid
only under iid; Lo (2002) shows annualized Sharpe overstated up to ~65% under
serial correlation.
D. Drawdown & recovery (equity-curve functionals)
E(t) = cumulative equity, HWM(t) = running peak, DD(t) = E(t)−HWM(t). No
trade list required.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Max drawdown | \max_t(\mathrm{HWM}-E) (level) or /\mathrm{HWM} (%) |
E |
stat-arb/MN <10–15%, HF <15–20%, L/S unacceptable >25–30%, >40% very high [C; no universal cutoff] |
| Recovery asymmetry | DD fraction d needs gain d/(1-d) |
a DD depth | exact: −20%→+25%, −50%→+100% [L, algebraic] |
| Average drawdown | (1/n)\sum\lvert\mathrm{DD}\rvert (=Pain Index) OR episode-mean of trough depths |
E, episode segmentation |
comparative; AvgDD/MaxDD≈1 → worst typical |
| Duration / time-to-recovery / time-underwater | per-episode index arithmetic; total UW = #(DD<0)/n | E + time index |
heuristic recovery ≈ MDD/per-period-return; flag censored open DD [C] |
| Longest drawdown | \max_k(t_{\mathrm{recover}}-t_{\mathrm{peak}}) |
episode segmentation | shorter better; often a different episode than max-depth |
| Ulcer Index | \sqrt{(\sum_i {D'_i}^2)/n}, D'_i=100\cdot(E_i-\mathrm{max\_so\_far})/\mathrm{max\_so\_far} |
E |
=0 no DD; > ~10 severe; comparative [C] |
| Ulcer Perf Index / Martin | (R_p-R_f)/\mathrm{UlcerIndex} |
r, E |
higher better; no band; more overfit-robust objective [L] |
| Pain Index | (\sum_i\lvert D'_i\rvert)/n, n = total obs |
E |
comparative; < MaxDD. [CORR: it EMBEDS duration — a remedy, NOT a metric that "ignores time"; Bacon 2008 p.89; n=total obs else it collapses to Average Drawdown**]** |
| Pain Ratio | (R_p-R_f)/\mathrm{PainIndex} |
r, E |
higher better; Bacon 2008 p.91 [CORR page] |
| Recovery Factor | \mathrm{NetProfit}/\lvert\mathrm{MaxDrawdown}\rvert |
final E / MaxDD |
poor <1, good 2–5, excellent ≥5 [formula L; thresholds C] |
| Expected MaxDD (Magdon-Ismail) | Brownian: linear (μ<0), \sim\sqrt{T} (μ=0, exact \sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T}), log (μ>0) |
μ,σ from r; classify regime |
not graded — a yardstick; observed ≫ E[MaxDD] = regime break / fat tails [L asymptotics] |
Load-bearing point: MaxDD is a downward-biased, sample-length-dependent point estimate — the cure is Monte-Carlo (report the distribution of MaxDD, not one point) and walk-forward (report OOS DD separately). Do NOT argmax Calmar/Recovery-Factor (single-point fragile); Ulcer/Pain-based ratios integrate the whole path and are more robust objectives.
E. Trade-level & efficiency metrics
Require a trade list. A "trade" spans from a position opening (or sign-flip) to
its close; P&L = Σ r_t over the span; direction = sign of the held position.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Win rate p | \mathrm{wins}/\mathrm{total\_trades} |
trades | not gradable alone; vs breakeven 1/(1+R) |
| Profit factor | \mathrm{gross\_profit}/\mathrm{gross\_loss} |
trades | <1 losing, 1.0–1.2 breakeven, 1.2–1.5 realistic, 1.5–2.0 good, 2.0–3.0 exceptional-but-scrutinize, >3 overfit flag. [CORR: swing target HIGHER than day-trade; ~200+ trades needed; live degrades PF ~10–20%] |
| Expectancy | (p\cdot\mathrm{AvgWin})-((1-p)\cdot\mathrm{AvgLoss}) |
trades | only hard threshold is sign >0 |
| Payoff ratio R | \mathrm{AvgWin}/\mathrm{AvgLoss} |
trades | good = exceeds breakeven (1−p)/p |
| PF↔win-rate identity | \mathrm{PF}=(p\cdot R)/(1-p), inverse p=\mathrm{PF}/(\mathrm{PF}+R) |
p, R | exact identity (holds iff R = ratio of averages) [L] |
| Breakeven win rate | 1/(1+R) |
R | 50%@1:1, 33.3%@1:2, 25%@1:3, 16.7%@1:5 [L] |
| SQN (Van Tharp) | \sqrt{N}\cdot\operatorname{mean}(\text{R-mult})/\operatorname{std}(\text{R-mult}), N often capped 100 |
per-trade R-multiples | <1.6 below, 1.6–2.0 avg, 2.0–2.5 good, 2.5–3.0 excellent, >7 overfit [C] |
| MAE / MFE | max adverse / favorable excursion = −min/max(running unreal. PnL) |
per-bar running P&L since entry | no cutoff; winners cluster at small MAE; large MFE-vs-realized gap → exiting early |
| Trade efficiency | exit \mathrm{ActualGain}/\mathrm{MFE}; range (\text{Exit}-\text{Entry})/(\text{High}-\text{Low}) |
per-trade MFE/MAE/range | 0–100%; <50% exit eff poor [C] |
| Turnover | \sum\lvert\Delta\mathrm{position}\rvert (or SEC \min(\mathrm{buys},\mathrm{sells})/\mathrm{avg\_assets}) |
position series | style-dependent; net-of-cost expectancy must stay >0 |
| R-multiple / expectancy-in-R | per trade \mathrm{PnL}/1R; mean p\cdot R-(1-p) |
trades + defined 1R | sign >0; ~+0.5R strong [C] — needs a defined risk/stop |
F. Tail, distribution & risk-of-ruin
From r (ruin from Monte-Carlo or the trade list). Real returns are negatively
skewed and leptokurtic, so iid-normal measures understate tail risk. Prefer
empirical / Monte-Carlo estimates.
| metric | formula | computed from | threshold |
|---|---|---|---|
| Historical VaR_α | -(1-\alpha) empirical quantile of r (5th @95%, 1st @99%) |
sort r, index floor((1−α)N) |
Basel traffic-light @250d/99%: GREEN 0–4, YELLOW 5–9, RED 10+ exceptions [L] |
| Parametric VaR | -(\mu+\sigma\cdot\Phi^{-1}(1-\alpha))\cdot W; z = 1.645@95, 2.326@99, 1.282@90 |
μ,σ of r |
same grading; iid-normal trap [L multipliers] |
| Historical ES/CVaR | mean of worst (1−α)N returns |
sorted r, k=ceil((1−α)N) |
coherent (VaR is not); FRTB uses 97.5% ES ≈ 99% VaR under normality [L] |
| Parametric ES | \mu+\sigma\cdot\varphi(\Phi^{-1}(\alpha))/(1-\alpha); mult ~2.063@95, 2.665@99 |
μ,σ | worse than VaR for fat tails [L] |
| Tail ratio | q95(r)/\lvert q05(r)\rvert |
percentiles of r |
>1 favourable, <1 dangerous [L 95/5] |
| Skewness | (1/N)\sum(r-\mu)^3/\sigma^3 |
r |
sign-driven; persistent negative = red flag even with high Sharpe |
| Excess kurtosis | (1/N)\sum(r-\mu)^4/\sigma^4 - 3 |
r |
0=normal; >0 leptokurtic (typical, dangerous); daily 3–10+ [L] |
| Downside deviation | \sqrt{(1/N)\sum[\min(0,r-T)]^2}, divisor total N |
r, T |
feeds Sortino [L, divisor=total N] |
| Risk of ruin (closed) | ((1-A)/(1+A))^U, A=p\cdot R-q, U=\text{capital}/\text{risk-per-trade} |
trades p, R | <1–5%, desks <1%, pros <0.1%; >5–10% reject [C] |
| Risk of ruin (empirical) | fraction of N Monte-Carlo paths breaching the ruin threshold |
multi-run dist. | preferred; sidesteps iid-normal; default R=50% capital |
| Kelly / optimal-f | f^*=(b\cdot p-q)/b; f^*=(\mu-r)/\sigma^2 |
r/odds |
deploy 0 < f ≤ ~0.5·f*; above full Kelly strictly dominated [L] |
| Gain-to-Pain | \sum(r)/\lvert\sum(\text{neg } r)\rvert |
r |
(monthly) ~1 acc, ~2 outstanding, 3 excellent, ~4 world-class, <1 poor [C; state the basis] |
| Common-Sense ratio | \mathrm{TailRatio}\cdot\mathrm{GainToPain} |
the two above | >1 robust, <1 fragile [L breakpoint] |
G. Walk-forward analysis
| metric | formula | computed from | threshold |
|---|---|---|---|
| WFE (return) | (\mathrm{OOS\_profit}/\mathrm{OOS\_bars})/(\mathrm{IS\_profit}/\mathrm{IS\_bars}) |
per-fold IS-optimal/OOS profit, bar counts | ≥0.5 robust, ~1.0 ideal, >0.7 excellent, 0.5–0.7 good, 0.3–0.5 mediocre, <0.3 poor; >1.0 is a warning (under-fit IS / lucky OOS), not superiority |
| WFE (Sharpe) | \mathrm{OOS\_Sharpe}/\mathrm{IS\_Sharpe} |
per-fold Sharpe over r |
>0.5 robust, ~1.0 ideal, <0.3 red flag |
| IS-OOS gap | \mathrm{IS\_Sharpe}-\mathrm{OOS\_Sharpe} per fold, averaged |
per-fold Sharpes | >0.5/cycle = overfitting; healthy mean <0.3 |
| Stitched OOS curve | chain each fold's OOS segment, re-based | family of OOS runs | net profitable; max DD <40% (TradeStation) |
| Parameter stability | dispersion of argmax params across folds; plateau check | per-fold winners | qualitative — broad plateau, clustered |
| Fraction profitable OOS windows | \#(\mathrm{OOS}>0)/\#\text{windows} |
per-window OOS P&L | majority >50%; worst-window OOS Sharpe >0 |
| Pardo verdict | net-profitable AND WFE ≥50% AND maxDD <40%; filter %Prof ≥40, PF ≥1.5 |
the above | composite pass/fail [software-default convention] |
Sizing: IS:OOS 2×–5×; ≥6 windows (10+ preferred); non-overlapping OOS for a
clean stitch; trade floors IS ≥100, OOS ≥30. Rolling vs anchored. Advanced: a
sweep of WFA configs (a "family of families") proves the verdict is not a lucky
schedule.
H. Monte-Carlo
| metric | formula | computed from | threshold |
|---|---|---|---|
| Trade-order shuffle | permute r, rebuild equity |
seeded reorder | additive sum INVARIANT — only path/DD stats vary |
| Bootstrap (with replacement) | resample T from r |
seeded resample | changes total → genuine return distribution; CI=[2.5, 97.5] pct |
| Equity percentile fan | per-horizon {5,25,50,75,95} pct | N curves | descriptive; viability: 5th-pct ending equity >0 |
| Stress (95th-pct) MaxDD | 95th pct of N MaxDD | per-run MaxDD | <25% good, >35% red flag; size capital off this [C] |
| Empirical VaR/ES | 5th-pct (1st @99) final P&L; ES = mean beyond | per-run terminal | 5th-pct ending equity >0 |
| Probability of ruin | \mathrm{ruin\_count}/N (MaxDD>R, default 50%) |
per-run MaxDD | <1% excellent, 1–5% acceptable, >5% not recommended [C] |
| Prob. of a losing period | \#(\text{period}<0)/N |
per-run period returns | >~30–40% hard to hold |
| MCPT p-value | (z+1)/(N+1), z=#perm ≥ real |
permute bar-to-bar log returns, keep first price, re-run | <0.05 min, <0.01 strict, >0.10 red flag, ~0.5 discard [L: +1 in numerator and denominator] |
| White RC / Hansen SPA | bootstrap max-stat over L rules, recentered | per-rule rel-perf + benchmark | p<0.05; SPA more powerful than the conservative RC |
Conventions: N = 1,000 min, 5,000 recommended, 10,000+ for tail stability.
Plain shuffle assumes serial independence — if r is autocorrelated use the
block/stationary bootstrap (Politis-Romano). The permutation test permutes
log returns (preserving the marginal, destroying ordering) → tests specifically
time-ordering edges.
I. Stress/scenario, regime, cost/capacity, benchmark-relative, sanity, live-vs-backtest
| tool | produces | key metric + formula | computed from | threshold |
|---|---|---|---|---|
| Stress/scenario | conditional worst-case | per-scenario MaxDD, profit, recover-time | strategy over crisis windows / shocked synthetic data | fail if stress DD > risk limit; excellent if net-positive through crises [C; Fed CCAR] |
| Regime | per-state metrics + fragility verdict | profit/Sharpe/DD masked by regime label | a regime-label series (vol/trend classifier or HMM) | edge only in one regime + negative elsewhere = fragile [qualitative] |
| Cost/slippage | net-vs-cost curve + break-even | c^*\text{ where net}=0\approx\mathrm{gross\_profit}/n_\text{trades}; d(\text{net Sharpe})/d(\text{cost}) |
a cost-modelling backtest, cost swept | reject if edge survives only at zero cost; break-even ≳2× real cost [C] |
| Capacity/impact | AUM ceiling + turnover | \text{impact}\approx Y\cdot\sigma\cdot\sqrt{Q/V}, Y~O(1); capacity = AUM where impact = gross alpha |
order volumes + ADV | capacity ≫ target = scalable; per-day order < ~5–10% ADV [C] |
| Benchmark-relative | alpha/beta/IR + significance | \beta=\operatorname{Cov}/\operatorname{Var}; Jensen alpha = regression intercept; \mathrm{IR}=\text{alpha}/\mathrm{TE} |
r regressed on a market series R_m |
alpha positive + significant (` |
| Null/sanity | "beat random?" | MCPT p=(\#\text{perm}\ge\text{real})/N; random-entry: real in right tail |
permuted input + a random-entry null strategy | p<0.05 min, ≤0.01 strict, >0.10 red flag, N≥1000 [L] |
| Live-vs-backtest | overfitting/decay verdict | haircut =(\mathrm{BT\_Sharpe}-\mathrm{live\_Sharpe})/\mathrm{BT\_Sharpe} |
live series vs backtest prediction | <30% acceptable, 30–50% typical, >50–70% largely overfit [C; verify primary source] |
J. Overfitting, multiple-testing & CPCV
The significance layer over the analysis axes. Treats "I ran many variants and kept the best" as the statistical problem it is.
| metric | formula | computed from | threshold |
|---|---|---|---|
| PSR(SR*) | \Phi[(\mathrm{SR}_{hat}-\mathrm{SR}^*)\cdot\sqrt{T-1}/\sqrt{1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2}] |
per-period SR_hat, T, g3=skew, g4=kurtosis (full, normal=3) of r |
>0.95 significant [L] |
| MinTRL | 1+(1-g_3\cdot \mathrm{SR}_{hat}+((g_4-1)/4)\cdot \mathrm{SR}_{hat}^2)\cdot(Z_\alpha/(\mathrm{SR}_{hat}-\mathrm{SR}^*))^2 |
as PSR + target α | need T ≥ MinTRL [L] |
| E[max SR_N] | \sqrt{V}\cdot((1-\gamma)\Phi^{-1}(1-1/N)+\gamma\Phi^{-1}(1-1/(Ne))), \gamma\approx0.5772 |
cross-trial Sharpe variance V, effective N | the moving hurdle; grows with N [L] |
| DSR | PSR at \mathrm{SR}^*=\mathrm{SR}_0=E[\max \mathrm{SR}_N] |
winner SR_hat/T/g3/g4 + V across trials + effective N | >0.95 survives deflation [L; ex: N=100,V=½,T=1250,g3=−3,g4=10 → DSR~0.90<0.95 REJECT] |
| MinBTL | \le 2\cdot\ln(N)/E[\max]^2 years |
N, target SR | 5y + >~45 independent trials → guaranteed overfit [C/L] |
| PBO (CSCV) | matrix (T\times M); S even blocks; C(S,S/2) combos; \omega=(\mathrm{rank}-0.5)/M; \lambda=\ln(\omega/(1-\omega)); \mathrm{PBO}=P[\lambda\le0] |
per-trial r series |
<0.5 sanity, ~0.5 alarm, <0.1 excellent [L] |
| Harvey-Liu-Zhu haircut | t=\mathrm{SR}\cdot\sqrt{T}; adjust p (Bonferroni \min(M\cdot p,1) / Holm / BHY); \mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t}) |
SR, T, M | required new-factor t ~3.0 (not 2.0) |
| CPCV paths | splits =C(N,k); paths \varphi=(k/N)C(N,k) |
grouped purged+embargoed windows | N=5–10 groups, k=2; purge: drop train obs whose label span overlaps test; embargo h~1% of T (forward only) [L] |
Master catalogue (compact index)
Inputs: r = per-period returns; E = equity curve; trades = closed trades; b = benchmark; multi = multi-run distribution.
Return — Total \sum r [C] · CAGR \operatorname{mean}(r)\cdot P [C] · Volatility \operatorname{std}(r)\cdot\sqrt{P} [C].
Risk-adjusted — Sharpe \sqrt{P}\cdot(\operatorname{mean}(r)-r_f)/\operatorname{std}(r-r_f) [CORR denom] · Sortino
\ldots/\sqrt{\sum\min(0,r-\mathrm{MAR})^2/N}, good >1 [CORR] · Calmar \operatorname{mean}(r)\cdot P/\lvert\mathrm{MaxDD}\rvert,
<1/1–3/>3 [CORR] · MAR \mathrm{CAGR}_{\text{inc}}/\lvert\mathrm{MaxDD}_{\text{inc}}\rvert [CORR window+numerator] · Sterling
\mathrm{CompoundROR}/\lvert\mathrm{avgAnnMaxDD}-10\%\rvert [CORR] · Burke (r_P-r_F)/\sqrt{\sum D^2} [L] · Omega
\sum\max(0,r-\theta)/\sum\max(0,\theta-r) [L] · Info-Ratio \operatorname{mean}(r-b)/\operatorname{std}(r-b)\cdot\sqrt{P} [L/C] · UPI
(R_p-R_f)/\mathrm{UlcerIndex} [L] · Pain-Ratio (R_p-R_f)/\mathrm{PainIndex} [CORR page] ·
Recovery-Factor \mathrm{NetProfit}/\lvert\mathrm{MaxDD}\rvert [L/C] · Gain-to-Pain \sum r/\lvert\sum\text{neg }r\rvert [C] ·
Common-Sense \mathrm{TailRatio}\cdot\mathrm{GainToPain} [L] · SQN \sqrt{N}\cdot\operatorname{mean}(R)/\operatorname{std}(R) [C].
Drawdown — MaxDD \max(\mathrm{HWM}-E) [C] · Recovery-asymmetry d/(1-d) [L] · AvgDD
\operatorname{mean}\lvert\mathrm{DD}\rvert [C] · Ulcer \sqrt{\sum D^{\prime 2}/n} [C] · Pain \sum\lvert D^\prime\rvert/n (n=total) [CORR embeds
duration] · Expected-MaxDD μ=0 \sqrt{\pi/2}\cdot\sigma\cdot\sqrt{T} [L].
Trade-level — Win-rate \mathrm{wins}/\mathrm{total} [C] · Profit-Factor \mathrm{gross\_profit}/\mathrm{gross\_loss},
<1/1.5–2/>3 overfit [CORR swing>day] · Expectancy p\cdot\mathrm{AvgWin}-(1-p)\cdot\mathrm{AvgLoss},
sign>0 [L] · Payoff \mathrm{AvgWin}/\mathrm{AvgLoss} [L] · PF-identity \mathrm{PF}=(p\cdot R)/(1-p) [L] ·
Breakeven-win 1/(1+R) [L] · MAE/MFE [C] · Trade-eff \mathrm{ActualGain}/\mathrm{MFE} [C] · Turnover
\sum\lvert\Delta\mathrm{position}\rvert [C].
Tail/distribution — Hist-VaR -q_{1-\alpha}(r), Basel traffic-light [L] · Param-VaR
-(\mu+\sigma\cdot z), z 1.645/2.326 [L] · Hist-ES \operatorname{mean}(\text{worst }(1-\alpha)N) [L] · Tail-Ratio
q95/\lvert q05\rvert [L] · Skew \operatorname{mean}((r-\mu)^3)/\sigma^3 [C] · Excess-kurtosis \operatorname{mean}((r-\mu)^4)/\sigma^4-3 [L]
· Downside-dev \sqrt{\sum\min(0,r-T)^2/N} [L] · Risk-of-ruin ((1-A)/(1+A))^U or
empirical, <1% pro [C] · Kelly (\mu-r)/\sigma^2, ≤0.5·f* [L].
Overfitting — PSR \Phi[(\mathrm{SR}-\mathrm{SR}^*)\sqrt{T-1}/\sqrt{1-g_3\cdot\mathrm{SR}+((g_4-1)/4)\mathrm{SR}^2}], >0.95 [L]
· MinTRL [L] · DSR = PSR at \mathrm{SR}^*=\operatorname{E}[\max \mathrm{SR}_N], >0.95 [L] · MinBTL \le 2\ln(N)/\operatorname{E}[\max]^2
[C/L] · PBO P[\ln(\omega/(1-\omega))\le 0], <0.5 sanity / <0.1 excellent [L] · HLZ-haircut
\mathrm{HSR}=\mathrm{SR}\cdot(\mathrm{adj\_t}/\mathrm{orig\_t}), t~3.0 [C/L] · CPCV \varphi=(k/N)C(N,k), embargo h~1% [L].
References
External, source-checked references per section (each link verified reachable and on-topic). Where a formula or threshold was corrected, the citation is in the Metric Verification Log.
A / B — Parameter sweep, optimization & robustness: Sensitivity analysis · Hyperparameter optimization · Pseudo-mathematics and financial charlatanism — Bailey, Borwein, López de Prado, Zhu · Deflated Sharpe Ratio (objective choice)
C — Core & risk-adjusted ratios: Sharpe · Sortino · Calmar & MAR · Sterling (PerformanceAnalytics) · Burke (PerformanceAnalytics) · Omega · Information ratio · Treynor · Volatility / sqrt-of-time · Lo (2002), The Statistics of Sharpe Ratios — CFA Institute
D — Drawdown & recovery: Drawdown (max / average / duration) · Ulcer Index & UPI · Pain Index · Pain Ratio · Average drawdown · Recovery Factor (Fidelity glossary) · Expected MaxDD — Magdon-Ismail et al.
E — Trade-level & efficiency: Win rate vs profit factor · Expectancy · Payoff & breakeven · PF↔win-rate identity · SQN (Van Tharp) · MAE · MFE · Trade efficiency (TradeStation) · Turnover (CFI)
F — Tail, distribution & risk-of-ruin: Value-at-Risk · Expected Shortfall / CVaR · Skewness · Kurtosis · Downside deviation · Risk of ruin · Kelly criterion · Gain-to-pain · Basel backtesting / traffic-light (BIS)
G — Walk-forward analysis: Walk-forward optimization · WFE / parameter stability / criteria (TradeStation) · Anchored vs rolling windows
H — Monte-Carlo & permutation tests: Bootstrapping (incl. block/stationary) · Permutation test · Monte-Carlo Permutation Test (BuildAlpha) · White's Reality Check (1999) · Hansen's SPA test (2005)
I — Stress, regime, cost/capacity, benchmark-relative, sanity: Stress testing (financial) · Regime / Markov switching · Slippage · Transaction-cost backtesting (QuantStart) · Square-root market-impact law (arXiv) · CAPM (alpha/beta) · Jensen's alpha · Live-vs-backtest haircut — Bailey & López de Prado, Deflated Sharpe (PDF)
J — Overfitting, multiple-testing & CPCV: Deflated Sharpe Ratio & False Strategy Theorem · Probabilistic Sharpe Ratio (PerformanceAnalytics) · Minimum Track Record Length · PBO / MinBTL via CSCV (pbo vignette) · Harvey-Liu-Zhu haircut (quantstrat) · Bonferroni · Holm-Bonferroni · Benjamini-Hochberg / FDR · Purged cross-validation (CPCV, purging, embargo) · López de Prado, Advances in Financial ML, ch. 7
Backtesting Engine Architecture
- Architecture overview
- Event-Driven Backtesting
- Look-Ahead Bias & Causality
- Determinism & Reproducibility
- Reactive Streaming Dataflow
- Columnar Data Layout
- Signal, Exposure & Execution
- Graph Compilation & Optimization
- Authoring & Deployment Lifecycle
Strategy Analysis & Validation