Systematic Trading, Backtesting and Model Validation
Systematic Trading
Signal to portfolio
A systematic process converts a hypothesis into a measurable signal, standardises or ranks it, builds a portfolio with position sizing and constraints, and executes with cost-aware trading. Distinguish time-series (absolute) from cross-sectional (relative) signals, and true alpha from risk premia such as value, momentum, carry and quality.
Implementation
Position sizing via volatility targeting, risk parity or fractional Kelly. Rebalancing frequency trades signal decay against turnover. Model costs explicitly: commission, spread, market impact and slippage; capacity shrinks as assets under management grow. Execution algorithms (TWAP, VWAP, POV, implementation shortfall) manage the trade-off between market impact and timing risk.
Traps
- Presenting gross returns when turnover is high; costs can consume the whole edge.
- Ignoring short borrow costs, locate availability and hard-to-borrow constraints.
- Universe construction contaminated by survivorship or look-ahead bias.
- Over-tuning thresholds or lookback windows to one historical sample.
- Labelling beta or a well-known risk premium as alpha.
Worked example
A monthly strategy trades 100% one-way turnover with a 10 bp round-trip cost. Monthly cost is 1.0 x 0.0010 = 10 bp (0.10%), or about 1.2% a year. If gross alpha is 3.0%, net alpha is roughly 1.8%; doubling turnover without improving signal quality halves the net edge.
Backtesting and Model Validation
Protocol
A credible backtest uses point-in-time data, a survivorship-bias-free universe including delisted names, realistic costs, borrow and financing charges, and an explicit universe and rebalance schedule. Separate in-sample design from out-of-sample and walk-forward evaluation, and hold out a final period untouched until the end, or use purged, embargoed combinatorial cross-validation. Report results net of costs against a relevant benchmark, with capacity and sensitivity analysis.
Overfitting and multiple testing
Every additional trial raises the expected best result under the null, so a single strong backtest is weak evidence. Deflated Sharpe ratios, Bonferroni or false discovery rate corrections and White's Reality Check address data snooping. Pre-register hypotheses and count trials honestly.
Traps
- Look-ahead bias: using data unavailable at decision time (restatements, index membership, same-day closes).
- Ignoring delisting returns and corporate actions.
- Assuming fills at the close or at mid with no market impact.
- Survivorship bias in stock or fund universes.
- Reporting only the best of many parameter settings.
Worked example
With N independent trials and no true skill, the expected maximum of N standard normal statistics is roughly sqrt(2 ln N). For N = 100 this is about 3.0, so the best t-statistic of a large search is expected to be near 3 even when nothing works; the significance hurdle must be raised accordingly.