Twelve Configurations of a Trend Strategy: Deflating the Best Sharpe
A commodity trading advisor has backtested a volatility-targeted trend-following strategy on 15 liquid futures over 10 years of daily data. The research process tried 12 configurations: three lookback windows (20, 60, and 120 days), two volatility targets (10% and 15%), and two execution assumptions (next-open and VWAP). The best configuration has a gross Sharpe of 1.10, annualised volatility 11%, maximum drawdown 18%, and negative skew of -0.8. It performed well in 2008 and 2022 but lost 22% in the 2020 COVID crash before recovering. Parameter sensitivity shows the Sharpe falling to 0.4 when the lookback is shifted by 10 days. The investment committee likes the crisis-alpha story but is worried about data mining. They ask for a statistically defensible view of whether the edge is real and what minimum evidence would justify a 5% allocation.
1. Adjust the 1.10 Sharpe for the 12 configurations tested and state whether it is statistically distinguishable from zero. Show the reasoning, not just a number.
Read the worked answer
The reported Sharpe is the maximum of 12 trials, so its sampling distribution is not the same as a single pre-specified strategy. Under a null of no skill, the expected maximum Sharpe across N independent trials is approximately
SR_max ≈ sqrt(V[SR]) * [(1 - γ) Φ^-1(1 - 1/N) + γ Φ^-1(1 - 1/(N e))],
where γ is the Euler-Mascheroni constant. With 10 years of daily data, the standard error of an annualised Sharpe is roughly 1/sqrt(10) ≈ 0.32, so the noise scale is large. For N = 12, the expected maximum under the null is on the order of 0.6 to 0.8, depending on the correlation between configurations.
A point estimate of 1.10 is above that threshold but not overwhelmingly so, and the trials are positively correlated because they share data and signals, which lowers the effective number of independent trials but also reduces the information content of the maximum. The negative skew and the 2020 drawdown increase the standard error of the Sharpe estimate. A White's reality-check or bootstrap-based max-Sharpe test should be run. Preliminary conclusion: the edge is suggestive but not established at conventional confidence; it does not yet meet a 'distinguishable from zero after selection' standard.
2. Interpret the parameter sensitivity and negative skew for the investment committee. What allocation, if any, is justified?
Read the worked answer
The parameter sensitivity is the more damaging evidence. A Sharpe of 1.10 at the chosen lookback falling to 0.4 when the lookback shifts by 10 days indicates a narrow peak rather than a broad plateau. Robust trend strategies typically show gradual performance decay across neighbouring lookbacks; a cliff suggests the result depends on a few historical episodes or on data-mined timing. The committee should ask for a heat map of Sharpe across the full parameter grid, not just the chosen point.
Negative skew of -0.8 is consistent with trend following's convex payoff in sustained crises but also with crash exposure when trends reverse sharply, as in the 2020 episode. The crisis-alpha narrative is partially supported by 2008 and 2022, but the 22% drawdown shows the strategy is not a reliable hedge.
Allocation view: a 5% allocation is not justified on the current evidence as a standalone return driver. It could be justified as a small, risk-budgeted diversifier if (a) the plateau test passes across neighbouring parameters, (b) the max-Sharpe test survives multiple-testing adjustment, and (c) the allocation is sized by drawdown tolerance rather than by the backtest Sharpe. Staged funding with kill criteria is more defensible than a full 5% commitment.
3. Design a staged out-of-sample protocol with kill criteria before committing 5%.
Read the worked answer
Pre-register the strategy before any further testing: fix the signal definition, the volatility-targeting rule, the execution assumption, and the parameter grid. Then run a staged protocol:
- Hold-out and paper trading. Reserve the most recent data as a true hold-out and run 6 to 12 months of paper trading with the pre-registered rules. Record live slippage, not assumed VWAP execution.
- Small live allocation. If paper trading is consistent with the backtest, fund 1% of the portfolio and monitor for at least one quarter.
- Scale conditionally. Move toward 5% only if live Sharpe is positive, drawdown stays within the pre-agreed envelope, and realised volatility tracks the 10-15% target.
Kill criteria should be explicit and mechanical: for example, stop if live drawdown exceeds 12%, if live Sharpe over the evaluation window falls below 0.3, if realised slippage exceeds the modelled cost by more than 50%, or if the parameter plateau breaks down in the hold-out. Report the max-Sharpe test and bootstrap confidence interval alongside the live results. The committee should treat the 5% as a ceiling earned by evidence, not a starting allocation.