Crossover System Whipsaw in a Range Regime
You run a trend-following overlay on a diversified commodity ETF using a 20/50 EMA crossover, long-only, 100% notional, with 6 bps round-trip cost. From January last year through June this year the ETF has oscillated between 24 and 29 without a sustained trend. The system generated 14 round trips: 4 winners averaging +4.8% and 10 losers averaging -1.6%. Profit factor is 0.82, maximum drawdown 9.4%, and the longest winning streak was two trades. During the prior six-year trending period the same rules produced a profit factor of 1.65. The ETF's realized volatility has fallen from 22% to 13% annualized, and average daily range is now 0.9%. Your committee wants to add a 200-day filter and an ADX(14) above 20 rule and re-optimize the EMA lengths on the full sample.
1. Distinguish a regime change from a broken system. What evidence would you require before concluding the rules no longer work?
Read the worked answer
Fourteen round trips with a 29% hit rate inside a 24-29 range is close to the expected behavior of a crossover system in a non-trending regime, not proof of breakage. Trend systems typically have low win rates and depend on a few large winners; the prior profit factor of 1.65 over six trending years shows the edge is conditional on autocorrelation. The key evidence is regime: realized volatility fell from 22% to 13%, average daily range is 0.9%, and price oscillated around the EMAs. Before declaring the system broken, compare the current profit factor with a distribution generated from shuffled returns of the same volatility. If the observed result sits inside that random range, the system has no edge in this regime, but the rules are mismatched rather than wrong.
2. Critically evaluate the committee's proposal to add two filters and re-optimize EMA lengths on the full sample.
Read the worked answer
Re-optimizing EMA lengths on the full sample after observing the drawdown is in-sample curve fitting and will overstate future performance. Adding both a 200-day filter and an ADX rule addresses the same problem-trend absence-and may over-filter, cutting the trade count so low that statistical significance disappears. The 200-day filter has economic logic: trade with the long-term trend. ADX is lagging and parameter-sensitive, and a fixed threshold of 20 on a commodity ETF may be too low or too high across regimes. More importantly, changing lengths, filter, and threshold simultaneously makes attribution impossible: if performance improves, you will not know which change caused it, and if it fails, you cannot diagnose why.
3. What testing protocol would you use instead, and how would costs and trade count affect your conclusion?
Read the worked answer
Test one change at a time with a pre-specified protocol: split the data into a design window (first 60%) and a holdout (last 40%), or use walk-forward optimization with rolling windows and no peeking. Evaluate profit factor, MAR, average trade net of 6 bps, and the number of independent trades. Stress costs at 10 and 15 bps; the edge must survive. Add a regime filter only if it improves out-of-sample profit factor without reducing trade count below roughly 30. Compare 20/50, 50/200, and a Donchian breakout as robustness checks. If all trend variants fail in the range but work in trends, accept that the overlay needs a regime switch and size it as a diversifier rather than a return engine. Re-optimize annually, not after a drawdown.