A Regression That Stops Being Significant When the Standard Errors Are Fixed
An analyst at a fund of hedge funds regresses 60 monthly excess returns of a strategy on the excess return of a broad equity index:
R_strategy = 0.31 + 0.85 × R_index, R² = 0.62, n = 60
The software reports a standard error of 0.22 on the slope, giving t = 3.86, and the memo concludes "equity beta is significant at the 1% level." A reviewer reruns the standard errors with a Newey–West correction and obtains 0.45. The Durbin–Watson statistic is 1.15. Separately, the analyst flagged a second strategy whose alpha had a t-statistic of 2.10; both strategies were selected from 12 candidates screened in the same study. Strategy returns are built partly from monthly appraisal valuations.
1. Show whether the beta is significant at the 5% level, and explain what the Durbin–Watson statistic implies.
Read the worked answer
Significance. With the corrected standard error:
t = 0.85 / 0.45 = 1.89
With n = 60 and one regressor, df = 58 and the two-tailed 5% critical value is ≈ 2.00 (p ≈ 0.064). The null of zero beta cannot be rejected at 5%. The original t of 3.86 was an artefact of the standard error, not of a stronger relationship: the point estimate of 0.85 never changed.
What the DW statistic is telling you. For n = 60 and one regressor the lower bound is approximately 1.55. A DW of 1.15 falls below it, which is evidence of positive first-order serial correlation in the residuals (for reference, DW ≈ 2(1 − ρ̂), so ρ̂ ≈ 0.43).
Why it matters:
- OLS coefficients remain unbiased and consistent if the regressors are strictly exogenous; only the inference breaks.
- With positively autocorrelated errors, the usual OLS standard errors are biased downward, so t-statistics are inflated - exactly the pattern observed.
- Positively autocorrelated regressors (here, a filtered/smoothed strategy return and a market index that itself trends) amplify the problem.
- The underlying economics matters too: appraisal-based marks are the standard cause of serial correlation in hedge fund returns, so the residuals are signalling stale pricing, not just a technical nuisance. A stale-priced series understates its true beta and volatility.
The correct action is Newey–West (or another HAC) standard errors plus, ideally, unsmoothing the return series before re-estimating.
2. The second strategy's alpha has a t-statistic of 2.10. How should that be interpreted given the screening process?
Read the worked answer
A t of 2.10 corresponds to a two-sided p-value of roughly 0.04, which clears a naive 5% threshold - but the test was not run in isolation.
Multiple testing. Twelve candidates were screened. If none had genuine skill, the probability of seeing at least one |t| > 2.10 is 1 − (1 − 0.04)^12 ≈ 39%. That is close to a coin flip; the result is what pure noise looks like when you shop around.
A simple correction. Bonferroni: test each at α = 0.05/12 = 0.00417. The corresponding two-tailed critical t with df ≈ 58 is about 2.9. The observed 2.10 does not survive. (False discovery rate control gives a slightly less severe but similar conclusion.)
Deeper issues.
- The 12 candidates are not independent: overlapping universes, correlated styles, and shared data windows mean the effective number of tests is lower than 12, but the bias is still real.
- Selecting the maximum t-statistic and then reporting that t-statistic is a selection bias; the reported value is a maximum of a random variable, so its sampling distribution is shifted right.
- If the strategy was also chosen because its history looked good, then any subsequent in-sample metric is contaminated. The only clean evidence is out-of-sample, ideally on data the analyst had not seen when the hypothesis was formed.
Conclusion: treat the 2.10 as a candidate for further investigation, not as evidence of alpha.
3. What should the risk team do with the factor model before it is used for beta hedging and capital allocation?
Read the worked answer
The hedge ratio is the model's single most consequential output, so the sequence matters:
- Fix the inference first. Re-report all coefficients with HAC standard errors. Publish confidence intervals, not just point estimates. An 0.85 beta with a 95% interval of roughly [−0.05, 1.75] is not a hedging input; it is a warning.
- Unsmooth the returns. Estimate the moving-average lag structure of the reported series, invert it, and re-run the regression. Expect a higher beta and a lower R² - the smoothed series overstates explanatory power by attributing lagged market moves to the strategy.
- Stress the hedge. Compute hedge effectiveness across a range of plausible betas (say 0.6 to 1.4) and across a crisis window where correlations rise. If the residual exposure is unacceptable at the low end of the interval, hedge more conservatively or use a different instrument.
- Do not capitalise model risk away. Capital allocation should use a beta and volatility from unsmoothed data plus an explicit model-risk add-on; the 3.86 t-statistic should never appear in a committee paper again.
- Document the selection process. Record all 12 candidates and their results so that later reviews can see the multiple-testing context, and pre-commit to the test that will be used before the next screen.