Quantitative Methods
Building the regression
Multiple regression estimates y = b0 + b1x1 + ... + bkxk + e by ordinary least squares. Interpret each slope as the effect of a one-unit change in that independent variable, holding the others constant. Assess the model with the F-test (all slopes jointly zero) and individual t-tests, and compare R-squared with adjusted R-squared, which penalizes added regressors and can fall.
Assumption violations
- Heteroskedasticity: residual variance varies with the regressors. Coefficients stay consistent, but standard errors are wrong - use White/Hansen robust errors.
- Serial correlation: residuals are correlated across observations, detected with the Durbin-Watson statistic, DW roughly 2(1 - r). Positive serial correlation understates standard errors and inflates t-statistics.
- Multicollinearity: high correlation among regressors inflates standard errors, producing insignificant t-tests alongside a significant F-test and high R-squared.
- Misspecification: omitted variables, wrong functional form, or an inappropriate dependent variable.
Qualitative and time-series dependent variables
Binary dependent variables call for logit or probit models, evaluated with a confusion matrix and a cutoff. Time series require covariance stationarity; test with unit root tests, correct with first differencing, and model seasonality with lagged seasonal terms. Autoregressive forecasts are judged by out-of-sample RMSE and checked for ARCH.
Worked example
A regression of returns on two factors reports R-squared of 0.62 with an F-test significant at 1%, but both t-statistics below 1.5. The likely diagnosis is multicollinearity, not an insignificant model. Remedies: drop one correlated regressor, or use the joint model for forecasting only.
Rule of thumb: read regression output for the direction of the bias - serial correlation and heteroskedasticity rarely bias coefficients, but they reliably bias inference.