Regression and Correlation
This domain examines how two measured variables move together and how one can be predicted from the other: correlation coefficients, the least-squares line, diagnostics, and the assumptions that make inference on a slope valid.
Correlation
- Pearson r equals the covariance divided by the product of the standard deviations; it lies between minus 1 and 1 and is unit-free.
- It measures linear association only. A strong curved relationship can give r near zero.
- r-squared is the proportion of variation in y explained by the linear fit.
- Spearman rank correlation is Pearson r applied to ranks; it suits ordinal data or monotone but nonlinear relations and resists outliers.
Simple linear regression
- Model: y = a + b x + error. The least-squares slope is b = Sxy/Sxx, which also equals r times sy divided by sx; the intercept is a = y-bar minus b times x-bar.
- With an intercept fitted, residuals sum to zero and are uncorrelated with x.
- Assumptions for tests and intervals: linearity, independence of errors, constant variance, and roughly normal residuals. Residual plots reveal violations.
- Points with high leverage and a large residual are influential; Cook's distance summarises the combination.
Worked example
With x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5: x-bar = 3 and y-bar = 4. Then Sxx = 10, Syy = 6 and Sxy = 6, so b = 0.6 and a = 4 minus 1.8 = 2.2. The correlation is 6 divided by the root of 60, which is 0.775, and r-squared is 0.60. The slope means y rises by 0.6 units per unit of x, and 60% of the variation in y is explained by the line.
Traps
- Predicting far outside the observed range of x, where the fitted line has no support.
- Reading r as a percentage, or squaring it twice.
- Interpreting the intercept when x = 0 is meaningless or impossible.
- Treating a high correlation as evidence of causation, without a designed comparison.
- Ignoring curvature, which correlation alone will not flag.