Regression Analysis
The least-squares line
A scatterplot is examined first for direction, form, strength, and outliers. The least-squares regression line ŷ = a + bx minimizes the sum of squared residuals; slope b = r(s_y/s_x) and intercept a = ȳ − b·x̄. The slope is interpreted as the predicted change in y per one-unit increase in x - a statement about prediction, not causation. The intercept is often meaningless when x = 0 is outside the data range. The correlation r (between −1 and 1) measures only the strength and direction of a linear relationship; it is unaffected by changing units, and r² is the fraction of variation in y explained by the linear model.
Residuals and diagnostics
A residual is observed − predicted. The residual plot should show no pattern; curvature, funnels, or clusters signal that a line is not appropriate. Influential points can change the regression line substantially; high leverage depends on x-position, while an outlier depends on y. Removing a point can raise or lower r - a classic conceptual trap.
Inference for slope
Inference for the slope uses t = b/SE_b with df = n − 2, testing H₀: β = 0 (no linear relationship). Conditions: linearity, independence, normal residuals, equal variance, and random sampling. Extrapolation beyond the observed x-range is risky and a common wrong-answer choice.
Worked example. With r = 0.8, s_x = 2, s_y = 5, b = 0.8(5/2) = 2.0. If x̄ = 10 and ȳ = 30, then a = 30 − 2(10) = 10, so ŷ = 10 + 2x; each one-unit increase in x predicts a 2-unit increase in y.