Experimental Design and Analysis of Variance
Core concepts
Understand the principles of experimental design: randomisation, replication, and blocking. Know completely randomised designs, randomised block designs, factorial designs, and split-plot designs. Be able to set up the ANOVA table for one-way and two-way layouts, decompose total sum of squares into treatment and error components, compute degrees of freedom, mean squares, and the F statistic. Understand fixed versus random effects, interaction effects, multiple comparison procedures (Tukey, Bonferroni), and the assumptions of ANOVA: independence, normality, and homogeneity of variance.
Traps
- Forgetting that df for error in a one-way ANOVA is N − k, not N − 1.
- Running many pairwise t-tests instead of an appropriate multiple comparison procedure, inflating Type I error.
- Ignoring a significant interaction and interpreting main effects alone.
- Confusing a randomised block design with a repeated-measures design.
- Applying ANOVA to severely heteroscedastic data without transformation.
- Treating a random factor as fixed, which changes the error term.
Worked example
Three treatments with 5 observations each: N=15, k=3. Total df = 14, treatment df = 2, error df = 12. If treatment SS = 60 and error SS = 120, then MS_treatment = 30, MS_error = 10, and F = 3.0 with (2,12) df. The critical value at α=0.05 is 3.89, so we fail to reject. Rule of thumb: the F statistic is always a ratio of mean squares, and its numerator df equals the number of groups minus one.