Interpreting a Significant t-test with Unequal Variances
A researcher compares the mean scores of two teaching methods on a standardized test. Group A (traditional lecture, n=30) has mean 72, SD 12. Group B (active learning, n=25) has mean 78, SD 18. A two-sample t-test assuming equal variances yields t = 1.45, p = 0.15. The researcher concludes there is no significant difference. However, Levene's test for equality of variances gives p = 0.02. The researcher is unsure whether to trust the equal-variance t-test or use Welch's t-test. The sample sizes are unequal, and the SDs differ substantially. The researcher also notes that the distributions are roughly normal within each group.
1. Should the researcher use the equal-variance t-test or Welch's t-test? Justify your answer based on the information provided.
Read the worked answer
The researcher should use Welch's t-test. Levene's test indicates unequal variances (p = 0.02 < 0.05), and the sample sizes are unequal (30 vs 25) with substantially different SDs (12 vs 18). The equal-variance t-test assumes homogeneity of variances; violating this assumption, especially with unequal n, can lead to inflated Type I error or reduced power. Welch's t-test does not assume equal variances and is more robust in this situation. Even though the equal-variance test gave p = 0.15, Welch's test may yield a different p-value. The researcher should report Welch's result and not conclude 'no difference' based solely on the equal-variance test.
2. Given the data, calculate Welch's t-statistic and approximate degrees of freedom. What conclusion would you draw at α = 0.05?
Read the worked answer
Welch's t = (meanA - meanB) / sqrt(sA^2/nA + sB^2/nB) = (72 - 78) / sqrt(144/30 + 324/25) = -6 / sqrt(4.8 + 12.96) = -6 / sqrt(17.76) = -6 / 4.214 = -1.424. Degrees of freedom (Welch-Satterthwaite): df = (sA^2/nA + sB^2/nB)^2 / [ (sA^2/nA)^2/(nA-1) + (sB^2/nB)^2/(nB-1) ] = (17.76)^2 / [ (4.8)^2/29 + (12.96)^2/24 ] = 315.42 / [23.04/29 + 167.96/24] = 315.42 / [0.794 + 6.998] = 315.42 / 7.792 = 40.48. So df ≈ 40. Critical t at α = 0.05 (two-tailed) for df=40 is about 2.021. |t| = 1.424 < 2.021, so p > 0.05. We fail to reject the null hypothesis. Conclusion: no statistically significant difference at the 5% level, but note the effect size (difference = 6, pooled SD ~15, Cohen's d ≈ 0.4) suggests a moderate effect that might be practically meaningful; the study may be underpowered.