Hypothesis Testing
This domain tests the logic of statistical tests: hypotheses, error probabilities, p-values, power, and the construction of common tests and confidence intervals.
Core rules
- H0 is the default; H1 is the research claim. A test statistic measures discrepancy with H0.
- Significance level α=P(reject H0|H0 true), the Type I error rate. Power=1−P(Type II error)=P(reject H0|H1 true).
- The p-value is the probability, under H0, of a statistic at least as extreme as observed. It is not P(H0 true).
- For a normal sample with unknown variance, use t=(X̄−μ0)/(S/√n)~t_{n−1} under H0.
- A two-sided 100(1−α)% confidence interval contains all μ0 not rejected by the corresponding two-sided test.
Worked example
Suppose n=16 observations give x̄=52.3 and s=4.1. Test H0:μ=50 versus H1:μ≠50 at α=0.05. The statistic is t=(52.3−50)/(4.1/4)=2.24. The critical values are ±t_{15,0.975}=±2.131. Since 2.24>2.131, reject H0. Equivalently, the 95% interval 52.3±2.131×1.025=(50.12,54.48) excludes 50. The evidence suggests a mean above 50.
Traps
- Interpreting a p-value as the probability the null is true.
- Using a one-sided critical value for a two-sided alternative.
- Wrong degrees of freedom, especially in paired or two-sample t tests.
- Confusing statistical significance with practical importance; with large n, tiny effects can be significant.