Hypothesis Testing with Sample Data: Interpreting p-values and Errors
A company claims that the mean weight of its cereal boxes is 500 grams. A consumer group takes a random sample of 40 boxes and finds a sample mean of 495 grams with a sample standard deviation of 12 grams. They conduct a one-sample t-test with null hypothesis (H_0: \mu = 500) and alternative (H_1: \mu < 500). The calculated test statistic is (t = \frac{495-500}{12/\sqrt{40}} \approx -2.635) with 39 degrees of freedom. The p-value is approximately 0.006. The consumer group concludes that the mean weight is less than 500 grams. The company argues that the result is not practically significant because the difference is only 5 grams.
1. Interpret the p-value in the context of this test. What does the p-value of 0.006 tell us? Is the consumer group's conclusion statistically justified at the 1% significance level?
Read the worked answer
The p-value is the probability of observing a sample mean as extreme as 495 grams or less, assuming the null hypothesis is true (i.e., the true mean weight is 500 grams). A p-value of 0.006 means that if the true mean were 500 grams, there is only a 0.6% chance of getting a sample mean of 495 grams or less in a sample of 40 boxes. Since 0.006 < 0.01, the result is statistically significant at the 1% level. Therefore, we reject (H_0) in favour of (H_1), concluding that there is strong evidence that the mean weight is less than 500 grams. The consumer group's conclusion is statistically justified at the 1% significance level.
2. Discuss the difference between statistical significance and practical significance in this context. What additional information might help assess practical significance? Also, explain what a Type I and Type II error would mean here.
Read the worked answer
Statistical significance indicates that the observed difference is unlikely to have occurred by chance if the null hypothesis were true. Practical significance asks whether the difference is large enough to matter in real life. Here, the difference is 5 grams (1% of 500g). Whether this is practically significant depends on context: for a cereal company, a 5g shortfall per box might translate to significant cost savings or consumer dissatisfaction, but it might be within acceptable tolerance. Additional information such as the cost impact, regulatory tolerances, or consumer perception would help. A Type I error would occur if we reject (H_0) when it is actually true-i.e., we conclude the mean is less than 500g when it is actually 500g. A Type II error would occur if we fail to reject (H_0) when it is false-i.e., we conclude there is not enough evidence that the mean is less than 500g when it actually is. The probability of Type I error is the significance level (0.01), and Type II error depends on the true mean and sample size.