The Online Subscription Trial
A company tests a new website design to increase the proportion of visitors who sign up for a free trial. In a random sample of 200 visitors to the old design, 40 signed up. In a random sample of 250 visitors to the new design, 65 signed up. The company wants to know if the new design leads to a higher sign-up rate. They plan to conduct a two-proportion z-test at the 5% significance level. However, a data analyst notices that the sample sizes are not equal and wonders if that affects the validity of the test.
1. State the null and alternative hypotheses for the test, and check whether the conditions for inference are met.
Read the worked answer
Let p1 be the true proportion of sign-ups for the old design and p2 for the new design. Hypotheses: H0: p1 = p2 (or p1 - p2 = 0) versus Ha: p1 < p2 (or p1 - p2 < 0). Conditions: Random samples: met (both samples are random). Independence: The samples are independent of each other, and the 10% condition is met if the populations of visitors are large (more than 2000 and 2500 respectively). Normality: Check the success-failure condition using the pooled proportion. The pooled proportion is (40+65)/(200+250)=105/450≈0.2333. Then np̂ = 200(0.2333)=46.67, n(1-p̂)=200(0.7667)=153.33; and for the second sample, 250(0.2333)=58.33, 250(0.7667)=191.67. All are at least 10, so the condition is met. Unequal sample sizes do not invalidate the test.
2. The test statistic is calculated as z = -1.62 with a p-value of 0.0526. Interpret the p-value in context and make a conclusion at the 5% significance level.
Read the worked answer
The p-value of 0.0526 means that if there were no difference in the true sign-up rates (p1 = p2), there is about a 5.26% chance of observing a difference in sample proportions as large as or larger in the direction of the alternative (old minus new ≤ -0.075) purely due to random variation. Since 0.0526 > 0.05, we fail to reject the null hypothesis. There is not enough evidence at the 5% significance level to conclude that the new design has a higher sign-up rate than the old design.