Survey Majority with Nonresponse and Coverage Bias
A principal sends a schedule-change survey to 400 families randomly selected from the 2,000 families in the school. After two weeks, 228 families respond; 150 of the respondents support the change. The principal announces, 'A clear majority-65.8% of families-supports the new schedule.' A parent notes that only 57% of the selected families responded and that families who oppose the change may have been more motivated to reply. The survey was sent by email, and 85% of families have email on file; district data show that families without email are more likely to be lower-income and may have different schedule needs. The principal asks for a statistical memo before the school board meeting. The board wants to know what the survey actually estimates, whether the majority claim is supported, and what follow-up would strengthen the evidence. The district uses a 95% confidence level.
1. Compute and interpret a 95% confidence interval for the proportion of respondents who support the change. Does the interval support the principal's claim of a 'clear majority' among all families? Explain.
Read the worked answer
p̂=150/228=0.6579. Standard error=sqrt[p̂(1−p̂)/n]=sqrt(0.6579×0.3421/228)=sqrt(0.2251/228)=sqrt(0.000987)=0.0314. For 95% confidence, margin=1.96×0.0314=0.0615. Interval: 0.6579±0.0615=(0.596,0.719), or 59.6% to 71.9%. Among respondents, this supports a majority because the lower bound exceeds 50%. It does not automatically support a 'clear majority among all families,' because the interval applies to the respondent population; nonresponse and coverage problems can shift the true proportion outside this interval.
2. The principal wants to apply the result to all 2,000 families. Identify two sources of bias and explain the direction each might push the estimate. Why is a high response rate not a guarantee of an unbiased estimate?
Read the worked answer
Nonresponse bias: only 228 of 400 responded. If families who oppose the change were more motivated to reply, support is overestimated; if supporters were more motivated, it is underestimated. Coverage bias: the survey was email-only, and 15% of families lack email on file; if those families have systematically different schedule needs, the estimate is biased in a direction determined by their unmeasured opinions. A high response rate reduces nonresponse bias but does not fix coverage bias, question wording, or measurement error. Even a 100% response from the selected sample would not represent families with no email unless they were reached by another mode.
3. Design a follow-up that would allow a more defensible estimate for all 2,000 families. Specify sampling method, contact mode, and one question you would change. Explain how you would report uncertainty.
Read the worked answer
Use a stratified random sample with strata defined by grade level and by email/no-email status, then contact selected families by email plus phone or mail for the no-email stratum. Follow up nonrespondents at least twice. Replace the yes/no support question with a neutral three-option item: 'Do you support, oppose, or are you undecided about the proposed schedule change?' and add one open-ended reason. Report the weighted proportion, the 95% confidence interval, the response rate by stratum, and a nonresponse analysis comparing early and late respondents. If the no-email stratum is small, report a separate estimate for it rather than assuming it matches the email stratum.