The Survey With a Missing Option
A team collects responses from 1,200 students about how they get to school. The form offers exactly four choices: car, bus, bike, walk. After cleaning, they report that 62% come by car, 30% by bus, 8% by bike, and 0% walk. A teammate wants to publish the finding 'no students walk to school.' A second teammate notices that the form was distributed only during first-period classes in the north wing of campus, that the response rate was 1,200 of 2,400 students, and that a free-text 'other' field was never included. The team must decide what claim, if any, the data supports.
1. Evaluate the proposed claim 'no students walk to school.' Is it supported by the data? Justify using the specific numbers.
Read the worked answer
No - the claim overreaches on two independent grounds.
- The instrument cannot produce the observation. With no 'other' option, a student who walks has no truthful way to answer, so the 0% may reflect the form's design rather than student behavior. A zero on a question that cannot be answered truthfully is not evidence of zero.
- The sample is not the population. 1,200 of 2,400 students responded, and only from north-wing first-period classes. The 50% who did not respond are unobserved, and their transportation could differ systematically (e.g., students who walk may be less likely to be in those classes, or may have arrived too late to answer).
A defensible claim is narrower: 'Among 1,200 respondents from north-wing first-period classes, 0% selected walk from the four provided options.' That is a statement about the data, not about students.
2. The team wants to improve the study. Describe two concrete changes and explain what each one would and would not fix.
Read the worked answer
Change 1: Add an 'other' / 'prefer not to answer' option and a free-text field. This fixes the instrument problem - it removes the forced-choice bias that made 'walk' unselectable-in-practice. It does not fix sampling: a better question asked of a biased sample is still biased.
Change 2: Sample across the whole campus and follow up non-respondents. This addresses the coverage problem (north wing, first period) and the 50% non-response. It does not fix measurement - if the question is still forced-choice, the answers remain constrained.
Good data work fixes instrument and sampling separately, because each contaminates results in a different way and neither masks the other's flaw.