The Breakfast and Grades Survey
A school newspaper reports: 'Students who eat breakfast before school get higher grades. We surveyed 80 students in the cafeteria at 8:00 a.m. and found that 68 of them had eaten breakfast. Those 68 reported an average grade of 82%, while the 12 who had not eaten breakfast reported an average of 74%. The newspaper concludes that eating breakfast raises grades by 8 percentage points and urges all students to eat breakfast.'
The survey was conducted on a single Tuesday. Students who arrive late or skip the cafeteria were less likely to be included. The 12 non-breakfast students included several who said they felt unwell that morning. The school has 640 students. A parent asks whether the conclusion is justified and what a better study would look like. The editor has 200 words to respond.
1. Identify two threats to the validity of the newspaper's conclusion and explain how each could produce the 8-point gap even if breakfast has no effect.
Read the worked answer
One threat is selection bias. The sample is 80 students in the cafeteria at 8:00 a.m. on one Tuesday, not a random sample of the 640 students. Students who arrive late, eat at home, skip the cafeteria, or are absent are less likely to be counted. If students with stable morning routines are both more likely to eat breakfast and more likely to have higher grades, the 8-point gap can appear without breakfast causing anything. A second threat is confounding by health: the 12 non-breakfast students included several who felt unwell. Illness can reduce both appetite and test performance, so the non-breakfast group is not comparable. The 8-point difference is therefore a descriptive gap in a selected sample, not an estimated causal effect.
2. The newspaper says breakfast 'raises grades by 8 percentage points.' Explain why this causal wording is not supported by the data, and rewrite the claim as a cautious association.
Read the worked answer
The phrase 'raises grades by 8 percentage points' asserts causation and a precise effect size. The data are observational: the newspaper compared two self-selected groups at one moment. There was no random assignment, no control for prior grades, sleep, illness, or income, and the grades were self-reported. The 8 points is just 82 minus 74 in this sample; it is not the result of an intervention. A cautious rewrite is: 'In a small cafeteria survey, students who reported eating breakfast also reported an average grade 8 percentage points higher than those who did not. The survey cannot show that breakfast caused the difference.' This keeps the number but removes the causal claim.
3. Design a better study in 3-4 sentences that could test whether eating breakfast affects grades. Include one randomization or control feature and one measurement detail.
Read the worked answer
A better study would recruit a random sample of students and randomly assign consenting volunteers to a standardized school breakfast or to continue their usual morning routine for six weeks. Random assignment makes the groups more comparable on sleep, health, and prior achievement. The study should measure grades on common assessments before and after, and record breakfast attendance so the researchers know whether the assigned breakfast was actually eaten. It should also pre-register the main outcome and analysis to prevent cherry-picking. If random assignment is not allowed, a randomized encouragement design could compare students urged to attend breakfast with those not urged, while tracking uptake.