Interpreting Conflicting Evidence on Breakfast and Concentration
A Year 11 student is investigating whether eating breakfast improves concentration. She finds three sources: (1) a cross-sectional survey of 500 students showing breakfast eaters reported better concentration; (2) a randomised controlled trial of 80 students where half were given breakfast and half were not, with a 20-minute attention test; the breakfast group scored higher, but the difference was small and the confidence interval crossed zero; (3) a blog post citing a single study and claiming 'breakfast makes you 30% smarter'. Her teacher asks her to evaluate the evidence and design a small school-based investigation.
1. Evaluate the strength and limitations of each of the three sources.
Read the worked answer
Source 1 - cross-sectional survey (500 students):
- Strength: large sample, easy to collect, shows an association.
- Limitations: cannot establish causation (correlation only); confounding - breakfast eaters may differ in income, sleep, routine or parental support, any of which could explain concentration; self-report bias in both breakfast and concentration measures; no control for prior ability.
Source 2 - randomised controlled trial (80 students):
- Strength: randomisation distributes confounders between groups, so it supports causal inference far better than the survey; objective attention test reduces self-report bias.
- Limitations: small sample (80) gives imprecise estimates; the confidence interval crossing zero means the effect is not statistically significant - the data are compatible with no effect; short 20-minute test may not reflect real classroom concentration; blinding is difficult (students know if they ate); short duration.
Source 3 - blog post:
- Weakest. Single study, no citation detail, '30% smarter' is a distorted claim - attention tests are not 'smartness', and a single study cannot support a population claim. Likely sensationalised and not peer-reviewed.
Overall ranking: RCT > cross-sectional survey > blog. The RCT is strongest but inconclusive; the survey is suggestive but confounded; the blog should be disregarded as evidence.
2. Design a small investigation the student could run at school to test the question more rigorously, addressing the weaknesses you identified.
Read the worked answer
Research question: Does eating breakfast before school affect performance on a 10-minute attention task in Year 11 students?
Design:
- Randomised crossover trial - each participant acts as their own control, which controls for individual differences. Assign students randomly to 'breakfast' or 'no breakfast' on day 1, then swap on day 2, with a washout of at least one day.
- Standardised breakfast: same food and portion for everyone (e.g. 40 g oats with 200 mL milk), eaten at a set time, to remove variation in breakfast type.
- Standardised test: same attention task, same time of day, same room and conditions, to control for time-of-day and environment effects.
- Blinding: the researcher scoring the tests should not know which condition each student was in (single-blind).
- Sample: aim for at least 30 participants; report the number and any exclusions.
- Controls: record sleep hours, prior-night dinner, caffeine intake and whether students usually eat breakfast, as these are potential confounders.
Analysis: compare mean scores between conditions using a paired test; report the difference and a confidence interval, not just 'better' or 'worse'.
Ethics: informed consent from students and parents, no student forced to skip breakfast if it causes distress, and a plan to feed all participants after testing.
3. The student's results show a mean difference of 0.8 points out of 20, with a wide confidence interval. How should she report this, and what cautions apply?
Read the worked answer
Reporting:
- State the estimate and its uncertainty: 'a mean difference of 0.8 points out of 20, with a wide confidence interval' - do not report it as a definite effect.
- Distinguish statistical from practical significance: 0.8/20 is a small effect; even if statistically significant, it may not matter for real classroom concentration.
- Avoid causal overreach: a small school sample with a short task cannot support claims about academic performance or 'smartness'.
- Contextualise against the literature: note that her result is consistent with the RCT's small, uncertain effect, strengthening the overall picture that any effect is modest.
Cautions:
- Small sample and short duration limit generalisability beyond her school.
- Demand characteristics: students may guess the hypothesis and perform accordingly.
- Crossover carryover: if the washout was too short, day-1 effects may contaminate day 2.
- Confounders not fully controlled: sleep and caffeine may still differ between conditions.
- Ethical framing: do not publicise 'breakfast doesn't matter' - the honest conclusion is that the evidence for a concentration effect is weak and mixed, and breakfast still matters for overall nutrition.
Conclusion to report: 'The effect was small and uncertain; this study does not provide strong evidence that breakfast improves attention, but it does not rule out a modest benefit.'