What the sleep survey can and cannot say
A school with 640 students on its roll runs a health survey. On one morning, 600 students are present and each records how long they slept the previous night and whether they failed the end-of-term mathematics test. The 40 students absent that day are not surveyed.
Of the 200 students who reported less than 6 hours of sleep, 30 failed mathematics. Of the 400 who reported 6 hours or more, 20 failed.
A student writes in the school paper: "Short sleep triples your risk of failing: 15 % against 5 %. And 60 % of all failures are students who slept under 6 hours, so most failures are caused by late-night study." The vice-principal fears the article will be read as medical advice, and asks for a check of whether the data, as collected, support either sentence.
1. Compute P(fail | short sleep), P(fail | 6 h or more), the relative risk and the absolute difference. Then compute P(short sleep | fail) and P(short sleep) over the surveyed students, and say precisely what the 60 % figure does and does not show.
Read the worked answer
From the table: 50 students failed in total (30 + 20).
- P(fail | short) = 30/200 = 0.15
- P(fail | long) = 20/400 = 0.05
- Relative risk = 0.15/0.05 = 3.0; absolute difference = 10 percentage points
- P(short | fail) = 30/50 = 0.60
- P(short) = 200/600 = 0.333
So the ratio of 3 is arithmetically correct, and the 60 % statement is also arithmetically correct - but it is the reverse conditional. The student has taken P(short | fail) = 0.60 and read it as if it were P(fail | short) = 0.15, a classic inversion. The honest comparison is 60 % of failures sleeping short against a base rate of 33 % of all students sleeping short: short sleepers are over-represented among failures, by roughly a factor of 1.8, not by a factor of 3 in that direction.
The practical point: 10 percentage points of extra risk is real but modest, and the ratio of 3 sounds far more alarming than it is.
2. A classmate claims the survey proves that short sleep causes failure, and suggests the school should run a campaign telling students to sleep more. Evaluate that inference, naming the specific weaknesses of this design and what each one would do to the numbers.
Read the worked answer
The design supports association, not causation.
- Timing/reverse causation. Sleep is recorded for the night before a mathematics test. Anxiety about the test plausibly shortens sleep, so the arrow may run from the test to the sleep rather than the other way. A habit measure (a week's log) is needed.
- Confounding. Illness, part-time work, noise at home and general study habits could all raise both short sleep and failure. Without measuring them, the 15 % versus 5 % contrast cannot be attributed to sleep.
- Selection bias. The 40 absentees are excluded. If absentees include the ill and the most sleep-deprived, both the failure rate and the short-sleep group are distorted, and the direction of the distortion is unknown.
- No control group and no intervention. Nothing here shows what happens if sleep is increased.
A stronger design would follow the same students over a term with weekly sleep logs, include absentees, and record plausible confounders. Only an intervention - for example, a sleep-hygiene programme assigned to some classes - would begin to justify the causal wording.
3. The school repeats the survey properly and finds the same 15 % versus 5 % contrast. Write, in one short paragraph suitable for the school paper, what can honestly be concluded, and state the one extra number a reader would need in order to judge how much the finding matters.
Read the worked answer
A defensible wording: "In this survey, students who slept under 6 hours before the test failed mathematics three times as often as those who slept 6 hours or more (15 % against 5 %). The survey cannot tell us whether short sleep is the cause, since other factors were not measured; it shows a link worth investigating, not a guaranteed effect of going to bed earlier."
The extra number is the absolute size of the effect in students, not the ratio: 10 extra failures per 100 short sleepers. That is what a reader needs in order to judge the practical importance, since a ratio of 3 on a base rate of 0.5 % would mean almost nothing, while 10 percentage points is worth acting on. Reporting only the ratio is the commonest way a real finding gets inflated in a school newsletter.