Sleep survey: sampling bias and conditional probability
A student at a Belfast secondary school wants to find out whether pupils get enough sleep. At 11 pm on a Sunday she posts a survey link in a Year 11 social media group, and 40 pupils reply. She writes in the school newsletter that '70% of students get less than 8 hours of sleep a night'. The school has 240 pupils across Years 11 and 12, of whom 150 are in Year 11. Her 40 replies show 28 pupils getting less than 8 hours; of those 28, 20 are in Year 11. Of the 12 pupils getting at least 8 hours, 4 are in Year 11. A separate random sample of 50 pupils, selected so that year groups are represented in proportion to their size, finds 26 pupils getting less than 8 hours. The student concludes that Year 11 pupils sleep less than Year 12 pupils, and that the school should start an hour later.
1. Give three specific reasons why the 70% figure is unlikely to be reliable, and describe one improvement to the method.
Read the worked answer
Specific problems with this sample:
- Self-selection: pupils choose whether to reply, and those who are awake at 11 pm on a Sunday are more likely to be short of sleep. The sample is biased towards the very group being measured.
- Coverage: the link is only in a Year 11 group, so Year 12 is excluded entirely. A claim about 'students' at the school cannot be based on Year 11 alone.
- Sample size and response: 40 replies out of 240 pupils is under a fifth of the population, and non-response is high, so random variation and non-response bias are both large.
Improvement: select pupils at random from the full school list (or use a stratified random sample by year group) and follow up non-responders, rather than relying on a social media poll at night.
The key judgement is that the 70% is a description of 40 volunteers, not an estimate for the school.
2. Use the random sample of 50 to estimate how many of the 240 pupils get less than 8 hours, and show how a stratified sample of 50 would be split between the year groups.
Read the worked answer
From the random sample, the proportion is 26 ÷ 50 = 0.52, or 52%.
Estimated number in the school = 0.52 × 240 = 124.8, so about 125 pupils.
Stratified split of 50 in the ratio 150 : 90 (Year 11 : Year 12):
- Year 11: (150 ÷ 240) × 50 = 31.25, so 31 pupils
- Year 12: (90 ÷ 240) × 50 = 18.75, so 19 pupils
31 + 19 = 50, and the rounding is adjusted so the total matches the required sample size.
The estimate of about 125 is far below the 70% claim: 0.70 × 240 = 168 pupils. The difference (about 43 pupils) shows how misleading the biased sample is.
3. From the student's own 40 replies, work out P(Year 11 | under 8 hours) and compare Year 11 and Year 12 rates. Does the data support her conclusion?
Read the worked answer
Her table, from the figures given:
- Under 8 hours: 28 in total, 20 in Year 11 and 8 in Year 12
- At least 8 hours: 12 in total, 4 in Year 11 and 8 in Year 12
- Year 11 total = 24, Year 12 total = 16
P(Year 11 | under 8 hours) = 20 ÷ 28 = 5/7 = 0.714 (71.4%)
For a fair comparison, compare rates within each year group:
- Year 11 under 8 hours: 20 ÷ 24 = 0.833 (83.3%)
- Year 12 under 8 hours: 8 ÷ 16 = 0.500 (50.0%)
Her figures do suggest a difference, but the conclusion is not safe. The groups are small (24 and 16), the sample is self-selected and taken at 11 pm, and Year 12 is barely represented. A difference of this kind could easily arise from bias rather than from a real year-group effect.
A causal claim that a later start would improve sleep also goes beyond the data: this is an observational survey, so it cannot establish cause. A stratified random sample with a larger Year 12 group would be needed before any timetable decision.