Choosing a sample and writing a questionnaire in a sixth form
A sixth form has 240 students: 140 in Year 12 and 100 in Year 13. A student wants to estimate the mean number of minutes of homework done per night by the whole sixth form. Three plans are proposed. Plan A: post a link in the Year 13 group chat. Plan B: stand at the library door from 08:00 to 08:30 and ask every fourth student who comes in. Plan C: number an alphabetical list of all 240 students and use a random number generator to select 40. The questionnaire asks: 'How long do you spend on the unreasonable amount of homework you are set each night?' with tick boxes 1–2, 2–3, 3–4, 4+ hours.
1. Which of the three plans gives the most reliable estimate of the mean for the whole sixth form? Give one specific reason why each of the other two plans will not.
Read the worked answer
Plan C is the only one that samples the whole population at random.
- Why C works: the alphabetical list is a complete sampling frame, so every one of the 240 students is listed exactly once and has an equal chance of being selected. The random number generator removes the researcher's choice, so the sample is unbiased, and 40 out of 240 is large enough that the sample mean should be reasonably close to the population mean.
- Why A fails: only Year 13 students are in the chat, so all 140 Year 12 students have no chance of being selected (undercoverage). On top of that, only students who see the message and choose to reply take part - that is a self-selecting (volunteer) sample, and volunteers may differ systematically from the rest.
- Why B fails: students who arrive by another entrance, arrive after 08:30, or never use the library are excluded, so the sample is not drawn from all 240. Taking 'every fourth student' is systematic sampling, but the order in which students enter the library at that time is not random with respect to homework habits - early library users are likely to be more studious, so the estimate will be biased upwards.
2. Give two faults with the survey question and its tick boxes, and write a corrected version that would allow the mean to be estimated.
Read the worked answer
Faults:
- Leading (loaded) wording. 'The unreasonable amount of homework' tells the respondent what answer is expected and invites exaggeration, so responses will be biased upwards.
- Overlapping and incomplete boxes. A student doing exactly 3 hours fits both '2–3' and '3–4', so the data are ambiguous; there is no box for less than 1 hour, so a student doing 30 minutes cannot answer at all.
- Also worth noting: the stem asks in minutes but the boxes are in hours, and 'each night' requires recall over a variable week rather than one defined night.
Corrected version:
How many minutes did you spend on homework last night? Tick one box.
0–29 / 30–59 / 60–89 / 90–119 / 120 or more
The boxes do not overlap and cover every possible value. Because the answers are grouped, the mean is estimated by using the midpoints of the boxes (and a sensible value such as 150 for the open-ended top box), so the result is an estimate rather than an exact mean.
3. The 40 replies give a mean of 96 minutes, and a governor announces that 'the sixth form averages 96 minutes of homework a night'. Give two reasons why this statement may still be wrong even though sampling was done properly.
Read the worked answer
- Sampling variability: a different random sample of 40 would almost certainly give a different mean. The sample mean is only an estimate of the population mean, and with only 40 out of 240 the likely error could be several minutes either way.
- Non-response: the 40 who replied are not necessarily the 40 selected - students who ignored the link may do much less (or much more) homework, so the responders may not be representative.
- Measurement error: self-reported homework time is a guess, not a measurement, and students tend to round or exaggerate.
- Atypical night: one particular night may be unusual (a deadline day, a day with a late fixture), so it may not represent the typical night the governor is describing.
- Grouped midpoints: if the answer was calculated from tick-box midpoints, it is an approximation of the sample mean.
Any two of these, each explained in terms of how it distorts the estimate, earn full credit.