The Survey With 38 Responses
The student wellbeing group at a large co-ed school wants to improve access to mental health support. They wrote a 12-question survey and shared it via the school app and two Instagram stories. Of 600 students, 38 responded. Of those 38, 26 (68%) said they "don't know where to get help," and 31 said they would use a drop-in service at lunchtime. The respondents were mostly Year 13 (24 of 38) and mostly students from one whānau house (19 of 38). The group has drafted a proposal to the board asking for $4,000 to set up a lunchtime drop-in, quoting "68% of students don't know where to get help." Their teacher has asked them to check the data before they present. The board meets in three weeks.
1. Evaluate how trustworthy the finding is, and identify two things that affect whether it can be generalised to the whole school.
Read the worked answer
Trustworthiness is low for the claim being made. 38 out of 600 is a 6% response rate. Generalising to "students" requires a sample that resembles the population; this one does not. Two things affect generalisation: (1) non-response bias - students who are struggling or disengaged are less likely to answer an app or Instagram survey, so 68% may over- or under-state the real figure, and we cannot tell which; (2) selection within the sample - 24 of 38 were Year 13 and 19 of 38 came from one whānau house, so the result mostly reflects one year level and one house. A third issue is that "don't know where to get help" was a single item with no definition, so respondents may mean different things.
What the data does support: 26 students in this group said they did not know where to get help, and 31 said they would use a drop-in. That is enough to justify testing a drop-in, not enough to claim 68% of the school. The proposal's wording is the problem, not the idea.
2. Recommend what the group should do before launching the campaign or presenting to the board, and explain your reasoning.
Read the worked answer
Next steps before the board: (1) re-run the survey with a proper sampling method - for example, a tutor-group census in form time, on paper and digitally, with a 70% response target, reported by year and house; (2) add two or three focus groups, deliberately including Year 9s and students who are often absent, to find out what "help" means and what would stop them using it; (3) check whether the school already has a guidance service students do not recognise - the problem may be visibility, not absence.
Reasoning: the board is deciding on $4,000 of recurring money. Presenting a 6% sample as a school-wide figure is the kind of claim that gets a good idea declined, and it can misdirect the money - a lunchtime drop-in may not suit the students who most need it. A small, defensible pilot is more persuasive than an overstated one. The group should also state clearly what it still does not know, because that is what a good inquiry looks like and it makes the request harder to dismiss.
3. If the group had to act now with only $200, what would you advise and why?
Read the worked answer
With $200 and three weeks, I would not launch the service. I would run one tutor-group census (free, on paper) in four tutor groups chosen to cover Year 9–13, and two lunchtime focus groups with kai. The census gives a defensible number; the focus groups give the design. $200 covers printing, kai and maybe a voucher for participants.
Reasoning: the scarce resource is not the $200 - it is the board's trust and the term's time. A defensible 200-student sample with clear questions is worth more than a $200 drop-in that runs twice and closes. If the group were forced to spend on action rather than inquiry, I would spend it on visibility - a simple card or poster listing existing services - because that is low-risk, fast, and tests the hypothesis that the problem is visibility rather than absence. Either way, the decision should be tied to what the evidence can currently support, not to what the group hopes is true.