Attendance Records, a 16% Response Rate and a Causal Claim
A Year 12 student at a Maltese state college is designing an extended essay on why some Year 9 students repeatedly miss school. The college's data officer gives her anonymised attendance records for 240 Year 9 students, of whom 31 missed more than 15 days in the autumn term. She plans to email an online questionnaire to all 240 through the school system, asking about family structure, parents' jobs, bedtime and whether they 'feel school is worth it'. After two weeks she has 38 replies; the mean absence of respondents is 4 days, against a college mean of 9. She then intends to interview the ten students with the highest absence, whose names the data officer reluctantly supplies, and to write in her conclusion: 'Truancy at the college is mainly caused by family breakdown.' Her teacher is worried about both the design and the conclusion.
1. Identify and explain two features of the questionnaire stage that weaken the evidence, and state what each does to the conclusion the student wants to draw.
Read the worked answer
Non-response and self-selection. Thirty-eight replies out of 240 is roughly 16 per cent. Students who read and answer a school email are likely to be those who attend and engage. The respondents' mean absence of 4 days, against a college mean of 9, is therefore not evidence that absence is low; it is evidence that the sample is skewed on the very variable being studied. Because the non-response is not random, the result cannot be generalised to the 240, let alone to the college.
Coverage error. The questionnaire is delivered through the school system, but the persistent absentees - the population of interest - are precisely those least likely to see or answer a school email. The sampling frame (the roll of 240) and the accessible population (those who read school email) diverge, so the group most relevant to the research question is systematically missing.
Leading wording. 'Do you feel school is worth it?' invites a socially desirable answer and presupposes that students hold a single settled attitude.
Effect on the claim: the 38 responses describe a self-selected group of attenders. They cannot support a description of truancy at the college, and they contain no information at all about the absentees whose behaviour the conclusion is about.
2. The student interviews the ten highest-absence students and concludes that family breakdown is the main cause. Explain why this design cannot support that conclusion, and outline a stronger design.
Read the worked answer
Selection on the dependent variable. All ten cases have the same outcome (very high absence). With no variation in the outcome there is nothing to compare, so no cause can be identified - the design can only describe, not explain.
No comparison group. Without regular attenders from the same year and classes, the student cannot tell whether the family circumstances she hears about are more common among absentees than among everyone else. If half the year group also has separated parents, the 'cause' disappears.
Retrospective accounts. Interviews produce students' own explanations, which may be rationalisations offered to an adult in a school setting.
Truancy is not defined. Many of the 31 may have authorised absence for illness. A stronger design would separate authorised from unauthorised absence using the records, then compare a sample of high-absence students with a matched sample of low-absence students in the same classes, using a short questionnaire plus a few interviews sampled across the whole range of attendance, and triangulating with form-teacher and parent accounts.
3. The data officer supplies the names of the ten students. Discuss the ethical issues this raises and say how the student should proceed.
Read the worked answer
Consent. The students are 13 or 14. Valid consent requires parental or guardian consent plus the young person's own assent; either alone is insufficient. Participation must be genuinely voluntary, which is hard in a school where an adult's request can feel compulsory, so the invitation should make clear it is not disciplinary and will not be seen by year tutors.
Confidentiality versus anonymity. The questionnaire promised anonymity; the interviews cannot, because the researcher will know who is speaking. This must be explained in advance, along with the limits of confidentiality: if a student discloses abuse or serious neglect, the researcher has a safeguarding duty to pass it on. Promising absolute secrecy would be unethical because it could not be kept.
Data protection. Names are personal data; under data protection rules she should collect only what is necessary, store identifiers separately from responses, and delete them when the study ends. She needs gatekeeper permission from the head and the college's data protection officer, and ethics approval from her institution.
Proceeding: pseudonymise records, interview volunteers from among the absentees rather than the ten she has named, offer a neutral room and a known adult nearby, and tell participants they may stop at any time.