The Mindfulness App Trial at Te Kura o te Awa

Threat 1: history/maturation confounded with time. The design is a single-group pre-post comparison, so the 'treatment' is indistinguishable from everything else that happened between March and September. Term 3 coincides with derived grade examinations and external assessment deadlines, which is plausibly the single largest driver of assessment anxiety in the year. Anxiety scores could rise because of the exams despite a real app effect, or fall because students have finished externals by the second testing point. There is no comparison group, so there is no way to separate the app from the calendar.

Threat 2: self-selection and differential attrition. The 62 students who declined were removed entirely rather than analysed. Students who decline a wellbeing app are not a random subset - they are plausibly more anxious, more avoidant, or more cynical about school initiatives, and the most anxious students are also the most likely to drop out of the app partway through. Restricting analysis to engagers and completers inflates the apparent effect because the remaining group is systematically different before the app is even switched on. This is compounded by the researcher's stake and the visibility of app minutes to form teachers.

Why these outrank the alternatives. Regression to the mean, demand characteristics and reactivity on a self-report scale are all real, but each is a secondary distortion layered onto the two structural problems. Neither the calendar confound nor the selection bias can be removed by better measurement or better statistics after the fact; they make the comparison itself uninterpretable. Researcher bias matters too, but it operates mainly by magnifying a result that is already uninterpretable.

Use a randomised waitlist design. Randomly assign students within each form class to 'app now' or 'app in Term 3' using a concealed allocation generated by someone who does not teach them. Randomisation is the change that does the most work: it makes the two groups comparable on anxiety, motivation and class-teacher effects before the app starts, which converts the calendar problem from fatal to manageable, because both groups experience the same exam period.

Keep both groups in the analysis. Analyse by allocated group (intention-to-treat) rather than by minutes used. Otherwise the compliant-subset bias returns.

Separate the researcher from the data. The deputy principal should not collect or analyse her own students' data. Have data de-identified and coded by an independent person, and analyse blind to condition. This removes expectancy effects on scoring and on informal influence over students.

Fix the measure and the timing. Use a validated instrument with published reliability rather than a bespoke 10-item scale, administer at identical time of day and term week for both groups, and add at least one non-self-report indicator (e.g. attendance on assessment days, or a brief physiological or behavioural measure). Self-report alone cannot distinguish a change in anxiety from a change in willingness to admit anxiety.

State the hypothesis and analysis in advance, including a power calculation. With roughly 124 per arm, the school can detect only a moderate effect; it should decide in advance what size of effect would justify funding, so the decision is not made after seeing the data.

The core problem is voluntariness. The deputy principal has authority over these students' assessment, and form teachers have pastoral authority. When a student knows their non-use will be visible to those people, 'consent' becomes compliance. The Health and Disability Code of Rights and standard research ethics both require consent that is free from coercion, and this design builds the coercion into the data flow. Reporting minutes also breaches confidentiality in a way that is disproportionate to the purpose: the research question is about anxiety, not about compliance monitoring.

There is a second, subtler problem: stigma and secondary harm. Students who disengage from a mental-health intervention can be marked out as not trying, which may worsen the very anxiety being studied.

Resolution. Consent must be genuinely refusable - students should be able to decline or withdraw without any academic or pastoral consequence, and the withdrawal must be invisible to teachers. Data should be de-identified before it leaves the research team, with the app logging aggregated to the group level only. If the school genuinely wants engagement monitoring for pastoral reasons, that should be a separate, transparent process with its own consent, not smuggled in as a research instrument. Most Year 13 students are 17 and can consent on their own behalf, but the power relationship still makes independent, written, plain-language information and an opt-in process essential. The school should also consider what happens if a student's responses suggest significant distress, and state that duty of care pathway in advance.