The Breathing App Trial
A wellbeing team tests a new breathing app on 60 Year 11 students. Thirty volunteers are assigned to use the app for 20 minutes daily; the other 30 continue usual routines. After four weeks, the app group reports lower anxiety on a 10-item self-report scale. However, the app group also received weekly check-in calls from a researcher, while the control group did not. Attendance at school was unchanged in both groups. The team concludes the app reduces anxiety and wants to roll it out to the whole school. A sceptical teacher says the result is 'just placebo'.
1. Identify the design flaw and explain how it could produce the reported result without the app being effective.
Read the worked answer
The app group received weekly researcher calls while the control group did not, so the groups differ on two variables: app use and social contact/attention. The attention could reduce anxiety through support and expectation, a confounding variable. Expectancy and placebo effects also operate: volunteers who chose to participate may expect improvement and report lower scores. Self-report is susceptible to demand characteristics - participants may guess the hypothesis and answer accordingly. So the causal claim 'the app reduces anxiety' is not supported; the design cannot separate app effects from attention and expectancy.
2. Redesign the trial to control for these factors and state what result would support rollout.
Read the worked answer
Give both groups equal researcher contact (e.g., weekly calls), use random allocation to app versus an active control (a sham app or relaxation reading), keep participants and the outcome assessor blind where possible, use an objective or multi-source measure alongside self-report, and pre-register the primary outcome. A result supporting rollout would be a statistically and practically meaningful anxiety reduction in the app group versus the active control, sustained at follow-up, with no greater dropout in one group. A judgement call: even a clean trial in 60 volunteers is a small sample; rollout should be framed as provisional pending replication, not proven.