Appraising a Cluster-Randomized Fall Prevention Trial
An 8-ward, 240-bed community hospital is considering a multifactorial fall prevention program. A vendor-supplied cluster RCT randomized four wards to the program and four to usual care (240 patients; 120 per arm). Falls occurred in 18% of usual-care patients and 12% of intervention patients (p=0.04). The trial reports 22% attrition, no blinding of staff or patients, and analyzes outcomes at the patient level although randomization was by ward. At baseline, intervention wards had more patients with dementia (38% vs 27%) and more bed alarms in use. The vendor advertises a “33% relative reduction in falls” and offers a 15% discount if signed within 30 days. Your quality committee has 6 months of local fall data showing 3.1 falls per 1,000 patient-days and a post-fall huddle program already in place. The CNO asks whether to purchase the program.
1. Identify the main threats to internal validity in this trial and explain the likely direction of each threat.
Read the worked answer
Threats cluster around the cluster design. Randomization by ward means patients are not independent; analyzing at the patient level inflates precision and produces a unit-of-analysis error, so the p=0.04 is likely too small. Baseline imbalance is serious: intervention wards had more dementia (38% vs 27%), which should increase falls, yet falls fell more-so confounding could bias either way, but the lack of adjustment leaves the effect uncertain. Attrition of 22% without blinding risks differential ascertainment: staff who know the program may record falls differently or implement co-interventions. No blinding increases detection and performance bias. The vendor’s 33% relative reduction is a crude RR (0.12/0.18 = 0.67), but the absolute reduction is only 6 percentage points, and the confidence interval is not reported. Cluster-level analysis with a random-effects model, adjustment for dementia and bed alarms, and intention-to-treat would likely widen the CI and could cross no effect. The trial is suggestive, not definitive.
2. Interpret the reported effect, including absolute risk and precision, and recommend a course of action for the committee.
Read the worked answer
ARR = 18% − 12% = 6 percentage points; crude RR = 0.67; crude NNT ≈ 17 patients to prevent one fall, but this ignores clustering, attrition, and baseline imbalance, so the true NNT is probably larger and imprecise. The committee should not purchase on a 30-day discount. I would request the cluster-adjusted CI and intraclass correlation, the CONSORT extension for cluster trials, and a per-ward analysis. Locally, the hospital already has post-fall huddles and 3.1 falls per 1,000 patient-days; the vendor program should be compared with strengthening existing interventions. A defensible option is a 6-month pilot on two wards using a stepped-wedge or matched comparison, with fall rate per 1,000 patient-days, injurious falls, dementia-adjusted analysis, staff time, and cost per fall avoided. Reject the claim of 33% reduction as established; report it as a hypothesis. If the pilot shows a clinically important reduction with acceptable cost and workload, then consider adoption.