Discharge Protocol: Simpson's Paradox, Standardisation, and Effect Measures
A hospital trust reviews whether a new discharge protocol reduced 30-day readmissions. Over 12 months, 1,000 patients followed the protocol and 116 were readmitted (11.6%); 1,000 patients discharged before rollout formed the comparison group and 214 were readmitted (21.4%). A Pearson chi-square for the 2×2 table gives χ² = 34.8, p < 0.001, and the audit summary concludes the protocol cut readmissions by 9.8 percentage points. The protocol was introduced first on the medical ward, where baseline readmission risk is low, and only later on the surgical ward, where baseline risk is roughly 25%. Of protocol patients, 800 were medical and 200 surgical; of comparison patients, 200 were medical and 800 surgical. Ward-specific rates: medical ward, protocol 64/800 = 8.0% versus comparison 14/200 = 7.0%; surgical ward, protocol 52/200 = 26.0% versus comparison 200/800 = 25.0%. A secondary table cross-classifies 96 adverse events by ward (2 levels) and severity (4 levels), with expected counts below 5 in 5 of the 8 cells.
1. Reconcile the crude result with the ward-specific results. Compute a standardised comparison and state what the data show.
Read the worked answer
The crude rates are weighted by very different ward mixes. 80% of protocol patients were low-risk medical patients, while 80% of comparison patients were high-risk surgical patients. The crude comparison is mostly a comparison of wards.
Direct standardisation to the combined ward distribution (1,000 medical, 1,000 surgical; weights 0.5/0.5):
- protocol: 0.5 × 8.0% + 0.5 × 26.0% = 17.0%
- comparison: 0.5 × 7.0% + 0.5 × 25.0% = 16.0%
Adjusted difference = +1.0 percentage point against the protocol; adjusted risk ratio 17.0/16.0 = 1.06. Compare the crude difference of −9.8 pp (crude RR 11.6/21.4 = 0.54).
Note that the standardised difference equals the stratum-specific differences: +1.0 pp in the medical ward and +1.0 pp in the surgical ward. There is no variation in effect across wards to explain; the crude and every stratum-specific estimate point in opposite directions. This is a textbook case of Simpson's paradox, and the χ² = 34.8 mostly measures the ward imbalance.
The data therefore show no evidence that the protocol reduced readmissions. If anything, the point estimate is 1 pp worse in both wards. Precision: the medical-ward comparison (64/800 vs 14/200) has SE ≈ √(0.08×0.92/800 + 0.07×0.93/200) ≈ 0.021, so a 1 pp difference is indistinguishable from zero. Report "no detectable effect; crude comparison confounded".
2. Identify the confounding mechanism, state which estimate should be published, and comment on the risk-difference versus risk-ratio choice.
Read the worked answer
The confounder is ward. Ward is strongly associated with the exposure (rollout order: medical first, so protocol patients are disproportionately medical) and strongly associated with the outcome (baseline readmission ~7–8% medical versus ~25% surgical). That is exactly the definition of a confounder, and it is severe enough to reverse the sign - not a minor imbalance to be adjusted away casually.
What to publish:
- The ward-specific and standardised estimates (+1.0 pp adjusted), with a confidence interval, alongside the crude figures and an explicit explanation of why they differ. Never publish the crude 9.8 pp "reduction" alone.
- A clear statement that the historical comparison group carries its own temporal confounders: secular trends in readmission, learning effects as staff adopt the protocol, and case-mix drift over the 12 months. The design cannot separate the protocol from these. A defensible study would use concurrent controls with matching or stratification, or a difference-in-differences analysis if pre/post data are available for both wards.
- Check whether the wards differ in other ways that matter (age, comorbidity, discharge destination). If ward is a proxy for these, stratification by ward only partly controls confounding.
Risk difference versus risk ratio:
- Quote the adjusted risk difference (≈ +1 pp, CI spanning zero) for policy statements, since decisions are about numbers of readmissions avoided.
- Do not lead with a risk ratio. Here a roughly constant risk difference on baselines of 7% and 25% translates into very different ratios (1.14 in the medical ward versus 1.04 in the surgical ward), so an RR quoted without the baseline invites misreading. When baselines differ this much, the RD is the more stable and interpretable summary; report both, labelled.
3. The secondary table has 96 adverse events in a 2×4 layout with expected counts below 5 in 5 of 8 cells. What is the appropriate analysis, and what should be avoided?
Read the worked answer
With 96 events across 8 cells the average expected count is 12, so the problem is uneven marginal totals rather than small n overall. The asymptotic χ² approximation is unreliable where several expected counts are below 5.
Options, in order of preference:
- If severity is ordered (it usually is), use a trend test such as Cochran–Armitage rather than an unordered χ². It uses one degree of freedom, is more powerful against a monotone gradient, and is far less sensitive to sparse extreme cells.
- Use an exact or Monte Carlo test for the 2×4 table (Fisher–Freeman–Halton exact test, or a permutation p-value with a fixed seed and enough resamples). At this table size exact computation is feasible.
- Collapse adjacent severity levels only if the collapsing rule is clinically defensible and stated in advance; report the collapsed and uncollapsed results together.
What to avoid:
- Do not merge categories purely to push expected counts above 5. Arbitrary collapsing can destroy the gradient being tested and inflate type I error if the choice is data-driven.
- Do not collapse across ward: that reintroduces the confounding identified in the readmission analysis and would rebuild the Simpson's paradox in a new table.
- Do not substitute the likelihood-ratio χ² as a rescue; it carries the same large-sample requirement.
Reporting: name the test used, give exact counts as well as percentages, state how sparse cells were handled, and present the p-value with the effect estimate rather than the p-value alone.