ED waiting-time headline after a rapid assessment zone
A district health board publishes monthly median ED waiting time from triage to treatment. In the 12 months before a new rapid assessment zone opened, monthly medians ranged from 54 to 72 minutes; in the 12 months after, from 48 to 80 minutes. The posted headline says the median fell by 9 minutes. Two post-period months are flagged internally: one included a cyber incident that slowed registration, and one was the winter respiratory surge. The rapid assessment zone was also accompanied by diverting low-acuity patients to a primary care clinic, so the ED denominator changed. Monthly ED attendances fell from about 7,100 to 6,300. A clinician notes the median is insensitive to long waits. The communications team wants to know whether the 9-minute fall is a real improvement and how to describe it publicly.
1. Assess whether the 9-minute fall supports a claim of improved ED timeliness. Discuss at least three specific alternative explanations.
Read the worked answer
The 9-minute fall is a comparison of two 12-month periods, not a controlled estimate. The denominator changed: attendances fell from 7,100 to 6,300 because low-acuity patients were diverted, so the remaining patients may be sicker and the median could change without any improvement in flow. The median is insensitive to the tail, so long waits may worsen while the median falls. The cyber incident and winter surge could push the post-period median either way; excluding them post hoc would be selective unless pre-specified. Seasonal and secular trends are uncontrolled. The monthly medians also have sampling variability; a 9-minute difference may be within noise. The headline picks the most favourable comparison and ignores the range (48–80 post versus 54–72 pre).
2. Design an analysis and a public statement that would be more defensible. Include what measures you would add and how you would handle the two flagged months.
Read the worked answer
Analyse an interrupted time series with at least 24 pre and 24 post monthly points, including a control hospital or region unaffected by the zone. Model level and slope changes, adjust for seasonality and case-mix, and report the median plus the 90th percentile and proportion seen within target, stratified by triage acuity. Use a statistical process control chart to distinguish signal from common-cause variation. Pre-specify handling of the cyber and winter months: include flag indicators and present sensitivity analyses with and without them, not a single chosen result. The public statement should say what changed, for whom, over what period, with uncertainty, and note the denominator change. Avoid 'waiting times improved by 9 minutes' as a causal claim.
3. A board member asks for a single number 'percentage improvement'. Explain the risks of that request and give a better summary.
Read the worked answer
A single percentage improvement would hide which part of the distribution changed and whether case mix changed. It also invites false precision. Better: report a small dashboard of median, 90th percentile, proportion within target, and ED attendance/case-mix, with confidence intervals or control limits. Explain that a median fall with a rising 90th percentile is not a uniform improvement. If a summary is required, give a range and state the comparison period, the population, and the caveats. Do not let a board-friendly number replace the distributional evidence.