Reading the quality report: rates, bases and a causal claim
A hospital quality report gives the following figures for one month.
| Clinic | Patients seen | Nurses on duty (total shifts) | Reported medication incidents |
|---|---|---|---|
| A | 1200 | 8 | 24 |
| B | 800 | 5 | 20 |
| C | 1500 | 15 | 30 |
The report's summary states: "Clinic C recorded the most incidents, so it is our least safe unit. Clinic B, with fewer than half of C's incidents, is the safest. The new checking protocol introduced in Clinic B on the first day of the month is therefore clearly working."
1. Calculate incidents per 100 patients and incidents per nurse for each clinic, and rank the clinics by rate.
Read the worked answer
Per 100 patients.
- A: 24 ÷ 1200 × 100 = 2.0
- B: 20 ÷ 800 × 100 = 2.5
- C: 30 ÷ 1500 × 100 = 2.0
Per nurse (8, 5 and 15 shifts respectively).
- A: 24 ÷ 8 = 3.0
- B: 20 ÷ 5 = 4.0
- C: 30 ÷ 15 = 2.0
Ranking. By both denominators the order is B (highest) > A = C. The absolute counts rank C (30) > A (24) > B (20) - almost the reverse of the rate ranking, because the counts track how many patients each clinic sees rather than how often something goes wrong.
2. Evaluate the report's first two sentences. Which clinic is actually the least safe by these figures, and what does that imply about comparing raw counts?
Read the worked answer
The first sentence is a base-rate error. C has the largest count (30) mainly because it treats 1500 patients - 25% more than A and nearly twice B. Once volume is accounted for, C's rate is 2.0 per 100 patients, identical to A's, the lowest in the table. C is not shown to be the least safe; it is shown to be the busiest.
B is the outlier in the other direction. Its 20 incidents look reassuringly small in absolute terms, but against only 800 patients that is 2.5 per 100 - 25% above A and C. Per nurse (4.0) it is double C's. On both normalisations, B is the clinic with the highest incident rate.
Implication. Raw counts are not comparable between units of different size; they measure exposure, not safety. Any comparison must use a rate, and the choice of denominator changes what the rate means:
- per patient answers "how likely is an incident for someone treated here?"
- per nurse answers "how risky is a shift here?" - though the report gives shifts, not hours, so this denominator is only a proxy and would be distorted by part-time staffing.
Reporting the counts alone, as this report does, produces a conclusion that reverses the truth.
3. Evaluate the claim that the new checking protocol is "clearly working". List the specific information you would need before accepting it.
Read the worked answer
The claim is not supported, and the data actively argue against it.
- Wrong direction. B has the highest incident rate in the table (2.5 per 100 patients, 4.0 per nurse). If the protocol were working, you would expect the opposite, so the report is citing B's small absolute count as evidence while ignoring the rate.
- No baseline. There is no figure for B before the protocol. A rate of 2.5 could be an improvement from 5.0 or a deterioration from 1.5; without the prior value the number carries no information about change.
- No comparison group. All three clinics are observed in the same month with no control. Clinics differ in case mix, staffing, acuity and reporting culture, any of which could produce the gap.
- No time for an effect. The protocol was introduced on day 1 and measured within the same month - too short to separate a real effect from month-to-month noise, and vulnerable to regression to the mean if B was chosen because its numbers were bad.
- Reporting artefact. A new checking protocol often increases the reporting of incidents before it reduces them. A rise in recorded incidents may reflect better detection.
Information needed. B's pre-protocol rate over comparable months; whether the definition of "reported incident" changed; patient acuity and case mix; nurse hours rather than shift counts; the number of incidents that reached the patient versus near-misses; and a concurrent comparison clinic that did not adopt the protocol. Until then the honest statement is that B records the highest rate and the cause is unknown.