The Dashboard That Looks Fine
You are the quality director. Your hospital's sepsis mortality dashboard shows a rate of 9.1%, below the national benchmark of 11%. The board has cited this as evidence of strong performance. A nurse manager privately tells you that the sepsis alert fires so often that clinicians ignore it, and that the mortality figure excludes patients who were coded as septic only after transfer to the ICU. She also notes that the hospital changed its sepsis case definition eight months ago. The dashboard has not been updated to reflect the change. The next board meeting is in three weeks.
1. How do you assess the validity of the 9.1% figure, and what do you do before the board meeting?
Read the worked answer
The 9.1% is not trustworthy as presented. Treat it as a measurement problem first.
Validity concerns:
- Definition change (the biggest issue): if the case definition changed eight months ago, the numerator and denominator are not comparable across the reporting period. A rate computed across a definition change is mixing two populations.
- Exclusion bias: excluding patients coded septic only after ICU transfer systematically removes the sickest/latest-recognized cases - which lowers measured mortality. That is a selection artifact, not performance.
- Alert fatigue: a high-firing, low-specificity alert degrades the very behavior the metric is meant to capture.
Before the board:
- Recompute the rate using a consistent definition and stratify by recognition timing (sepsis present on admission vs. hospital-onset).
- Quantify the effect of the exclusion rule by running the rate with and without those patients.
- Review alert firing volume and positive predictive value.
Do not present a number you cannot defend.
2. What do you tell the board, and how do you frame the correction without destroying trust?
Read the worked answer
Frame it as a measurement-integrity issue, not a failure confession.
Message structure:
- State plainly: the 9.1% is not comparable to the benchmark because of a mid-period definition change and an exclusion rule that removes high-risk cases. The prior board citation should be corrected.
- Show the corrected picture: present the stratified rates and explain what they mean.
- Name the systemic drivers: alert fatigue and late recognition are the real quality targets.
- Propose actions with owners and dates: alert redesign, a sepsis recognition bundle, and a standing data-validation review before any metric goes to the board.
Trust framing: boards lose confidence when they discover a metric was wrong and no one said so. Disclosing proactively, with a fix, builds credibility. Add a governance safeguard: every board metric carries a definition and a data-quality footnote.
Judgment tested: measurement literacy and the courage to correct a flattering number.