The Dashboard Nobody Trusts
A quality department publishes a monthly readmission dashboard used by six service lines. The dashboard pulls from the data warehouse, which refreshes nightly. Two service line leaders report that the dashboard's 30-day readmission rate (11.2%) disagrees with the rate they calculate from their own EHR reports (9.4%). Investigation finds: (1) the warehouse excludes readmissions to outside hospitals, (2) the dashboard counts observation stays as readmissions while the service line reports do not, and (3) the warehouse's nightly load occasionally drops records when the source system is down at 2 a.m. The quality director wants to 'just pick one number and move on.' Clinicians are beginning to ignore the dashboard entirely.
1. How do you reconcile the two numbers, and what do you tell the quality director who wants a single number?
Read the worked answer
The two numbers measure different things, so 'picking one' without defining the measure would institutionalize a misleading metric. Reconcile by making the definition explicit and reproducible.
Reconciliation steps:
- Define the measure precisely. Specify: index population, 30-day window, inclusion/exclusion of observation stays, inclusion of outside-hospital readmissions, and denominator rules. Adopt a standard definition (e.g., aligned to a national specification) and document it.
- Quantify each discrepancy. Attribute the 11.2% vs 9.4% gap to the three known causes: outside-hospital exclusion, observation-stay handling, and dropped records. Show the arithmetic so leaders see where the difference comes from.
- Fix the data defect. The dropped-record issue is a reliability bug, not a definitional choice; add load monitoring, retry logic, and a completeness check that blocks publication if source feeds are incomplete.
- Publish a data dictionary alongside the dashboard so the definition travels with the number.
Message to the director: A single number is fine only if it is a defined, validated number. Recommend one governed definition for the enterprise dashboard, with a footnote on known limitations (outside-hospital readmissions), and a separate view for service-line operational use if needed. Transparency restores trust; an unexplained number does not.
2. What data quality dimensions should you monitor, and how would you design ongoing validation so this does not recur?
Read the worked answer
Data quality dimensions to monitor:
- Completeness: proportion of expected records present; feed-level record counts vs source counts; detect the dropped-record pattern.
- Accuracy: sampled chart validation against source; agreement rate.
- Consistency: cross-system agreement on the same patient/encounter (e.g., warehouse vs EHR).
- Timeliness: refresh latency and on-time publication.
- Validity: values within expected domains; conformance to the data dictionary.
- Uniqueness: duplicate rate.
Ongoing validation design:
- Automated reconciliation controls at each load: row counts, checksums, and a completeness gate that halts publication and alerts the data team if a source feed is short.
- A monthly sampled audit (e.g., 30–50 charts) comparing dashboard output to source documentation, with results reported to the data governance committee.
- Definition versioning: any change to the measure definition is logged with an effective date and communicated to users.
- User feedback channel with triage SLA, so clinicians can report discrepancies and get a response.
- Dashboards display data vintage and known limitations, so users can interpret the number correctly.
Tie these to a data governance charter with named owners for each dimension.