The Board Wants Proof After Six Months
A community health center enrolls 220 adults with type 2 diabetes and at least one avoidable ED visit in the prior year into a telephonic case management program. At six months: 30-day readmissions fell from 18% to 15%; ED visits rose from 1.9 to 2.1 per patient-year; the proportion with HbA1c above 9% fell from 42% to 33%; only 58% of enrollees completed a care plan; and actual caseload is 1:85 against a planned 1:50. Two other initiatives launched in the same quarter: a pharmacist-led medication reconciliation clinic and extended evening clinic hours. The board asks whether to expand the program to 600 patients. The medical director says the readmission drop proves the program works.
1. What is wrong with the conclusion that the readmission drop proves the program works?
Read the worked answer
Several things.
- Size. 18% to 15% of 220 patients is roughly 40 events down to 33 - seven fewer readmissions. With denominators that small the confidence intervals overlap substantially; the change is compatible with noise.
- Selection and regression to the mean. Enrollees were chosen because they had a prior avoidable ED visit. High utilizers drift back toward the mean on the next measurement regardless of intervention. Some of the drop would have happened with no program at all.
- No comparison. There is no concurrent control, no pre-specified baseline, and no interrupted time series. Two other initiatives started the same quarter, and admissions have seasonal patterns.
- Measurement drift. If the definition of a readmission, the attribution window, or data capture changed, the rate changed for reasons unrelated to care.
- Counter-evidence ignored. ED visits rose, care plans were completed for only 58%, and caseload is 1:85 against a planned 1:50. A program delivered at roughly 60% of designed dose, with a worsening balancing measure, is not a demonstrated success.
The honest statement to the board: favorable movement, not yet distinguishable from noise or secular trend.
2. Design the measurement set the board should see before deciding on expansion.
Read the worked answer
Use structure-process-outcome, and add balancing measures - the program's weakest link is dose, and dose is a process measure.
Structure. Staffing ratio and credentials, caseload per case manager, training hours, registry and EHR capability.
Process. Care plan completed within 30 days; 7-day post-discharge follow-up visit; medication reconciliation completed; reach rate (attempts versus completed contacts); proportion of the eligible population identified by a written risk rule.
Outcome. All-cause 30-day readmission with an explicit definition (index admission, planned readmissions and hospice excluded); ED visits per patient-year; HbA1c below 8% and above 9% on the same panel; blood pressure control; patient-reported outcomes; cost per enrollee.
Balancing. Staff turnover and burnout, patient experience, unintended disenrollment, and whether other populations were neglected because staff were pulled into this program.
For every measure, write the numerator, denominator, exclusions, source, frequency, and owner. Display as a run chart or statistical process control chart over at least 12 months, not as a before-and-after bar. Note that the HbA1c shift is the strongest signal in the current data - and check that the same patients were retested, since differential testing can produce that pattern by itself.
3. The medical director later says, 'Our patients are sicker than the benchmark's, so the comparison is unfair.' How should the case manager respond?
Read the worked answer
The claim is testable, so test it rather than argue about it.
Define 'sicker.' Measure it: HCC risk scores, LACE index, prior utilization, and social risk (housing, food, transportation, insurance churn). Compare the center's case mix to the benchmark's.
Choose the right comparison. An observed-to-expected ratio, stratification by risk tier, or an internal comparison of enrolled versus eligible-but-not-enrolled patients is fairer than a raw rate against a national number.
One caution. Risk adjustment can absorb disadvantage. If social risk is folded into the model, the center quietly stops being accountable for the patients who are hardest to serve. The better practice is to adjust for clinical case mix and then report outcomes separately by social risk, so disparities become visible instead of invisible.
Also ask which benchmark. A national all-condition readmission rate is not a fair peer for a center serving a high-poverty panel; a peer group of similar centers, or a trend over time, may be.
Finally, name the ambiguity. A legitimate case-mix caveat and an excuse look identical until someone measures. Ask the medical director to sponsor the measurement - that request usually distinguishes the two.