Deciding Whether to Adopt a New Clinical Technology
Your 150-bed hospital is considering purchasing an AI-based sepsis early-warning system. The vendor cites a published study showing a 20% relative reduction in sepsis mortality at a large academic center. The system costs $450,000 per year plus implementation effort. Your current sepsis bundle compliance is 70%, and your EHR vendor offers a simpler, cheaper alerting tool. The IT department is already stretched supporting a recent EHR upgrade. The chief medical officer is enthusiastic; the chief nursing officer worries about alert fatigue, noting the unit already receives dozens of low-value alerts per shift. The board wants a decision within 60 days.
1. What questions would you ask about the vendor's evidence before recommending adoption?
Read the worked answer
I would interrogate the evidence for applicability and validity:
- Setting and scale: the study was at a large academic center. Does the effect generalize to a 150-bed community hospital with different case mix, staffing, and EHR?
- Study design: was it a randomized trial, a before-after study, or a quality-improvement bundle? If the mortality reduction came bundled with other interventions (protocols, staffing, education), the AI tool may not be the active ingredient.
- Outcome definition: what counted as sepsis, and how were alerts acted upon? A tool that fires and is ignored has no effect.
- Alert burden: what is the alert rate, positive predictive value, and false-positive rate? This directly addresses the CNO's concern.
- Workflow fit: does it integrate with our EHR, and what is the implementation and maintenance burden on IT?
- Local validation: can we run it on our own historical data to estimate alert volume and performance before committing?
I would not accept a single-site study as sufficient justification for a $450,000 commitment.
2. Compare the AI system with the cheaper EHR alerting tool. What is your recommendation and rationale?
Read the worked answer
I would recommend a staged approach: pilot the cheaper EHR tool first, with a defined evaluation, rather than committing to the AI system now.
Rationale:
- The binding local constraint is alert fatigue and IT capacity, not a proven lack of sepsis detection. Our bundle compliance is 70%, meaning a meaningful share of the gap is execution, not recognition.
- The cheaper tool can be implemented with existing EHR infrastructure, less IT burden, and a faster timeline.
- The AI system's evidence is from a different setting and may bundle other interventions; the incremental benefit over a well-tuned EHR alert is unproven here.
- A $450,000 annual commitment plus implementation strain is hard to justify without local validation.
Pilot design: run the EHR tool for 90 days on selected units, measure alert volume, PPV, time-to-antibiotics, bundle compliance, and sepsis mortality. If performance is inadequate, revisit the AI system with local data in hand.
3. The CMO is enthusiastic and the board wants a decision in 60 days. How do you manage the decision process and the relationships?
Read the worked answer
I would reframe the 60-day deadline as a decision about the path, not a forced yes/no on the purchase.
- Acknowledge the CMO's goal: we both want to reduce sepsis mortality; the disagreement is about the fastest reliable route, not the goal.
- Propose a decision at 60 days on the pilot and evaluation plan, with the purchase decision deferred to the pilot results. This meets the board's timeline while avoiding an irreversible commitment on weak local evidence.
- Bring data: present alert-volume estimates, IT capacity constraints, and the evidence-applicability concerns in writing.
- Engage the CNO: her alert-fatigue concern is a design requirement for the pilot, which builds alliance.
- Set explicit success criteria in advance so the pilot is not a popularity contest: if the EHR tool meets thresholds, we keep it; if not, we have a documented case for the AI system.
I would document the rationale so the board sees a reasoned process, not resistance to innovation.