The Sepsis AI Pilot: Evidence, Stopping Rules, and a 10-to-2 Vote
A regional health system's AI governance committee is reviewing a third-party sepsis early-warning model that runs inside the EHR and pushes a risk score to nurses' mobile devices. The vendor reports an AUROC of 0.87 from a single-site retrospective study; there is no prospective external validation, and the training data came from an academic medical center with a different case mix. Local nurse leaders worry about alert fatigue and about nurses being blamed for ignoring the alert. The vendor offers a 90-day free pilot, and the CMO is enthusiastic. You are one of two nurses on a 12-member committee, and the vote is in three weeks.
1. What evidence and safeguards would you require before endorsing even a free 90-day pilot? Organize your requirements by category.
Read the worked answer
Free is not the relevant cost; nurse attention and patient safety are.
Performance evidence. External prospective validation at a site demographically and operationally like ours. Calibration, not just discrimination - an AUROC of 0.87 says nothing about whether a score of 0.6 means 60%. Positive predictive value at our sepsis prevalence, which is usually far lower than a study cohort's, and subgroup performance by age, sex, race, and language. Projected alert volume per nurse per shift, stated as a number, not a range.
Actionability. What is the nurse expected to do, within what timeframe, with what escalation path, and does that action differ from what our existing screening already triggers? An alert with no distinct action is noise.
Safety. Behaviour with false negatives, interaction with the current sepsis screen (two competing alerts is worse than one), override capture with structured reasons, and a plan for the patient in whom the alert is dismissed and deterioration follows.
Governance and contract. Data use agreement, PHI handling and vendor access, model version control and notification of retraining, liability and indemnification language, and exit terms including data return.
Evaluation. Pre-specified metrics and a comparison unit, agreed before launch.
Workforce. Education, a named nurse champion per unit, and explicit language that override with documented rationale is clinically appropriate, not non-compliance.
2. How would you design the pilot's monitoring and stopping rules, and who should own them?
Read the worked answer
Pre-register the metrics before go-live, so that success and failure cannot be redefined afterward: alert volume per nurse per shift; PPV; override rate with reason distribution; time to antibiotics and to fluids; ICU transfer and mortality as balancing measures; subgroup performance; measured minutes of added nurse workload; and a short staff survey at day 30 and day 90.
Comparison: run on two intervention units with two matched comparison units rather than system-wide, so an effect can be estimated at all.
Stopping rules must be numeric and automatic. For example: sustained override rate above 80% for two consecutive weeks; alert burden above a stated ceiling per nurse per shift; PPV below the value used in the business case; any serious safety event attributable to a missed or erroneous alert; or failure of subgroup performance to meet a pre-agreed floor. Include a rule that a single sentinel event pauses the pilot pending review.
Ownership: shared and named - the nurse informaticist or clinical lead owns workflow and safety signals, the CMIO owns model performance, quality owns outcomes - but the authority to pause must sit with the governance committee, not the vendor and not a single executive sponsor. Write it into the pilot agreement. Weekly dashboard review by a subgroup that includes at least two bedside nurses, with minutes circulated.
3. You are outvoted 10 to 2 and the pilot is approved. What are your professional obligations, and what would you do?
Read the worked answer
Being outvoted does not dissolve accountability, but it does change the form it takes.
Document. Ask that your dissent and your conditions be entered in the minutes verbatim, and state which specific safeguards you consider prerequisites. A recorded dissent is not obstruction; it creates the record that later informs review.
Convert dissent into conditions. Move the committee to attach your stopping rules, the comparison design, and the nurse co-ownership of the pause authority as formal conditions of approval. Most committees will accept safeguards they would reject as objections.
Escalate only if warranted. If the conditions are refused and the risk is material, raise it in writing to the CNO and the patient safety officer, framed around patient risk rather than process grievance. Escalation is proportionate, not automatic.
Then support safe use. Once approved, your job is to make the pilot as safe and as informative as possible: build the override reason set, train the champions, watch the dashboard, and report honestly. Sabotage or quiet non-participation would be a breach of professional obligation and would also destroy your credibility for the next decision.
Protect the nurses. Ensure the messaging to staff states that documented override is a clinical judgement, and that override rates are reviewed as a system signal, not as individual performance. If the pilot fails, argue for discontinuation on the pre-registered metrics, which is a far stronger position than arguing from alarm.