Telehealth risk criteria built for a different kind of harm
A regional health system (four hospitals, 380,000 patients) is launching virtual urgent care. The clinical governance committee must approve the risk assessment before go-live in ten weeks. The risk team proposes the corporate 5x5 matrix: likelihood measured over twelve months; consequence measured in dollars with $2m defined as major. Clinicians object that the matrix rates 'missed sepsis diagnosis' as low because such events are rare, and roughly equal to 'vendor platform downtime' of similar dollar cost. Volume data for the new service comes from a six-week pilot of 900 calls. Two committee members hold equity in the proposed technology vendor. The committee has one two-hour session scheduled.
1. Critique the proposed risk criteria and specify how likelihood, consequence and thresholds should be redefined for this decision.
Read the worked answer
The criteria were designed for financial materiality and are being applied to clinical safety - a criteria design failure, not a scoring error.
Consequence. A single dollar scale allows compensation between unlike harms: a missed sepsis diagnosis and a platform outage of the same dollar value are not equivalent. Use separate consequence categories - patient safety and clinical outcome, regulatory and accreditation, financial, reputational, service continuity - and a rule that the highest applicable category drives the rating. Irreversibility and harm to patients should attract a floor rating regardless of frequency.
Likelihood. Event frequency over twelve months systematically demotes rare catastrophic events to 'low' and screens them out before evaluation. Reframe likelihood as the frequency of the exposure (thousands of triage encounters per month, each an opportunity for the event) and require scenario-based analysis for low-frequency, high-severity events, including a credible-worst-case screen.
Thresholds and appetite. $2m as major is a financial construct. Criteria should also state what level of clinical risk is tolerable without escalation, what must go to the medical director, and what constitutes a stop condition.
Horizon. Twelve months is too short for a new service where harm may emerge only after scaling and where detection lags. Define horizon per category and set re-estimation points.
Finally, criteria must state how uncertainty in the analysis is to be expressed, or the 5x5 will manufacture false precision from 900 calls.
2. How should the process handle the vendor equity interests and the very limited data?
Read the worked answer
Conflicts of interest. Both members declare their holdings in writing before the session, and the declarations are recorded in the minutes. They recuse from vendor selection, contract approval and any vote on the technology component; they may stay for the clinical discussion. This is transparency and inclusivity working as intended - excluding them entirely would lose clinical judgement and signal that disclosure is punished. Check whether the holdings trigger any regulatory or procurement disclosure obligation; if so, that is a parallel process, not a substitute.
Sparse data. Under best available information, six weeks and 900 calls supports directional conclusions only, and the pilot population may not represent future users. Do not place point estimates in matrix cells. Instead:
- Triangulate: pilot data, published incident experience from comparable virtual services, national reporting data, and structured expert elicitation with the clinicians who will staff the service.
- Express likelihood as ranges or qualitatively, with the reasoning recorded.
- State assumptions and the sensitivity of each rating to them; a rating that flips on one assumption is not a basis for acceptance.
- Define re-estimation triggers - for example re-run the analysis at 5,000 encounters and after any serious safety event.
The recording requirement matters here: the dissent of the clinicians, the assumptions and the data limitations all belong in the record, so a later reviewer can see what the committee actually knew.
3. One two-hour session, ten weeks to go-live. What can the committee legitimately do?
Read the worked answer
It cannot approve 'risk assessment complete'. It can approve a conditional go-live with a defined safety case, which is both honest and operationally workable:
- The revised criteria, agreed and minuted, so future assessments use them.
- Clinical safeguards evidenced as treatment: red-flag symptom escalation pathways, defined clinician staffing and supervision ratios, response-time standards, and an audited route for patients diverted to in-person care.
- Technology continuity requirements in the contract: minimum availability, failover, notification obligations and remedies.
- A monitoring dashboard with pre-agreed stop or pause triggers - not merely indicators to watch.
- An independent review at three months, plus the re-estimation points agreed above.
- Explicit residual risk acceptance by the accountable executive (medical director or chief clinical officer) for anything exceeding the stated tolerance, rather than silence by the committee.
Two structural observations should be minuted. First, a two-hour single session cannot discharge the evaluation duty for a service of this scale; split it into a technical review and a decision session. Second, the ten-week date is a constraint on the process, not a criterion for acceptance - if the safety case is not ready, the honest output is a delayed go-live with reasons, and that decision should be taken deliberately rather than by default.