Ransomware at 06:00: Failover or Continue Downtime
A 400-bed hospital experiences a ransomware incident at 02:10 on a Saturday. The electronic health record is unavailable, and downtime procedures are in effect. The EHR recovery time objective is four hours and the recovery point objective is 15 minutes. Immutable backups exist, but the last full restore test was 14 months ago. At 06:00, three of twelve interfaces remain down, including the lab interface; paper order entry is creating duplicate orders. The incident commander must decide whether to authorize failover to the disaster recovery site, which engineers estimate will take six hours and may lose the last 90 minutes of documentation, or continue downtime until the primary environment is validated. A local news outlet has called, and the state health department requires notification within 24 hours of a confirmed breach. Clinicians report that two patients have had delayed medication orders, neither with harm.
1. Recommend a decision at 06:00 and justify it against RTO, patient safety, and data integrity.
Read the worked answer
The decision should be patient safety first, then data integrity, then restoration speed. The RTO was four hours and it is already exceeded at 06:00. The lab interface is down, paper ordering is producing duplicates, and two medication orders have been delayed. Those are leading indicators that continued downtime is creating active clinical risk.
Failover is preferable if two conditions are met: the backups are confirmed clean and the malware is contained so the DR environment will not be reinfected. Engineers estimate six hours and possible loss of the last 90 minutes of documentation. That is a real data-integrity cost, but it is bounded and can be mitigated with paper reconciliation, a documented gap list, and clinician review after restoration. Continued downtime has an unbounded safety cost because interface failures and duplicate orders can multiply.
If containment is not confirmed, failover could reintroduce the attacker. In that case the commander should authorize a staged failover to a clean environment, prioritize the lab interface and medication ordering, and keep paper procedures for the remaining functions. The decision should be documented with the facts known at 06:00, the alternatives considered, and the accepted data-loss window. The commander should not wait for perfect information; the RTO breach and clinical signals already justify action.
2. Sequence the first five actions for the next two hours and assign owners.
Read the worked answer
Convene a clinical safety huddle led by the chief medical officer or nursing lead. Identify patients with pending labs or medication orders and set a manual workaround for the lab interface. This is the highest-priority action because safety signals are already present.
Confirm malware containment and backup integrity with the security and infrastructure leads before failover. The commander needs a go or no-go recommendation on whether the DR environment is clean. Document the last known good backup and the expected 90-minute documentation gap.
Preserve evidence and maintain incident chronology through the security operations lead. Do not wipe or rebuild systems before forensic capture. The health department notification decision depends on confirmed facts.
Notify the incident executive team and legal counsel, and prepare a holding statement for the news outlet through communications. The statement should acknowledge an incident and patient care continuity without speculating on data exposure.
Assign a regulatory lead to assess the 24-hour health department notification requirement and prepare a factual timeline. The notification decision may not be final within two hours, but the assessment and draft should be underway.
Each action needs a named owner and a time check. The commander should reconvene at a fixed time rather than let the response drift.
3. What should the post-incident review examine, and which finding is most likely to change the risk register?
Read the worked answer
The review should examine detection and containment, backup restore testing, interface dependencies, downtime procedure effectiveness, clinical workarounds, communications, and regulatory notification timing. It should distinguish what worked from what was assumed to work.
The most likely risk-register change is the untested restore capability. The hospital had immutable backups and a four-hour RTO, but the last full restore test was 14 months ago. That is an assurance gap, not merely an operational delay. The register entry should be updated to reflect that recovery capability is unproven, with a corrective action to test full restoration at a defined frequency and record actual restore time.
A secondary finding is the lab interface dependency. The downtime procedure did not prevent duplicate paper orders, which suggests the procedure was not designed for a prolonged multi-interface outage. That should produce a corrective action and a tabletop exercise focused on interface loss.
The review should also test whether the decision to fail over was reasonable given the information at the time. A good outcome does not prove a good decision, and a bad outcome does not prove a bad one. The goal is to improve future judgement under uncertainty, not to assign blame.