Thirty-Six Hours Against an Eight-Hour RTO

The gap is a finding about business tolerance, not primarily about backup technology. An RTO written by the IT department four years ago without clinical input has no authority as a business requirement; equally, the 36-hour restore estimate does not redefine what the hospitals can tolerate. The correct response is to re-establish the requirement with the people who bear the consequence - clinical directors, pharmacy, radiology, patient administration - and then decide how to meet it.

The analysis should decompose the eight hours. Which clinical functions actually fail at eight hours, and which can run on downtime procedures for 24 or 72? That produces tiered recovery objectives per application and interface rather than one number for the EHR. Then evaluate three routes together: reduce recovery time through warm standby, immutable backups and tested interface rebuild; extend survivable time through practised downtime procedures, pre-printed forms, and cached local access to medication and allergy data; and formally accept residual risk where the cost of the technology exceeds the harm of the gap.

The credibility of the downtime route depends on the 20% training figure. Downtime procedures that staff have not practised are a plan on paper, and clinical staff under pressure will improvise. A rolling simulation programme, including one unannounced exercise per site per year, is a cheap and high-yield investment that often closes more of the gap than infrastructure.

What not to do: do not silently rewrite the RTO to 36 hours to close the finding, and do not buy replication technology before the business has stated what it needs. Both make the register look clean while leaving patients exposed. If the gap cannot be funded, it should appear as a formally accepted risk owned by the chief medical officer or COO, not by IT.

The delay was not a failure of courage; it was a failure of documented delegation. At 02:00 nobody could point to a sentence saying who may disconnect a hospital network, so the decision escalated to people who were asleep.

The fix is a written delegation of authority, approved by the executive team and referenced in the incident response plan:

• Named roles with deputies, not named individuals: for example the on-call security manager, the on-call IT operations manager, and the incident commander, each with a deputy for every shift. Publish the roster and the escalation numbers on the same page.
• Defined triggers that authorise action without further approval: confirmed ransomware on two or more hosts, mass file-encryption alerts, domain administrator compromise, or loss of integrity on clinical systems. Triggers should be observable facts, not judgements, so the on-call person is not asked to assess severity alone at 02:00.
• A pre-approved action list: isolate a network segment, disable an account, block a hash at the endpoint or mail gateway, suspend interfaces between EHR and ancillary systems, and switch to downtime procedures. State explicitly that clinical safety and preservation of evidence both outrank service continuity in the first hour.
• Obligations attached to the authority: notify the incident commander and executive on-call within 30 minutes, open the incident log immediately, and record the decision, the trigger and the expected impact.
• Retrospective review within five working days, which is what prevents the delegation from being abused or becoming unilateral.

The delegation must be exercised at least once in an exercise, or it will not be believed. Run the next tabletop at night with the actual on-call roster and measure the time from trigger to action; that interval, not the existence of the document, is the control.

There are three parallel tracks - technical containment, legal and regulatory assessment, and communications - and the common mistake is to run them as a sequence. They share one input: a documented timeline of what was known, when, and by whom.

On obligations: the insurance notification (24 hours from suspicion) has the lowest threshold and should be issued as a preliminary notice early, since suspected incidents qualify and the notice can be updated. The regulatory notification (72 hours from awareness) requires an assessment of whether personal data was affected, which needs forensic scope - but the clock runs from awareness, not from confirmation, so begin drafting at hour one and reserve the option to submit an initial notification with a phased update. If the EHR vendor is a processor, contractual notification clauses to them and from them apply on a separate timeline; identify who notifies patients and who notifies the regulator, because two parties notifying creates inconsistent messaging. In some jurisdictions the insurance policy is invalidated by late notice, which makes the 24-hour obligation commercially as well as legally significant.

Sequencing that preserves evidence: capture volatile data (memory, running processes, network connections) and preserve relevant logs before reimaging or rebuilding any host, and place a hold on log rotation and mailbox deletion for the affected accounts. Restoration and forensic imaging must be planned together so that restoring service does not destroy the source of the scope assessment. Document each decision with a timestamp and a rationale; regulators assess the reasonableness of the process as much as the outcome.

Finally, fix the communications gap shown in the exercise: a call tree that includes the duty executive, legal counsel, the communications lead, the regulator liaison and the insurer, tested quarterly, with pre-approved holding statements. The absence of any route to media, regulator or patients is a finding that will read badly in any post-incident review.