Saturday Ransomware: Restore Now or Scope First?
At 02:14 on a Saturday, EDR flags a finance workstation for LSASS credential access, then SMB connections to two file servers, then an attempt to run vssadmin delete shadows. The SOC triage runbook auto-closes LSASS alerts tagged with a well-known backup agent's process name, and an analyst closed this one at 02:20 under that rule without checking the parent process.
At 09:40 the file servers begin mass file renames with a .locked extension. Backups are nightly Veeam to an on-prem repository that is domain-joined and reachable with the compromised service account. The last verified restore test was 7 months ago; 24 hours of incrementals are unverified. The CFO wants restoration started immediately to hit Monday trading. The CISO is unreachable for 3 hours.
1. Sequence the first 60 minutes. Address the CFO's request directly.
Read the worked answer
Order: contain, preserve, verify, then restore. Restoring into a still-compromised environment re-encrypts the restored data and destroys the evidence of the initial access vector - it buys a Monday and loses the case.
- Contain the three known hosts by network isolation, not shutdown. Powering off loses volatile memory (running processes, injected code, credential material in RAM), which is the most perishable evidence in a credential-theft intrusion. Isolate at the EDR/security-group layer while preserving a management path for the responder.
- Disable and rotate the compromised service account and any credential used for backup access. Assume the credentials from LSASS are in the attacker's hands - they are, that is what LSASS access is for.
- Test whether the backup repository is already affected before using it. It is domain-joined and reachable with the compromised account: check for .locked files, altered or missing restore points, and Veeam job modifications. This is the single most important check of the hour and it is the one most often skipped.
- Ask, then act on, the domain controller question. If Tier 0 compromise is plausible, plan a double KRBTGT reset with replication time between the two resets (~10 hours), plus privileged credential rotation - scheduled deliberately, not mid-containment.
To the CFO: give a decision timeline, not a refusal. Commit to a scoping checkpoint (e.g. 4 hours) and a restore-in-isolated-network for validation. A documented promise at 12:00 beats a failed trading day on Monday plus an unrecoverable environment.
2. Was this a detection failure or a process failure? What would you change?
Read the worked answer
Detection worked. Triage failed. The alert fired at 02:14. The clock that matters here is not MTTD - it is time to acknowledge (02:14 to 09:40, about 7.5 hours of dwell) and time to contain.
Root causes:
- A suppression rule keyed on a spoofable, weak attribute. Process name is trivially matched by malware masquerading as a backup agent. Any auto-close rule must key on signature/hash, canonical path, and parent-process lineage - not a string a binary can choose.
- Credential-access alerts were eligible for auto-close at all. LSASS access belongs on a never-auto-close list; the cost of that policy is analyst minutes, and the cost of the alternative is measured in days of encryption.
- Single-analyst closure with no second set of eyes on a high-severity class. Human factors - 02:20 on a Saturday, thin staffing - are exactly the conditions in which the process must compensate.
Changes:
- Reclassify LSASS access as tier-1, no auto-close, mandatory parent-process and command-line review.
- Add an automated enrichment step (is the signing certificate valid? does the hash match the known agent?) so the analyst starts with evidence rather than judgement under time pressure.
- Track auto-closed alerts by rule as a standing control metric and audit a sample monthly - the rule was probably wrong for months before it mattered.
Judgement: do not present this as a tooling gap. Buying another detection product would not have changed 02:20.
3. The CFO wants trading restored by Monday. What do you commit to, and what must be true before any restore is declared complete?
Read the worked answer
Commit to a business-prioritised, isolated recovery, not to a full restore on a fixed date. The recovery time objective is a business decision; the recovery method is not negotiable.
What to commit to:
- A named restore subset that actually enables trading (order management, payment path, inventory) rather than all systems - a full restore is the slowest possible route to a Monday.
- Validation in a clean, non-domain-joined recovery network before anything rejoins production.
- A checkpoint by a stated hour, with honest status, including 'we do not yet know if the backups are intact'.
Must be true before declaring complete:
- Backup integrity verified by test restore, not by the existence of files. Seven months without a verified restore means all backups are unverified; the 24 hours of incrementals are additionally unproven.
- No .locked artefacts, no unexpected changes to backup jobs or retention, and repository credentials rotated off the compromised account.
- Reintroduction is clean - rebuilt from known-good images, patched, with the compromised service account replaced and Tier 0 credentials rotated; otherwise the restore is a reinfection with extra steps.
- Target architecture changes committed: immutable/air-gapped repository, backup credentials outside the production domain, at least one offline copy, and a restore test on a defined cadence with the result recorded.
Judgement: a restore that succeeds on Monday and is encrypted again on Wednesday is worse than a two-day outage, because it consumes the backups you will not have a second time.