ESXi Cluster Ransomware with a Memory-Only Service and a Doubtful Backup

Powering off is the wrong first move. Encryption has already completed on the affected datastores, so the marginal containment gain is small, while a shutdown destroys VM memory (including the licensing service), in-guest logs still buffered, and the session state that shows what the attacker still holds.

Order of work:

1. Isolate at the network layer: firewall and port-group rules blocking egress and lateral SMB/RDP/WinRM, while keeping a management path for responders. Record every change in UTC.
2. Preserve host state before touching accounts: run vm-support on each host to external media, copy /var/log/vmware/{hostd,vpxd,shell,vmkernel}.log, and capture the active session table (esxcli network session list, who, last).
3. Only then disable the abused local ESXi account and rotate vpxuser, expecting the attacker to notice.
4. Handle the licensing VM deliberately: suspend it to produce the .vmss memory file, copy it off, hash it, then resume. A snapshot with memory is an alternative. Document that suspending changed system state.
5. Shut down remaining hosts only if isolation genuinely cannot be achieved.

Keep a single responder command log, photograph the consoles, and treat the jump host as compromised infrastructure, not a trusted collection platform.

Not automatically. The backup's success timestamp only proves the job ran, not that the repository is clean. The jump host had an SMB mount to the NAS, so the repository is in the attacker's reach and may contain encrypted or renamed restore points, or a restored ERP that already carries the persistence used for initial access.

Checks before restore: mount the NAS read-only from a clean host; inventory restore points and look for renamed files or ransom notes; confirm immutability or offline copies; review Veeam job logs for configuration changes; identify the earliest evidence of attacker presence (the SSH login time on the ESXi hosts) and confirm the restore point predates it.

Decision rule: restore only from a copy that is both verified intact and earlier than first confirmed attacker access, not merely earlier than encryption. Restore to an isolated VLAN, run database consistency checks, compare row counts and last transaction identifiers against expectations, and scan the restored filesystem for webshells, scheduled tasks and new service accounts before promoting to production. If no pre-access copy exists, rebuild the ERP from clean media and re-import data through validated extracts rather than restoring a potentially poisoned image. Communicate the revised RTO as soon as that determination is made, because the 10:00 target may be unachievable and the business needs to plan around it.

Treat it as the highest-volatility asset and capture it before any host-level remediation. The correct primitive is a suspend-to-memory operation: suspending the VM writes its full memory image to a .vmss file on the datastore while preserving the running state so the service can be resumed. Copy that file to write-once external media, hash it immediately, and record the VM's configuration file alongside it. A snapshot that includes memory is an acceptable alternative if suspend is judged too disruptive, but be explicit about which one you used and what it changed.

Three constraints govern the choice. First, the datastore holding the .vmss may be the one being encrypted, so confirm free space and target a datastore that is not affected, or export directly. Second, suspending is a state change that a defence expert will probe, so log the exact time, the operator, the vSphere task ID and the resume time; the operational need for continuity is a legitimate justification when documented. Third, do not rely on VMware Tools process listings as a substitute for memory: they show names, not resident keys or in-process configuration.

If suspend risks the service failing to resume, capture memory anyway on a copy of the VM registered on an isolated host, and state the trade-off in the report. Also ask the vendor whether the licence state can be reissued; that determines how much operational risk the preservation step is worth.