A Stolen Prism Admin Credential Deleted Production and DR Copies

Principle: the identity that protects data must not be the identity that can delete it.

• Split the roles. Backup/replication operators get create-and-restore rights but not delete. Snapshot and protection-domain deletion lives in a small custom role held by very few people, requiring MFA and approval.
• Break the shared trust between sites. The DR cluster should be administered by a separate Prism Central with its own directory integration, or at minimum a distinct account set with no shared local admin and no credential reuse. The exercise succeeded because one credential was valid in two places - that is a design defect, not an attacker capability.
• Scope the replication identity. A service account bound only to the protection domains and clusters it replicates, with rotated API keys and source-IP restriction, so it cannot be repurposed into a deletion tool.
• Remove standing Prism Admin from human accounts. Just-in-time elevation with an approval and an audit trail; standing admin is what made a single phish into a full destructive event.
• Separate duties for retention. The person who runs backups should not be able to shorten or remove retention. Otherwise the control is only as strong as the most over-privileged operator.

Diagnose first: the attack succeeded because every copy was reachable, deletable and controlled by the same identity plane as production. Replication did not help - it faithfully replicated the deletion. So the fix is not 'more replication'.

Changes:

• An immutable or offline copy. Target object storage with retention lock/WORM, or a copy pushed one-way to a repository whose credentials do not exist in the production directory. If the attacker cannot authenticate to it, they cannot delete from it.
• Retention longer than plausible dwell time. Ransomware operators typically sit for weeks before detonating. If retention is 7 days and dwell time is 30, the only surviving copy is already encrypted. Set the immutable window deliberately against that assumption and state it in the design.
• Keep the DR cluster but reclassify it. It is a availability control, not a ransomware control, unless the replicas themselves are immutable and protected by separate credentials.
• Test restores on a schedule with defined scope: a single VM, a file-level restore from the file server, and a full DR failover, each with time-to-restore recorded. An untested backup is a hypothesis, and the 2 TB file server that has never been restored is the most likely failure in the real event.

Detection (the gap was alerting, not logging):

• Confirm audit logs leave the cluster and land in a store the production admin credential cannot modify or delete; if the attacker can wipe the log, you have no evidence.
• Alert on rate, not just events: protection domain or snapshot deletion above a threshold in a short window, mass VM deletion, storage container changes, role and permission changes, new admin account creation, local and break-glass logins, SSO failures, and replication target changes.
• Verify time synchronisation end to end. Timestamps that drift make an incident timeline indefensible.
• Test each high-value rule with a benign action and keep the test record; a rule that has never fired is an assumption.

Evidence package for the auditor:

1. Role-to-group mapping with the separation-of-duties statement and the quarterly access review results.
2. Immutability/retention configuration for the protected copies, with the retention window justified against assumed dwell time.
3. Restore and DR test reports with dates, scope, time-to-restore, and issues raised and closed.
4. Alert rule inventory with the test evidence for each.
5. Audit log retention period and proof of off-cluster storage and access control.
6. The incident runbook, including who can invoke break-glass and how that use is reviewed.

Sequencing judgement: do the credential separation and the immutable copy first. Detection without a protected recovery copy still ends in a ransom decision.