Three Miners Behind an Auto Scaling Group
The organization spans 41 accounts. GuardDuty, delegated to the security account, raises CryptoCurrency:EC2/BitcoinTool.B! against three instances in the production account within six minutes of each other. All three run in one Auto Scaling group behind an ALB, share an instance profile granting s3:GetObject on the backup bucket plus broad ssm permissions, and have no direct internet route - egress goes through a NAT gateway shared with the load-test fleet. An organization CloudTrail trail, VPC Flow Logs to a central S3 bucket, and SSM Session Manager are in place. There is no bastion and no EDR agent. The CISO requires containment within 30 minutes and preservation of evidence suitable for legal hold. The on-call engineer's instinct is to terminate all three instances immediately.
1. The CISO wants containment inside 30 minutes with evidence preserved. Sequence the first actions and justify why terminating the three instances is the wrong default here.
Read the worked answer
Rough order: snapshot → capture volatile state → isolate network → freeze the ASG → constrain credentials → investigate.
- EBS snapshots of all three instances from the security account via API. This needs no agent and no bastion, and it is the only step that survives an accidental termination.
- Volatile capture before you cut the SSM channel, if memory matters: an SSM Run Command writing an image to S3. Treat that artifact as untrusted - the same agent and instance role the attacker can reach produced it, so output can be falsified. VPC Flow Logs (and Traffic Mirroring, if it was pre-enabled) are the only out-of-band sources you can rely on.
- Isolate: quarantine security group with no ingress and no egress, and deregister from the ALB target group. Stop nothing yet - stopping destroys memory and muddies the snapshot timeline.
- Suspend the ASG's Launch, Terminate and HealthCheck processes. Terminating without this step just manufactures three fresh hosts from a launch template or user data that may itself be poisoned - the classic way to "contain" a miner into permanence.
- Constrain credentials: attach an explicit Deny on s3:, ssm: and ec2:Describe* to the instance role, conditioned on
aws:TokenIssueTimeso existing role sessions are hit too.
Terminating destroys memory, the binaries and dropped tooling on the volumes, and the instance's metadata history, while giving an ASG the opportunity to rebuild the compromise. Terminate only after snapshots exist and the launch template has been cleared.
2. You have no EDR and no bastion. How do you capture and validate volatile evidence, and what is the risk in the only channel you have?
Read the worked answer
The only execution path onto those hosts is SSM, which runs as the instance's own role. That means the acquisition tool and its output travel over the same control plane a host-resident adversary can influence.
Practical sequence:
- Prefer out-of-band artifacts first because they cannot be tampered with from the host: Flow Logs for byte volume and destinations, ALB access logs for request patterns, CloudTrail for every API call the role made (including SSM SendCommand targets, if the role had that right), and the GuardDuty finding timeline.
- Then memory, via a Run Command document that writes to an S3 bucket the instance role cannot delete from (write-only, with Object Lock). Take a hash on the host, but do not treat host-computed hashes as integrity proof.
- Corroborate: if the host's memory image shows no miner but Flow Logs show steady outbound traffic to a mining pool IP on a non-standard port, believe the network. Absence of host artifacts is not exculpatory when the host controls the collector.
- Image the volumes and later mount snapshots in an isolated forensic VPC in a dedicated account for analysis, so the production account's blast radius does not extend to your investigation.
The judgement point: with no EDR, your evidentiary strength comes from network and API logs, not from the host. Rank them accordingly and say so in the report.
3. The finding mentions the backup bucket. What evidence can prove or disprove that objects were read or deleted, and what do you tell legal if data events were never enabled?
Read the worked answer
Separate what was recorded from what was not.
Recorded: CloudTrail management events prove whether the bucket policy, ACL, lifecycle or versioning state changed, and whether the bucket itself was deleted. GuardDuty S3 Protection findings, if any, indicate anomalous access patterns but are detections, not a per-object ledger. VPC Flow Logs show bytes and destination prefix lists - volume, never object identity.
Not recorded: with data events disabled, there is no authoritative record of GetObject or DeleteObject calls. Server access logs would have helped, but they were not enabled on this bucket, and even where enabled they are best-effort with multi-hour delivery and known gaps.
What you can say to legal, precisely: "We can establish that the bucket's configuration was unmodified and that no bucket-level destructive action occurred. We can bound the volume of data transferred to S3 endpoints during the window. We cannot state which objects were read or by whom, and we cannot exclude reads."
Then fix the gap in a way that matches the answer you just had to give: CloudTrail advanced event selectors scoped to this bucket for read and write data events, Object Lock or versioning plus MFA delete to make deletion provable, and an access-logging baseline for every bucket that holds regulated data.