340 Megabytes to a Three-Week-Old Domain
Fabrikam Analytics runs 40 AWS accounts. VPC Flow Logs ship to a central S3 bucket; Route 53 Resolver query logging was enabled three weeks ago. Egress proxies log about 1.2 TB per day of HTTP and HTTPS metadata but not full URLs. GuardDuty is enabled in all regions. A threat hunter, working a tip about a cloud-storage exfiltration campaign, finds one EC2 instance in a production account sending TCP/443 to 203.0.113.47 - roughly 8 to 14 KB every five minutes with small jitter - for 30 days, totalling 340 MB. The destination's SNI is a domain registered 19 days ago whose A record sits in a /24 adjacent to, but not inside, a range attributed to a known C2 operator. The instance runs a Java service that normally talks only to an internal API, and its IAM role holds s3:GetObject and s3:ListBucket on a bucket holding customer support attachments. GuardDuty raised a Low-severity Trojan:EC2/BlackholeTraffic finding for a different instance 12 days ago; an analyst archived it as expected scanner traffic.
1. Assess whether this is exfiltration. Give at least three competing hypotheses and the specific evidence that would discriminate between them.
Read the worked answer
The volume is small enough to be anything and regular enough to be something. Competing hypotheses:
H1 - C2 beaconing with staged exfiltration. Beacons are small and regular; the 340 MB is the stolen data trickling out. H2 - A legitimate third-party agent or SaaS integration deployed by the application team (APM, license check, log shipper). H3 - A misconfigured application retry loop hitting an external endpoint it was never meant to reach. H4 - A security scanner or pen-test tool from a sanctioned exercise.
Discriminators, cheapest first:
- Payload symmetry. Beacons are asymmetric: small outbound tasking, occasional larger inbound or outbound bursts. A retry loop sends near-identical request sizes. Pull per-flow byte counts by direction from Flow Logs for the whole 30 days and look for the burst pattern.
- Correlate with CloudTrail data events. S3 GetObject events on the support-attachments bucket timestamped within seconds of egress bursts converts H1 from plausible to demonstrated. This is the single highest-value query.
- Process attribution. SSM into the instance and map the connection to a PID and command line. If it is the Java service as shipped in the golden AMI, H2/H3 rise; if it is a child process, a script, or a binary outside the AMI, H1 rises sharply.
- Domain age and infrastructure. A 19-day-old domain that a production Java service has been talking to for 30 days is internally contradictory - either the flow dates are wrong or the domain is a fast-flux replacement. Check passive DNS and certificate transparency for prior resolutions.
- Baseline prevalence. Do any other instances in the 40 accounts talk to this IP or SNI? Zero prevalence across a fleet of this size is not normal for a legitimate vendor.
- JA3/JA4 fingerprint compared with known tooling, and whether TLS is to an IP rather than a name in some flows.
Conclude on convergence, not on a single indicator, and place a low-cost containment (deny egress to that IP) while the discriminating queries run, so the timer stops.
2. Given the IAM role's data access, what is your containment sequence, and what could you destroy by doing it wrong?
Read the worked answer
Preservation comes before containment in cloud, because the most useful evidence is ephemeral or deletable.
Sequence:
- Snapshot the EBS volume and capture instance memory if the platform supports it. Do not terminate the instance - termination destroys the disk and the ephemeral evidence, and it also tips off the operator.
- Preserve logs out of reach: confirm CloudTrail, Flow Logs, and Resolver logs are in a bucket the compromised role cannot write to or delete from, and export the relevant window to a separate forensic account.
- Contain egress, not the host. Apply a deny-all security group or a NACL rule blocking the destination, keeping management access via SSM so you retain visibility. Full isolation that severs your own access is containment that blinds you.
- Neutralise the credentials properly. Role credentials are temporary, so deleting the role does nothing to sessions already issued. Attach an explicit deny policy conditioned on
aws:TokenIssueTimeearlier than now, or apply an SCP, so existing sessions are revoked rather than merely blocked at the next refresh. - Quantify before you clean. Query CloudTrail data events for
s3:GetObjectby that role over 30 days to establish what was actually taken - the 340 MB is a lower bound and may be unrelated. - Hunt persistence: newly created IAM users or access keys, Lambda functions, EC2 user-data changes, Systems Manager documents, new trust policies, and other instances launched from the same AMI or launch template.
What you can destroy by moving fast: memory-only implants, the ability to attribute, and the only record of which objects were read. Terminating the instance is the most common self-inflicted wound in cloud response.
3. Was archiving the GuardDuty Low finding defensible, and how would you change triage of cloud findings?
Read the worked answer
The archive decision may have been correct in outcome but was indefensible in method: the closure reason was expected scanner traffic, which is an assertion with no owner, no evidence link, and no expiry. That is the pattern that lets a real finding disappear - and here a related finding appeared 12 days later on a different instance in the same account family.
Changes to cloud triage:
- GuardDuty severity is not your risk. Its Low/Medium/High reflects confidence and potential impact in the abstract. Re-score with asset criticality, whether the instance's role can read sensitive buckets, whether the instance is internet-facing, and account sensitivity.
- Require evidence and an owner to suppress. Any finding closed as expected must reference the scanner's source CIDR or the drill ticket number, name an owner, and expire. Re-open automatically if the same finding type appears on a host outside that documented scope.
- Tune at the source where the finding is genuinely noisy. GuardDuty suppression rules scoped to a specific scanner's IP range and account are auditable; a free-text archive reason is not.
- Baseline expected egress per account so a new destination is visible without waiting for a threat-intel tip. The 19-day-old domain was discoverable by Resolver logs from day one.
- Treat adjacent infrastructure as a weak signal, not evidence. Being in a /24 next to known C2 means little on its own; it justifies the hunt that produced the real evidence, and should not itself drive an incident declaration.