A Leaked Key, Ninety Days of Logs, and One Anomalous Region
A developer pushed an AWS access key to a public repository 11 days ago. GuardDuty has raised an anomaly: the key was used from an IP in a region the company has never operated in. The key belongs to an IAM user with s3:GetObject on three buckets, one of which holds customer contract PDFs. CloudTrail is on with 90-day retention; S3 server access logging is off; VPC Flow Logs are off. One production EC2 instance runs a web application with an instance profile granting full S3 access. The bucket policy allows * on GetObject for one bucket; block public access is not enabled at account level. Your incident lead wants the key disabled immediately, but the forensic analyst wants logs preserved first.
1. Sequence the response. Who is right about the order, and why?
Read the worked answer
The forensic analyst is right about the sequence, but the window is measured in minutes, not days.
Order:
- Preserve first, in parallel with containment. Export the relevant CloudTrail events to a separate, write-once location in a different account or bucket. This takes minutes and is irreversible-proof: 90-day retention is a rolling window, and 11 days of exposure already sits inside it.
- Contain the key. Disable (deactivate) rather than delete the access key. Deactivation is reversible, immediately stops use, and preserves the key ID and any attached metadata for the investigation. Deleting it destroys linkage you may need.
- Rotate anything the key could reach - the IAM user's credentials, and check whether the key was used to create new users, roles, or keys of its own. Revoking the original key does nothing if a persistence key was minted on day two.
- Contain the public exposure - enable account-level block public access, then remediate the
*bucket policy. Note that block public access may break legitimate workloads; confirm before enabling. - Eradicate and recover - review the instance profile, scan the web app for SSRF paths that could reach the metadata service, then restore normal operation.
Disabling first would be defensible only if you had already captured the logs. You have not.
2. How do you establish blast radius given 90-day CloudTrail retention, no S3 access logs, and no flow logs?
Read the worked answer
Build the picture from what exists, and be explicit about the gaps.
Available evidence:
- CloudTrail management events: every API call made with that key - source IP, user agent, timestamp, region, and the specific objects requested. This answers "what did they touch and from where" for object-level
GetObjectcalls, because S3 data events recorded via CloudTrail include the bucket and key name if data events were enabled - check this first; if they were not, you only see management-plane activity. - GuardDuty findings history: the original detection plus any correlated findings in the account.
- Repository history: when the key was committed, whether it was ever truly public, and whether it was scraped. Commit timestamps bound the exposure window.
- The IAM user's permission set: simulate the policy to enumerate exactly which buckets and prefixes were reachable. That defines the maximum possible loss.
Gaps to state plainly:
- With S3 server access logging off and no data events, you may be unable to say which specific PDFs were downloaded. Report that as an evidence gap, not as "nothing was taken."
- With no flow logs, internal lateral movement via the instance profile cannot be ruled out; recommend enabling both going forward.
The honest deliverable distinguishes "confirmed accessed" from "could have accessed". Treat the delta as the breach-notification scope.
3. Leadership asks who is responsible for the exposed bucket. What is the accurate answer, and what do you recommend?
Read the worked answer
The customer is responsible for the bucket - the exposure is a configuration decision, not a provider failure.
Under the shared responsibility model, the provider secures the infrastructure, the availability zones, and the managed service itself; the customer owns identity, permissions, bucket policies, ACLs, and data classification. A bucket policy granting * on GetObject with account-level public-access blocking off is squarely a customer-side misconfiguration. Do not let the conversation drift toward "AWS let this happen" - it will not survive contact with the provider's own documentation.
Recommendations, in order of effect:
- Account-level block public access, then re-enable selectively only where a public bucket is genuinely required and reviewed.
- Replace
*principals with explicit principals on the offending bucket; if public read is required for a website, serve it through CloudFront with an origin access control. - Remove full S3 access from the EC2 instance profile and scope it to specific buckets and prefixes. A web application with instance-wide S3 rights is one SSRF away from the same outcome, and it will not appear in a key-leak report.
- Add S3 data events and Flow Logs, and extend CloudTrail retention or archive to an immutable store.
- Secret scanning in CI with push protection, plus pre-commit hooks - the cheapest control in this entire list, and the one that would have prevented the incident.