I85PREP
⌘ K
CCSO study materials
CCSO study resources

Nine Days of Stolen Keys, a Muted Mailbox, and a 14-Day SIEM

CCSO · Put your knowledge to work in context.

All case studies

Nine Days of Stolen Keys, a Muted Mailbox, and a 14-Day SIEM

Marrow Retail has 12 AWS accounts and an organization trail delivering CloudTrail management events to a log-archive account's S3 bucket with Object Lock in compliance mode for 400 days. GuardDuty runs in 9 accounts. VPC Flow Logs go to a self-managed SIEM that costs $38,000 a month - 74% of the security tooling budget - and the SIEM team has cut retention to 14 days. In a recent incident, an access key belonging to a developer's IAM user was used from an anonymizing proxy for nine days. During that window the actor created three IAM users, attached AdministratorAccess to them, launched 40 GPU instances, and deleted the S3 bucket that held the developer's own access logs. The first signal was a billing alert on day nine. The review found GuardDuty raised four findings on day one, delivered to a shared security@ mailbox nobody owns; no SIEM alert existed for IAM user creation.

1. Explain the detection failure by control layer and rank the three changes with the highest risk reduction per unit of effort.

Read the worked answer

Layer-by-layer, the failure is not one missing control but four.

  • Preventive: long-lived IAM user access keys are permitted for humans, no permissions boundary caps what a compromised identity can do, and iam:CreateUser is unrestricted. The actor's escalation path existed because it was allowed to exist.
  • Coverage: GuardDuty runs in 9 of 12 accounts and the SIEM ingests management events for 2 of 12. Detection cannot be uniform when telemetry is not.
  • Analytic: there was no rule for CreateUser, CreateAccessKey, AttachUserPolicy, or console login without MFA. These are generic privilege-escalation signals that would have fired on day one regardless of who the actor was.
  • Response routing: GuardDuty did its job - four findings on day one - and delivered them to an unowned mailbox. A finding that reaches no on-call rotation is telemetry, not a control. The 14-day SIEM retention against a 9-day dwell time is also luck, not margin.

Top three by risk reduction per unit of effort:

  1. Route findings to a pager with an owner. Cheap, immediate, and it converts four ignored findings into a day-one page. Severity-based routing with a named rotation and an acknowledgement SLA.
  2. A short list of high-signal CloudTrail detections - CreateUser, CreateAccessKey, AttachUserPolicy, PutBucketPolicy, StopLogging, DeleteTrail, snapshot or KMS grant creation by an unusual principal - plus org-wide GuardDuty auto-enable and full-account CloudTrail delivery. These are portable and catch the next actor, not just this one.
  3. Preventive guardrails: an SCP denying IAM user and key creation outside a break-glass role, permission boundaries on developer roles, and migration of humans to Identity Center with short-lived credentials.

Deleting the shared mailbox is not a fix; naming an owner is.

2. The SIEM budget must fall. Redesign the telemetry economics: what do you keep, where, and what do you filter?

Read the worked answer

Separate 'must alert' from 'must be investigable'. The audit requirement for 400 days is already satisfied by S3 Object Lock in the log archive, so paying SIEM retention prices to satisfy it is waste.

  • Keep in the SIEM: CloudTrail management events for all 12 accounts (low volume, high value), GuardDuty and Security Hub findings, identity-provider sign-in logs, and Route 53 Resolver DNS query logs for production VPCs - DNS is consistently the highest signal per dollar for command-and-control and exfiltration.
  • Move out of the SIEM: full VPC Flow Logs. Land them in S3 as partitioned Parquet via Firehose and query with Athena when a case demands it. If some flow data must be hot, ingest only REJECT records and only for internet-facing subnets and specific ENIs.
  • Be selective with data events: S3 object-level events are enormous. Do not ingest them all; enable them for the sensitive buckets, store them in CloudTrail Lake or the archive tier, and alert on a narrow set of patterns (bulk GetObject, public ACL changes, bucket policy changes).
  • Tier retention deliberately: 30 days hot in the SIEM for hunting and alerting, 400 days cold and queryable in the archive. Anything you would need on day 90 is a forensic question, not an alerting question, and forensics can wait for an Athena query.

Rationale to give the CFO: the target is not a percentage cut, it is cost per detectable event. Cutting flow logs entirely would be a mistake - losing the network record for the next incident is exactly how this one became a nine-day dwell. Moving them from ingest-and-store to store-and-query keeps the evidence and removes the premium.

3. The key is confirmed compromised. Give the containment and evidence order of operations, and say what you explicitly avoid doing.

Read the worked answer

Sequence: scope, preserve, contain, eradicate, recover, notify.

  1. Scope before you touch anything. Query CloudTrail by access key ID across every region, not just the ones you use. Attackers pick unused regions because nobody looks there. Capture source IP, user agent, and error codes to distinguish reconnaissance from success.
  2. Verify the evidence is intact. The actor had AdministratorAccess, so assume they could reach logging. Confirm the organization trail is still delivering, check for StopLogging, DeleteTrail, PutBucketPolicy, or lifecycle changes, and validate the archive account's Object Lock and its SCP protection. The 400-day archive in a separate account is your durable record; the 14-day SIEM window is not.
  3. Contain by disabling, not deleting. Deactivate the access key, attach an inline deny-all policy, revoke the user's console sessions, and rotate any other credential that developer could have exposed. Deactivating preserves reversibility and keeps the key ID searchable while you confirm scope. Do not delete the three backdoor users yet - capture their creation events, attached policies, and any keys first, then disable and remove them. Leave no live credential behind you.
  4. Enumerate persistence, not just spend. New IAM roles with trust to unknown accounts, modified trust policies, KMS grants, shared snapshots, new Lambda functions, EventBridge rules, and additional access keys. Crypto-mining is the visible cost; the hidden cost is a re-entry path you miss.
  5. Eradicate and recover from known-good. Terminate the instances, but snapshot one for forensics first. Rebuild affected resources from IaC rather than cleaning them.
  6. Notify on the clock. Legal and privacy immediately: if the actor read or exfiltrated personal data, the regulatory clock is running. Also notify the developer whose logs were deleted and the platform owner.

Explicitly avoid: deleting the key or the account as a first move, terminating instances before snapshotting, relying on the 14-day SIEM as your record, and closing the case without an owner for the routing and coverage fixes.