Seven-day logs and a suspicious mass download
A security team routes IBM Cloud Activity Tracker events to IBM Cloud Logs in one region with 7-day retention. There are no alerts. A customer reports that 2,000 objects were downloaded from a COS bucket at 02:00. Investigation shows a service ID API key was used from an IP address never seen before, and the same key attempted to delete a Key Protect root key 20 minutes later. The key is still active. Logs for other regions are not centralized, and VPC flow logs are enabled but sent to a bucket the security team cannot read. The incident responder must preserve evidence while stopping the activity, and leadership wants a 90-day retention design and better detection. The company has IBM Cloud Logs, Activity Tracker, COS, and Secrets Manager, but no formal incident-response runbook.
1. Outline the immediate incident-response actions. What do you do first, and what must you avoid?
Read the worked answer
First, declare an incident and contain the confirmed identity: disable or revoke the service ID API key, but preserve its metadata and recent Activity Tracker records before deleting anything. Do not delete the Key Protect root key or the COS bucket in a rush; that destroys evidence and may cause an outage. If the key deletion attempt is still pending, cancel or block it, confirm dual authorization and deletion protection, and alert the key administrators. Restrict the source IP through context-based restrictions if that can be done without breaking legitimate workloads, and isolate the service ID from further use.
Second, preserve evidence: export Activity Tracker events for the service ID, IAM, COS, and Key Protect to a separate evidence bucket with restricted access and retention. Capture the API key inventory, IAM policies, bucket policy, and key state. Identify the objects accessed, the bucket, the time window, and whether any other service IDs or IPs appear. Check for persistence: new API keys, new IAM policies, new service IDs, and changes to logging or key settings.
Third, scope the data impact and notify the data owner and privacy or legal team. Recovery includes rotating the key, validating the application, and monitoring for re-use. Avoid the common mistakes: deleting the key immediately, overwriting logs, or rotating every credential at once and losing the ability to distinguish the attacker's activity from the response.
2. Redesign logging and evidence retention to support a 90-day investigation window.
Read the worked answer
Centralize Activity Tracker event routing from every region to a security-owned logging destination, with a documented archive path. IBM Cloud Logs can provide search and alerting, but the design should not depend on a single regional instance with 7-day retention. Set retention to at least 90 days for security-relevant events, and export or archive to a COS bucket with retention controls and restricted access so that logs survive accidental deletion or a compromised operator account.
VPC flow logs must go to a bucket the security team can actually read, ideally in a security account or resource group with read-only access for responders and write-only access for the network team. Protect the log destination with its own IAM boundaries and context-based restrictions; a responder should not need Administrator on the whole account to read logs. Record the time synchronization and the regions covered so that events from different regions can be correlated.
Finally, define evidence handling: who can export, where exports are stored, how integrity is checked, and how long evidence is kept for an incident. The 90-day retention is a minimum, not a substitute for legal hold. Test the path by searching for a known event from each region and confirming it appears in the central destination.
3. What detections and responder access model would you put in place?
Read the worked answer
Detections should focus on behaviors that matter in IBM Cloud: creation of new API keys, IAM policy changes that grant Administrator, access-group membership changes, Key Protect key disable or deletion attempts, COS bucket policy changes, and unusual data download volume. Add behavioral alerts for a service ID used from a new country or IP range, a service ID reading many objects in a short period, and a key operation outside the normal change window. Tune thresholds with a baseline so the team does not drown in alerts.
For responder access, create a security access group with read-only visibility to Activity Tracker, Cloud Logs, COS logs, and security posture results. Keep containment actions such as key disable, API key revocation, and network restriction behind a break-glass role with MFA and alerting. Do not give every responder account-level Administrator; that recreates the risk being investigated. The runbook should state who can declare an incident, who can revoke a key, who can contact the data owner, and who can approve evidence export.
After the incident, run a lessons-learned review: were the 7-day logs sufficient, were flow logs readable, and did any alert fire? If the answer is no, fix the logging and access path before closing the incident. The goal is a repeatable capability, not a one-time cleanup.