SIEM Pipeline Gaps, Retention and a Silenced Cloud Guard Recipe
Metro Transit's SOC ingests OCI logs through a Service Connector Hub into a third-party SIEM. Sources are tenancy Audit logs, VCN Flow Logs for two subnets, and Object Storage read/write logs for a bucket holding fare data. In March an analyst hunting a suspected exfiltration found no flow-log records for either subnet after 14 February. The investigation showed an application engineer had deleted the flow log during a tuning exercise and re-created it a week later. The Logging service retention is set to 30 days and the SIEM keeps 90 days, while the regulator requires security events to be retained for 13 months. In January the platform team cloned the Cloud Guard detector recipe, disabled two detectors in the clone, and attached it to the tenancy after a flood of problems about an unused instance principal. Nobody recorded a review date. The SOC lead wants to know whether the February gap is a reportable incident.
1. Is the flow-log gap recoverable, and how would you redesign so that the next deletion is both attributable and loud?
Read the worked answer
Not recoverable. Log content is not retroactive: OCI does not buffer flow records for later download, and re-creating a flow log starts collecting from that moment. Whatever crossed those subnets between 14 February and the re-creation date is gone. Say so plainly rather than implying the SIEM can backfill.
What you can still prove. Audit retains tenancy API activity for 365 days and is immutable to the customer. It will show the DeleteFlowLog call with the caller's OCID, source IP and timestamp, and the matching create. That gives you attribution and a defensible timeline for the reportable-incident question, even though the traffic records are lost. Whether it is reportable depends on your regulator's definition - data exposure is what triggers notification, not the monitoring gap itself, so the answer hinges on whether fare data actually left.
Redesign.
- Move log configurations into a SOC-owned compartment where application engineers get
readonly; the deletion above happened because the engineer had manage rights on logging resources in a compartment he owned. - Add an Events rule on Logging service delete and update operations that fires a Notification to the SOC and invokes a Function to record the change.
- Configure Cloud Guard to watch logging configuration changes and keep a baseline of expected flow logs in Terraform with drift detection.
- Alert on the absence of data, not just on bad data: a Service Connector Hub delivery alarm catches a silent pipeline faster than a human hunt does.
2. Design the retention architecture to meet 13 months while keeping 90 days immediately searchable, and explain why the Logging service alone cannot satisfy it.
Read the worked answer
Why Logging alone fails. The Logging service is a collection and routing layer with a bounded, configurable retention window; it is not an archive and you cannot extend it to 13 months. The same is true of Audit: OCI retains audit events for 365 days and does not let you change that, so month 13 evidence exists only if you copied it elsewhere.
Architecture.
- Service Connector Hub keeps streaming Audit, flow and Object Storage logs to the SIEM (90 days hot, searchable).
- Add a second Service Connector Hub target - or a second connector if you want independent failure domains - writing the same sources to an Object Storage bucket in an SOC-owned compartment.
- Lifecycle that bucket to the Archive tier after 90 days; the archive is your 13-month evidence store, and it is cheap.
- Protect it with an Object Storage retention rule and versioning so a delete during the retention period is refused, and encrypt it with a key in the SOC's vault, not the platform team's.
Two traps. First, a second connector doubles ingestion cost - confirm whether the SIEM vendor can export cold data to Object Storage instead and pick one path deliberately. Second, an archive nobody can query is not evidence; test a retrieval of a known February timestamp and record the time it takes in the runbook, because regulators ask for the data, not for a bucket name.
3. Was disabling the Cloud Guard detectors acceptable, and what should the tuning process have looked like?
Read the worked answer
Acceptable mechanism, unacceptable execution. Cloning a detector recipe to tune it for production is the supported, documented way to adapt Cloud Guard - you do not edit the managed Oracle recipes. The failure was the decision itself: disabling the detectors removes coverage for every resource in scope, forever, with no expiry and no compensating control. If a detector was firing 300 problems, the noise was a signal about a stale instance principal that should have been deleted, not a reason to switch the detector off.
What good tuning looks like.
- Fix the root cause first: retire the unused instance principal and the dynamic group behind it.
- Keep the detector enabled. Tune the problem threshold in the cloned recipe, or narrow the recipe's target compartments, so it still fires on genuine findings.
- If a detector genuinely must be disabled, it goes in the clone with a ticket, a named owner, an expiry date, and a compensating detective control - for example a SIEM correlation rule that catches the same behaviour.
- Review the clone quarterly against the Oracle-managed version; recipes drift and new detectors appear.
Also check the responder side. A responder recipe that auto-remediates or auto-isolates converts a detection into a control, but only if it was tested; an untested responder that deletes a bucket during an incident is a second outage. Record the review date now - the missing date is the finding.