Logging Ingest Cost and Retention Conflict
A tenancy runs 150 compute instances across prod, stage, and dev compartments. The OCI Logging agent uses one agent configuration that collects /var/log/app/*.log at TRACE level and sends all entries to log group app-logs. A service connector hub forwards every log object to Object Storage bucket app-archive and to a SIEM through Streaming. Daily ingest is 4.2 TB. Logging and storage now account for 38% of monthly spend, more than compute. The security team requires 365-day retention for authentication and audit logs, and 90-day searchability for security investigations. The application team needs DEBUG-level logs only during incidents, for no more than 4 hours. The current Object Storage lifecycle rule deletes all objects after 30 days, which already conflicts with the security requirement.
1. What changes would you make first, and how do you avoid losing security evidence?
Read the worked answer
First, split the single agent configuration into separate configurations by log type. Put authentication, audit, and other security-relevant logs in a security log group with a controlled retention and forwarding path. Put application logs in an app log group at INFO by default, with a temporary DEBUG configuration that expires after 4 hours during incidents. Add a Service Connector Hub filter so DEBUG entries are not forwarded to the archive bucket or the SIEM. Keep the security logs flowing throughout the change, and verify with test events that audit logs still reach the SIEM. Do not delete existing buckets or log groups until the new paths are validated, because the current archive is the only copy of older security logs.
2. Design retention and storage tiers for 365-day security retention while controlling cost.
Read the worked answer
Use the Logging service for short-term search and investigation: for example, 30 to 90 days for security log groups, depending on the investigation requirement. Use Object Storage lifecycle policies by prefix to move security archives to an archive or auto-tiering tier for the remainder of the 365 days, and keep application logs on a shorter lifecycle. Correct the current blanket 30-day delete rule so it does not apply to the security prefix. Document retrieval time for archived objects so investigators know the expected delay. If some security data must be searchable for the full 365 days, keep only the required fields in Logging and archive the full raw logs. This separates fast search from cheap long-term retention.
3. How would you measure success and allocate the remaining logging cost?
Read the worked answer
Apply cost-tracking tags to log groups, buckets, and service connector hubs by compartment and purpose. Use Cost Analysis to compare ingest volume and storage cost before and after the change, and set a budget with alerts on the logging cost center. Monitor daily ingest volume and object counts by prefix, and alarm if application DEBUG volume reappears outside an approved window. Review the top log producers weekly. Success is lower ingest volume and storage cost while security logs still meet 365-day retention and 90-day searchability, with no gap in SIEM delivery.