The Audit Log Evidence That Does Not Add Up
While assessing practice 3.3.1, audit logging, the OSA provides: a 12-month-old policy stating logs are retained for one year; a screenshot of a Windows event log setting showing 'Overwrite events as needed' with a 20 MB maximum; and a SIEM dashboard showing 30 days of retention. In an interview, the system administrator says logs are kept 'at least 90 days' and that the SIEM forwards everything. When you ask for a log from 100 days ago for a specific CUI server, the administrator cannot produce one and says, 'The SIEM vendor changed and we lost some history.' The OSA security lead tells you the policy is the authoritative evidence and the screenshot is 'just a default.'
1. Evaluate the sufficiency and appropriateness of the evidence provided for 3.3.1.
Read the worked answer
Conclusion: The evidence is not sufficient for a 'met' conclusion. It is conflicting and partly contradicted by direct testing.
- The policy is appropriate for intent but not sufficient to show implementation. It is 12 months old and may not reflect current practice.
- The screenshot shows overwrite-as-needed with a 20 MB maximum. That is configuration evidence for one system, but it contradicts the one-year retention policy and the administrator's 90-day statement.
- The SIEM dashboard showing 30 days directly conflicts with both the policy and the interview.
- The inability to produce a 100-day-old log for a named CUI server is direct evidence that retention is not operating as claimed.
- Sufficiency means enough evidence from enough sources. Here the sources disagree, so the assessor cannot conclude the practice is met. The security lead's assertion does not override technical evidence.
2. What additional evidence and interviews would you conduct, and how would you resolve the conflict?
Read the worked answer
Additional evidence:
- SIEM retention and index configuration, not just the dashboard.
- Exported logs from several dates, including an attempt to retrieve an event older than 90 days.
- Windows event-log settings from a sample of CUI servers, with timestamps and hostnames.
- Change tickets or vendor-migration records that explain the lost history.
- Backup or archive configuration for logs.
Interviews and observation:
- Interview the SIEM administrator and a SOC analyst separately from the system administrator.
- Observe a live search for an event from 100 days ago.
Resolution:
- Reconcile the policy against the implemented configuration. A policy may be aspirational; the assessment tests the implemented state.
- If the OSA cannot produce logs sufficient for monitoring, analysis, investigation, and reporting, that supports a finding.
- Document each source, date, and discrepancy. Do not infer compliance from an absent record.