The Change Log That Shrinks When You Ask For It
You are the audit lead for a review of a retail bank's loan-origination system. You requested the full change log for the past 12 months - roughly 1,400 changes - to test whether changes were approved before deployment. The application owner supplied a filtered extract of 380 records, stating the remainder were 'minor configuration updates' and out of scope. When you asked the database team for the raw log, they said retention is 90 days and older records have been purged. That same application owner is the individual who approves changes. The engagement objective is broad: assess whether changes are authorized, tested and implemented in a controlled manner. Fieldwork is 70% complete and the report is due in three weeks.
1. How should you classify the unavailable change records, and what should you do in the next 48 hours?
Read the worked answer
Classify it precisely, because the classification drives the reporting.
This is a scope limitation arising from a control failure, not a management-imposed restriction. The distinction matters: a refusal would trigger immediate escalation as an integrity issue; here the cause is that the entity does not retain evidence long enough for the audit to use it. That is itself a finding against the change-management control environment.
Next 48 hours:
- Put the request and the response in writing to the application owner and IT operations, including what was asked, what was supplied, and what was withheld or unavailable. Contemporaneous documentation protects the conclusion later.
- Confirm the retention policy versus reality - obtain the documented log-retention standard. If policy says 12 months and logs survive 90 days, you have a policy non-compliance finding independent of the audit's needs. If policy says 90 days, you have a design deficiency in the standard itself.
- Test whether alternative evidence exists before accepting the gap: CI/CD pipeline records, deployment tickets, release notes, server file timestamps, service-desk records for the 'minor configuration' changes. Often the population can be reconstructed from a second, independent source.
- Escalate to the CAE, and flag for the audit committee that a scope limitation is likely, so the report's wording is not negotiated at the last minute.
- Do not accept the owner's characterisation that the excluded 1,020 records are 'minor'. Whether they are minor is precisely what the audit is meant to determine.
2. Can you base a conclusion on the 380-record extract? Explain what must happen first.
Read the worked answer
No - not as delivered, and the reason is independence of selection, not sample size.
The extract was defined by the same individual who approves changes. That person had both the motive and the opportunity to exclude records that would fail testing. A sample drawn from a management-filtered population can only ever support a conclusion about that population, and you cannot currently define what that population is.
Before any testing:
- Reconcile the extract to an independent source - deployment pipeline logs, ticket counts, or file-level timestamps on the production servers. The reconciliation result (matched / unmatched / unexplained) is itself audit evidence and is arguably more persuasive than the sample test.
- Only then sample, from the reconciled population, using a selection method you control.
- Cover the excluded categories directly. Pull a judgmental sample from the 'minor configuration updates' bucket and determine whether any of them could affect calculation logic, credit decisioning, or access. If even one does, the scoping statement is false and that is a finding.
If reconciliation cannot be completed, state the conclusion as limited to the tested 380 records and say explicitly that completeness of the population was not established. Never generalise a clean result on a management-selected subset into assurance over the full year.
3. Draft the reporting treatment. What goes in the report, in what form?
Read the worked answer
Three separate items, not one blended observation:
- Scope limitation on the engagement objective. State the objective, the evidence that could not be obtained, the reason, and the resulting restriction on the conclusion - e.g. 'we were unable to obtain sufficient evidence that all changes during the period were authorised; our conclusion on this objective is limited accordingly.'
- Control deficiency - log retention. Insufficient retention of change logs to support retrospective review. Include the fact that retention was materially shorter than the audit period the control is expected to evidence.
- Control deficiency - segregation of duties. The change approver also controlled the evidence population. This is the more serious finding because it undermines the detective capability of the control, not just one audit.
Practical points:
- Communicate to those charged with governance, not only to the application owner, since the limitation affects the committee's own reliance on management's assertions.
- Obtain a management response with a named owner and a date for both findings; a deficiency without an owner and date will reappear next cycle.
- Resist folding this into 'observations' or 'housekeeping'. The temptation at 70% fieldwork with a deadline is to soften the language to keep the report on schedule. That trade - deadline against the integrity of the conclusion - should be escalated, not made quietly by the audit lead.