Three Deviations in a Change-Management Sample: Reliance, Extrapolation, and the Deadline Squeeze

Point estimate vs. conclusion. The observed rate is 3/60, or 5%, which sits exactly on the tolerable rate. That is not a pass. The upper deviation limit at the chosen confidence level will materially exceed 5%, so the sample does not support the proposition that the true deviation rate is below tolerable. The auditor cannot extrapolate 5% and call the control effective, and cannot call it ineffective on the point estimate alone.

Nature of the exceptions matters more than the count. All three failures fall on one weekend and one vendor patch bundle. That pattern is inconsistent with random, dispersed control failure and consistent with a specific bypass path, such as an emergency change route or a scheduling backlog. This should drive a targeted response rather than a mechanical increase in sample size: test the full population of changes executed in that weekend window, which is likely small and can be examined completely within the available hours.

The email argument. Whether approval occurred is a different question from whether the control as designed operated. If the documented control requires approval recorded in the change system, an approval buried in a personal mailbox is not the defined evidence and the mailbox has no retention policy, so it cannot be relied on for the period. The auditor should inspect the emails if produced, but should conclude the control did not operate as designed while noting that authorisation may have existed informally.

Reportable position. Pending the targeted testing, the supportable conclusion is that operating effectiveness is not established. The auditor should tell the financial auditor immediately so that reliance can be reduced and substantive procedures extended over the affected applications, and should consider whether a compensating control, such as pre-deployment QA sign-off, mitigates the risk before the audit committee meeting.

A compensating control is only usable if it addresses the same risk at a comparable level of precision, over the same period, and has itself been tested. Work through four tests.

Same risk, same direction. The failed control is preventive authorisation: stopping unauthorised change from reaching production. QA testing is largely detective and answers whether the change works, not whether it was requested by someone entitled to request it. It reduces the chance that unapproved functionality reaches production only because a reviewer sees the change content. It does not prevent a malicious or self-serving change that the developer describes innocently. The auditor should conclude it is a partial mitigation, at best, and say so explicitly rather than treating it as a substitute.

Independence and precision. Who employs QA, to whom does it report, and has QA ever rejected a change? A QA function inside IT, signing off on its own colleagues' work, is not independent. Test a sample of the QA evidence itself rather than accepting the description.

Coverage of the actual exceptions. Did QA test the three deviant changes, and did it review authorisation or only functionality? If QA evidence is missing for exactly the population at risk, the mitigation does not apply where it is needed.

Period and evidence retention. The control must have operated throughout the reliance period and must leave evidence the auditor can re-perform. If QA records are stored in the same unprotected mailboxes, the same completeness problem recurs. Only after these four are satisfied can the auditor document the compensation and reduce, not eliminate, the assessed risk.

Reducing the sample after seeing the results is not a scoping decision, it is a result-driven one, and it would leave the auditor defending a conclusion that the evidence cannot carry. The response should be framed around what the conclusion needs, not around refusing to cooperate.

Alternatives that fit the budget.

• Replace the reduced random sample with complete examination of the high-risk sub-population: the weekend of the vendor patch bundle, plus all emergency changes in the period, which is where the risk concentrates and where testing is cheap per item because the volume is small.
• Test the automated elements of the control (workflow-enforced approval fields, system-enforced segregation of developer and deployer access) using a smaller sample, since automated controls can be tested at lower volume once the configuration and change history are verified.
• Re-perform only the review step with a focused attribute test, and separately corroborate with the change advisory board minutes for the deviant weekend.

Documentation. The auditor should record the original design, the reason for any change in approach, and the resulting limitation, so that a reviewer can see the decision was driven by risk and evidence rather than by the calendar.

Escalation. If the partner insists on a reduced sample that cannot support reliance, the auditor documents the disagreement and the effect: the financial auditor is told reliance is withdrawn on this control. Scope limitations of this kind belong in communication with those charged with governance, not in silence. The deadline is a fact to be managed; it is not a basis for changing what the evidence supports.

The report should be written so a board member can see the difference between what was found, why it matters, and what is being asked.

Observation (facts only). Of 60 changes sampled from 1,240, three lacked approval recorded in the change system. All three occurred in the same weekend window and related to a single vendor patch bundle. Approvals are asserted to exist in individual mailboxes; no retention policy applies. Testing of the identified weekend population is in progress or complete, with results stated.

Risk (consequence). Unauthorised or undocumented change can reach production without a record of who accepted the business risk, which undermines both the integrity of the change pipeline and the audit evidence supporting the IT general controls opinion. Because the exception clusters rather than scatters, the exposure is likely a bypass route available to all staff during high-volume periods, not three isolated lapses.

Recommendation (specific and owned). Require approval to be recorded in the change system as the sole evidence of authorisation, with emergency changes captured in the same record and reviewed retrospectively within a defined window; eliminate mailbox-based approval for production change; and assign an owner and date for each action.

Management response and status. Include management's position, including the claim that email approvals exist, without adopting it as fact. State the auditor's current conclusion: operating effectiveness not established pending completion of targeted testing. Note the effect on the financial audit so the committee understands the consequence is not administrative. Avoid naming individuals; the finding is about the process.