Twenty-Three of Twenty-Five
During a compliance assessment of a payment platform, the assessor reviews account termination. Policy requires accounts disabled within 24 hours of separation. HR exported 412 separations from the past year; the assessor sampled 25 and found 2 accounts still enabled 9 and 14 days after the termination date, both retaining production read access. The identity team's ticket log shows the deprovisioning task was created automatically every time and closed manually by the technician, with no system check of actual account state. The control owner calls the two cases 'exceptions' and asks that they be excluded from the report because 23 of 25 passed and the overall rate is 'over 90%.'
1. Determine whether the control is effective and respond to the exclusion request.
Read the worked answer
The control is not operating effectively, and the exclusion request must be declined.
The policy states a 24-hour requirement. That is a binary obligation, not a percentage target, so 'over 90%' is not the standard being measured. Two failures in 25 samples is an 8% observed failure rate; more importantly, the presence of any failure means the assessor cannot conclude the population meets the stated requirement, and cannot state that the remaining 387 separations were handled correctly.
Exclusions apply to scope - a component or period genuinely outside the boundary - never to samples that failed. Removing failures from a sample is a misrepresentation of the test result and would invalidate the report.
The correct output is a finding: the control is other-than-satisfied, with both instances documented, their access level noted as material (production read), and the exposure period recorded. The control owner may add a management response; they do not get to edit the result.
2. Distinguish design effectiveness from operating effectiveness here, and identify the root cause.
Read the worked answer
Design looks adequate: a policy exists with a defined window, HR feeds separations, and ticket creation is automated, so the triggering mechanism does not depend on someone remembering.
Operation failed. The gap is that ticket closure is a manual act with no verification that the account state changed. The ticket can be closed while the account remains live - and in both instances it was. That is a process design flaw surfacing as an operating failure: closure is decoupled from evidence of the outcome.
Root cause: the workflow measures task completion rather than effect. The technician's action is not validated against the identity provider, and no independent reconciliation compares HR separations to active accounts.
Remediation should address the cause, not the symptoms: require closure to be gated on an automated query of account state; add a monthly reconciliation of HR separations to active directory accounts with a named reviewer; and investigate the remaining 387 separations to determine whether these two are isolated or indicative. Cleaning up the two accounts alone would leave the failure mechanism intact.
3. The control owner proposes re-testing a fresh sample of 25 and, if clean, reporting the control as effective. Respond.
Read the worked answer
Re-testing is useful but it does not retract the original result.
The earlier sample was drawn properly and its failures are facts about the control during the audited period. They remain in the record as the finding for that period. A clean re-test, run after remediation, evidences the remediated state - it is remediation evidence and can be reported as such, with dates making clear which period each result covers. It cannot be used to overwrite the prior period, and suggesting otherwise is the kind of pressure an assessor should document.
Conditions for the re-test to be meaningful:
- The root-cause fix must already be in place, not just the two accounts disabled. A clean sample drawn from a broken process is luck.
- The new sample must be independently selected, not nominated by the control owner.
- Evidence must include the system check that gates ticket closure, plus the reconciliation output for at least one cycle.
If remediation is not yet complete, the honest position is: finding stands, corrective action open, re-test scheduled after implementation.