The Sampling Plan That Missed the Breach

Defect 1 - dormant accounts excluded. The population is incomplete. Dormant accounts are exactly where stale, excessive, or orphaned access hides. A review that excludes them cannot support a conclusion about the account population. This is a population completeness defect and it is the most fundamental, because every downstream conclusion inherits it.

Defect 2 - self-review. Managers reviewing access they approved is not independent. It creates a bias toward 'appropriate' and removes the challenge function. This is a design defect.

Defect 3 - 1,100 sign-offs in a day. At even 30 seconds per account, that is over 9 hours of continuous review. It is not credible that each account received individual consideration. This is a rubber-stamping indicator and suggests the evidence is not what it purports to be.

Defect 4 - arbitrary 2.5% sample. No statistical basis means no defined confidence or margin of error. The sample may be adequate by luck, but the auditor cannot state what the result supports.

Defect 5 - 4 of 30 terminated users still active, all in the reviewed population. This is the most damaging finding, because it is direct contradicted evidence: accounts marked 'appropriate' were in fact inappropriate. It converts the review from 'performed but flawed' to 'performed and ineffective.'

Which most undermines the conclusion? Defect 5, because it is outcome evidence that the control failed. But Defect 1 is the root cause of the scope failure, and Defect 2 is the root cause of the quality failure. In a report, lead with Defect 5 as the finding, then explain that Defects 1–4 are why it happened.

Response to the significance argument: the argument mistakes direction for magnitude. Four failures in 30 is a 13% error rate with a wide confidence interval, but the finding does not depend on precision - it depends on existence. Even one active credential for a terminated employee is a control failure, because the control's purpose is zero tolerance. Statistical significance is the wrong frame for an attribute where the acceptable rate is zero. The 4 cases also came from the 'reviewed' population, which directly falsifies the review's conclusion regardless of rate.

Designing the test:

• Define the population: all terminated employees in the period (say, 2,400), not just a convenience sample. Exclude rehires and contractors in a separate stratum.
• Census where feasible: termination events are a bounded, high-risk population. A full reconciliation of HR termination dates against active directory accounts is cheaper and more conclusive than sampling. Recommend the census.
• If sampling: use attribute sampling with a 0% tolerable deviation rate and, say, 95% confidence. For zero expected deviations, a sample of ~59 gives 95% confidence of detecting a 5% rate; but for a zero-tolerance control, the auditor should state that any deviation is a finding and sample enough to estimate the rate (e.g., 60–100), not to 'pass.'
• Stratify by termination date (recent vs. older than 90 days) and by role sensitivity (payroll admin vs. standard).
• Test the mechanism, not just the outcome: does the deprovisioning job run, and does it reconcile against HR? A control test on the job's logs is stronger evidence than a user sample.

Report framing: the 4 cases are the finding; the census is the remediation validation; the review redesign (independent reviewer, dormant accounts in scope, credible sign-off volume) is the corrective action.

A testable control needs a population, a performer, a frequency, a criterion, and evidence. Draft:

Control description: 'On a quarterly basis, the identity governance team generates a complete access report covering all active and dormant accounts (dormant defined as no login in 90 days), including terminated-employee accounts identified by HR reconciliation. Each account is reviewed by a manager who did not approve the original access request, or by the application owner where no independent manager exists. Reviewers must make an explicit keep/modify/revoke decision per account; bulk approval of more than 50 accounts requires a documented justification and secondary approval. Revocations are executed within 5 business days and confirmed by a system-generated report. The identity governance lead signs off on completion, and results are retained for 3 years.'

Why each element is testable:

• Complete population including dormant - auditor can reconcile the report to the HR and directory extracts.
• Independent reviewer - auditor can compare reviewer identity to the original approver.
• Explicit per-account decision - auditor can check for bulk-approval patterns and the 50-account threshold.
• 5-day execution - auditor can measure the gap between decision and system change.
• System-generated confirmation - auditor can test the report against directory state, not just the spreadsheet.
• Sign-off and retention - auditor can verify the evidence exists and is complete.

Test procedure for next cycle: obtain the population, reconcile to HR/directory, sample 60 accounts across strata, verify independent review and decision, trace revocations to system logs, and independently test 30 terminated users. Pass criterion: no active credentials for terminated users and no bulk approvals above threshold without justification.