Control Testing Evidence That Does Not Support the Rating
A payments firm's RCSA records a preventive control: 'all new payees require independent verification before first payment.' The control is rated 'effective' and drives a residual score reduction from 12 to 6. The evidence file contains: a procedure document dated two years ago, a screenshot of the payee verification screen, and a manager's statement that 'the team always does this.' No sample testing was performed. Separately, the firm's quality assurance function reviewed 15 RCSA assessments this quarter and found 9 had at least one control rated effective without transaction-level evidence. The head of first line argues that sampling is impractical because payee volumes are high and the control is embedded in the system. The QA lead must decide what to report and what remediation to require.
1. Assess whether the evidence supports the 'effective' rating and explain what evidence would.
Read the worked answer
The evidence does not support the rating. A procedure document proves the control is designed and documented, not that it operates. A screenshot proves the screen exists. A manager's assertion is self-assessment without independent corroboration and is the weakest form of evidence. None of the three shows that verification actually happened on actual payees.
Evidence that would support the rating: a sample of new payees created during the period, tested against the requirement, showing the verification step was performed by someone other than the payee creator, with a timestamp and identifiable verifier. A reasonable sample for a high-volume, system-embedded control might be 25 to 40 items drawn across the period and across teams. If the control is genuinely automated, the evidence should be a system configuration review plus a test that the configuration blocks payment when verification is absent - an attempted bypass is stronger evidence than a screenshot.
The rating should be downgraded to 'partially effective' or 'not tested' until evidence is produced, and the residual score of 6 is unsupported. The reduction from 12 to 6 should be reversed pending evidence.
2. The first line argues sampling is impractical because the control is system-embedded. How should QA respond, and what should the QA report say about the 9 of 15 finding?
Read the worked answer
QA should accept the premise and change the method, not waive the evidence. If the control is system-embedded, the appropriate test is a configuration and design test plus a negative test: attempt a payment to an unverified payee and confirm the system blocks it. That is cheaper than transaction sampling and stronger evidence. 'Impractical to sample' is not a reason to rate a control effective on documentation alone; it is a reason to select a different test.
The QA report should state the 9 of 15 rate plainly as a systemic theme, not as nine individual lapses. A 60% incidence suggests the RCSA methodology does not define what evidence is acceptable, so assessors default to documents and assertions. The report should recommend: an evidence standard by control type (manual, automated, detective), mandatory downgrade where evidence is absent, and a re-performance of the affected assessments. It should also note the downstream effect - residual scores across the portfolio are likely overstated, which affects capital and board reporting. Naming the systemic cause is more useful than listing the nine.