The Third Quarter Is Clean Because the Sample Was Small

The methodology has a structural flaw that invalidates the ratings it produces.

Self-scoring by control owners. Control effectiveness is assessed by the manager whose performance depends on the control appearing effective. That is not an assessment; it is an assertion. Control effectiveness should be scored by compliance or internal audit on the basis of testing evidence -- sample results, exception rates, and issue history -- with business input limited to describing the process.

Mulitplicative model hides the problem. When inherent risk is multiplied by a control score that is always strong, the residual risk always lands low or moderate. The ratings in January are therefore predictable outputs of the mechanics, not conclusions about the bank. A useful check: if every risk stripe scores moderate or lower across multiple cycles, the scoring scale is not being used.

Segregation of duties. The analyst who writes policy should not be the person who monitors adherence to it. Monitoring needs an independent reviewer, or the results need independent validation.

Testing quality. Eleven of nineteen controls tested by walkthrough interview means the bank confirmed that a procedure exists and that someone described it, not that it operated. Walkthroughs are a first step; transaction testing is the evidence. Nineteen of nineteen effective with that mix of methods is a finding about the testing program, not about the controls.

Answer to the chair. The deposit regulations rating is no longer supportable on its face. The appropriate response is to reopen the rating, not to explain it away: the complaint describes a control failure in the adverse action process, and the bank has no evidence base to say how widespread it is.

Immediate targeted testing on the account closure and adverse action process:

• Pull all deposit account closures in the last 12 months and identify which involved a consumer reporting agency report, a fraud suspicion, or a risk-based decision.
• For each, verify whether a notice was sent, whether it identified the consumer reporting agency where applicable, and whether the timing met the requirement.
• Test the same workflow at each branch to determine whether the failure is one branch or systemic.
• Review complaint records for the same theme over 24 months; a single complaint often surfaces a pattern the bank did not measure.

Adjacent testing, because the same weakness likely recurs:

• Reg E dispute timeliness.
• Reg DD change-in-terms notices.
• Reg CC exception holds and funds availability disclosures.
• Account screening and de-risking decisions for fair lending and UDAAP exposure.

Feeding back into the assessment.

• Raise the inherent risk for deposit regulations if the product mix or screening practices warrant it, and lower control effectiveness to reflect tested reality.
• Re-score every control marked effective on walkthrough only, and mark it unevaluated pending testing.
• Add an issue to the issue log with an owner and a date, and reflect it in the MRA and audit plan.
• Document the methodology change in the risk assessment so the committee can see why the ratings moved.

Testing plan redesign. Require transaction testing for any control designated key, set minimum sample sizes by control frequency, and require the tester to be independent of both the control owner and the policy author.

Present it as a self-identified issue with an evidence base, and present it early.

Framing. The bank found the problem through its complaint process, investigated, quantified the scope, and is remediating. That is the control environment working, even though a control failed. The alternative framing -- waiting for the examiner to find it during the next deposit review -- removes all the credit and adds the appearance of concealment.

What to bring:

• A written description of the issue, when it was identified, and how.
• The scope of the lookback and the results, including the number of accounts affected and any remediation paid.
• Corrected procedures and the training completed.
• The revised risk assessment with the reasoning for the changed ratings.
• The methodology changes -- independent control testing, segregation of monitoring from policy, transaction testing requirements.
• A commitment date for the follow-up testing that confirms the fix operated.

What to avoid. Do not describe the risk assessment revision as a refinement if it was triggered by a failure; examiners compare prior and current assessments and will see the movement. Do not attribute the failure to a single employee if the procedure permitted it. And do not assert that the fix is complete before the follow-up testing is done -- say what has been fixed, what is being tested, and when results will be available.

Internal governance. The audit committee should approve the response and receive the follow-up results. If the lookback shows consumer harm, the bank should be prepared to discuss restitution proactively rather than wait to be asked.