The Risk Assessment That Ranked the Wrong Thing
A payments company runs an annual enterprise compliance risk assessment. The methodology scores each business line on inherent risk (transaction volume, geography, customer type) and control effectiveness, then multiplies to get residual risk. This year, the highest residual score went to the small crypto pilot, largely because its control score was rated weak by a single reviewer who had never assessed that unit before. The largest business line, domestic card processing, scored low because its controls were rated strong, but that rating was based on a self-assessment the unit submitted without evidence. The compliance team has 6 staff for 14 business lines. The chief risk officer wants the results finalized in three weeks for the risk committee, and the crypto pilot's launch depends on the outcome. The head of the crypto pilot has emailed twice asking to discuss the rating.
1. What are the methodological problems with this assessment, and how should the compliance team address them before finalization?
Read the worked answer
Two problems stand out. First, the crypto pilot's high residual score rests on a single reviewer's control rating with no calibration. A single unreviewed rating is not a reliable control assessment, and it is doing a lot of work: it is driving the top residual ranking and affecting a launch decision. The team should require a second reviewer, a documented rationale, and evidence for the rating, then recalibrate.
Second, the domestic card line's low residual score rests on a self-assessment with no evidence. Self-assessment is an input, not a control test. A 'strong' rating without evidence is unsupported, and because the line is the largest by volume, an understated residual score there is more consequential than an overstated one in the pilot. The team should require evidence for the self-assessment or downgrade the control rating to 'not validated' and treat the residual score as provisional.
The team should also check whether the inherent-risk inputs are consistent across lines, since geography and customer type may be scored differently by different reviewers. With 6 staff and 14 lines, full re-testing is not feasible in three weeks. The team should prioritize: validate the two lines that drive the top and bottom of the ranking, document the basis for every rating that changes, and flag unvalidated ratings in the report rather than presenting them as final.
2. How should the compliance team handle the crypto pilot head's requests to discuss the rating, and what should the risk committee be told?
Read the worked answer
The pilot head has a legitimate interest in the rating because a launch decision depends on it, but the timing and repetition of the requests create pressure risk. The team should respond in writing, explain the methodology and the specific basis for the rating, and offer a structured calibration meeting with the second reviewer present. The team should not negotiate the score in a private conversation, and should not change the rating because of the emails. Any change should be driven by evidence, documented with the reason.
The team should also separate two questions: whether the rating is accurate and whether the launch should proceed. A high residual score does not automatically block a launch; it triggers conditions, such as enhanced monitoring, transaction limits, or a pre-launch control test. The team should give the risk committee both the rating and the recommended conditions rather than treating the score as a verdict.
To the risk committee, the team should present: the methodology, the two lines with unvalidated ratings, the actions taken to validate them, the residual uncertainty, and the launch recommendation with conditions. The report should state plainly which ratings are evidence-based and which are provisional. Presenting a provisional score as final, especially where a business launch depends on it, would mislead the committee and invite a challenge the team cannot defend.