The Metric That Looked Great
A compliance team reports monthly to the board. Its headline metric is '98% of staff completed annual compliance training.' The board has accepted this as evidence of a healthy culture for three years. A new Chief Compliance Officer digs in: completion is measured as 'clicked through the module,' the pass mark is 60%, and 31% of completions were done in the final 48 hours before the deadline. A staff survey shows only 44% could correctly answer a scenario question about reporting a suspected bribe. Two whistleblowing reports in the past year were mishandled at the intake stage. The board's audit committee chair says, 'But training completion is 98% - what's the problem?'
1. Evaluate whether the 98% metric is a valid measure of compliance culture.
Read the worked answer
No. The metric measures exposure to a module, not learning or behavior. Three defects:
- Completion ≠ comprehension. Clicking through with a 60% pass mark and mass last-minute completion indicates box-ticking, not understanding.
- The survey contradicts the metric. Only 44% could answer a basic anti-bribery reporting scenario correctly. That is the outcome measure that matters, and it is poor.
- Behavior evidence is worse. Two mishandled whistleblowing reports suggest the reporting channel - the behavior the training is supposed to enable - is failing in practice.
A 98% completion figure alongside 44% scenario accuracy and mishandled reports is not a healthy culture; it is a metric that has drifted from the risk it was meant to represent. Presenting it to the board for three years without challenge is a reporting-assurance failure: the board was given comfort it had not earned. The metric isn't wrong to collect, but it cannot be the headline evidence of culture.
2. How should the CCO reframe reporting to the board?
Read the worked answer
Reframe around outcomes and leading indicators, and be honest about the old metric's limits.
Proposed reporting set:
- Knowledge outcome: % passing a scenario-based assessment (not click-through), with a meaningful bar.
- Behavior/leading indicators: volume and nature of whistleblowing reports, time-to-acknowledge, intake quality audits, speak-up survey results.
- Channel health: were reports handled correctly? Track mishandling as a defect.
- Lagging: substantiated cases, regulatory inquiries.
Framing to the audit committee chair: explain that 98% completion tells you people saw the module; the 44% scenario accuracy and the two mishandled reports tell you whether the program works. Offer to keep completion as a process metric but demote it from headline.
Also fix the underlying program: raise the pass mark, add scenario content, and audit the whistleblowing intake process, since that is where behavior is actually failing. The reporting change and the program change should be presented together so the board sees cause and corrective action, not just a metric swap.