The Training That Everyone Passed
A regional health plan's compliance team launches its annual fraud, waste, and abuse (FWA) and privacy training in October. Completion is tracked in the LMS. By the December 31 deadline, 96% of 1,800 employees have completed it; the 4% who have not are mostly in two acquired clinics whose managers say 'the LMS doesn't recognize their old email addresses.' In February, an internal audit finds that a claims-adjustment unit of 22 people has been sharing one login to complete training 'to save time,' and that the module's final quiz can be retaken unlimited times with the answer key visible after the first attempt. The chief compliance officer must report training effectiveness to the audit committee in March.
1. Is the 96% completion figure a valid measure of training effectiveness? Explain what it does and does not tell the CCO.
Read the worked answer
No - 96% is a completion metric, not an effectiveness metric, and here it is also inflated.
What it tells the CCO: most employees have a record of having been exposed to the content, which supports a defensible 'we trained them' position for regulators and supports the element of effective education.
What it does not tell the CCO:
- Whether learning occurred. Unlimited retakes with a visible answer key means a 'pass' can reflect trial-and-error, not knowledge.
- Whether the right people completed it. Shared logins mean 22 people may have one completion record; the acquired-clinic gap means the denominator and identity mapping are unreliable.
- Whether behavior changed. Completion says nothing about whether staff can recognize an FWA red flag or a privacy incident in their actual work.
The CCO should report completion with explicit caveats and pair it with at least one effectiveness indicator (e.g., quiz first-attempt pass rates by unit, post-training scenario assessment, audit findings, or help-desk question trends). Presenting 96% as effectiveness to the audit committee would overstate assurance and is itself a reporting risk.
2. Identify the specific training-program control failures and the corrective actions for each.
Read the worked answer
Control failures and fixes:
- Shared logins / one completion for 22 people. This defeats individual attestation and creates a records-integrity problem. Fix: enforce unique credentials, block concurrent sessions, require individual attestation, and re-train the 22 with individual records. Consider whether the unit's behavior signals a workload or access barrier that drove the shortcut.
- Unlimited retakes with visible answer key. This destroys the assessment's validity. Fix: limit attempts, randomize question order/answers, hide the key until the attempt is submitted, and require a passing threshold on a first or limited attempt. Keep a remediation path for those who fail.
- Acquired-clinic identity/email mapping. The 4% gap is a data-integration failure, not employee noncompliance. Fix: reconcile the HR roster to the LMS, fix identity provisioning for acquired staff, and set a hard escalation for non-completion that distinguishes 'didn't do it' from 'couldn't access it.'
- No effectiveness measurement. Fix: add a post-training scenario assessment and track first-attempt pass rates and downstream indicators by unit.
Each fix should have an owner and a completion date so the March report can show remediation status, not just the original number.
3. How should the CCO frame the March audit-committee report so it is honest without being alarmist?
Read the worked answer
Frame it as assurance with known limitations and a remediation plan. Structure:
- Headline: 'Training completion is high (96%), but our measurement has gaps we are correcting; completion should not be read as proof of effectiveness.'
- What we know: completion by population; the 4% gap is an access/provisioning issue being fixed.
- What we found: shared logins in one 22-person unit and an assessment design that permits answer-key viewing and unlimited retakes - these undermine the validity of the pass records.
- What it means for risk: the organization cannot currently demonstrate that these employees can recognize FWA/privacy issues; that is a control gap, not evidence of wrongdoing.
- Remediation with dates and owners: identity enforcement, assessment redesign, re-training of the affected unit, roster reconciliation.
- How we will measure going forward: first-attempt pass rates, scenario-assessment results, and audit/help-desk indicators.
This is honest (it discloses the weakness), proportionate (it does not claim fraud or a crisis), and forward-looking (it commits to measurable fixes). The committee should leave knowing both the number and its limits.