The Sampling Plan That Missed the Risk
A compliance auditor must audit outpatient claims for a multi-site physician group with 40,000 claims/year across 6 sites. Last year's audit used a random sample of 60 claims and found a 4% error rate. This year, a whistleblower alleges upcoding of level-5 visits at one site (Site 3). The auditor proposes repeating the same random 60-claim sample across all sites. The budget allows 100 chart reviews. Leadership wants a 'statistically valid' error rate for the whole organization to report to the board.
1. Evaluate the proposed sampling plan. What are its weaknesses, and what sampling approach would you recommend?
Read the worked answer
Weaknesses of the proposed plan:
- Ignores the risk signal. A purely random 60-claim sample across 40,000 claims will almost certainly miss or under-detect a site-specific upcoding pattern. Random sampling is inefficient for targeted risk.
- Sample size too small for site-level conclusions. 60 claims across 6 sites ≈ 10 per site - far too few to detect a meaningful error rate at Site 3.
- Statistical validity claim is overstated. 60 of 40,000 gives a very wide confidence interval; a 4% finding could plausibly be 1%–10%. Reporting a single organization-wide rate to the board masks site variation.
Recommended approach - stratified/targeted design:
- Stratify by site and by code level. Over-sample Site 3 and level-5 codes (the alleged risk area).
- Use a discovery sample at Site 3 (e.g., 30–40 claims) to estimate the error rate; if elevated, expand.
- Maintain a random baseline sample across other sites for the organization-wide estimate.
- Allocate the 100 reviews: e.g., 40 targeted at Site 3 level-5 claims, 60 stratified random across other sites.
- Document rationale so the design is defensible as risk-based, not arbitrary.
2. How should the auditor report results to the board given the design constraints?
Read the worked answer
- Separate the two questions: (1) organization-wide error rate (from the random/stratified baseline) and (2) Site 3 targeted findings (from the focused sample). Do not blend them into one number.
- State confidence intervals, not just point estimates. Say '4% ± X% at 95% confidence' rather than '4% error rate.'
- Flag limitations: small sample, targeted oversampling, non-extrapolatable targeted findings.
- If Site 3 shows a high error rate, recommend a full population review of that site's level-5 claims rather than extrapolating from a small sample.
- Board communication: lead with the risk finding and the corrective action plan, not just a rate. Boards need to know what is being done, not only a statistic.