340 Alerts, Two People
Veritas Health Plans runs compliance analytics over 1.2 million expense transactions per year. One rule flags any expense between $4,750 and $4,999 - just below the $5,000 approval threshold - with a round-hundred dollar amount. It produced 340 alerts last quarter. The compliance department has 2 FTE for monitoring. Last year 290 alerts were reviewed, 11 led to substantiated findings, and 6 resulted in discipline. Business units complain the reviews are slow and disruptive. The rule was written in 2020 and has never been tuned. There is no documented sampling methodology, and reviewers keep notes in personal spreadsheets.
1. How do you triage 340 alerts with 2 FTE? Defend the approach as defensible coverage, not just workload management.
Read the worked answer
The constraint is real: 340 alerts, roughly 2 FTE, and a historical substantiation rate of about 3.8% (11 of 290). Reviewing everything superficially and reviewing a subset properly are not equivalent - but neither is quietly dropping alerts without a record.
Tiered approach:
- Tier 1 - 100% review. Any alert involving a government official, a healthcare professional, a third-party intermediary, a submitter or approver with a prior finding, or a vendor with a single-source relationship. These are small in number and carry the highest consequence.
- Tier 2 - 100% review of repeat patterns. Same submitter, approver, or cost center appearing multiple times across the quarter; split invoices (two transactions just under $5,000 with the same vendor).
- Tier 3 - stratified random sample of the remainder. Sample across business units and quarters rather than taking the first 80 by date. Document the population, the sample size, the selection method, and the rationale in the workpaper.
Defensibility comes from documentation, not from volume. The question a regulator or auditor asks is "how did you decide what to review, and can you show it?" A written stratification rule answers that. An undocumented backlog does not.
Also reduce the friction causing the business complaints:
- A 15-minute triage form that clears obvious legitimate items (pre-approved contract pricing, recurring vendor charges) and escalates the rest.
- A published service-level target and a named contact, so business units know what to expect.
- Track precision by tier and feed it back into rule tuning.
2. How would you determine whether this rule is still effective, and what would you change?
Read the worked answer
Measure three things before touching the rule: precision, unique contribution, and cost.
- Precision. 11 findings from 290 reviews is roughly 3.8%. Low - but precision alone is not a reason to retire a rule.
- Unique contribution. Back-test the rule against the last two years of substantiated expense cases. If it caught cases no other rule would have caught, it is earning its place. If the findings were also caught by other analytics, it is redundant.
- Cost. Estimate investigator hours per alert and compare to the severity of the findings. A low-precision rule that catches high-severity conduct can still be worth running; a low-precision rule that only catches $200 of unsupported spend is a candidate for retirement or redirection.
Do not conclude "low precision, kill it." The rule also has a deterrent effect that is invisible in the precision metric, and removing it without replacement reduces monitoring coverage.
Tuning options:
- Narrow the round-hundred condition, which is probably driving most of the volume through legitimate pricing patterns.
- Add complementary analytics that catch what this rule misses: split-payment detection across vendors, duplicate vendor master records, weekend and holiday submissions, after-hours approvals, and approver override patterns.
- Segment the rule by business unit so high-volume, low-risk areas are sampled rather than fully reviewed.
Reasoning to state: rule effectiveness is a portfolio question, not a per-rule question. The right output is a documented decision - keep, tune, or retire - with the data behind it.
3. What do you do about the personal spreadsheets and the absence of a documented methodology? Prioritize the fixes.
Read the worked answer
Prioritize by what creates the most exposure per unit of effort.
- Stop the note-keeping in personal spreadsheets first. This is the highest-risk item because it is a records integrity problem: notes on personal devices may be lost, are not access-controlled, may contain sensitive HR or personal data, and cannot be produced reliably in a legal or audit request. Move to a shared, access-controlled case repository with a standard workpaper template (alert ID, triage decision, evidence, disposition, reviewer, date).
- Write the sampling methodology down. One page: population definition, stratification rules, sample sizes, escalation criteria, and who approves deviations. This converts ad hoc judgment into a repeatable control.
- Define retention. Specify retention periods for alerts, workpapers, and dispositions in line with the company's records policy and legal-hold requirements; make deletion a controlled action, not a convenience.
- Build a metrics pack. Alerts received, reviewed, substantiated; median cycle time; findings by category; cost per finding; and referrals to policy, training, or remediation. Report trends to the Audit Committee, not raw quarterly counts.
- Close the loop. Every substantiated finding should trigger a decision about the underlying cause: policy wording, approval threshold design, training, or manager accountability. Monitoring that produces findings but no program change is the most common failure mode in this domain.