The analytics cluster
A personal injury insurer's new network-analytics model flags 62 claims in 90 days, clustered around three repairers and two medical clinics. The model uses claimant postcode, payment method, time of claim submission and links between phone numbers. Its reported precision is 71%, but that figure came from validation on 2022 data and has not been re-tested. The SIU has six investigators; a full investigation costs about €1,500 and takes three weeks. The head of claims wants to place all 62 claims on hold, send letters stating that the claims are 'suspected fraudulent', and report the three repairers to the police. A journalist has asked whether insurers are 'using algorithms to spy on honest claimants'. The board's fraud savings target for the year is €4m.
1. Evaluate the model output as evidence. Which features create risk, what does 71% precision mean in practice, and how would you triage the 62 claims?
Read the worked answer
The model output is intelligence, not evidence of fraud by any individual.
Feature risk
- Postcode is a proxy for income and ethnicity. It has low individual predictive value, high fairness risk, and is hard to defend to a regulator or a journalist.
- Time of submission and payment method are weak indicators with obvious benign explanations - shift work, direct debit cycles, vulnerability.
- Phone-number links are more defensible, but only where the link is meaningful. A shared clinic switchboard is not the same as a number used by a claimant and a repairer.
- Validation on 2022 data means the model has not been tested against current behaviour; concept drift since then may have degraded it further.
What 71% precision means. Roughly one in three flagged claims is not fraudulent. Across 62 flags that could mean around 18 innocent claimants. That number belongs in the board paper, not only in the SIU.
Triage
- Score each claim against independent indicators: same-day reporting and treatment, treatment duration and modality, solicitor and agency links, repair estimate anomalies, prior claims, invoice inconsistencies, distance between home and treatment.
- Select the strongest 8–12 for full investigation, not all 62.
- Run a cheap validation: take five low-scoring flagged claims and five unflagged claims and check outcomes, to test the model against reality.
- Each hold needs its own reason and its own human decision-maker, recorded on the claim file.
- Ask the model owner to remove or fairness-test postcode and to re-validate before further use.
2. Advise the head of claims on the proposed bulk hold, the wording of the letters and the police report. What are the risks and what would you do instead?
Read the worked answer
Bulk hold and the wording. Holding 62 claims and writing that they are 'suspected fraudulent' is not proportionate. Many will be genuine. The letter is an allegation of dishonesty to consumers, several of whom will complain or litigate, and it will be quoted by the journalist. It also weakens the insurer if a claim later proves sound.
Better letters say what is needed, not what is suspected: 'We are unable to complete our assessment until we receive...', a revised decision date, a named contact, and the complaint and ombudsman route. Hold only where there is a claim-specific reason.
Undisputed elements. Do not freeze rehabilitation, undisputed repair costs or interim payments where liability is clear, unless there is a claim-specific reason. Withholding money to force settlement is a conduct risk and in some jurisdictions a regulatory breach.
Police report. The model does not support reporting three repairers. Report only after documentary evidence of false invoices or a pattern supported by witness evidence, and after legal advice. A report made on model output risks reputational damage and, if it is wrong, legal exposure for the insurer and the individuals named.
Media. Give a short factual statement about protecting customers from fraud and about paying genuine claims quickly; never discuss individuals. Prepare for the question 'how many innocent people have you accused?' - the honest answer must not begin with 62.
Tell the board the estimated false-positive count, the complaint exposure and the average hold duration, and make clear the €4m target must not be met by counting voidances.
3. What governance, controls and ethical safeguards should surround the model and the fraud savings target?
Read the worked answer
Model governance
- Documented purpose and lawful basis; a data protection impact assessment; version control; a re-validation schedule with defined performance thresholds.
- Feature justification: every feature must be explainable and defensible; postcode should be removed or fairness-tested.
- Fairness testing on outcomes - who is held, who is paid, how long each group waits - not just on model accuracy.
Human control
- No automated adverse decision. A named decision-maker per claim, with authority limits and a written reason.
- Independent audit of a random sample of non-flagged claims, to detect missed fraud and drift.
Outcome monitoring
- Overturn and complaint rates, hold duration, the proportion of holds producing a confirmed fraud finding, and distribution of holds by postcode, ethnicity proxy and vulnerability.
- Board reporting on all of these, not only on savings achieved.
Incentives and ethics
- Savings targets drive the wrong behaviour. Split the KPIs: detection quality (confirmed fraud per investigation), false-positive rate, customer outcomes, complaint uphold rate. Remove individual bonuses tied to voidances or repudiations.
- A protected route for staff to challenge a hold or a target without penalty, and a whistleblowing channel.
- Senior accountability: one executive owns the model and its customer outcomes.
The ethical position is not softness. Wrongful accusations cost more in complaints, litigation and trust than the claims saved, and the insurer's licence to investigate depends on being seen to investigate fairly. A model is a reason to look; it is never a reason to conclude.