Automation and AI: Choosing the Right Technique for a Claims Process
An insurance company processes 10,000 claims per month. The process involves: receiving claims via email (70%) and web form (30%), extracting claim details, validating policy coverage, assessing damage from photos, and approving or denying the claim. Currently, 80% of claims are straightforward and could be automated. The company has an RPA program but limited AI expertise. They are considering three options: (1) RPA only with rule-based extraction, (2) RPA + OCR + ML for damage assessment, (3) RPA + conversational AI for email handling + ML for damage assessment. The cost and complexity increase with each option. The company wants to reduce processing time from 5 days to 1 day and improve accuracy. They have a budget of $500,000 for the first year. Regulatory requirements mandate explainability of claim decisions.
1. Evaluate the three options against the company's goals and constraints. Which option would you recommend and why? Consider technical feasibility, cost, explainability, and time to value.
Read the worked answer
Option 1: RPA only
- Feasible for structured data, but emails are unstructured; rule-based extraction may have low accuracy. Damage assessment from photos not possible. Likely insufficient to meet 1-day target and accuracy. Low cost, quick to implement. Explainability high but limited scope.
Option 2: RPA + OCR + ML for damage assessment
- OCR can extract from emails and forms; ML can assess damage. More accurate and faster. Explainability of ML model is a concern; need to use explainable AI techniques (e.g., LIME, SHAP) or rule-based fallback. Cost moderate; requires ML expertise. Time to value 6-9 months.
Option 3: RPA + conversational AI + ML
- Conversational AI can handle email queries, but complexity high. Overkill for initial phase. High cost, long time to value. Explainability of conversational AI is less critical but still needed.
Recommendation: Option 2. It balances capability and cost. Start with OCR for extraction and a simple ML model for damage assessment with human review for low-confidence cases. Ensure explainability by using interpretable models or providing feature importance. This can achieve 1-day processing for straightforward claims. Option 1 is too limited; Option 3 is too risky given limited AI expertise. Also, consider a phased approach: implement Option 2 first, then add conversational AI later if needed.
2. How would you ensure explainability of the ML-based damage assessment to meet regulatory requirements? What design choices would you make?
Read the worked answer
Design choices for explainability:
- Use interpretable models (e.g., decision trees, logistic regression) instead of black-box models like deep neural networks, or use hybrid models.
- If using complex models, apply post-hoc explanation techniques (SHAP, LIME) to generate explanations for each decision.
- Provide a human-readable rationale for each claim decision, e.g., 'Damage detected in left fender based on image analysis; confidence 85%; key features: dent size, location.'
- Implement a human-in-the-loop for borderline cases or when model confidence is low.
- Maintain an audit trail of model inputs, outputs, and explanations.
- Regularly validate model fairness and accuracy.
- Document model development and validation for regulators.
Judgment: Explainability is non-negotiable. The company should prioritize models that are inherently interpretable or use explanation tools. Also, the damage assessment ML should be trained on a diverse dataset to avoid bias. The cost of explainability tools should be included in the budget.