The Underwriter Who Says the Model Is Wrong
Your company has deployed a machine-learning model to score small commercial property risks. It was validated on three years of historical data and performs well on aggregate loss ratios. A senior underwriter with 25 years of experience flags that the model systematically under-prices properties in a coastal county that recently changed its building code. She cannot articulate the statistical reason but insists the model is 'blind to something.' The model's vendor says the model is 'explainable' and the underwriter is 'resisting change.' Two months later, a broker submits a large book of business in that county, and the model prices it attractively. You have a production deadline, a skeptical vendor, and an underwriter whose concerns you cannot yet quantify.
1. How should you weigh the underwriter's unstructured concern against the model's validated performance, and what immediate action do you take on the broker's submission?
Read the worked answer
Treat the underwriter's concern as a valid signal of model risk, not as resistance. Her inability to articulate a statistical reason is expected: expert judgment often detects distributional shift before it is quantified. The model was validated on historical data; a building code change is a structural break that makes the training data non-representative for that county - exactly the kind of shift aggregate validation misses.
Immediate action on the submission:
- Do not auto-bind on the model price. Route the book to manual underwriting and reference the code change explicitly.
- Segment the data: compare the county's loss experience before and after the code change; check whether the model's features include code vintage or construction standards.
- Ask the vendor for feature-level behavior on coastal/code-affected risks, not just aggregate explainability claims.
- Document the challenge in the model risk log.
You do not need to prove the model wrong to justify a hold; you need to show the assumption of representativeness is questionable for this segment.
2. Design a governance process so that expert challenges like this one are captured and acted on before a production deadline forces a decision.
Read the worked answer
Build a model risk management loop with defined roles and triggers:
- Challenge channel: any underwriter can log a model challenge with segment, symptom, and evidence. Challenges are triaged within a set period (e.g., 5 business days) by a model risk committee including underwriting, actuarial, and data science.
- Segmentation testing: require periodic out-of-sample review by geography, construction type, and code vintage - not just aggregate loss ratio.
- Change triggers: external events (code changes, catastrophe experience, regulatory shifts) automatically trigger a segment review before the model is used for that segment.
- Human-in-the-loop thresholds: model scores above a size threshold require manual sign-off until segment concerns are resolved.
- Vendor accountability: contracts should require segment-level performance reporting and timely response to challenges.
- Feedback culture: recognize challenges that surface real issues; do not label them resistance.
The goal is to make expert judgment a structured input to model governance, so the deadline pressure never becomes the deciding factor.