Forty-Five Days to Launch: Claims Triage Across Two Regimes
Meridian Health Insurance plans to deploy ClaimAssist, an LLM assistant that summarises medical records and recommends approve / deny / escalate for 1.2 million claims a year across Spain, Ireland, Texas and Ohio. It is a vendor-hosted API model with retrieval over 900 internal policy documents; Meridian does no training or fine-tuning. The programme office has a 45-day launch window tied to open enrolment and a contractual penalty clause.
The compliance file contains the vendor's SOC 2 Type II, an ISO/IEC 42001 certificate covering the vendor's AI management system, and DPIAs for the Spanish and Irish entities. It does not contain a classification decision for the EU AI Act, a fundamental rights impact assessment, a human-oversight design document, or any state insurance department filing. The master services agreement gives Meridian no audit rights, no model-change notification, and no restriction on the vendor using submitted claims text for service improvement. Medical coding staff have been told their role becomes "exception handling", but no workflow has been published.
1. Is ClaimAssist a high-risk AI system under the EU AI Act, and what must Meridian produce before go-live? Show the classification reasoning rather than citing a single article.
Read the worked answer
Step 1: role. Meridian does not place the system on the market under its own name and does not fine-tune it, so it is a deployer, not a provider. That single answer changes the obligation set more than the classification does.
Step 2: Annex III fit. Point 5(b) covers AI used for risk assessment and pricing of natural persons in life and health insurance. Claims adjudication is adjacent but not identical; the deciding factor is whether the output feeds coverage, underwriting or pricing decisions, or only the payment of an already-covered claim. A "recommend deny" output that a coder rubber-stamps is functionally an eligibility decision and should be treated as in scope.
Step 3: the Article 6(3) filter. It is unavailable if the system performs profiling or materially influences the decision. An LLM summarising a psychiatric record and recommending denial fails that test.
Consequences. A fundamental rights impact assessment before first use (Article 27 applies to deployers of Annex III 5(b)), a documented human-oversight design (Article 26), log retention of at least six months, post-market monitoring, serious-incident reporting, and informing affected persons and workers' representatives. Meridian must also obtain and review the provider's instructions for use and Annex IV technical documentation.
What does not count. The ISO/IEC 42001 certificate attests to the vendor's AI management system; it is not evidence of Meridian's deployer compliance and does not substitute for the FRIA. SOC 2 covers controls, not rights.
Timing. Annex III obligations phase in from 2 August 2026 in the current text; confirm any adopted amendment before relying on that date. Until then the DPIA and Article 22 GDPR analysis carry the load.
2. Which contract terms are non-negotiable before any claim is processed, and what is your fallback if the vendor refuses each one?
Read the worked answer
Rank the gaps by what they make impossible to do, not by legal tidiness.
- Model-change notification with version pinning. Without it, nothing Meridian validates stays valid. Ask for 30 days' notice of material change, the right to pin a version, and a re-validation trigger. Fallback: pin the version contractually for the term and treat any forced migration as a re-procurement event.
- No training on Meridian data plus retention limits. This is a GDPR purpose-limitation problem that a DPIA cannot paper over. Fallback: no realistic one at scale. Pseudonymising claims text before submission destroys the clinical context and the product with it; if refused, the EU deployment does not happen.
- Audit and testing rights. Needed to evidence oversight and to run your own red-team prompts. Fallback: a shared assessment report plus a right to receive penetration-test summaries annually, and a contractual commitment to remediate named findings.
- Incident notification inside 24 hours with model-specific detail, so Meridian can meet its own downstream clocks. Fallback: 72 hours plus a named security contact.
- Indemnity and regulatory-fine allocation, plus an exit plan and data-export format.
The test for each fallback is whether it leaves Meridian able to evidence a control to a supervisor. If it does not, it is a risk acceptance for the accountable executive, not a negotiation outcome for the programme office.
3. The CEO wants a go/no-go recommendation in 48 hours. Give your recommendation and the reasoning structure behind it.
Read the worked answer
Recommendation: split the decision by jurisdiction and by decision authority, not a single yes or no.
EU (Spain, Ireland): no production deployment of deny recommendations. Missing FRIA plus no training-data restriction is not a documentation gap, it is a lawful-basis problem. Permitted interim state: internal analyst use on de-identified historical claims to build the evaluation set.
US (Texas, Ohio): conditional go in advisory mode only. The model drafts a summary and a suggested disposition; a licensed adjuster makes and signs the decision; both the suggestion and the override are logged. This preserves the business date without creating an automated decision. Confirm with counsel whether either state's insurance department requires a filing or disclosure for AI used in claims before the first decision is issued.
Structure the reasoning in four questions. (1) Is the model good enough? Answer with the validation evidence, false-decline impact by segment, and the override rate from the pilot. (2) Can we evidence compliance? Today, no, in the EU. (3) What is the cost of being wrong? Wrongful denial of a health claim is a harm to a person, regulatorily visible and hard to reverse. (4) Who owns the residual risk?
Conditions of the go. Named accountable executive sign-off with an expiry date; published oversight workflow agreed with the coding team's representatives; weekly monitoring on override rate, denial rate by segment, and complaint volume; automatic rollback to advisory-only if denial rate for any segment moves more than two standard deviations from the pilot baseline.
Do not let the 45-day window convert a compliance question into a schedule question.