The Tier 3 That Wasn't: Classifying a Vendor LLM Under a Legacy Model Risk Policy

Not defensible. The tier was derived from the wrong variables.

Why the reasoning fails

• The policy's only tier driver is financial materiality. For AI, the defensible drivers are impact on customer outcomes, degree of autonomy, scale and repeatability of errors, reversibility, and affected population - not whether funds move.
• SOC 2 Type II attests to security, availability and confidentiality controls. It says nothing about model behaviour, accuracy, fairness or vendor change discipline. It is not evidence of low model risk.
• 'Vendor-hosted' describes delivery, not impact. The bank still owns the outcome.
• The tool suggests offers and drafts customer-facing communication at 45,000 chats a month. Errors are systematic, not one-off, and non-deterministic behaviour defeats sampling-based sign-off.

Classification Treat as high tier (Tier 1) under an interim impact-based rubric while the policy is amended. The customer-conduct dimension (possible unfair treatment, mis-selling of offers, complaint exposure) is sufficient on its own.

Policy gap to name explicitly The 2019 definition ('quantitative method producing an estimate') structurally excludes generative and agentic tools, so the policy cannot classify them at all - this is a scope defect, not a business-unit failure. Recommend amending the definition to cover any algorithmic system that informs a decision, including vendor, embedded and shadow AI, and to require impact-based tiering.

Flag separately: 14 recorded models against an estimated 120 means tiering is happening without an inventory - a governance finding in its own right.

The business unit cannot self-determine the tier, and neither can the vendor's certification.

Roles

• First line: owns the risk, the control design and the pilot.
• Second line (model risk, compliance): independently challenges the tier and validates; cannot be overruled by the pilot sponsor.
• AI governance committee: the decision forum for tier, conditions and any policy exception.

Sequence

1. Second line issues a written tier challenge with rationale - do this now, not at the next committee.
2. If the committee cannot convene before the pilot date, the policy's delegated authority (typically the CRO, sometimes jointly with the CIO) decides the tier and conditions, and the decision is ratified at the next meeting. Decide in advance which route applies and record it; 'the committee meets once a quarter and the business moves faster' is itself the governance finding.
3. Escalate to the board risk committee because three triggers are met: new risk type, a likely policy exception, and board AI principles (transparency, human oversight) that the current design does not evidence.

Anti-patterns the examiner is testing

• Letting a pilot proceed 'while governance catches up'. A pilot on live customer chats is a production deployment with a smaller user count.
• Allowing the vendor's unwillingness to disclose to become a reason to lower the tier. Opacity raises risk; it does not reduce it.
• Treating the six-week deadline as a constraint on the decision rather than on the build.

Approval should be conditional and time-boxed, with pre-agreed stop criteria.

Conditions precedent

• Human-in-the-loop: every drafted response reviewed by an agent before sending; no auto-send for the pilot's duration.
• Scope restriction: no offers, pricing or eligibility recommendations in phase one - draft and summarise only.
• A frozen, versioned evaluation set spanning customer intents and segments, scored before go-live for accuracy, hallucination and differential treatment by segment.
• Logging of prompts, retrieved context and outputs, with defined retention.
• A kill switch and a tested rollback path.
• Pilot limits: one queue, no more than 50 agents, eight weeks, written exit criteria.

Contract changes to require before signature

• 90-day advance notice of model changes, with release notes and evaluation evidence, and a right to defer an update.
• Right to test and audit; access to evaluation methodology.
• Prohibition on training on bank data; data residency; incident notification window.
• Exit and data-portability terms; indemnity for third-party claims.

Monitoring Weekly: sampled hallucination rate, complaints, escalation rate, containment, agent override rate, intent-mix drift, fairness slices. Define thresholds now - e.g. a material error rate above 2% in a weekly sample triggers pause. Report to the AI governance committee at each stage gate against the written go/no-go criteria, so continuation is a decision rather than a drift.