Tiering a Vendor Chatbot Under a Half-Built AI Policy

Tier assignment should follow capability and access, not vendor reputation. The assistant generates customer-facing text and reads PII, but it has no decision authority, no payment function, and no write access to accounts. Under Meridian's own policy the PII trigger is dispositive: read access to name, address and card last-four is still PII access, so Tier 1 applies unless the design changes. The reviewers diverged because the rubric asks for judgements ('is the vendor trustworthy?') rather than observable facts ('which data classes can the system read?', 'can it act?'). Observable criteria produce reproducible scores.

Note also that a support chatbot is not, by function, a high-risk system under the EU AI Act's Annex III - it is not credit scoring or employment screening - but transparency duties to disclose AI interaction apply to the EU customers, and the subprocessor and residency position creates a separate transfer problem. Recommendation: Tier 1, conditional on PII minimisation (tokenised account references, field-level retrieval) and a revised rubric scored on data class, autonomy, reversibility, and population size. If the committee instead re-scopes the system to hold no PII, Tier 2 is defensible.

Launch-blocking issues: (1) no documented training-data provenance - without it Meridian cannot assess copyright exposure, PII memorisation, or poisoned-data risk, and cannot answer a regulator; (2) weekly updates with no changelog or pre-release evaluation, which defeats Tier 1 change management; (3) subprocessors and residency inconsistent with policy and transfer rules; (4) no data-processing terms prohibiting the vendor from training on Meridian data. These are not documentation gaps, they are unassessed risks.

Satisfiable within 10 days: PII minimisation in prompts; prompt and response logging with defined retention and access control; an incident clause that defines 'incident' to include model compromise; a subprocessor list with 30-day change notice; a tested kill switch and fallback to a human-only queue; a pre-launch red-team covering prompt injection and data exfiltration; and a model card. Each condition needs a named owner and evidence, not 'best efforts'. If the vendor will not move on provenance and change notice, the correct decision is to delay or run a human-supervised pilot with PII masked - Meridian's own policy is the framework a regulator will test it against.

Treat the vendor model as a controlled component with a known version, not a service that silently changes underneath the bank. Controls: (a) contract a 'stable' channel with semantic version pinning and at least 7 days' notice for material changes, defined as changes to base model, fine-tuning data, safety policy, or tool permissions, with security patches exempt but logged; (b) require a model card and an eval delta per release; (c) run Meridian's own regression suite before each version is exposed - injection attempts, PII-extraction probes, refusal tests, and task accuracy on a golden set; (d) canary 5% of traffic with automatic rollback on error, complaint, or eval thresholds; (e) record the model version with every response so incidents can be scoped.

Fallback if the vendor refuses: pin a version under a long-term-support commitment, evaluate continuously against the pinned version, and place a gateway in front that can route to a second vendor or a human queue. Document the residual risk and compensating controls for the risk committee. The principle is that 'the vendor updates it' does not transfer the bank's accountability; post-market monitoring duties fall on the deployer.