Operational Risk, Third-Party, and Model Lifecycle
What this domain tests
Operational, third-party, and lifecycle questions test resilience and control across outsourcing, change, and retirement.
Core rules
- The lifecycle runs from ideation and inventory through development, validation, deployment, monitoring, change, and decommissioning.
- Third-party risk includes due diligence, contract terms, ongoing monitoring, audit rights, concentration risk, subcontractors, and exit plans.
- Material model changes, including data, features, vendor version, or use case, need impact assessment and revalidation.
- Model inventory should record owner, purpose, tier, dependencies, versions, and status.
- Decommissioning needs data retention, access removal, archival, and replacement validation.
Worked example
A firm uses a vendor LLM API for customer support. Risks include data leakage, silent model updates, service outage, and vendor failure. Controls include a data-processing agreement, no sensitive prompts without approval, notification of material model changes, a fallback workflow, output logging, and an exit plan. The firm retains residual risk because it cannot audit the vendor full training pipeline.
Traps
Relying on annual reviews; treating vendor certification as eliminating residual risk; ignoring fourth parties; and allowing shadow AI outside the inventory.