AI Risk Monitoring, Reporting, and Regulatory Compliance
Core concepts
Monitoring detects drift, degradation, and emerging risk after deployment. Candidates must know data drift, concept drift, model performance decay, output distribution shifts, and behavioral anomalies. Monitoring must cover inputs, outputs, and outcomes, and include fairness monitoring for subgroup performance over time.
Reporting translates monitoring into decisions: KRIs and KPIs, dashboards, incident reporting, escalation thresholds, and periodic model risk reporting to the board or risk committee. Understand model inventory reporting, exception management, and audit trails.
Regulatory compliance requires mapping obligations to controls. Know the EU AI Act obligations for high-risk systems (risk management, data governance, technical documentation, transparency, human oversight, accuracy/robustness/cybersecurity, post-market monitoring), GDPR implications (automated decision-making, data subject rights), and sector rules (financial, healthcare). Understand transparency and disclosure duties for AI-generated content and chatbot interactions.
Traps
Questions often treat monitoring as only performance tracking; fairness and drift monitoring are equally required. Another trap: assuming compliance is a one-time certification; it is continuous. Watch for reporting that lacks defined thresholds and owners.
Rule of thumb
Every monitored metric needs a threshold, an owner, and an action. A metric without an escalation path is not a control.
Worked example
A fraud model's AUC remains stable, but approval rates for one demographic drop 15%. Performance monitoring alone would miss this. Fairness monitoring with a defined threshold triggers review and escalation.