AI Incident Response and Monitoring
Domain 7 tests incident response and monitoring for AI systems: how to detect AI-specific incidents, contain them, preserve evidence, and meet notification duties. Expect playbook and monitoring-design questions.
Core rules and concepts
- Extend the incident taxonomy: model inversion, data extraction, prompt injection, poisoning, harmful output, drift causing unsafe behavior, agent tool misuse, and third-party model outage.
- Monitor inputs, retrievals, tool calls, outputs, model versions, and drift metrics. Alert on anomalies such as unusual extraction patterns, sudden refusal-rate changes, or unexpected tool invocations.
- Define severity with business impact and safety impact, not just technical anomaly. Include a threshold for declaring an AI incident.
- Containment options: disable a tool, revoke credentials, roll back a model or prompt, block a data source, rate-limit, or take the service offline.
- Preserve evidence: prompts, retrieved documents, model version, logs, and decision records. Maintain chain of custody.
- Coordinate with legal, privacy, communications, and the model vendor; know regulatory and contractual notification timelines.
Worked example
A RAG assistant starts returning another customer contract text. Monitoring shows a spike in retrieval results with mismatched tenant metadata. The playbook: declare a privacy incident, disable the retrieval path, roll back the index change, preserve logs, identify affected tenants, notify privacy and legal, and issue customer notices per contract. Root cause is a metadata filter regression in deployment. The fix includes a regression test and a deployment gate. The exam wants containment and evidence preservation before root-cause analysis.
Traps that cost marks
- Using a generic IT playbook with no model rollback or prompt/version controls.
- Waiting for root cause before containing a data leak.
- Not logging prompts and retrievals, making scope impossible to determine.
- Treating vendor incidents as solely the vendor problem.
- Missing notification timelines in contracts and regulation.