AI Life Cycle Risk Management
AI Life Cycle Risk Management
This domain tests whether you place the right risk activities in each phase: plan and design, data preparation, development, verification and validation, deployment, operation and monitoring, decommissioning. Risk work is continuous, not a gate passed once.
Core rules
- Design fixes most of the risk: intended use, prohibited uses, affected populations and the human-oversight model. Undocumented intended use makes later validation meaningless.
- Data risk covers quality, provenance, representativeness, consent and leakage between training and test sets.
- Model risk covers performance, robustness, adversarial manipulation, hallucination and drift.
- Validation is independent of the build team and measured against pre-agreed acceptance criteria, including disaggregated performance across groups.
- Documentation travels with the model: model card, AI bill of materials for third-party components, version and change history.
- Monitoring needs defined triggers - drift thresholds, incident rates, override rates - that force revalidation or retirement.
Worked example. A credit-scoring model. At design, record intended use (consumer lending) and excluded uses. At the data stage, trace lineage and test labeled outcomes for historical bias. At validation, run disparate-impact analysis across protected groups against a pre-set threshold, not a post-hoc judgment. At deployment, require adverse-action explanations and human review of declines. In operation, monitor population stability and outcome drift quarterly; a breach triggers revalidation. At decommission, revoke access and archive documentation for the retention period.
Traps. Validating only at go-live; testing aggregate accuracy while ignoring subgroup performance; feedback loops where the model's own decisions become the training labels; treating explainability as a documentation checkbox; and forgetting decommissioning entirely.