Accepted Risk That Was Never Treated: Reading the Risk File Against Reality
Paylane, a payments provider certified to ISO/IEC 42001 for two years, runs the "Sofia" fraud-scoring model, retrained monthly since a January architecture change. During surveillance you find: the AI risk assessment, scheduled for annual review, still describes the pre-January architecture; the risk treatment plan lists six actions with target dates between March and June, all now overdue; July top-management minutes record acceptance of residual risk for three of those risks; the AI system impact assessment for Sofia dates from the original design and has not been repeated; and one treatment action depends on a data-processing agreement with a new vendor that is still unsigned, although the vendor has supplied training data since April. No incidents have been logged in the period.
1. Which requirements does the stale risk assessment engage, and how do you separate that from the overdue treatment actions?
Read the worked answer
Two distinct conclusions, two distinct findings - do not merge them.
Risk assessment. The requirement is to review the assessment at planned intervals and when significant change occurs. A major architecture change in January, followed by monthly retraining, is a significant change by any reasonable reading, and the assessment still describes the old architecture. The finding is that the risk assessment does not reflect the system as it operates. That is a failure of the process itself, and it contaminates everything downstream that relies on it - including the July risk acceptance.
Treatment actions. Overdue actions are a separate question: are the actions themselves still appropriate, and is the plan being managed? Delay may be a resourcing problem or may indicate the plan was never realistic. Check whether progress is monitored, whether revised dates were approved, and whether anyone assessed the risk of the delay.
Aggravating fact. The unsigned agreement means a treatment credited in the plan is not actually in place while the vendor has been supplying data since April. The stated residual risk therefore does not exist; the real exposure is higher than the file says. Grade accordingly - the combination of a stale assessment and an unimplemented treatment supporting an acceptance decision is systemic, and a major grading should be considered.
2. Can you accept the July record of residual-risk acceptance? Explain how far the auditor may go in judging management's decision.
Read the worked answer
Risk acceptance is a legitimate treatment option and management's prerogative - but only when it is informed and made within the organisation's own criteria.
The auditor does not substitute their risk appetite for the client's. The test is whether the decision stands up on process grounds:
- Was it made by top management, or by someone with delegated authority under defined criteria?
- Was it based on a current assessment? Here it was not - the assessment describes the pre-January architecture, so the decision was taken on information that no longer matches the system.
- Did the acceptance account for the fact that one credited treatment is not implemented? On the evidence, no; the residual figure is overstated in the client's favour.
- Is it documented and traceable to the specific risks?
Conclusion. The acceptance does not satisfy the requirement that residual risk be accepted by top management on the basis of the risk assessment, because the assessment was stale and the treatment position misstated. Raise a nonconformity joining the finding to the underlying process failure, and ask for the risk to be re-assessed and re-accepted on correct information. If, on re-assessment, management still accepts the risk within its criteria, that is a valid outcome and the auditor must say so in the file.
3. The impact assessment has not been repeated since design, and the vendor has been sending training data under an unsigned agreement. How do you structure these findings and what is your overall audit conclusion?
Read the worked answer
Impact assessment. The requirement is to keep the impact assessment current, repeating it at planned intervals and on significant change. A January architecture change and monthly retraining constitute such change; a design-phase assessment is no longer evidence about the system in operation. Raise a nonconformity: the organisation cannot demonstrate that impacts on individuals have been reconsidered for the current model. Ask what triggers the organisation has defined - if none exist, the process gap is systemic.
Third-party data. Training data supplied since April under an unsigned agreement indicates that activities affecting AI data were performed before the relationship was defined and controlled. Frame it against the organisation's own supplier and data controls: due diligence, agreed terms, and monitoring. Whether a legal exposure exists is for others; the audit finding is the absence of an agreed, controlled basis for an operating data flow.
Overall conclusion. No incidents in the period does not indicate effectiveness - the risks were misdescribed, so nothing was set up to detect them. Taken together, the stale assessment, the unimplemented treatment underpinning an acceptance, the outdated impact assessment and the uncontrolled data flow show the operation has drifted from the documented system. Recommend that the AIMS is not effective in this period and set out what must be re-verified before the next decision point.