AI-RRS common mistakes
AI-RRS · Understand the traps and how to avoid them.
Treating 'AI risk' as synonymous with model accuracy/performance risk, so answers focus on model error rates when the scenario is about fairness, privacy, or third-party/supply-chain risk.
Candidates with ML engineering backgrounds anchor on the technical failure of the model rather than the broader organisational and societal harm taxonomy the exam uses.
What to do instead
Before answering, classify the scenario against the full AI risk taxonomy (performance, bias/fairness, explainability, privacy, security, third-party, regulatory, reputational, operational). Practice mapping one scenario to multiple risk categories and picking the one the stem emphasises.
Confusing inherent risk with residual risk, or rating a control's effectiveness as if it reduced inherent risk before controls were applied.
The two-stage rating (inherent then residual) is easy to collapse into a single gut-feel score, especially under time pressure.
What to do instead
Always run the two-step calculation explicitly: rate likelihood and impact assuming no controls (inherent), then re-rate after crediting only controls with evidence of operating effectiveness (residual). Write both numbers down before selecting an option.
Mixing up governance bodies' roles, e.g., assigning model approval to the risk function, or assigning independent validation to the first-line development team.
Three-lines-of-defence terminology overlaps with generic corporate governance language, and candidates default to 'risk owns risk decisions'.
What to do instead
Memorise the division: first line owns and operates the AI system, second line sets frameworks and challenges, third line provides independent assurance; the governing body/committee approves risk appetite and material model decisions. Test yourself by assigning each activity in a scenario to a line.
Selecting a framework-specific artefact (e.g., a particular clause number or document name) as the answer when the question is testing the underlying principle.
Candidates over-memorise framework structures and then pattern-match keywords instead of reading what the question actually asks.
What to do instead
Answer the principle first, then check which option expresses it; treat framework names as context, not as the answer key. When two options are both 'correct-sounding', choose the one that satisfies the stated objective rather than the one that cites a document.
Assuming a control that is 'implemented' is also 'effective', and recommending documentation or policy updates as the mitigation for a technical risk.
Policy and documentation are the most familiar controls, so they are chosen by default even when the risk is a live model behaviour.
What to do instead
Match control type to risk type: technical risks need technical/preventive controls (validation, monitoring, guardrails, access control), while governance risks need process controls. Ask whether the proposed control would actually change the outcome in the scenario.
Skipping the risk-identification step and jumping straight to a mitigation, so the answer addresses a symptom rather than the root cause described in the scenario.
Experienced practitioners recognise the scenario type and recall a familiar remedy, short-circuiting the analysis.
What to do instead
Force a one-line root-cause statement before looking at options ('the risk is X because Y'). Eliminate any option that mitigates a consequence rather than the identified cause.
Choosing the most restrictive control (e.g., disable the system, full human review of every output) as the 'safest' answer regardless of proportionality.
Risk-aversion bias plus the intuition that stricter equals better; candidates ignore cost, feasibility, and business impact stated in the stem.
What to do instead
Apply proportionality explicitly: controls should be commensurate with the risk rating and the system's materiality. Prefer tiered or risk-based controls (e.g., human review for high-risk decisions only) over blanket restriction unless the stem signals unacceptable risk.
Confusing monitoring with testing/validation, or proposing one-off validation as the answer to an ongoing-drift scenario.
Model validation and model monitoring are often taught together, and both involve performance metrics.
What to do instead
Separate the lifecycle: pre-deployment validation establishes fitness; ongoing monitoring detects drift, degradation, and emerging harms in production. If the scenario mentions time, change, or post-deployment behaviour, the answer is a monitoring control.
Missing the reporting/escalation path - answering with 'log the issue' or 'retrain the model' when the scenario involves a threshold breach that requires escalation to a committee, regulator, or client.
Candidates focus on the technical remediation and forget the governance obligation triggered by the event.
What to do instead
For any monitoring scenario, ask three questions: who must be told, within what timeframe, and through what channel? Check whether a defined threshold, materiality trigger, or regulatory notification duty is implied before choosing a purely technical response.
Applying a single jurisdiction's rules to a cross-border scenario, or missing that an AI system used in a regulated activity inherits existing sectoral obligations on top of AI-specific ones.
Candidates learn one regulatory regime in depth and treat it as universal, overlooking that AI rules layer onto financial, health, or data-protection requirements.
What to do instead
For regulatory questions, identify the jurisdiction, the sector, and the role (provider vs deployer/user) before evaluating options, and check whether existing sectoral or data-protection duties still apply alongside AI-specific ones.