CAIS common mistakes
CAIS · Understand the traps and how to avoid them.
Answering governance scenarios by naming a body ('the AI governance committee reviews it') without specifying the tiering threshold that triggers escalation, the named accountable role, the artefact produced, or who formally accepts residual risk.
Committee-and-policy phrasing is easy to recall and feels safe under time pressure, so candidates substitute organisational vocabulary for control mechanics they have not actually mapped to a deliverable.
What to do instead
For every governance scenario, force out four elements: the artefact (model inventory entry, risk assessment, exception record), the owner by role, the trigger threshold (e.g. model touches regulated data or makes autonomous decisions), and the review cadence. Rehearse rewriting 'should be governed' into 'who decides, on what threshold, recorded where, reviewed when'.
Misclassifying adversarial attacks - labelling data poisoning as evasion, or conflating model inversion with membership inference - and then selecting a control that addresses the wrong attack class.
The taxonomy gets memorised as a vocabulary list in isolation from attacker goal, access level and lifecycle phase, so superficially similar names collapse into each other during recall.
What to do instead
Build a four-column matrix (attacker goal, access: black/grey/white box, lifecycle phase: training/tuning/deployment/inference, detectable indicator) and place each attack in exactly one row. Then write the single control that only works for that row, which makes the distinguishing feature concrete rather than verbal.
Treating prompt injection as a model-quality defect to be fixed with better system prompts or fine-tuning, and overlooking indirect injection arriving through retrieved documents, tool outputs, web content or stored memory.
The LLM framing pulls candidates toward prompt-engineering solutions, and a training-phase fix sounds authoritative and complete, so the input trust boundary is never examined.
What to do instead
Re-frame every injection question as an input trust-boundary problem: classify each context source as trusted or untrusted, then require output-side controls - tool allow-lists, human confirmation before state-changing actions, privilege separation between retrieval and execution, and scoped credentials. State explicitly that prompt hardening reduces but never removes the risk.
Answering re-identification questions with 'the data was anonymised' or 'pseudonymised' without addressing linkage attacks, quasi-identifiers, small-group uniqueness, or the fact that a trained model can memorise and later regurgitate training records.
De-identification is a comfortable checkbox from privacy training, and candidates equate applying a technique with eliminating the risk, so residual inference paths are never named.
What to do instead
Structure the answer as technique, residual risk, compensating control, legal test: name the specific method (aggregation, noise addition with the privacy budget stated, generalisation), the residual vectors (linkage, inference, memorisation, canary extraction), the mitigation (output filtering, extraction testing, access constraints), and the regulatory standard being applied.
Assuming the MLOps stack is secure by default - deserialising model artefacts from public hubs, using pickle-format weights, storing API keys in notebooks, and giving pipeline service accounts broad object-store access across projects.
Tooling demonstrations optimise for velocity and say nothing about provenance, and model supply chains are far less familiar than software dependency management, so each hop is treated as trusted infrastructure.
What to do instead
Enumerate the model supply chain hop by hop (dataset, base model, fine-tuned adapters, container image, dependencies, registry, endpoint) and attach one control to each: signature and provenance verification, safe serialisation formats, secrets manager with short-lived scoped credentials, pinned hashes, and network egress restriction on training jobs.
Describing ML lifecycle security as a single pre-deployment review, so retraining runs, adapter updates, prompt and context changes, and dataset refreshes pass without re-approval or re-evaluation.
The classic software SDLC with a go-live gate is over-learned, and models that appear to change no code feel like configuration rather than a security-relevant change event.
What to do instead
Define 'model change' explicitly to include data refresh, hyperparameter change, adapter swap, system prompt revision and retrieval corpus update, then state the re-evaluation trigger set and how evaluation suites run in CI with pass thresholds that block promotion.
Treating drift monitoring as a performance concern only, tracking accuracy decay while ignoring behavioural security signals such as sudden refusal-rate shifts, output-length anomalies, unusual tool-call sequences, or clustered extraction-style probing from a single tenant.
The data-science framing of MLOps dominates candidate preparation, so monitoring is mentally filed under quality rather than under detection and response.
What to do instead
Pair every drift metric with a security-relevant counterpart and name the data source for it - gateway and prompt logs, egress monitoring, tool-call audit trails, per-tenant request profiling - then state the alert threshold, the triage owner and the escalation path when both fire together.
Running a generic incident-response playbook on an AI incident without AI-specific containment and evidence steps: no endpoint or model rollback, no version hash pinning, no capture of the prompt, response, retrieved context and tool calls, and no decision on retrain versus quarantine.
The textbook detect-contain-eradicate-recover sequence is so well rehearsed that the AI-specific artefacts and options never get inserted into it under exam conditions.
What to do instead
Write an AI incident addendum in advance listing the evidence set to preserve (prompt, completion, model version hash, retrieval set, tool calls, tenant identifiers), the containment options ranked by blast radius (disable endpoint, roll back version, restrict tools, rate-limit), and the criteria for retrain versus rollback versus permanent shutdown.
Conflating responsible-AI testing with security assurance - offering fairness metrics, explainability reports or harmful-content red-teaming as the mitigation for adversarial manipulation, extraction or data poisoning.
The two vocabularies overlap heavily and vendor material markets everything as AI assurance, so candidates reach for whichever term surfaced most recently in their study material.
What to do instead
Keep two separate assurance tracks in your notes: safety and harm evaluation (bias, toxicity, disclosure, human oversight) versus security assurance (adversarial robustness testing, extraction and evasion testing, access control and pen-test evidence, supply-chain attestations). When a question asks for security assurance, cite only the second track and say why the first does not substitute.
On 'FIRST' and 'BEST' scenario questions, choosing the most thorough remediation over the correct earliest step, or recommending an action that presumes scope is already confirmed.
Engineering instinct rewards completeness, so well-specified fix-it options look stronger than short containment or validation steps, and the qualifier word is skimmed rather than parsed.
What to do instead
Underline the qualifier before reading the options, then rank candidates by sequence - confirm and scope, contain, assess, remediate, notify - and eliminate any option that acts before scope is established. Where two options are both defensible, prefer the one that preserves evidence and reversibility.