CAICSO common mistakes
CAICSO · Understand the traps and how to avoid them.
Treating AI governance frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) as interchangeable checklists and answering a framework-specific question with controls from a different framework.
Candidates memorize control names in isolation and never map which framework is voluntary vs. regulatory, or which function (Govern, Map, Measure, Manage) a given activity belongs to.
What to do instead
Build a one-page crosswalk table: rows are activities (risk assessment, data provenance, human oversight, logging), columns are NIST AI RMF function, ISO/IEC 42001 clause, and EU AI Act obligation. Practice classifying a described activity into the correct framework and function before choosing an answer.
Classifying a system's risk tier from its technology (e.g., 'it uses an LLM, so it is high-risk') rather than from its use context, affected population, and decision impact.
Candidates anchor on model capability and vendor marketing instead of the regulatory and organizational criteria that actually determine risk tier.
What to do instead
For every scenario, write the use case, the decision it informs, who is affected, and whether a human can contest the outcome. Decide tier from those facts first, then note the technology. Drill scenarios where the same model is minimal-risk in one deployment and high-risk in another.
Confusing inherent risk with residual risk, and proposing controls without stating the residual risk that remains after those controls.
Risk registers are often read as flat lists, so candidates treat 'we added a filter' as eliminating risk rather than reducing it to a stated level with an owner and acceptance decision.
What to do instead
For each risk item, state likelihood and impact before controls, the specific control, and the post-control rating with a named acceptor. In practice questions, eliminate options that claim a control removes risk entirely; prefer options that quantify and formally accept residual risk.
Answering prompt-injection and jailbreak questions with generic 'sanitize input' advice instead of distinguishing direct injection, indirect injection via retrieved or tool-returned content, and model-level jailbreaks.
The threat taxonomy is compressed in candidates' minds into one 'bad input' category, so they miss that indirect injection bypasses input filters because the payload arrives through trusted data channels.
What to do instead
Memorize three distinct attack surfaces: user prompt, retrieved/tool content, and model behavior under adversarial prompting. For each, know the matching control class (input/output filtering, content provenance and trust labeling, least-privilege tool scoping, human confirmation for high-impact actions). Practice labeling a described attack with its surface before selecting a control.
Choosing 'add a classifier guardrail' as the primary defense against data poisoning or backdoor attacks on training data.
Guardrails are the most familiar control, so candidates reach for them even when the compromise occurs upstream in the data supply chain where inference-time filters cannot see it.
What to do instead
Map attacks to lifecycle stage: poisoning and backdoors are training-time, so controls are data provenance, source allowlisting, deduplication, outlier and influence analysis, and evaluation on trigger-pattern test sets. Reserve guardrails for inference-time behavior. State the stage explicitly before answering.
Designing AI security architecture around the model endpoint alone, omitting the surrounding retrieval store, vector index, feature pipeline, agent tool permissions, and model registry.
Candidates picture 'the AI' as the model API and forget that most real exposure sits in the data and orchestration layers that the model calls.
What to do instead
Draw the full reference architecture: data sources, ingestion, feature/embedding store, model registry, serving endpoint, orchestration and tool layer, output handling, logging. For each component, name the trust boundary and the control at that boundary. Answer architecture questions by locating the component in this diagram first.
Treating privacy compliance as satisfied by de-identifying the training set, while ignoring membership inference, model memorization, and sensitive attribute leakage from embeddings.
Candidates apply classic data-protection habits (remove names, hash IDs) that do not address model-level leakage, and they underestimate that embeddings can be inverted or linked back to individuals.
What to do instead
Separate data-level controls (minimization, de-identification, consent basis) from model-level controls (differential privacy, memorization and extraction testing, embedding access controls, output filtering). For any privacy scenario, ask both 'what data went in' and 'what can be extracted out'.
Describing MLOps security as CI/CD for code, missing model artifacts, training data versions, and the model registry as supply-chain assets requiring signing, lineage, and access control.
Candidates reuse software supply-chain intuition and stop at source code scanning, so they miss that the model weights and datasets are the actual deployable artifacts.
What to do instead
List the AI supply chain artifacts explicitly: datasets, feature definitions, training code, hyperparameters, weights, evaluation reports, and deployment configs. For each, state integrity control (signing, hashing), provenance record, and access restriction. Practice questions by asking which artifact is unprotected in the scenario.
Writing incident response that stops at detection and containment, without AI-specific triage criteria, rollback to a prior model version, or post-incident retraining and evaluation gates.
Candidates inherit classic IR phases but do not adapt them to a system whose behavior can degrade silently and whose 'fix' is a redeployment rather than a patch.
What to do instead
Build an AI IR playbook with named triggers (quality drift, extraction attempts, prompt-injection success, bias complaints), a rollback path to a signed prior model version, evidence preservation for prompts and outputs, and a re-evaluation gate before restoring service. Practice by ordering these steps for a given incident.
Framing AI security communication to executives as a technical control list rather than as business risk in their language, with no quantified exposure, no decision requested, and no clear owner.
Candidates default to demonstrating technical depth and omit the risk-appetite and accountability framing that leadership needs to act.
What to do instead
Structure every leadership answer as: business outcome at risk, likelihood and impact in financial or regulatory terms, options with trade-offs, recommended option, owner, and decision deadline. Keep technical detail in an appendix. Practice converting a technical finding into a two-sentence executive ask.