AI Security and Data Protection: Threats, Controls and Privacy
Domain 4 - AI Security and Data Protection. The exam tests AI-specific threat modelling alongside ISO/IEC 27001 and ISO/IEC 27701 controls, and whether you can connect privacy principles to training, inference and logging.
Core rules
AI expands the attack surface: data poisoning, adversarial inputs, model extraction, membership inference, prompt injection, insecure model supply chains and model theft. Controls include provenance checks, least-privilege access, input and output validation, rate limiting, secure MLOps, red teaming and vendor assessment. Privacy controls include lawful basis, purpose limitation, minimisation, retention limits, data-subject rights and DPIA for high-risk processing. Training data can contain personal data, and synthetic data is not automatically anonymous.
Worked example
A customer-support LLM retrieves internal documents. Threat model: prompt injection via user input; retrieval with excessive permissions; personal data leaking into logs. Controls: least-privilege retrieval scoped to the requester, output filtering, PII redaction in logs, adversarial testing, a DPIA, and contractual security terms with the model vendor. Reasoning links each control to a named threat rather than adding blanket encryption.
Traps
Assuming a vendor-hosted model transfers all responsibility; treating anonymisation as guaranteed; securing the model but not the data pipeline; ignoring logs as personal data; confusing confidentiality with the full set of privacy principles.