Securing AI Systems and Data
What you must know
This domain covers the technical controls that protect models, training data, prompts, and inference infrastructure. Expect questions on data protection across the lifecycle: classification, minimization, de-identification, encryption in transit and at rest, and preventing sensitive data from entering prompts or training corpora. Know privacy-enhancing techniques at a conceptual level - differential privacy, federated learning, synthetic data - and their tradeoffs (utility vs. privacy).
Model protection includes access control and authentication for model endpoints, rate limiting and query monitoring to blunt extraction and inference attacks, model watermarking/fingerprinting for provenance, and secure model storage and signing to resist tampering. Understand prompt and output controls: input validation, allow-listing, output filtering/DLP, and guardrails placed outside the model rather than relying on instructions inside it. Know secrets management (never embed credentials in prompts or code), tenant isolation in multi-tenant inference, and logging with redaction so telemetry does not become a data leak.
Traps
- Trusting the model to enforce its own safety; controls must be architectural and external.
- Logging full prompts and responses, creating a secondary sensitive-data store.
- Confusing encryption at rest with protection against a legitimate over-privileged user.
- Believing de-identification is irreversible; re-identification via linkage is a common exam theme.
Rule of thumb
Treat the model as an untrusted component behind a trusted boundary: authenticate, validate, filter, log with redaction, and limit what the model can reach.