AI Data Security and Privacy
Domain 5 tests data protection across the AI lifecycle: collection, preparation, training, retrieval, inference, and deletion. Expect questions on sensitive data in embeddings, memorization, access control, and privacy-by-design.
Core rules and concepts
- Classify data before use: public, internal, confidential, restricted; include prompts, outputs, logs, embeddings, and fine-tuning data.
- Data minimization and purpose limitation apply to training and RAG corpora. Do not ingest data you cannot lawfully use or secure.
- De-identification is not a binary. Pseudonymized data can still be re-identified; embeddings can leak semantic content and sometimes reconstruct text.
- Enforce access control at retrieval and inference time, not only at the database. Per-user filtering prevents RAG from surfacing unauthorized documents.
- Protect against memorization and extraction: limit sensitive training data, test for regurgitation, rate-limit queries, and monitor extraction patterns.
- Manage retention and deletion across model artifacts, caches, vector indexes, and backups. Deleting a source document does not automatically remove its embedding.
Worked example
An HR RAG system indexes employee relations case files. The team plans to store embeddings in a shared vector database. The exam wants the privacy issue identified: embeddings are derived personal data and may reveal case details; shared index access bypasses source permissions. Controls: separate collections per sensitivity tier, per-user metadata filters, encryption, restricted admin access, deletion pipeline that removes vectors when source records are deleted, and extraction monitoring. The residual risk is accepted only after legal review.
Traps that cost marks
- Treating embeddings as non-sensitive because they are numeric.
- Assuming anonymization is permanent and irreversible.
- Ignoring prompts, logs, and outputs in data inventories.
- Deleting source data without deleting derived artifacts.
- Relying on vendor privacy terms as the only control.