AI Data Security, Privacy and Responsible Data Handling
AI data security, privacy and responsible handling
This domain tests protection of training, fine-tuning and retrieval data across its lifecycle, plus privacy engineering choices that survive regulatory scrutiny.
Core rules
- Classify data, minimize collection, limit purpose, and keep provenance so a source can be audited or removed.
- Pseudonymization is reversible; anonymization must resist re-identification. k-anonymity and l-diversity reduce but do not eliminate linkage risk.
- Differential privacy gives a formal guarantee tuned by epsilon: lower epsilon means more noise and stronger privacy.
- Models memorize. Extraction and membership-inference attacks mean training data can surface in outputs; test for memorization before release.
- RAG indexes are data stores: enforce document ACLs at retrieval, apply retention and deletion to embeddings, and map chunk IDs back to source records for erasure.
- Treat prompts, completions and logs as potential personal data; redact before logging or training.
Worked example
Support transcripts containing account numbers are proposed for fine-tuning. Reasoning: run a DPIA, redact with NER plus an allowlist check, prefer aggregate or synthetic data where quality allows, and add a no-train contractual clause with the vendor. If a customer later exercises erasure, deletion from the dataset is not enough if a model memorized the record: rebuild or unlearn, then verify with extraction probes, and rebuild affected index shards.
Traps
- Calling pseudonymized data anonymous and dropping controls.
- Assuming embeddings cannot leak personal data.
- Forgetting the vector index in retention and deletion schedules.
- Ignoring cross-border transfer rules when calling a hosted model API.