RAG Permission Bypass Over Sealed HR Documents
A law firm deploys a retrieval-augmented generation assistant over 50,000 internal documents across 8 practice groups. Document-level access controls are synchronized from the document management system every 24 hours. During a red-team test, a paralegal in the litigation group asked for 'all severance terms for departing partners' and the assistant returned a verbatim excerpt from a sealed HR memo. The memo had been moved to a restricted folder 10 hours earlier. The vendor says the retrieval index is 'eventually consistent' and recommends adding a disclaimer. The firm's CIO wants to keep the system live for all staff because 400 lawyers use it daily. The AI security lead must decide on containment and the technical control changes needed before re-enabling broad access.
1. Recommend immediate containment and explain why a disclaimer is insufficient.
Read the worked answer
Containment: suspend broad access immediately; restrict the assistant to a small incident-response group or to documents marked public; disable retrieval over the HR and sealed-matter repositories; purge retrieval caches and generated summaries; preserve logs of the red-team query, retrieved chunks, and user sessions; notify ethics and privilege counsel and the affected practice group; assess whether the sealed memo's contents were further disclosed.
A disclaimer is insufficient because the harm is unauthorized retrieval, not user reliance on a caveat. The access-control failure is in the retrieval layer: the index was stale for 10 hours, so a document moved to a restricted folder remained retrievable. Adding a disclaimer does not fix the permission model and may increase the firm's notice of the defect. The system should stay restricted until query-time authorization and index freshness are demonstrated by a repeat red-team test.
2. What technical control changes are required before re-enabling?
Read the worked answer
Move from index-time to query-time permission filtering: retrieve only chunks whose document ACL grants access to the requesting user, and enforce the same check on generated citations. Reduce ACL synchronization lag to minutes or trigger reindexing on permission change. Maintain a revocation queue and purge embeddings for restricted documents. Use document-level security trimming before the LLM sees content, not after.
Add a permission-aware test suite with canary documents, sealed matters, and HR memos. Test cross-practice-group queries and 'moved 10 hours ago' scenarios. Consider separating sensitive repositories from the general index. Monitor retrieval logs for denied documents and anomalous queries. Require a vendor SLA on index consistency and the right to audit. Re-enable first to a pilot group of 30 lawyers with DLP alerts, then expand only after two clean red-team cycles.
3. What governance and monitoring measures should accompany the technical fix?
Read the worked answer
Assign a product owner and a security owner, and add the RAG system to the AI inventory as high risk because it spans privileged and HR data. Require access reviews quarterly, red-team exercises after any connector or ACL change, and incident criteria for unauthorized retrieval. Monitor denied retrievals, stale-index alerts, documents with changed ACLs, user queries touching restricted terms, and DLP events. Report monthly to the AI governance committee and include vendor performance. Update the vendor contract for audit, breach notification, and index consistency. Train users that the assistant is not a discovery tool. Define a privilege-preservation protocol so that a retrieval incident does not waive privilege.
The key governance judgement is that 'eventually consistent' is not acceptable for a legal privilege boundary. The firm should accept residual risk only with a documented exception and a time-bound remediation plan.