Retrieval Without Authorisation: A Clinical Assistant Over 8.4 Million Chunks
Northlake Health has deployed a retrieval-augmented assistant to 3,200 clinicians. Documents come from 14 source systems: EHR notes, scanned PDFs passed through OCR, HR grievance files, complaints and quality reports, and payer contracts. The pipeline has produced 8.4 million chunks in a managed vector database. Embeddings are generated through a third-party embedding API whose standard terms retain inputs for 30 days for abuse monitoring; an enterprise no-retention tier exists but was not purchased.
Each source system enforces its own access control. The index does not carry those permissions: retrieval filters on tenant_id only. During red-teaming, a nurse practitioner retrieved three paragraphs of a VIP patient's psychiatric note and a fragment of an HR grievance naming a manager. A full re-index runs quarterly and takes six hours, during which the index can serve permissions that were revoked upstream. A product team has now requested a "chat with your documents" upload feature for the same assistant.
1. Explain why per-tenant filtering failed and specify the control pattern that replaces it.
Read the worked answer
Why it failed. Tenant filtering answers "which hospital", not "which clinician". Authorization was enforced in the source systems and then discarded at ingestion, so the assistant became a channel that bypasses every ACL in the estate. Post-filtering the model's answer is not a fix either: the unauthorised text has already been retrieved, may already have been quoted in a citation the user can see, and its presence changes the generated answer even when it is trimmed afterwards.
The pattern: permission-aware retrieval.
- Carry the ACL as chunk metadata. At ingestion, store document_id, source system, owning group, and an ACL version stamp alongside each chunk. Resolve identity from the SSO assertion, not from anything the user types.
- Filter before the vector search on the caller's group membership, deny-by-default, with an explicit allowlist of source systems eligible for this assistant at all.
- Two-stage check on ACL staleness. ANN search returns roughly 50 candidates; re-verify those 50 against the live ACL store before generation. That is cheap and closes the six-hour re-index window.
- Revocation propagation. Push ACL change events to invalidate affected chunks, and set a maximum tolerable staleness (for example 15 minutes) with an alert when the sync job lags.
- Grounding check. After generation, verify every cited chunk was both authorised and actually retrieved; a citation to a chunk the user could not have seen is a control failure, not a formatting bug.
The red-team result was not a model problem. It was an authorization architecture problem that happened to surface through a model.
2. The CISO says the vector database is "just an index, not a system of record". Assess that position and the third-party embedding API.
Read the worked answer
The position is wrong on two counts.
Content. Embeddings of clinical text are derived from protected health information. Inversion research has repeatedly shown that text can be reconstructed from embeddings, and reconstruction quality rises with model size and with access to the embedding model itself. Treat the index as an in-scope data store under the HIPAA Security Rule: encryption at rest and in transit, access logging, key management separated from the application, and a business associate agreement with the vector database vendor. Dimensionality reduction is not a de-identification technique.
Scope. The index also contains HR grievances, complaints and payer contracts. Those were never intended for clinical use. That is a data-minimisation failure at ingestion, and no prompt instruction fixes it: the fix is to exclude those source systems from this assistant's corpus and, if they are needed, build a separate assistant with its own audience.
The embedding API. Sending clinical text to a third party for vectorisation is a disclosure. Under the standard terms, inputs are retained for 30 days and used for abuse monitoring. Either purchase the no-retention enterprise tier with a business associate agreement and data-residency commitments, or self-host an embedding model inside the network. Choose the second if the vendor cannot contractually exclude human review of retained inputs.
Two follow-on questions worth asking. Who approved the procurement without a security review, and does the same embedding pipeline feed any other product? The answer usually reveals a shared service with a wider blast radius than this one assistant.
3. The product team wants the "chat with your documents" upload feature shipped in three weeks. What conditions, if any, would you accept?
Read the worked answer
Accept, with conditions. Refusing outright pushes the feature into a shadow build; accepting unconditionally creates a second copy of the same authorization failure with user-controlled content on top.
Non-negotiable conditions.
- Inherited permissions. An uploaded file is visible only to the uploader and to groups the uploader explicitly names, and that ACL is written into the chunk metadata at ingestion. Uploaded content never enters the shared 8.4 million-chunk index.
- Isolation and TTL. A separate namespace or collection per upload, a defined retention period measured in days, and deletion that removes the vectors, not just the source file.
- Ingest scanning. Malware scan and text extraction in a sandbox, a size and page cap, and rejection of any file whose extracted text contains instruction-like patterns targeting the assistant. Flag those to a queue rather than silently stripping them.
- Prompt-injection containment. Uploaded text is untrusted input. The assistant must have no tool that acts on it and must not be able to call any downstream system from an upload session.
- Provenance in the answer. Every response cites file name and page so a clinician can verify rather than trust.
- Audit. Upload, retrieval and deletion events go to the same SIEM as EHR access, with the uploader's identity attached.
What I would say out loud in the review. Three weeks is enough for the isolation, TTL and logging work. It is probably not enough for file-type coverage and a defensible injection test. Ship with a narrow allowlist of formats, and put the rest behind a flag.