The Row Filter That Did Not Follow the Chunks
A global bank has 12 Databricks workspaces and one Unity Catalog metastore. A RAG assistant answers questions over HR policy, legal memos, and deal documents. Source tables carry a row filter on region and a column mask on compensation, both defined in Unity Catalog. The ingestion pipeline writes a denormalized chunks table that copies document text plus a JSON array of group names harvested from the source table's ACLs, and the Vector Search index is built from that table. During user-acceptance testing, an analyst in the Singapore group asked about relocation packages and received a chunk from a US deal memo containing a named employee's compensation. Separately, a scan found account-number patterns in 4% of chunks. Internal audit has asked for lineage from source document to answer, and for evidence that the control works.
1. Diagnose the leakage and design the fix. Why is the copied group-name array not an adequate control?
Read the worked answer
Root cause: enforcement lives on the source table, and the assistant does not query the source table. Unity Catalog row filters and column masks are applied when a principal reads that table with their own identity. The chunks table is a separate object owned by a service principal; it copies text that was already filtered for that principal, and it carries a snapshot of ACLs rather than a live rule. The mask on compensation never ran on the denormalized copy.
Why the group array fails on four counts:
- It is point-in-time. Anyone added to or removed from a group after the harvest is wrong until the next run.
- Group names may not match the account-level groups the user actually belongs to, especially with nested groups across 12 workspaces.
- It moves the decision into the application. A missing filter, a client-supplied filter list, or a code path that forgets to pass it returns everything, with no backstop.
- Region on a deal document may be derived from the document rather than the row, so the value itself can be wrong.
Preferred fix: keep enforcement in Unity Catalog. Build the index over a governed view where the row filter and column mask apply, and query with the end user's identity so the filter is evaluated per user at query time. Confirm the behavior for your workspace and vector index type empirically - do not assume.
Defense in depth where that is not possible: carry region and an allowed-principals column, construct filters server-side from the authenticated session only, fail closed when the filter is empty, log the filter used with every retrieval, and re-check returned chunk ids against the source table before they enter the prompt.
2. Account numbers are already embedded in 4% of chunks. What do you do about data that is already in vectors, and how do you prevent recurrence?
Read the worked answer
Embeddings cannot be redacted. There is no mask function for a vector: the account numbers were in the text that produced it, the information is distributed across dimensions, and the chunk text itself is still in the source table. Deletion and re-embedding are the only real remedies.
Remediation sequence:
- Identify affected chunk ids by scanning chunk text with a pattern detector, and widen the scan to the source documents - the chunks may only be a subset.
- Delete those rows from the source table so Delta Sync removes them from the index, and verify counts on both sides.
- Re-extract from the source with redaction applied before chunking and embedding, so the new vector is computed on redacted text.
- Notify the privacy office, check retention obligations for logs and model-serving inference tables that may have captured the same text, and record the incident timeline.
Prevention at the pipeline, not at the prompt:
- A redaction stage between extraction and chunking, using pattern-based detection for account numbers plus a named-entity model for person data.
- An expectation that fails the batch rather than silently masking when detection fires above a threshold, with counts routed to a quarantine table.
- A CI test on a seeded fixture set containing known patterns, so a pipeline change that removes the redaction stage fails before deployment.
- Note that the model provider is another exposure path: retrieved text is sent to the serving endpoint, so redaction at ingestion also reduces what leaves the boundary.
3. What evidence would you assemble to satisfy internal audit that the control works, and what would you make repeatable?
Read the worked answer
Auditors need lineage, an access test, and evidence that the test can fail. Three artifacts.
1. Lineage. Unity Catalog lineage from source table through the chunks table to the vector index, plus a documented description of each hop. For a single sampled answer, reconstruct: which source document, which chunk, which index, which query, which user, which filter. That requires the application to log chunk ids with each response - retrieval traces plus inference tables, or MLflow tracing on the serving endpoint - not just the final answer text.
2. An access-control test matrix, run and recorded. Users from each region, a user in nested groups, a user recently removed from a group, and a service principal. For each, a fixed set of probe questions with expected outcomes, including questions whose correct answer is a refusal. Store expected vs actual, timestamp, and reviewer. The Singapore analyst case must appear as a regression test that now passes.
3. Proof the test can fail. Include one probe where the filter is deliberately disabled in a test environment and the probe correctly leaks, showing the test detects the condition rather than passing vacuously. Auditors are rightly suspicious of a green suite that has never been red.
Make it repeatable: run the matrix in CI on every pipeline change, alert on any deviation, and retain results. Also retain audit-log evidence (system.access.audit) that index queries run under the correct identity - that log is the operating effectiveness evidence, not just the design document.