Governance, Security, and Data Quality for Context
This domain tests whether your context pipeline enforces permissions, protects sensitive data, and keeps retrieval evidence trustworthy. The exam emphasizes Unity Catalog controls, ACL-aware retrieval, PII handling, and quality gates that prevent bad context from reaching the model.
Governance controls
- Use Unity Catalog privileges on catalogs, schemas, tables, functions, models, and vector indexes. Grant least privilege and separate development from production.
- Apply row filters and column masks on Delta tables, and use tags for classification and discovery. Lineage shows which downstream indexes and models depend on a source.
- Audit access to governed objects and review unusual retrieval or tool patterns.
- Design ACL-aware retrieval: carry allowed user or group identifiers on chunks and filter at query time using the authenticated identity. Do not assume source-table row filters automatically apply inside a vector index.
Quality and freshness
- Add expectations in Lakeflow pipelines; quarantine records that fail critical checks instead of silently indexing them.
- Monitor null rates, duplicate chunks, stale versions, and documents past their effective date.
- Redact PII before embedding and propagate deletions and permission revocations to the index and caches.
Worked example
A legal assistant retrieves matter documents. Each chunk carries matter_id and allowed_groups derived from the document system. At query time, the application resolves the user’s groups from the identity provider, applies an allowed_groups filter to Vector Search, and logs the filter and returned chunk IDs. A nightly pipeline checks for revoked matter access and marks affected chunks deleted; the Delta Sync index propagates the change. Reasoning: the vector index is a separate serving system, so row-level security must be translated into retrieval metadata and enforced before chunks enter the prompt.
Traps
- Assuming Unity Catalog row filters on a source table automatically constrain vector search results.
- Embedding secrets, credentials, or unredacted PII.
- Enforcing access with prompt instructions instead of query-time filters and server-side checks.
- Failing to propagate deletions and revocations to indexes and caches.
- Indexing records that failed quality expectations.