Governance, security and responsible AI for generative AI applications
Governance is part of the architecture
Every component, including source documents, Delta tables, Vector Search indexes, models, endpoints, and inference logs, should live in Unity Catalog with owners, tags, and lineage. Grant least privilege: consumers need SELECT or EXECUTE on what they use, not ownership. Use service credentials and secrets for external access; never embed tokens in notebooks. Classify data as PII or confidential and apply row filters, column masks, and dynamic views before data reaches prompts.
Responsible AI controls
Use AI Gateway guardrails on serving endpoints for PII detection and redaction, safety and content filters, rate limits, and payload logging. Add application-level controls: input validation, prompt-injection defenses, output filtering, citation requirements, and human review for high-impact actions. Keep audit logs and document model cards, evaluation results, and intended use. For agents with write actions, require approval steps or narrowly scoped tools.
Traps
- Assuming index access equals table access; both must be granted.
- Guardrails that add latency or block legitimate content; test thresholds.
- Logging raw prompts containing PII into broadly readable tables.
- Confusing endpoint permissions with model or data permissions.
- No retention or deletion policy for logged payloads.
Rule of thumb
Default deny, then grant the narrowest object and action; redact PII before the model, not after. Example: a RAG application over HR data should query a filtered view, use a service credential scoped to that view, and enable PII filters on the endpoint.