Databricks Data Engineer: Governance and Security

The Unity Catalog model

The namespace has three levels: catalog.schema.table, plus volumes and functions. One metastore per region attaches to workspaces. Privileges are granted on securables: USE CATALOG, USE SCHEMA, SELECT, MODIFY, CREATE TABLE, ALL PRIVILEGES. Reading a table requires USE CATALOG on the catalog, USE SCHEMA on the schema, and SELECT on the table - a classic exam trap. Ownership implies all current and future privileges plus the right to grant them.

Data objects

• Managed tables: Unity Catalog owns the storage location; DROP TABLE deletes the data.
• External tables: point at a path; dropping removes metadata only.
• External locations require a storage credential (cloud IAM role) plus CREATE EXTERNAL LOCATION; users also need READ FILES / WRITE FILES on the location.
• Volumes provide governed file access without DBFS mounts.

Fine-grained controls

Row filters and column masks are SQL UDFs bound to a table, so callers see only permitted rows or masked values. Tags and comments drive discovery and cost attribution. Audit logs land in system.access.audit. Delta Sharing shares data outside the account without copying it.

Cluster access modes

Single user (one principal, all Spark APIs), Shared (many users, UC-only, no DBFS root, limited RDD/MLlib APIs), and Custom (jobs, service principals). Secrets belong in secret scopes, never in notebook code or widget defaults.

Traps

• Granting SELECT and expecting access without USE CATALOG and USE SCHEMA.
• Assuming dropping a managed table keeps its files.
• Writing PII to a shared cluster's local storage path.
• Granting ALL PRIVILEGES on a catalog when only one schema was intended.