Azure Databricks Data Engineer Associate study plan

Map each ingestion path (Auto Loader, COPY INTO, streaming table ingestion, batch connector reads) to the workload scenario where it is the correct choice, and write a one-page decision table in your own words.

Configure Auto Loader with cloudFiles schema inference and schema evolution modes (addNewColumns, failOnNewColumns, rescue) and explain in notes when each mode protects pipeline continuity.

Create a bronze ingestion notebook that lands raw JSON and CSV files into a Unity Catalog managed volume and writes to a bronze Delta table with ingestion metadata columns (source file, ingestion timestamp, rescue data).

Compare idempotent re-run strategies (checkpoint reuse, file notification vs directory listing, MERGE-based deduplication) and document the failure mode each one prevents.

Practice reading the Auto Loader monitoring metrics (numFilesOutstanding, numBytesOutstanding) and state what each metric tells you during a backlog.

Ingest from an external relational source using a JDBC-style batch read and land the result in a bronze table partitioned sensibly for downstream filtering.

Implement incremental ingestion of only changed rows using a watermark column, and prove a second run does not duplicate rows.

Handle malformed records by routing them to a quarantine table with the rescue schema, then write a query that counts and classifies the rejected records.

Set up a streaming ingestion that survives a deliberate schema change by adding a new column to the source files mid-stream.

Document the cost and latency trade-offs between continuous, triggered, and available-now trigger modes for your ingestion.

Design a medallion layout (bronze, silver, gold) for a small dataset and justify what belongs in each layer rather than copying raw data upward.

Write silver-layer transformations that deduplicate on a business key, cast types explicitly, and standardize nulls and timestamps.

Implement an upsert with MERGE INTO, including matched-update, not-matched-insert, and a delete-handling branch, and explain the semantics of each clause.

Diagnose and fix the small-file problem by compacting a table with OPTIMIZE and evaluating whether liquid clustering or partitioning is the better fit for the query pattern.

Compare a batch transformation with its Structured Streaming equivalent for the same logic and note where the code diverges.

Apply Delta Lake table constraints (NOT NULL, CHECK) and observe what happens to a violating write, then decide where enforcement belongs in the pipeline.

Use time travel to query a prior table version and explain how retention and VACUUM interact with that ability.

Build at least one gold-layer aggregate that answers a concrete business question, and validate the result against a hand-computed check.

Describe the workspace objects (notebooks, clusters, SQL warehouses, catalogs, volumes, jobs) and how they relate, in a diagram you draw yourself.

Explain the difference between a managed and an external table in Unity Catalog and pick the correct one for two given scenarios.

Convert your multi-notebook pipeline into a Lakeflow Job with explicit task dependencies and a defined task-level cluster configuration.

Add job parameters and reference them from task code so the same job can be pointed at different environments.

Configure retries, timeouts, and a failure notification so a broken task surfaces to a human instead of failing silently.

Add a conditional or dependent task that runs only when upstream data passes a row-count sanity check.

Schedule the job and explain how the trigger type you chose affects cost and freshness.

Store notebooks and job definitions in a Git-backed folder, create a feature branch, and open a pull request containing a pipeline change.

Define a Databricks Asset Bundle that declares your job, its clusters, and its parameters, then deploy it to a second target environment from the command line.

Explain how bundle targets, variables, and environment-specific overrides keep one definition usable across dev and prod.

Read the Spark UI for a slow query and identify the stage with the spill or skew, then state the specific fix you would apply.

Use the query profile or event log to find a task that retried, and explain what the retry indicates about the underlying data or code.

Create a Unity Catalog catalog, schema, and volume, then grant least-privilege access to a group so a user can read gold but not bronze.

Explain the relationship between catalog, schema, table, and storage credential, and where each privilege is actually granted.

Implement row-level filtering or column masking on a gold table and verify the restriction with a query run as a lower-privileged principal.

Explain how a Delta Live Tables or Lakeflow pipeline's expectations differ from table constraints in where they block bad data.

Set up a baseline monitoring approach: capture table row counts and freshness over time and describe how you would alert on a freshness breach.

Rebuild your end-to-end pipeline from an empty workspace using only your own notes and the bundle, to expose the gaps in your documentation.

Write and answer 40 or more original practice questions across all seven domains, weighted toward ingestion, transformation, and jobs, and log every miss by domain.

Re-study only the two domains with the lowest practice-question accuracy, focusing on the specific concepts you missed rather than re-reading broadly.

Explain aloud, without notes, the trade-off between Auto Loader and COPY INTO, between MERGE and overwrite, and between job clusters and all-purpose clusters.

Review the official exam skills outline one final time and confirm you can point to concrete work you have done for every listed skill.