Databricks Context Engineer Associate study plan

Map each of the seven exam domains to its weight and write a one-paragraph summary of what each domain likely tests, so you can allocate later weeks deliberately

Set up a Databricks workspace with a Unity Catalog catalog, a schema, and a personal compute cluster, then confirm you can create a notebook and run a trivial query

Define context engineering in your own words and distinguish it from prompt engineering and from RAG by writing a short comparison note

Explain how Delta Lake tables, notebooks, jobs, and Unity Catalog objects relate to one another, and sketch a diagram of a context pipeline that uses all four

Design a target schema for a retrieval corpus, including primary key, source metadata columns, chunk text, and a change-tracking column, and justify each column

Build a Delta table that stores document chunks and load at least 100 rows into it from a small set of source documents

Compare Delta Lake, Parquet, and a plain table format for storing context data, and record which properties (time travel, schema enforcement, upserts) matter for retrieval

Practice MERGE INTO to upsert changed chunks and demonstrate that unchanged rows are not duplicated

Ingest raw documents from at least two different source types (for example files in a volume and rows from a relational table) into a landing area

Implement a cleaning step that strips boilerplate, normalizes whitespace, and removes near-empty records, and log how many rows each rule removed

Choose a chunking strategy, implement it, and compare two chunk sizes on retrieval quality using a handful of hand-written queries

Model metadata for filtering (source, date, document type) and write queries that combine a metadata filter with a text match

Generate embeddings for your chunk table using a Databricks-hosted or Foundation Model API embedding endpoint and store them alongside chunk IDs

Create a Vector Search index over the chunk table and run at least five similarity queries against it

Explain the tradeoffs between storing embeddings in a Delta table versus a managed vector index, and note the refresh behavior of each

Measure embedding cost and latency for your corpus and extrapolate to a ten-times-larger corpus

Implement hybrid retrieval that combines vector similarity with keyword matching and compare its results to pure vector search on the same query set

Add metadata pre-filtering to your retrieval and verify that filtered queries return only permitted or in-scope documents

Tune retrieval parameters (number of candidates, similarity threshold, reranking) and record how each change affects the relevance of the top results

Document the retrieval design decisions and the failure modes you observed, such as chunks that split mid-sentence or duplicate near-identical passages

Build an end-to-end application flow that takes a user question, retrieves context, assembles a prompt, and returns an answer with cited source chunks

Implement prompt assembly rules that respect a token budget and define what happens when retrieved context exceeds it

Orchestrate the pipeline as a Databricks job or scheduled workflow rather than manual notebook runs, and handle a failure case gracefully

Add a simple interface layer, such as a notebook function or a small app endpoint, and test it with ten realistic questions

Apply Unity Catalog privileges so that only intended groups can read the chunk table and query the index, and verify access with a second identity

Implement data quality checks on your corpus (null rates, duplicate rate, minimum chunk length, staleness) and route failing records to a quarantine table

Define evaluation metrics for retrieval and for the final answer, then build a small labeled evaluation set of at least twenty questions with expected sources

Run your evaluation and produce a baseline scorecard that you can rerun after any pipeline change

Instrument the pipeline with logging for latency, token usage, retrieval hit rate, and failure counts, and review the logs for one full run

Change one pipeline component (chunking, embedding model, or retrieval settings) and quantify the effect on your evaluation scorecard

Review each of the seven domains against your week-one map and list the specific gaps that remain

Complete a timed practice set of mixed-domain questions and analyze every miss by domain and by root cause