Databricks ML Professional common mistakes
Databricks ML Professional · Understand the traps and how to avoid them.
Treating MLflow autologging as sufficient and then being unable to explain how to add a custom metric, tag, or artifact to a run when the prompt asks for it.
Candidates memorise that mlflow.autolog() exists and stop there, so they never practise the explicit mlflow.log_metric, mlflow.log_param, mlflow.set_tag, and mlflow.log_artifact calls that scenarios actually test. The exam asks candidates to choose the correct logging API or diagnose why a metric never appeared in a run.
What to do instead
Practise a full manual logging loop: open a run with mlflow.start_run(), log params, metrics, tags, and an artifact, then register the model. Deliberately disable autologging in one experiment so you feel the difference. Be able to state that autologging captures framework-specific signals (for example model signature and default metrics for supported libraries) while custom business metrics always require explicit calls.
Confusing the MLflow Model Registry concepts: describing a registered model, a model version, a stage or alias, and a run as if they were interchangeable.
The names sound similar and candidates learn them from a single diagram. Exam items ask which object a URI points to, or how to promote a version, and a fuzzy mental model produces the wrong answer. Confusion between the deprecated stage mechanism and the newer alias mechanism is a frequent trap.
What to do instead
Build a small registry by hand and inspect each object: a registered model is the named container, a version is an immutable snapshot tied to a source run and artifact path, an alias or stage is a mutable pointer to a version. Practise loading a model by alias URI, moving an alias to a new version, and explaining why the old version's artifacts are unchanged. Know that stages are legacy and aliases are the recommended replacement for deployment pointers.
Assuming a Spark DataFrame and a pandas DataFrame are interchangeable in feature engineering code, then failing questions about when to use a pandas UDF or a Feature Store function.
Candidates who mostly write scikit-learn code default to pandas idioms. On Databricks, distributed feature computation needs Spark transformations, and the exam probes whether the candidate knows which API operates at cluster scale versus on the driver.
What to do instead
Rewrite a feature pipeline twice, once with Spark column functions and once with pandas, and note where each breaks. Learn that pandas UDFs (including the iterator and grouped-map variants) vectorise Python code across partitions, whereas a plain Python UDF is row-by-row and slow. Know which Feature Store APIs accept a Spark DataFrame and how to write a training set that preserves point-in-time correctness.
Ignoring point-in-time correctness when creating a training set from a feature table, producing label leakage and then answering feature-store questions as if a simple join were enough.
The mechanics of a time-series feature join are easy to skim, and candidates rarely build one. The exam tests whether the candidate understands that features must reflect values known at the prediction timestamp, not values written later.
What to do instead
Create a feature table with a timestamp key and a label table with event timestamps, then build a training set with the time-series join and inspect the result for leakage. Be able to explain that the join matches each label row to the most recent feature value at or before its timestamp, and why a naive join on entity key alone inflates offline metrics and degrades online performance.
Describing model deployment only in terms of a REST endpoint, without being able to distinguish batch inference, streaming inference, and real-time serving, or to justify a choice.
Serving gets the most attention in tutorials, so candidates over-fit to it. Scenario questions reward the candidate who can match latency, throughput, and freshness requirements to the right pattern.
What to do instead
For each pattern, write one sentence on its trigger and cost: batch scoring on a schedule for large volumes where staleness is acceptable, structured streaming for continuous ingestion, real-time serving for low-latency per-request predictions. Practise mapping a stated requirement (for example sub-second response for a user-facing app versus nightly scoring of a million rows) to a pattern and naming the Databricks mechanism that implements it.
Treating model monitoring as a one-time evaluation and omitting drift detection, data quality checks, and retraining triggers from an MLOps design answer.
Candidates study training metrics and stop at deployment. The exam expects lifecycle thinking, and answers that never mention production monitoring read as incomplete even when the deployment mechanics are correct.
What to do instead
Design a monitoring plan explicitly: track input feature distributions against a baseline, track prediction distributions, track label-derived quality once ground truth arrives, and define thresholds that trigger investigation or retraining. Be able to name where these signals are stored and how they feed a retraining job. Practise describing a closed loop from monitoring alert to retrain to re-registration to redeployment.
Confusing Delta Lake table operations with file-level operations, for example describing how to update rows by rewriting Parquet files rather than using MERGE, UPDATE, or time travel.
Candidates with data-engineering gaps fall back to file manipulation. The exam checks whether the candidate can use transactional table semantics for feature tables and inference logs.
What to do instead
Practise MERGE INTO for upserts into a feature table, UPDATE and DELETE for corrections, DESCRIBE HISTORY and time travel for auditing and rollback, and OPTIMIZE with Z-ordering for layout. Be able to explain why these operations are safer than overwriting files and how the transaction log supports concurrent readers and writers.
Answering hyperparameter tuning questions by naming a library without explaining search strategy, search space, and how trials are parallelised or tracked.
Candidates know that Hyperopt or Optuna exists but have not run a distributed tuning job, so they cannot describe early stopping, the objective function contract, or how results are logged to MLflow.
What to do instead
Run a tuning job with a defined search space, an objective that returns the metric to minimise, and a fixed trial budget. Observe how trials are distributed across workers and how each trial appears as a nested MLflow run. Be able to discuss random versus Bayesian search, early stopping of poor trials, and the cost of an oversized search space.
Assuming a model that scores well offline will deploy unchanged, ignoring schema enforcement, input validation, and environment or dependency differences between training and serving.
Candidates treat deployment as a copy step. Real failures come from mismatched feature order, unexpected nulls, or library version drift, and the exam probes whether the candidate anticipates them.
What to do instead
Log a model signature with input and output schemas at training time and verify that serving rejects malformed input. Pin dependencies and confirm the serving environment matches the training environment. Practise describing what happens when a new feature column is added upstream and how signature enforcement surfaces the mismatch early rather than silently corrupting predictions.
Describing CI/CD for models as generic software CI, without covering data and model versioning, automated evaluation gates, and reproducible environment capture.
Candidates transfer software engineering habits directly, but ML pipelines have data and model artifacts that must be versioned and validated, and the exam tests that distinction.
What to do instead
Outline a pipeline stage by stage: version code and data, train in a reproducible environment, evaluate against a baseline and a threshold, register the model only if the gate passes, then promote via alias. Be able to say what artifact each stage produces and where it is stored, and why a passing unit test suite alone does not justify promotion.