Databricks Machine Learning: Platform Fundamentals
Orientation
The Databricks Machine Learning domain is the platform-literacy section: compute, workspace objects, and governance primitives you will lean on in every other domain. Know the difference between an all-purpose cluster (long-lived, interactive, shared by a team, terminated manually or by policy) and a job cluster (created for a single job run, isolated, cheaper, destroyed at the end). Know that Databricks Runtime ML ships MLflow, Hyperopt, and many ML libraries preinstalled, and that GPU workloads require a GPU-enabled runtime plus a GPU instance type.
Notebook mechanics
Notebooks attach to a cluster and support magic commands: %md, %sh, %fs, %run, %pip, %conda. %run inlines another notebook into the current one, so its variables and functions land in the caller's namespace. dbutils.notebook.run() launches a separate child run, accepts widget parameters, and returns a value set by dbutils.notebook.exit(). Widgets (dbutils.widgets.text/dropdown/get) parameterize notebooks and exist only inside a notebook context.
Governance
Unity Catalog uses a three-level namespace: catalog.schema.object, covering tables, views, volumes, functions, and registered models. Secrets live in secret scopes and are read with dbutils.secrets.get(); printed values are redacted.
Traps
Questions like to blur %run with dbutils.notebook.run, or imply widgets persist outside a notebook. After a %pip install, restart Python (dbutils.library.restartPython()) or imports may still fail.
Rule of thumb
Interactive exploration goes on an all-purpose cluster; scheduled retraining goes on a job cluster; shared, governed data goes in Unity Catalog, never ad hoc paths.