I85PREP
⌘ K
Databricks Data Analyst study materials
Databricks Data Analyst study resources

Databricks Data Analyst: Importing Data

Databricks Data Analyst · Work through your exam, one topic at a time.

All guide sections

Importing Data

The ingestion menu

Know which tool fits which scenario - questions are almost always phrased as "which approach is best."

  • UI upload / Create Table from file - quick, small files, good for demos and one-offs.
  • COPY INTO - SQL command that loads files from cloud storage into a Delta target. It is idempotent: already-loaded files are skipped, so re-running is safe. Supports schema evolution with mergeSchema.
  • Auto Loader (cloudFiles) - incremental, scalable streaming ingestion that discovers new files as they land; the right answer for continuously arriving data in cloud storage.
  • CREATE TABLE AS SELECT / INSERT INTO ... SELECT - ingesting from another table or a JDBC source via a connection.
  • Lakeflow Connect / partner connectors - managed connectors for SaaS sources such as Salesforce or SQL Server.

Formats and behavior

CSV (inferSchema, header, delimiter), JSON, Parquet, Avro, and Delta are all supported. Streaming reads require Delta or Auto Loader, not raw CSV via COPY INTO semantics - a common trip-up.

Traps

  1. Choosing COPY INTO for a target that must be a non-Delta table - it requires Delta.
  2. Using Auto Loader with a local workspace path; it needs a cloud storage location.
  3. Forgetting that UI upload has practical file-size limits and is not a production pipeline.
  4. Re-running a naive INSERT INTO ... SELECT from a landing folder and double-loading data.

Worked example

New CSV files appear hourly in s3://landing/orders/. Correct pattern: create a bronze streaming table with read_files/Auto Loader, set a schema location, enable schema evolution, and let it checkpoint so files are processed exactly once.

Rule of thumb

Files that keep arriving → Auto Loader. Files that arrive in batches and may be replayed → COPY INTO. Handful of files → UI upload.