I85PREP
⌘ K
Databricks Data Engineer study materials
Databricks Data Engineer study resources

Databricks Data Engineer practice questions

One question at a time. Build your Databricks Data Engineer understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Databricks Intelligence Platform

In Delta Lake, which component records every committed change to a table and serves as the basis for its ACID guarantees?

  1. The _delta_log directory Correct answer.
  2. The Spark driver's in-memory catalog of tables
  3. The Hive metastore entry holding the table definition
  4. The collection of Parquet data files in the table root

Explanation

Each commit to a Delta table is written atomically as a JSON entry in the _delta_log directory, with periodic Parquet checkpoints summarizing older entries. Readers reconstruct the current snapshot by replaying that log, which is what enables ACID semantics and time travel. The data files themselves carry no transactional metadata.

Question 2Databricks Intelligence Platform

Which fully qualified name correctly references a table through the Unity Catalog three-level namespace?

  1. database.catalog.schema.table
  2. catalog.schema.table Correct answer.
  3. schema.table
  4. workspace.catalog.table

Explanation

Unity Catalog organizes securables as catalog, then schema, then table or view, so a table is addressed as catalog.schema.table. Two-part names work only when a default catalog and schema are in effect, and the three-part form is the canonical qualified reference.

Question 3Databricks Intelligence Platform

Which magic command executes another notebook inline within the current notebook and shares its variables and functions with the calling notebook?

  1. %sh
  2. %fs
  3. %pip
  4. %run Correct answer.

Explanation

%run includes the referenced notebook in the current execution context, so its defined variables and functions become available to the caller. This differs from %pip and %sh, which act on the environment or shell, and from %fs, which issues file system commands.

Question 4Databricks Intelligence Platform

What is Photon in the Databricks Lakehouse Platform?

  1. A notebook scheduling service for recurring dashboards
  2. A change data capture connector for streaming sources
  3. A vectorized C++ query engine Correct answer.
  4. A governance layer that enforces column-level access control

Explanation

Photon is Databricks' native execution engine, written in C++ with vectorized processing, that accelerates SQL and DataFrame workloads on Delta tables. It is enabled per cluster or warehouse and is compatible with Spark APIs, so existing code does not need rewriting.

Question 5Databricks Intelligence Platform

A data engineer creates an all-purpose cluster with autoscaling enabled, a minimum of 2 workers, and a maximum of 8 workers. How does the cluster respond as its workload changes?

  1. It automatically moves the cluster between serverless and classic compute types
  2. It adds or removes worker nodes within the configured bounds Correct answer.
  3. It scales the driver node's core count between the configured minimum and maximum values
  4. It keeps node count fixed but resizes each worker's memory as needed

Explanation

Autoscaling adjusts the number of worker nodes between the minimum and maximum you set, allocating more workers when tasks are queued and releasing them when demand drops. The driver is not scaled by autoscaling, and the compute type of an existing cluster does not change.

Question 6Databricks Intelligence Platform

In a Delta Live Tables pipeline, which decorator enforces a data quality expectation by discarding records that violate it while allowing the pipeline update to continue?

  1. @dlt.expect_all_or_fail
  2. @dlt.table
  3. @dlt.expect_or_fail
  4. @dlt.expect_or_drop Correct answer.

Explanation

The expect_or_drop decorator records the violation in metrics and removes the offending rows from the result set, so the pipeline update keeps running. Failure variants such as expect_or_fail and expect_all_or_fail stop the update instead, while @dlt.table only defines the dataset.

Question 7Databricks Intelligence Platform

A Delta table has ten committed versions and the default retention settings. Which statement returns the table contents as they existed at version 6 without altering the table?

  1. SELECT * FROM t TIMESTAMP AS OF 6
  2. RESTORE TABLE t TO VERSION AS OF 6
  3. SELECT * FROM t AT SNAPSHOT 6
  4. SELECT * FROM t VERSION AS OF 6 Correct answer.

Explanation

Time travel uses the clause VERSION AS OF or TIMESTAMP AS OF after the table name, so SELECT * FROM t VERSION AS OF 6 reads historical data. RESTORE changes the current state of the table rather than only querying a past version, and AT SNAPSHOT is not valid Delta syntax.

Question 8Databricks Intelligence Platform

A managed table is created in a Unity Catalog schema. Where do its data files reside?

  1. In the default DBFS root of the workspace that ran the query
  2. In the managed storage location of the containing schema or catalog Correct answer.
  3. In the /tmp directory of the cluster's driver node
  4. In the external location attached to the notebook's workspace

Explanation

In Unity Catalog, managed tables are stored in a managed storage location defined at the schema or catalog level, with the metastore as the authority for lifecycle. Dropping such a table also removes its data files. DBFS root and driver-local paths are not used for managed tables under Unity Catalog.

Question 9Databricks Intelligence Platform

What is the primary benefit of running OPTIMIZE with Z-ORDER on a Delta table?

  1. It converts the table into a streaming source with exactly-once semantics
  2. It only merges small files into larger ones without changing row order
  3. It permanently deletes old file versions to reduce storage cost
  4. It co-locates related data in the same files to improve data skipping Correct answer.

Explanation

Z-ORDER is a multi-dimensional clustering technique applied during OPTIMIZE that colocates rows with similar values across the z-order columns into the same files. Queries filtering on those columns can then skip far more files. Compaction alone is what plain OPTIMIZE without Z-ORDER performs.

Question 10Databricks Intelligence Platform

A multi-task job completes except for its final task, which fails. Which action reruns only that failed task and its downstream dependencies without rerunning the tasks that succeeded?

  1. Repairing the failed run Correct answer.
  2. Deleting and recreating the job with identical settings
  3. Resetting the job's schedule
  4. Cloning the job into a different workspace

Explanation

A repair run re-executes the failed task and any tasks that depend on it while reusing the completed results of upstream tasks, which saves time and compute. It operates on an existing run and can be started with modified parameters. Recreating or cloning the job starts everything from the beginning.

Question 11Databricks Intelligence Platform

By default, for which columns of a Delta table does Databricks collect the statistics used for data skipping?

  1. The first 32 columns of the table schema Correct answer.
  2. Only numeric and date-typed columns
  3. All columns regardless of their position in the schema
  4. Only columns declared as partition keys

Explanation

Delta collects per-file statistics such as min, max, and null counts for the first 32 columns of the schema by default, and these statistics drive data skipping. Columns beyond that limit can be included by reordering the table or by configuring additional statistics columns explicitly. Partition columns are handled separately through directory pruning.

Question 12Databricks Intelligence Platform

An engineer needs to create an external table at a cloud storage path in Unity Catalog. Which objects must already exist and be granted before the table can be created at that path?

  1. Only a catalog and schema, since the path is validated automatically by the workspace admin
  2. A cluster configured with an instance profile that can read the bucket
  3. A storage credential and an external location covering the path Correct answer.
  4. A DBFS mount point plus a table ACL grant on the target directory

Explanation

Unity Catalog governs access to cloud storage through storage credentials that hold the cloud identity, and external locations that bind a credential to a storage path. A user needs a privilege on the external location plus CREATE TABLE on the schema. Mount points and cluster instance profiles are the legacy workspace-level approach and are not the governing objects in Unity Catalog.

Question 13Data Ingestion and Loading

Which value of the format/read source activates Auto Loader in a Structured Streaming read?

  1. cloudFiles Correct answer.
  2. autoLoad
  3. streamFiles
  4. fileStream

Explanation

Auto Loader is enabled by setting the source format to cloudFiles and supplying options prefixed with cloudFiles. (e.g. cloudFiles.schemaLocation). It is a streaming source designed for incremental file arrival in cloud storage.

Question 14Data Ingestion and Loading

Which statement describes the idempotency behavior of COPY INTO when run repeatedly against the same source directory?

  1. It tracks previously loaded files and skips them on later runs. Correct answer.
  2. It reloads every file found in the path on each execution of the command.
  3. It requires that source files be deleted manually before each subsequent run.
  4. It deletes source files from cloud storage once they have been loaded.

Explanation

COPY INTO keeps a record of the files it has already ingested for a target table, so re-running the same command against the same path loads only new files. This makes the command safe to schedule without duplicating data.

Question 15Data Ingestion and Loading

Which two file discovery modes does Auto Loader support?

  1. polling and push notifications
  2. schema inference and schema evolution
  3. directory listing and file notification Correct answer.
  4. batch and streaming reads

Explanation

Auto Loader can discover new files either by listing the input directory or by subscribing to file arrival events from the cloud storage notification service. Directory listing is the default and requires no extra cloud setup, while file notification scales better for very large volumes.

Question 16Data Ingestion and Loading

When reading a CSV file with the DataFrame reader, which option causes the first line to be used as column names?

  1. firstRowAsHeader
  2. columnNames
  3. header Correct answer.
  4. inferHeader

Explanation

The CSV data source uses the option header set to true to treat the first row as the column names. Without it, columns are named _c0, _c1, and so on.

Question 17Data Ingestion and Loading

Why must every Structured Streaming query use its own unique checkpoint location?

  1. It defines the output schema used when writing to the sink table.
  2. It stores the query results in a durable location for reuse.
  3. It caches the source files on the cluster for faster reads.
  4. It records stream progress so processing resumes where it stopped. Correct answer.

Explanation

The checkpoint directory holds the progress information (which files or offsets were consumed) and streaming state for a query. Sharing a checkpoint between queries corrupts that bookkeeping, so each query needs its own path.

Question 18Data Ingestion and Loading

Which of these file formats stores data in a columnar layout and is commonly used as a landing format for ingestion into Delta Lake?

  1. JSON
  2. CSV
  3. Text
  4. Parquet Correct answer.

Explanation

Parquet is a columnar format that stores data with an embedded schema, which makes reads selective and compression efficient. CSV, JSON, and plain text are row-oriented text formats without a typed schema.

Question 19Data Ingestion and Loading

In a MERGE INTO statement, which clause specifies what to do when a source row matches an existing target row?

  1. USING <source>
  2. ON <merge condition>
  3. WHEN MATCHED THEN UPDATE SET Correct answer.
  4. WHEN NOT MATCHED BY TARGET THEN INSERT *

Explanation

The WHEN MATCHED clause defines the action taken for source rows that satisfy the merge condition and already exist in the target. Actions for new rows are defined by WHEN NOT MATCHED, while ON supplies the join condition.

Question 20Data Ingestion and Loading

In the medallion architecture, which layer holds raw ingested data with little or no transformation applied?

  1. Silver
  2. Platinum
  3. Gold
  4. Bronze Correct answer.

Explanation

The bronze layer stores data exactly as it arrived from the source, preserving the raw records for replay and auditing. Cleansing and conforming happen in silver, and business-level aggregates are produced in gold.

Question 21Data Ingestion and Loading

Which value of cloudFiles.schemaEvolutionMode is the default for Auto Loader?

  1. failOnNewColumns
  2. rescue
  3. addNewColumns Correct answer.
  4. none

Explanation

By default Auto Loader uses addNewColumns, which updates the tracked schema and restarts the stream when a new column appears. The other modes either fail the stream, park unknown data, or ignore changes entirely.

Question 22Data Ingestion and Loading

While ingesting JSON files, Auto Loader encounters fields that do not match the tracked schema. Where is that data placed by default?

  1. The stream fails on the first mismatched record.
  2. In a column named _rescued_data. Correct answer.
  3. In a separate quarantine table named bad_records.
  4. The record is dropped and logged as an error.

Explanation

When a record cannot be parsed against the schema, Auto Loader preserves it by putting the mismatched fields into a column called _rescued_data instead of failing or dropping the record. This lets downstream logic inspect and remediate the bad data later.

Question 23Data Ingestion and Loading

A COPY INTO statement targets a Delta table name that does not exist yet. What is the result?

  1. The rows are written to a managed staging table instead.
  2. The table is created automatically using an inferred schema.
  3. A temporary view is created to hold the incoming rows.
  4. It fails; the target Delta table must exist first. Correct answer.

Explanation

COPY INTO loads data into an existing Delta table and does not create the target. The table must be created beforehand with the desired schema, after which COPY INTO can append to it.

Question 24Data Ingestion and Loading

A pipeline must continuously ingest millions of small JSON files that land in cloud storage, and the source occasionally adds new columns. Which ingestion approach best fits?

  1. A nightly full read of the directory with spark.read
  2. COPY INTO with a file PATTERN clause
  3. A JDBC batch read of the landing directory
  4. Auto Loader with the cloudFiles format Correct answer.

Explanation

Auto Loader is built for continuously arriving files at high volume and tracks schema changes incrementally, adding new columns without reprocessing everything. COPY INTO can work but becomes inefficient at very large file counts because it must evaluate the directory contents on each run.

Question 25Data Ingestion and Loading

Which permission is required to use Auto Loader file notification mode on AWS?

  1. A Kafka cluster running in the same region and account
  2. IAM permissions to create an SNS topic and an SQS queue Correct answer.
  3. A Unity Catalog volume mounted at the checkpoint path
  4. The landing files rewritten into Delta format first

Explanation

File notification mode subscribes to storage events through a cloud notification service, so the identity running the pipeline needs permissions to create and manage the topic and queue used for those events. Without them, Auto Loader must fall back to directory listing.

Question 26Data Ingestion and Loading

A streaming query is restarted with the same checkpoint location but pointed at a different source directory. What happens?

  1. It silently ignores the new source and keeps reading the old one.
  2. It reprocesses every file in the new source directory from the very beginning.
  3. It reads both sources and unions the results together.
  4. The query fails because the checkpoint is bound to the original source. Correct answer.

Explanation

The checkpoint stores the identity and progress of the source it was created for, so changing the source while reusing the checkpoint causes the query to fail. A new checkpoint location is required whenever the source definition changes.

Question 27Data Ingestion and Loading

In a Delta Live Tables Python pipeline, what determines whether a defined dataset is a streaming table rather than a materialized view?

  1. Whether the query reads a stream via dlt.read_stream Correct answer.
  2. The catalog and schema in which the table is created
  3. The prefix used in the table name, such as stream_ or mv_
  4. The storage location assigned to the table in pipeline settings

Explanation

DLT infers the dataset type from the defining query: a query that reads a streaming source produces a streaming table, while a query that reads a static snapshot produces a materialized view. Naming, location, and catalog choices do not affect that classification.

Question 28Data Ingestion and Loading

Which reader options allow a JDBC read to be split into multiple parallel partitions?

  1. pushDownPredicate and pushDownAggregate set to true
  2. fetchsize, batchsize, and a custom data source class
  3. query and dbtable, with an optional isolation level
  4. partitionColumn, numPartitions, and both bounds Correct answer.

Explanation

Specifying a numeric partition column together with a lower bound, upper bound, and partition count lets Spark issue parallel range queries against the remote database. Without them, the read is performed through a single connection.

Question 29Data Ingestion and Loading

Which approach ingests only new or changed rows from a relational source on each scheduled run?

  1. Read the entire remote table on every run and then drop duplicate rows
  2. Enable change data capture on the target Delta table
  3. Run COPY INTO directly against the remote database
  4. Filter the JDBC query on an increasing column like updated_at Correct answer.

Explanation

The common incremental pattern is to parameterize the JDBC read with a predicate on a monotonically increasing column and pass the highest value seen in the previous run. This transfers only newly modified rows instead of the whole table.

Question 30Data Ingestion and Loading

Which Structured Streaming feature allows a MERGE or a full overwrite to be applied inside a streaming query?

  1. foreachBatch Correct answer.
  2. option('mergeSchema', 'true')
  3. outputMode('complete')
  4. trigger(once=True)

Explanation

foreachBatch hands each micro-batch to a user function that receives a static DataFrame, so arbitrary table operations such as MERGE, delete, or overwrite can be performed per batch. Native streaming sinks only support append, update, or complete semantics.

Question 31Data Ingestion and Loading

Which approach removes duplicate events from a streaming DataFrame keyed on an event id?

  1. dropDuplicates on the event id key, plus a watermark Correct answer.
  2. Repartitioning the stream to a single partition
  3. A broadcast join against a static deduplication table
  4. Writing the stream with outputMode complete

Explanation

dropDuplicates on the key column keeps one row per key within the streaming state, and a watermark bounds how long that state is retained so it does not grow without limit. Without a watermark the state grows indefinitely for an unbounded stream.

Question 32Data Ingestion and Loading

Which write option controls the physical directory layout of a table by the values of a column?

  1. repartition
  2. clusterBy
  3. bucketBy
  4. partitionBy Correct answer.

Explanation

partitionBy writes one subdirectory per distinct value of the chosen column, which enables partition pruning on later reads. bucketBy hashes rows into files and repartition reshuffles in memory without affecting the on-disk directory structure.

Question 33Data Ingestion and Loading

A CSV column containing mixed values is inferred as STRING, but it must be ingested as INTEGER. Which Auto Loader setting forces the type?

  1. Set cloudFiles.schemaHints to col INT Correct answer.
  2. Set cloudFiles.schemaEvolutionMode to 'rescue'
  3. Set mergeSchema to 'true' on the read
  4. Set cloudFiles.inferColumnTypes to 'false'

Explanation

Schema hints let you declare the data type of specific columns while leaving the remaining columns to inference. This is the supported way to override an inferred type for a named column in a cloudFiles read.

Question 34Data Ingestion and Loading

An Auto Loader job using trigger(availableNow=True) fails partway through a large file backlog and is restarted with the same checkpoint and schema location. Why are rows not duplicated?

  1. Writing with mode overwrite deletes any duplicate rows
  2. The durable checkpoint records which files were already processed Correct answer.
  3. Delta's transaction log automatically removes duplicated rows
  4. Auto Loader deduplicates rows by a primary key in the target table

Explanation

Auto Loader persists the set of processed files in the checkpoint and schema log on durable storage, so a restarted query resumes with the unprocessed files only. Exactly-once behavior depends on keeping those metadata locations stable and on a transactional sink such as Delta Lake.

Question 35Data Ingestion and Loading

Which option causes COPY INTO to reload files that were previously ingested into the target table?

  1. COPY_OPTIONS ('force' = 'true') Correct answer.
  2. Delete the checkpoint directory before each run
  3. SET TBLPROPERTIES ('force' = 'true')
  4. Add FILEFORMAT = 'force' to the statement

Explanation

COPY INTO normally skips files already loaded, but the force copy option makes it re-ingest all files matched by the path, including previously loaded ones. This is useful after fixing a data problem in the source files, at the cost of duplicating rows.

Question 36Data Ingestion and Loading

When reading from a Kafka topic with the default configuration, what does the value column of the resulting DataFrame contain?

  1. A base64-encoded text value
  2. A parsed JSON string
  3. Raw bytes that must be deserialized or cast Correct answer.
  4. An Avro record with a schema

Explanation

Kafka messages arrive as byte arrays, so the value column holds binary data until an explicit cast or a custom deserializer converts it. key, topic, partition, and offset columns are provided alongside value for bookkeeping.

Question 37Data Ingestion and Loading

A user needs to create an external table over a cloud storage path in Unity Catalog. What must exist first?

  1. A Hive metastore deployed in the same cloud region
  2. A personal access token embedded in the table path
  3. A global temporary view referencing the storage path
  4. An external location backed by a storage credential the user can use Correct answer.

Explanation

Unity Catalog governs cloud storage access through storage credentials and external locations, and the principal must be granted privileges on them. Only then can external tables or volumes be created over that path.

Question 38Data Ingestion and Loading

Which format string is passed to Spark's readStream to enable Auto Loader?

  1. cloud_files
  2. delta
  3. cloudFiles Correct answer.
  4. autoLoader

Explanation

Auto Loader is activated by using cloudFiles as the data source format on readStream, and all of its behavior is then tuned through options prefixed with cloudFiles. The format name is case-sensitive and spelled as a single camel-case word.

Question 39Data Ingestion and Loading

With the default configuration of the Kafka source, what does the value column of the resulting DataFrame contain?

  1. The raw message payload as a binary column. Correct answer.
  2. The message payload already parsed into typed fields from the schema registry.
  3. The message payload decoded as a UTF-8 string.
  4. The key and value combined into a single struct column.

Explanation

The Kafka source exposes key and value as separate binary columns and performs no deserialization itself. Decoding requires an explicit cast, from_json, or a custom deserializer applied after the read.

Question 40Data Ingestion and Loading

What does the FROM clause of a COPY INTO statement reference?

  1. A temporary view in the current Spark session.
  2. Another Delta table in the same metastore.
  3. A cloud storage path containing the source files. Correct answer.
  4. A Kafka topic that streams records.

Explanation

COPY INTO loads files from a storage location into a Delta table, so the FROM clause takes a path (or an external location or volume path) rather than a relation. Copying between tables is done with INSERT ... SELECT, MERGE, or CREATE TABLE AS.

Question 41Data Ingestion and Loading

A developer writes a streaming DataFrame to a Delta table using writeStream. Which option must be supplied for the query to start?

  1. checkpointLocation Correct answer.
  2. queryName
  3. maxFilesPerTrigger
  4. trigger

Explanation

Every streaming write must record progress somewhere, so checkpointLocation is mandatory; the query cannot start without a durable offset and commit log. The other options influence scheduling, rate limiting, or identification but are optional.

Question 42Data Ingestion and Loading

Auto Loader is implemented on top of which Spark capability?

  1. The Spark SQL catalog and metastore.
  2. Photon vectorized execution.
  3. Structured Streaming sources. Correct answer.
  4. Delta Sharing.

Explanation

Auto Loader registers a Structured Streaming source that incrementally discovers and reads new files, so it inherits checkpointing, exactly-once file tracking, and trigger semantics from Structured Streaming.

Question 43Data Ingestion and Loading

A COPY INTO command is executed twice against the same source directory with no new files arriving between runs. Why does the target table not receive duplicate rows?

  1. COPY INTO compares every column value against the target and drops rows that already exist.
  2. Delta Lake refuses to insert rows whose values are identical to existing rows.
  3. COPY INTO deletes source files after a successful load, so nothing remains to be read.
  4. COPY INTO records the files it has already loaded and skips them on subsequent runs. Correct answer.

Explanation

COPY INTO keeps track of which source files have been consumed and processes only files not seen before, which makes repeated runs idempotent at the file level. New appends to an already-loaded file are not picked up, so files should be immutable.

Question 44Data Ingestion and Loading

Which statement correctly contrasts Auto Loader's directory listing mode with its file notification mode?

  1. Listing mode requires a cloud queue, while notification mode lists the directory on every trigger.
  2. Notification mode transfers file contents through the queue, while listing mode reads files directly from storage.
  3. Notification mode supports only Parquet files, while listing mode supports any file format.
  4. Listing mode calls the storage list API each trigger, while notification mode consumes file events from a queue and scales better to very large numbers of files. Correct answer.

Explanation

In listing mode the source repeatedly asks cloud storage what files exist, which becomes expensive as the directory grows. In notification mode the source subscribes to file-arrival events through a cloud queue and uses the event payloads to identify new files.

Question 45Data Ingestion and Loading

A developer creates a new Auto Loader stream over a directory that already contains 10,000 historical files but wants the stream to begin with only files that land after it starts. Which option should be set?

  1. cloudFiles.useNotifications = false
  2. cloudFiles.includeExistingFiles = false Correct answer.
  3. cloudFiles.maxFilesPerTrigger = 1
  4. cloudFiles.schemaEvolutionMode = 'addNewColumns'

Explanation

By default Auto Loader processes files already present when a fresh stream starts; setting cloudFiles.includeExistingFiles to false makes the first run ignore that backlog. The setting only affects the initial discovery, after which any new file is picked up normally.

Question 46Data Ingestion and Loading

A streaming query must begin at the oldest available offsets of a Kafka topic the first time it runs. Which option accomplishes this?

  1. includeHeaders = 'true'
  2. kafka.auto.offset.reset = 'earliest'
  3. startingOffsets = 'earliest' Correct answer.
  4. failOnDataLoss = 'true'

Explanation

The Kafka source decides its initial position from startingOffsets, which accepts 'earliest', 'latest', or a JSON map of per-partition offsets. Consumer-level offset reset properties are not used because Spark itself tracks consumed offsets in the checkpoint.

Question 47Data Ingestion and Loading

What is the purpose of the cloudFiles.schemaLocation option in an Auto Loader stream?

  1. It stores the inferred schema and tracks schema changes across runs. Correct answer.
  2. It stores the checkpointed offsets of files already processed.
  3. It is the directory where records that failed parsing are written.
  4. It holds the transaction log of the target Delta table.

Explanation

Auto Loader persists the inferred schema plus a history of detected changes in schemaLocation so that restarts and schema evolution are consistent. Malformed records are not written there; they are surfaced through the rescued data column.

Question 48Data Ingestion and Loading

Which COPY INTO clause restricts a load to a specific list of named files in the source directory?

  1. FILEFORMAT
  2. COPY_OPTIONS
  3. FILES Correct answer.
  4. PATTERN

Explanation

The FILES clause takes an explicit list of file names relative to the source path, which is useful for backfills of known files. PATTERN offers regular-expression matching over paths rather than an explicit list.

Question 49Data Ingestion and Loading

What is the effect of supplying an explicit schema through .schema(...) when reading with Auto Loader?

  1. Auto Loader requires that schemaHints also be supplied for every column.
  2. Auto Loader uses the supplied schema and skips inferring one from the data. Correct answer.
  3. Auto Loader rejects any file whose columns do not exactly match the supplied schema.
  4. Auto Loader ignores the supplied schema and always infers from a sample of files.

Explanation

Providing a schema turns off inference, so the stream starts with the declared types and does not pay the cost of sampling files. Columns that do not fit the declared schema are not fatal; they are captured in the rescued data column when that option is enabled.

Question 50Data Ingestion and Loading

Which JDBC read option lets you ingest the result of a SQL subquery rather than an entire table?

  1. query Correct answer.
  2. partitionColumn
  3. fetchsize
  4. dbtable

Explanation

The query option accepts a parenthesized subquery whose result becomes the source, which is how filtering and joins are pushed to the remote database. It is mutually exclusive with dbtable in the same read.