I85PREP
⌘ K
GCP Data Practitioner study materials
GCP Data Practitioner study resources

GCP Data Practitioner practice questions

One question at a time. Build your GCP Data Practitioner understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Prepare and ingest data

A data engineer wants to land raw CSV files into Cloud Storage before any transformation happens. Which statement describes the role of Cloud Storage in a typical ingestion pipeline on Google Cloud?

  1. It enforces row-level access policies on the records contained inside each file object.
  2. It automatically infers schemas and applies column types to the files as they are uploaded.
  3. It converts the files into columnar format and partitions them by ingestion date.
  4. It acts as a durable landing zone from which downstream services such as BigQuery or Dataflow read the raw files. Correct answer.

Explanation

Cloud Storage is an object store used as a durable, low-cost landing zone for raw files before transformation. Schema inference, format conversion, and partitioning are performed by tools such as BigQuery load jobs, Dataflow, or Dataproc, not by the bucket itself. Row-level security is a BigQuery table concept, not a property of stored objects.

Question 2Prepare and ingest data

A team needs to run a recurring ingestion job that reads files from Cloud Storage, applies transformations, and writes to BigQuery. They want a fully managed, serverless runner with no cluster provisioning. Which Google Cloud service fits best?

  1. Cloud SQL
  2. Cloud Dataproc
  3. Cloud Dataflow Correct answer.
  4. Compute Engine managed instance groups

Explanation

Dataflow is a fully managed, serverless service for batch and streaming pipelines built on Apache Beam, so no cluster provisioning is required. Dataproc provides managed Hadoop and Spark clusters, which still require sizing decisions. Compute Engine instance groups and Cloud SQL do not provide a managed data pipeline runtime.

Question 3Prepare and ingest data

An analyst loads a newline-delimited JSON file into BigQuery and wants the service to determine column names and types from the data itself. Which load option should be enabled?

  1. Allow quoted newlines
  2. Maximum bad records
  3. Schema autodetect Correct answer.
  4. Field delimiter

Explanation

Schema autodetect tells BigQuery to infer field names and data types from the source data during a load job. Maximum bad records controls error tolerance, field delimiter applies to delimited formats such as CSV, and quoted newline handling affects parsing rather than schema.

Question 4Prepare and ingest data

A pipeline must ingest clickstream events with sub-second latency and make them queryable in BigQuery with near-real-time freshness. Which ingestion approach is most appropriate?

  1. Import the events into Cloud SQL and run federated queries from BigQuery.
  2. Schedule an hourly batching job that exports files to Cloud Storage and loads them into BigQuery.
  3. Publish events to Pub/Sub and stream them into BigQuery through a Dataflow streaming pipeline. Correct answer.
  4. Use BigQuery external tables that point directly at the raw event files in Cloud Storage.

Explanation

Pub/Sub provides a scalable messaging layer for high-volume events, and a Dataflow streaming pipeline can transform and write them to BigQuery with low latency. Hourly batch loads add delay, external tables do not provide streaming inserts, and Cloud SQL is not designed for high-throughput event ingestion.

Question 5Prepare and ingest data

A Dataflow pipeline reads from Pub/Sub and writes aggregated results to BigQuery. The team wants exactly-once semantics for the BigQuery output when the pipeline runs in streaming mode. Which feature should they rely on?

  1. Pub/Sub message retention of seven days
  2. BigQueryIO with deterministic write and appropriate windowing, using the Storage Write API Correct answer.
  3. BigQuery load jobs triggered by a Cloud Scheduler cron
  4. Dataproc autoscaling policies

Explanation

BigQueryIO supports exactly-once streaming writes when used with deterministic writes and the Storage Write API, combined with suitable windowing and triggers. Cloud Scheduler load jobs are batch and not tied to pipeline checkpoints, message retention affects replay not write deduplication, and Dataproc autoscaling is unrelated.

Question 6Prepare and ingest data

An engineer wants to query CSV files stored in a Cloud Storage bucket directly from BigQuery without loading the data into native tables. Which BigQuery feature should be used?

  1. A materialized view
  2. A clustered table
  3. A snapshot table
  4. An external table Correct answer.

Explanation

External tables let BigQuery query data that resides in Cloud Storage without ingesting it into native storage. Materialized views, clustered tables, and snapshots all operate on data already stored in BigQuery datasets.

Question 7Prepare and ingest data

A CSV file contains a header row, and an analyst loading it into BigQuery wants the first row treated as column names rather than data. Which load configuration accomplishes this?

  1. Enable allow quoted newlines
  2. Set maximum bad records to 1
  3. Set the field delimiter to a comma
  4. Set the header rows to skip to 1 Correct answer.

Explanation

The skip leading rows setting (header rows to skip) tells BigQuery to ignore the first row so it is used as column names rather than data. Maximum bad records tolerates malformed rows, the field delimiter defines separators, and quoted newline handling affects parsing of embedded line breaks.

Question 8Prepare and ingest data

A Dataflow batch pipeline reads thousands of small files from Cloud Storage. The team notices poor throughput. Which change is most likely to improve read performance?

  1. Disable autoscaling on the Dataflow job
  2. Increase the number of Pub/Sub topics used by the pipeline
  3. Consolidate the small files into fewer larger files before the pipeline reads them Correct answer.
  4. Switch the output sink from BigQuery to Cloud SQL

Explanation

Many small files create per-file overhead that limits parallel read throughput in Dataflow. Consolidating them into larger files reduces that overhead and lets workers read more efficiently. Pub/Sub topics, autoscaling, and the output sink do not address small-file read overhead.

Question 9Prepare and ingest data

An organization needs a managed service to run Spark jobs that read from Cloud Storage and write to BigQuery, and they want to keep using existing Spark code. Which service is the best fit?

  1. Cloud Pub/Sub
  2. Cloud Data Fusion
  3. Cloud Dataproc Correct answer.
  4. BigQuery BI Engine

Explanation

Dataproc is a managed Spark and Hadoop service that lets teams run existing Spark code with minimal changes. Pub/Sub is a messaging service, Data Fusion is a graphical data integration tool, and BI Engine accelerates BigQuery queries rather than running Spark jobs.

Question 10Prepare and ingest data

A Cloud Storage bucket receives files continuously, and a pipeline should process each new object exactly once as soon as it arrives. Which mechanism triggers the pipeline on object creation?

  1. A BigQuery scheduled query
  2. A Dataproc initialization action
  3. A Cloud Storage event notification published to Pub/Sub Correct answer.
  4. A Cloud Scheduler job polling the bucket listing

Explanation

Cloud Storage can publish object change notifications to Pub/Sub, which a Dataflow or Cloud Functions consumer can use to react to each new object. Scheduled queries and cron polling add latency and do not guarantee exactly-once processing, and initialization actions only run at cluster startup.

Question 11Prepare and ingest data

A data engineer needs to load a large Avro file into BigQuery and preserve nested and repeated fields. Which load format best supports that requirement?

  1. Plain text with a pipe delimiter
  2. Avro Correct answer.
  3. CSV
  4. TSV

Explanation

Avro is a self-describing format that supports nested and repeated fields, making it suitable for loading complex schemas into BigQuery. CSV, delimited text, and TSV are flat formats that cannot represent nested records natively.

Question 12Prepare and ingest data

A Dataflow pipeline needs to enrich incoming records with reference data that changes daily. The reference data is small enough to fit in memory on each worker. Which approach is most efficient?

  1. Join against a Cloud SQL table for every element
  2. Use a side input to broadcast the reference data to all workers Correct answer.
  3. Write each record to Pub/Sub and read it back
  4. Perform a per-record API call to an external service

Explanation

Side inputs broadcast a small, slowly changing dataset to every worker so lookups happen in memory without per-element network calls. Per-record API calls and per-element database joins add latency and cost, and round-tripping through Pub/Sub is unnecessary.

Question 13Prepare and ingest data

An analyst must load data into BigQuery from a CSV file that contains a few malformed rows. They want the job to succeed while capturing the rejected rows for later review. Which combination of settings supports this?

  1. Enable schema autodetect only
  2. Use a wildcard URI for the source file
  3. Set the field delimiter to a comma
  4. Set maximum bad records to a positive value and enable the option to write rejected rows to a table Correct answer.

Explanation

Maximum bad records allows a load job to tolerate a specified number of malformed rows, and the rejected-rows option writes those rows to a separate table for review. Autodetect, delimiter, and wildcard URIs do not provide error tolerance or capture of bad records.

Question 14Prepare and ingest data

A team wants to ingest data from an on-premises relational database into BigQuery on a recurring schedule with minimal code. Which Google Cloud service is designed for this?

  1. Cloud CDN
  2. Cloud Functions triggered by HTTP
  3. Cloud Dataflow with a custom Apache Beam source
  4. BigQuery Data Transfer Service with a supported connector Correct answer.

Explanation

BigQuery Data Transfer Service provides managed, scheduled transfers from supported sources with little custom code. A custom Beam source requires engineering effort, Cloud Functions is a compute primitive, and Cloud CDN is a content delivery service.

Question 15Prepare and ingest data

A streaming pipeline must handle late-arriving events without dropping them and still produce updated aggregates. Which Apache Beam concept addresses this?

  1. A fixed field delimiter
  2. Allowed lateness with triggers on the window Correct answer.
  3. A BigQuery snapshot table
  4. A Dataproc initialization action

Explanation

Allowed lateness keeps window state available after the watermark passes so late elements can still update results, and triggers control when those updates are emitted. Delimiters, snapshot tables, and initialization actions do not affect windowing behavior.

Question 16Prepare and ingest data

An engineer needs to grant a service account permission to read objects from a specific Cloud Storage bucket used for ingestion, following least privilege. Which role is most appropriate?

  1. BigQuery Data Owner on the project
  2. Project Editor
  3. Storage Admin on the project
  4. Storage Object Viewer on the bucket Correct answer.

Explanation

Storage Object Viewer grants read access to objects and can be scoped to a single bucket, satisfying least privilege for reading ingestion files. Storage Admin, BigQuery Data Owner, and Project Editor grant broader permissions than needed.

Question 17Prepare and ingest data

A pipeline writes to BigQuery and the team wants to reduce cost and improve query performance by organizing rows on a commonly filtered column. Which table design should they use?

  1. A wildcard table over multiple datasets
  2. A clustered table on the filtered column Correct answer.
  3. A view with no partitioning
  4. An external table over CSV files

Explanation

Clustering physically organizes table data by the clustering columns, so filters on those columns scan less data and cost less. External tables, wildcard tables, and unpartitioned views do not provide that physical organization.

Question 18Prepare and ingest data

A team needs to transform data in BigQuery using SQL statements scheduled to run on a recurring basis without managing external orchestration. Which feature should they use?

  1. BI Engine
  2. External tables
  3. Scheduled queries Correct answer.
  4. BigQuery Omni

Explanation

Scheduled queries run SQL on a defined schedule inside BigQuery, providing lightweight recurring transformation without external orchestrators. External tables, BI Engine, and Omni address querying external data, query acceleration, and multi-cloud access respectively.

Question 19Prepare and ingest data

A Dataflow pipeline reads from a Kafka cluster hosted outside Google Cloud and writes to BigQuery. Which component provides the connection to Kafka within Apache Beam?

  1. A KafkaIO source connector Correct answer.
  2. A BigQueryIO side input
  3. A Dataproc initialization action
  4. A Cloud Storage notification

Explanation

Apache Beam provides KafkaIO as a source connector for reading from Kafka topics. BigQueryIO is a sink or source for BigQuery, GCS notifications are storage events, and initialization actions configure clusters.

Question 20Prepare and ingest data

A data engineer must ensure that duplicate records arriving from an at-least-once source are removed before loading into a BigQuery reporting table. Which approach is most appropriate?

  1. Increase the number of workers in the pipeline
  2. Use a wildcard URI when reading the source
  3. Set maximum bad records to a high value
  4. Deduplicate on a stable key using a GroupByKey or stateful DoFn before the write Correct answer.

Explanation

At-least-once sources can emit duplicates, so the pipeline must deduplicate on a stable key, for example with GroupByKey or a stateful DoFn, before writing. Adding workers, tolerating bad records, and wildcard reads do not remove duplicates.

Question 21Prepare and ingest data

An organization wants a graphical, code-light environment to build data integration pipelines that connect many sources to BigQuery. Which Google Cloud service is intended for this?

  1. Cloud Data Fusion Correct answer.
  2. BigQuery BI Engine
  3. Cloud Pub/Sub
  4. Cloud CDN

Explanation

Cloud Data Fusion provides a visual, code-light pipeline builder for data integration across many sources and sinks. Pub/Sub is messaging, BI Engine accelerates queries, and Cloud CDN delivers web content.

Question 22Prepare and ingest data

A pipeline loads data into a partitioned BigQuery table. The team wants to avoid scanning the full table for a date-range report. Which practice ensures partition pruning?

  1. Query through an unpartitioned view
  2. Filter on a non-partitioned string column
  3. Filter the query on the partitioning column with a date range Correct answer.
  4. Select all columns with SELECT *

Explanation

Partition pruning occurs when a query filters on the partitioning column, so only the relevant partitions are scanned. Filtering other columns, selecting all columns, and querying through an unpartitioned view prevent pruning.

Question 23Prepare and ingest data

A team is choosing between loading data into BigQuery with a batch load job and streaming inserts for a dataset that must be available within minutes but does not need sub-second latency. Which choice is most cost-effective while meeting the requirement?

  1. Manual ad hoc loads by an analyst
  2. External tables over raw files with no scheduling
  3. Continuous streaming inserts for every record
  4. Scheduled batch load jobs at a short interval Correct answer.

Explanation

When minutes-level freshness is acceptable, frequent batch loads are typically cheaper than streaming every record. Continuous streaming is priced for higher throughput and lower latency, external tables add query overhead, and manual loads are not reliable.

Question 24Prepare and ingest data

A data engineer needs to validate incoming files in Cloud Storage and reject files that fail a schema check before loading into BigQuery. Which approach keeps invalid files out of the target table?

  1. Run a validation step in a pipeline or Cloud Function that quarantines failing files before the load Correct answer.
  2. Load all files and rely on maximum bad records to discard bad rows
  3. Load all files and query them with an external table
  4. Load all files and enable schema autodetect

Explanation

A validation step before the load, implemented in a pipeline or Cloud Function, can quarantine files that fail schema checks so they never reach the target table. Relying on bad-record tolerance, external tables, or autodetect does not prevent invalid files from being loaded or referenced.

Question 25Prepare and ingest data

An engineer wants to copy data from a Cloud Storage bucket in one project to a BigQuery dataset in another project. Which permission must the loading identity have on the destination dataset?

  1. BigQuery Data Editor on the destination dataset Correct answer.
  2. BigQuery Job User on the source project only
  3. Storage Object Viewer on the source bucket only
  4. Project Viewer on the destination project

Explanation

Writing data into a BigQuery dataset requires an edit-level role such as BigQuery Data Editor on that dataset, in addition to read access on the source objects. Source-only read roles, job-user roles, and viewer roles do not permit writing table data.

Question 26Prepare and ingest data

A team wants to schedule a recurring extraction from an on-premises PostgreSQL database into BigQuery without writing or maintaining any pipeline code. Which Google Cloud service is designed for this task?

  1. Cloud Data Fusion
  2. BigQuery Data Transfer Service Correct answer.
  3. Cloud Composer
  4. Dataproc

Explanation

BigQuery Data Transfer Service provides managed, scheduled transfers from supported sources, including database connectors, directly into BigQuery without customer-managed pipeline code. Cloud Data Fusion and Dataproc require building and operating pipelines, and Cloud Composer orchestrates workflows rather than providing a built-in database transfer connector.

Question 27Prepare and ingest data

An analyst needs to load a newline-delimited JSON file into BigQuery while keeping nested objects and repeated arrays intact. Which load approach preserves this structure?

  1. Loading the file into a single STRING column
  2. Loading the file with the NEWLINE_DELIMITED_JSON format Correct answer.
  3. Loading the file with the CSV format and schema autodetect enabled
  4. Loading the file as an external table over CSV

Explanation

BigQuery's NEWLINE_DELIMITED_JSON load format maps JSON objects and arrays to RECORD and REPEATED fields, so nested structure is retained. CSV loading flattens data and cannot represent nested or repeated fields natively, and storing the payload as a single STRING column discards the schema.

Question 28Prepare and ingest data

A Dataflow pipeline reads from a Pub/Sub subscription. The team must ensure that no message is lost if a worker crashes mid-processing. Which Pub/Sub behavior supports this requirement?

  1. Messages are duplicated to a second subscription on delivery
  2. Messages are automatically archived to Cloud Storage on delivery
  3. Messages remain in the subscription and are redelivered until acknowledged Correct answer.
  4. Messages are deleted from the subscription as soon as they are delivered to a subscriber

Explanation

Pub/Sub retains messages in a subscription and redelivers them if they are not acknowledged before the ack deadline expires, which prevents loss when a worker fails. Deletion on delivery would make crashes cause data loss.

Question 29Prepare and ingest data

A data engineer wants to ingest change data capture events from an operational database into BigQuery with minimal latency and no custom code. Which service should they evaluate?

  1. Cloud Storage Transfer Service
  2. BigQuery reservation
  3. Datastream Correct answer.
  4. Data Catalog

Explanation

Datastream is a serverless change data capture and replication service that streams changes from operational databases into destinations such as BigQuery and Cloud Storage. The other options move files, allocate compute, or catalog metadata rather than capture database changes.

Question 30Prepare and ingest data

A pipeline must read a Cloud Storage object only after it has been fully written. Which Cloud Storage feature lets the pipeline react to object finalization events without polling?

  1. Bucket retention policies
  2. Pub/Sub notifications on the bucket Correct answer.
  3. Signed URLs
  4. Object lifecycle rules

Explanation

Cloud Storage can publish notifications to Pub/Sub when objects are finalized, which lets downstream pipelines react to complete objects instead of polling. Lifecycle rules delete or transition objects, retention policies prevent deletion, and signed URLs grant time-limited access.

Question 31Prepare and ingest data

An engineer needs to load a Parquet file from Cloud Storage into BigQuery and keep the column types defined in the file. Which load configuration achieves this?

  1. Load with the AVRO format and ignore the file schema
  2. Load with the ORC format and a single-column schema
  3. Load with the CSV format and a manually defined schema
  4. Load with the PARQUET format and schema autodetect enabled Correct answer.

Explanation

Parquet is a self-describing columnar format, and BigQuery can read its schema automatically when autodetect is enabled, preserving the file's column types. Loading Parquet as CSV or ORC would misread the data, and ignoring the schema discards type information.

Question 32Prepare and ingest data

A team wants to move millions of files from an on-premises S3-compatible store into Cloud Storage without writing transfer code. Which service should they use?

  1. Dataprep by Trifacta
  2. Storage Transfer Service Correct answer.
  3. Cloud Scheduler
  4. BigQuery Data Transfer Service

Explanation

Storage Transfer Service is built for large-scale object transfers from sources such as S3-compatible stores, HTTP endpoints, and other buckets into Cloud Storage. The other services transform data, move database rows, or trigger jobs on a schedule.

Question 33Prepare and ingest data

A Dataflow pipeline reads a bounded source and the team wants the runner to choose worker count dynamically based on backlog. Which configuration supports this?

  1. Using a single high-memory worker
  2. Enabling autoscaling with a minimum and maximum worker range Correct answer.
  3. Setting the pipeline to streaming mode
  4. Setting a fixed number of workers with autoscaling disabled

Explanation

Dataflow autoscaling adjusts the number of workers between configured minimum and maximum bounds based on the workload, which is appropriate for variable backlog. A fixed worker count or a single worker prevents scaling, and streaming mode does not by itself control worker count.

Question 34Prepare and ingest data

An analyst needs to query an external Parquet dataset in Cloud Storage from BigQuery without ingesting it. Which BigQuery feature should they use?

  1. A BigQuery external table Correct answer.
  2. A BigQuery reservation
  3. A BigQuery materialized view
  4. A BigQuery routine

Explanation

External tables let BigQuery query data that remains in Cloud Storage, including Parquet files, without loading it into native storage. Materialized views precompute results from BigQuery tables, routines hold SQL or JavaScript logic, and reservations allocate compute capacity.

Question 35Prepare and ingest data

A pipeline must reject records that fail validation and route them to a separate BigQuery table while valid records continue downstream. Which Dataflow pattern implements this?

  1. A window with a fixed duration
  2. A GroupByKey followed by a single write
  3. A side output that tags failed records Correct answer.
  4. A single ParquetIO write with no branching

Explanation

Side outputs let a ParDo transform emit failed records to a separate output collection, which can be written to a dead-letter table while the main output continues. Grouping or windowing does not separate valid from invalid records.

Question 36Prepare and ingest data

An engineer loads data into a BigQuery table and must ensure the load job does not partially commit when a bad row is encountered. Which load job configuration provides this guarantee?

  1. Enabling schema autodetect
  2. Setting maxBadRecords to a positive value
  3. Using a wildcard URI
  4. Using the default behavior that fails the job on any error Correct answer.

Explanation

By default a BigQuery load job fails and commits nothing when any row cannot be parsed, giving all-or-nothing behavior. Setting maxBadRecords tolerates bad rows and commits the rest, autodetect affects schema inference only, and wildcards change which files are read.

Question 37Prepare and ingest data

A team must ingest Avro files whose embedded writer schema differs slightly from the target BigQuery table schema. Which load option lets BigQuery reconcile the file schema with the table schema?

  1. Loading with the CSV format
  2. Disabling schema autodetect
  3. Setting ignoreUnknownValues to true
  4. Enabling schemaUpdateOptions to allow field additions Correct answer.

Explanation

schemaUpdateOptions allows a load job to add new fields to the destination table when the incoming data contains columns not yet in the schema. ignoreUnknownValues drops unknown fields rather than adding them, and disabling autodetect prevents schema inference.

Question 38Prepare and ingest data

A Dataflow streaming pipeline must join a high-volume event stream with a slowly changing lookup table stored in BigQuery. Which approach avoids querying BigQuery for every element?

  1. Using a side input loaded from BigQuery and refreshed periodically Correct answer.
  2. Using a GroupByKey on the event stream
  3. Calling the BigQuery API inside a DoFn for each element
  4. Writing events to BigQuery and joining later with SQL

Explanation

Loading the lookup table as a side input caches it in worker memory and refreshes it on a schedule, avoiding a per-element API call. Per-element queries add latency and cost, GroupByKey does not provide lookup data, and deferring the join changes the pipeline design.

Question 39Prepare and ingest data

An organization needs to copy data from a Cloud SQL for MySQL instance into BigQuery continuously with minimal operational overhead. Which service is purpose-built for this?

  1. Dataproc with a Sqoop job
  2. Datastream with a Cloud SQL source Correct answer.
  3. Cloud Storage Transfer Service
  4. BigQuery Data Transfer Service

Explanation

Datastream supports Cloud SQL as a source and continuously replicates changes into BigQuery with a managed, serverless architecture. Storage Transfer Service moves objects, BigQuery Data Transfer Service targets scheduled dataset copies, and Sqoop on Dataproc requires managing cluster jobs.

Question 40Prepare and ingest data

A pipeline writes Avro records to Cloud Storage and later loads them into BigQuery. Which property of Avro makes it a good choice for this ingestion path?

  1. It supports only flat single-column records
  2. It stores data as plain text with comma delimiters
  3. It embeds the schema with the data Correct answer.
  4. It requires no schema at all

Explanation

Avro files carry their writer schema alongside the data, so BigQuery can read the structure without a separately supplied schema. Avro is a binary format, is schema-based, and supports nested records, so the other descriptions are inaccurate.

Question 41Prepare and ingest data

A team wants a no-code visual environment to build a pipeline that joins Cloud Storage data with a BigQuery table and writes the result back to BigQuery. Which service fits best?

  1. Cloud Composer
  2. Dataproc
  3. Cloud Scheduler
  4. Cloud Data Fusion Correct answer.

Explanation

Cloud Data Fusion provides a graphical pipeline designer with connectors and transforms for sources such as Cloud Storage and BigQuery. Composer orchestrates code-defined workflows, Dataproc runs clusters, and Scheduler triggers jobs on a timetable.

Question 42Prepare and ingest data

A data engineer must grant a Dataflow pipeline permission to write to a BigQuery dataset. Which IAM role assignment is the least privileged that still allows inserting rows?

  1. roles/bigquery.admin on the project
  2. roles/bigquery.dataEditor on the dataset Correct answer.
  3. roles/bigquery.metadataViewer on the dataset
  4. roles/owner on the project

Explanation

The BigQuery Data Editor role on the specific dataset permits creating and modifying tables and inserting data while limiting scope to that dataset. Project-level admin or owner grants far more than needed, and metadata viewer is read-only.

Question 43Prepare and ingest data

A streaming pipeline must handle events that arrive after their window has closed without losing them. Which Dataflow feature addresses this?

  1. Allowed lateness on the window Correct answer.
  2. A shorter window duration
  3. A fixed number of workers
  4. Discarding mode for accumulation

Explanation

Allowed lateness keeps window state available after the watermark passes so late events can still update results. Shortening windows reduces the chance but does not handle late data, discarding mode drops prior panes, and worker count affects capacity only.

Question 44Prepare and ingest data

An analyst wants to load a CSV file into BigQuery where the first line contains column names. Which load option handles this?

  1. Setting skipLeadingRows to 1 Correct answer.
  2. Setting allowQuotedNewlines to true
  3. Setting maxBadRecords to 1
  4. Setting fieldDelimiter to a tab

Explanation

skipLeadingRows tells BigQuery to ignore a specified number of initial rows, which is the standard way to skip a CSV header. maxBadRecords tolerates malformed rows, allowQuotedNewlines handles embedded line breaks, and fieldDelimiter changes the separator.

Question 45Prepare and ingest data

A team needs to load data into BigQuery from a Cloud Storage bucket in a different project. Which permission does the BigQuery service account or user need on the source bucket?

  1. storage.buckets.delete on the bucket
  2. bigquery.tables.create on the dataset
  3. storage.objects.get on the bucket Correct answer.
  4. bigquery.jobs.listAll on the project

Explanation

Reading source objects requires storage.objects.get, typically granted through roles/storage.objectViewer. BigQuery table permissions govern the destination, bucket deletion is unrelated, and job listing does not grant object reads.

Question 46Prepare and ingest data

A pipeline must deduplicate records arriving from an at-least-once source before loading them into BigQuery. Which Dataflow approach removes duplicates within a window?

  1. Applying a Distinct transform after grouping by a unique key Correct answer.
  2. Applying a ParDo that logs each record
  3. Applying a fixed window with no grouping
  4. Writing all records and relying on BigQuery to deduplicate

Explanation

Grouping by a unique key and then applying Distinct removes repeated records within the window. Logging does not change the data, windowing alone does not deduplicate, and BigQuery append writes do not automatically remove duplicates.

Question 47Prepare and ingest data

An engineer wants to validate that incoming Cloud Storage files match an expected schema before loading them, and route failures to a quarantine bucket. Which design is most appropriate?

  1. A Cloud Function triggered by object finalization that validates and moves files Correct answer.
  2. A Cloud Storage lifecycle rule that deletes old objects
  3. A BigQuery scheduled query that deletes invalid rows
  4. A BigQuery reservation that limits query slots

Explanation

An event-driven function can inspect each finalized object, check its schema, and copy invalid files to a quarantine bucket before any load occurs. Scheduled queries act after data is loaded, lifecycle rules only manage object age, and reservations control compute.

Question 48Prepare and ingest data

A team needs to move data from an on-premises Oracle database to BigQuery nightly with transformations, and they want a managed orchestration layer with retries and dependency handling. Which combination fits best?

  1. Manual execution of load jobs by an operator
  2. A Cloud Storage lifecycle rule
  3. A single Cloud Scheduler job that runs one SQL statement
  4. Cloud Composer to orchestrate Datastream or Dataflow jobs with dependencies Correct answer.

Explanation

Cloud Composer manages workflow dependencies, retries, and scheduling across ingestion and transformation steps, which suits a recurring multi-stage pipeline. A single scheduled statement, manual runs, and lifecycle rules cannot express dependencies or retries across jobs.

Question 49Prepare and ingest data

A Dataflow pipeline reading many small files from Cloud Storage shows low throughput. Which change most directly improves performance without altering the source data?

  1. Increasing the number of output shards
  2. Adding a transform that groups files into larger bundles before parsing Correct answer.
  3. Disabling autoscaling
  4. Reducing the number of workers

Explanation

Grouping many small files into larger bundles reduces per-file overhead and improves read throughput. Output sharding affects writes, fewer workers reduce capacity, and disabling autoscaling prevents scaling to the workload.

Question 50Prepare and ingest data

An organization must ingest data from a SaaS application's REST API that enforces strict rate limits and returns paginated results. Which ingestion design is most robust?

  1. A Cloud Run job or Dataflow pipeline with backoff, pagination handling, and checkpointing Correct answer.
  2. A BigQuery scheduled query that calls the API
  3. A Cloud Storage lifecycle rule that triggers the API
  4. A Dataproc cluster that calls the API once per day without retries

Explanation

Handling pagination, exponential backoff, and checkpointing in a Cloud Run job or Dataflow pipeline respects rate limits and allows resuming after failures. Scheduled queries cannot call external APIs, lifecycle rules do not invoke APIs, and a no-retry job will fail on rate limiting.