GCP Data Practitioner flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Storage Transfer Service

Managed service that copies files on a schedule from Cloud Storage, Amazon S3, Azure Blob, HTTP/HTTPS sources, or on-premises file systems into Google Cloud destinations such as Cloud Storage or BigQuery.

Datastream CDC replication

Change data capture service that continuously replicates inserts, updates, and deletes from MySQL, PostgreSQL, SQL Server, or Oracle into BigQuery or Cloud Storage with low latency.

BigQuery load job formats

Load jobs support CSV, newline-delimited JSON, Avro, Parquet, and ORC. Avro and Parquet carry typed, nested schemas; CSV requires an explicit schema and has no nested type support.

BigQuery schema autodetect

BigQuery samples source data to infer column names and types. It is unreliable for ambiguous types, empty files, or strict typing needs, so an explicit schema is preferred in those cases.

BigQuery external tables

Tables whose data stays in Cloud Storage, BigLake, or Drive. Queries read the external files each run, so there is no BigQuery storage charge but repeated scan cost.

WRITE_APPEND vs WRITE_TRUNCATE

BigQuery write dispositions: WRITE_APPEND adds rows to the existing table, WRITE_TRUNCATE replaces all existing rows, and WRITE_EMPTY fails if the destination already contains data.

BigQuery Data Transfer Service

Scheduled ingestion from SaaS sources such as Google Ads, YouTube, and Amazon S3 into BigQuery, with configurable run frequency and backfill of historical date ranges.

Pub/Sub message retention

Pub/Sub keeps unacknowledged messages for 7 days by default; the retention window can be configured up to 31 days. Subscription retention can also replay acknowledged messages.

On-demand vs flat-rate pricing

BigQuery on-demand bills per byte scanned with a monthly free tier; flat-rate and slot-based editions bill for dedicated compute capacity regardless of bytes scanned.

BigQuery dry run

Validates a query and returns the estimated bytes it would process without executing it or incurring charges. Used to preview cost before running expensive queries.

BI Engine

BigQuery in-memory analysis layer that accelerates dashboard queries and reduces bytes scanned for Looker Studio and other BI tools, with reserved capacity sized in GB.

BigQuery materialized view

Precomputed, incrementally refreshed query result stored as a table. The optimizer can rewrite matching queries to read the view, reducing scan cost and latency.

APPROX_COUNT_DISTINCT

BigQuery function returning an approximate number of distinct values using HyperLogLog++ sketches; far cheaper than exact COUNT(DISTINCT) on very large datasets with small error.

QUALIFY clause

BigQuery clause that filters rows on the output of window functions, avoiding a subquery wrapper. It is evaluated after window computation and before ORDER BY.

Looker Studio data sources

Free reporting tool connecting to BigQuery, Looker, Google Sheets, Cloud SQL, and third-party connectors. It builds dashboards without LookML, unlike the Looker product.

Partitioning versus clustering

Partitioning splits a table by date or ingestion time to prune scanned data; clustering sorts data within partitions by chosen columns to reduce bytes read for filtered queries.

Cloud Composer

Managed Apache Airflow service. Pipelines are Python DAG files; task dependencies use the >> operator or set_upstream and set_downstream, and scheduling runs on managed workers.

Dataflow and Apache Beam

Dataflow is a managed runner for Apache Beam pipelines. The same pipeline code can run in batch or streaming mode, with worker autoscaling based on backlog.

Dataflow Flex template

Packaged Dataflow pipeline delivered as a container image plus metadata file, allowing parameterized launches without rebuilding code and easier dependency management.

Cloud Scheduler

Fully managed cron job service that triggers HTTP endpoints, Pub/Sub topics, or App Engine routes on a schedule. It invokes work but does not transform data itself.

Cloud Functions triggers

Event-driven serverless functions invoked by Cloud Storage object events, Pub/Sub messages, or HTTP calls. Suited to short tasks, not long-running data transforms.

Airflow sensor tradeoff

Sensors wait for a condition such as a file arriving, occupying a worker slot while waiting. Deferrable operators release the slot and resume when the condition is met.

Dataproc ephemeral clusters

Short-lived Dataproc clusters created for a single Spark or Hadoop job and then deleted. Workflow templates automate creation, job submission, and teardown to control cost.

Dataform SQL orchestration

Google Cloud service that manages SQL transformation workflows in BigQuery, with dependency graphs, assertions, and Git-based version control of SQLX definitions.

IAM role additivity

Google Cloud IAM grants are additive: a principal receives the union of all bound roles. There are no deny rules, so removing access means removing bindings.

BigQuery authorized view

A view granted access to a dataset's underlying tables without granting users direct table access. It exposes only the columns and rows the view selects.

Policy tags for column security

Taxonomy-based tags applied to BigQuery columns to restrict access at column level. Users need the fine-grained reader role on the tag to read tagged data.

Row-level security policy

BigQuery row access policy attaches a filter predicate to a table so different users see different row subsets. Users without the filter see no rows from the table.

Uniform bucket-level access

Cloud Storage setting that disables object ACLs and enforces permissions only through IAM bucket policies, simplifying and making access consistent across all objects.

VPC Service Controls perimeter

Security boundary around Google Cloud services such as BigQuery and Cloud Storage that blocks data movement outside the perimeter, mitigating exfiltration risk.

BigQuery time travel

Ability to query a table as it existed at any point in the past 7 days by default, using FOR SYSTEM_TIME AS OF, enabling recovery of deleted or changed data.

INFORMATION_SCHEMA.JOBS

BigQuery view listing job history including user, statement type, bytes processed, and slot milliseconds, used for cost attribution and audit of query activity.