GCP Data Engineer study plan

Sketch the end-to-end architecture of three reference pipelines (batch ELT, streaming event ingestion, hybrid CDC) and label every GCP service on each hop

Compare BigQuery, Cloud SQL, Spanner, Bigtable, Firestore, and Cloud Storage on access pattern, scale, and consistency, and produce a written selection rule for each

Choose a file format and partitioning/clustering strategy for two sample datasets and justify the choice against query cost and scan volume

List the IAM roles, service accounts, and network controls (private IP, VPC Service Controls, CMEK) each reference architecture needs

Map each of the three reference architectures to a concrete migration or build sequence, ordering the steps by dependency

Design the schema, partitioning, clustering, and expiration policy for a BigQuery warehouse layer (raw, staging, curated)

Identify where data quality checks and idempotency belong in each pipeline, and specify the checkpoints

Write out the cost model for one pipeline: storage, shuffle/compute, and query scan, with the two biggest cost levers

Build a batch pipeline that lands files from Cloud Storage into BigQuery and schedule it, then re-run it to prove idempotency

Build a streaming pipeline that ingests from Pub/Sub, applies windowing and late-data handling, and writes to BigQuery

Implement a CDC path using Datastream or a change-stream pattern and explain how deletes and updates are propagated downstream

Compare Dataflow, Dataproc, Cloud Data Fusion, and Dataproc Serverless for two given workloads and pick one for each with reasons

Add error handling and a dead-letter path to at least one pipeline

Tune one Dataflow job using autoscaling, worker type, and shuffle settings, and record the before/after metrics

Define the storage layout for a lakehouse-style zone: bucket structure, lifecycle rules, and format choices

Explain and configure retention, time travel, and failover behavior for one BigQuery dataset and one Cloud SQL or Spanner instance

Document how to move data between storage tiers and services (Storage to BigQuery, Bigtable to BigQuery, BigQuery to Cloud Storage export)

Pick a storage service for four given workload profiles (high-throughput key lookups, relational OLTP, analytical scans, document/mobile sync) with a written rationale for each

Configure a Bigtable or Spanner schema (row key or interleaving) for a high-write workload and explain the hotspot or distribution reasoning

Implement external tables or a federated query against Cloud Storage and Cloud SQL and note the performance limits you observe

Define a backup, restore, and disaster-recovery plan for two different storage services

Build a curated BigQuery layer using views, materialized views, and scheduled queries, and measure the scan cost before and after

Implement data quality and cleansing rules (deduplication, null handling, type casting, PII masking with DLP or SQL) on a sample dataset

Create a reporting asset in Looker Studio or a Looker-style semantic model that answers three defined business questions

Use BigQuery ML or a notebook-based workflow to train and evaluate one simple model on the curated data

Orchestrate two of your week 3 pipelines with Cloud Composer or Cloud Workflows, including retries and failure alerts

Instrument one pipeline with Cloud Monitoring dashboards and log-based alerts for latency, error rate, and data freshness

Automate deployment of the pipeline infrastructure with Terraform or an equivalent IaC setup and prove it recreates the environment from scratch

Write an SLA/SLO definition and an on-call runbook for one pipeline, covering common failure modes and rollback

Take two full-length timed practice exams and log every missed question by domain

Re-derive the selection rules for compute, storage, and ingestion services from memory, then correct them against your notes

Revisit the two weakest domains and rewrite the decision records from weeks 1 and 5 with the corrections

List every service you still cannot place confidently and close those gaps with focused reading