GCP GenAI Engineer cheat sheet

Vertex AI is the unified GCP ML platform: datasets, training, pipelines, endpoints, model registry, and monitoring live in one project-scoped service.

Choose BigQuery ML when the model trains on data already in BigQuery and SQL-first iteration is enough; choose Vertex AI custom training when you need arbitrary frameworks, GPUs, or custom containers.

Vertex AI Model Garden provides foundation models (Gemini, Imagen, Embeddings) and open models; use publisher endpoints for pay-per-token or deploy to a dedicated endpoint for guaranteed throughput.

Prefer a managed service ladder: Vision/Video/Natural Language APIs for single-task prediction, AutoML for custom labels without code, custom training for full control.

A Vertex AI Pipeline is a DAG of components executed by Vertex AI Pipelines; artifacts and parameters flow through ML Metadata, enabling lineage and reproducibility.

Region choice is constrained by service availability (e.g., Gemini model availability, TPU regions) and data residency; Vertex AI resources are regional, so co-locate data, training, and serving.

Quota design: check regional vCPU, GPU (e.g., A100, L4), and TPU quotas before architecture sign-off; custom training jobs and prediction endpoints consume separate quotas.

For low-latency online inference, deploy to an endpoint; for batch scoring use Batch Prediction; for scheduled retraining use Vertex AI Pipelines with a Cloud Scheduler trigger.

Security architecture: use service accounts with least privilege, VPC Service Controls perimeters around Vertex AI and BigQuery, and CMEK for data at rest.

Dataflow with Apache Beam is the managed runner for batch and streaming preprocessing; use it when transformations exceed SQL or need windowing, state, and exactly-once streaming.

BigQuery is the default store for tabular features and training data; use external tables only when data must stay in GCS, and prefer native tables for performance.

Vertex AI Feature Store stores feature values with entity IDs and timestamps; point-in-time correct retrieval prevents label leakage during training.

For unstructured data (images, text, audio), store raw files in Cloud Storage, reference them in a Vertex AI managed dataset, and let AutoML or custom training read from GCS.

Use Dataprep or Dataflow for cleaning; use Dataplex and Knowledge Catalog for discovery, lineage, and quality metadata across BigQuery and GCS.

Train/test/valid splits must be created before any fitting of scalers or encoders; fit preprocessing on the training split only and persist the transform for serving.

TFRecord or Parquet are preferred training formats; avoid CSV for large datasets because of parsing cost and type ambiguity.

Streaming ingestion pattern: Pub/Sub -> Dataflow (windowed aggregation) -> BigQuery or Feature Store; this supports near-real-time feature freshness.

Data quality checks belong in the pipeline: use Beam assertions or a validation component that fails the pipeline on schema or distribution drift.

Vertex AI custom training runs your training code in a prebuilt or custom container; specify a Python package or source distribution and a machine type with optional accelerators.

Use Vertex AI Experiments to log parameters, metrics, and artifacts; use Vertex AI TensorBoard for metric curves and profiling.

Hyperparameter tuning jobs in Vertex AI use Bayesian optimization by default; define the metric, goal, parameter ranges, and max trial count.

Distributed training: use reduction server or multi-worker strategies for large models; choose GPU or TPU topology that matches the framework (e.g., PyTorch DDP, JAX on TPU).

For tabular data, BigQuery ML supports linear/logistic regression, boosted trees (XGBoost), and AutoML Tables; export the model for custom serving if needed.

Gemini tuning: supervised fine-tuning adjusts a base model on labeled examples; use it for style/format adaptation, not for injecting large new knowledge bases (use RAG for that).

Model evaluation must use a held-out set; report task-appropriate metrics (AUC, F1, RMSE, BLEU) and compare against a baseline before promotion.

Explainability: Vertex AI supports feature attributions (Sampled Shapley, Integrated Gradients) for custom and AutoML models to validate feature reliance.

Vertex AI Pipelines compiles a KFP or TFX pipeline to a JSON spec and runs it on managed infrastructure; each component runs in its own container.

Use Cloud Scheduler + Cloud Functions or Pub/Sub to trigger pipeline runs on a cron or event; Vertex AI Pipelines itself does not schedule runs.

Pipeline components should be deterministic and idempotent; cache enabled components by setting enable_caching so unchanged steps are skipped.

ML Metadata tracks executions, artifacts, and lineage; use it to trace which dataset and code produced a model version.

Model Registry stores model versions and aliases; promote a model by assigning an alias (e.g., 'prod') that the serving deployment references.

CI/CD for ML: Cloud Build builds and tests training/serving images, then a pipeline step deploys to a staging endpoint for validation before production.

Use Vertex AI Pipelines with a conditional or exit handler to gate promotion on evaluation metrics; failed gates should stop deployment.

For recurring retraining, combine a scheduled pipeline with data freshness checks and automatic model evaluation before alias reassignment.

Vertex AI Endpoints support automatic scaling based on traffic; set min and max replica counts to control cold starts and cost.

Deploy multiple models to one endpoint with traffic split percentages to perform canary or A/B rollout.

Use dedicated endpoints for foundation models when you need guaranteed throughput or custom serving; use publisher endpoints for pay-per-token simplicity.

Batch prediction is cheaper for large offline scoring; online endpoints are for low-latency interactive requests.

Model serving containers must implement the Vertex AI prediction contract: health, predict, and optionally explain routes.

For GPU serving, choose an accelerator-optimized machine type and confirm the model fits in GPU memory with batch size headroom.

Private endpoints use VPC peering and are reachable only from your network; public endpoints are internet-reachable and should be protected with IAM.

Reduce serving cost by using smaller machine types, autoscaling to zero where acceptable, and batching requests client-side.

Monitor endpoint latency, error rate, and CPU/GPU utilization; use these signals to tune replica counts and machine shapes.

Vertex AI Model Monitoring detects training-serving skew and prediction drift using configured thresholds on input features and output distributions.

Skew compares live serving distribution to the training baseline; drift compares live distribution over time; both require a baseline and a monitoring schedule.

Set alerting via Cloud Monitoring policies on endpoint error rate, latency, and model monitoring anomalies; route to Pub/Sub or email.

Continuous evaluation for generative models uses Vertex AI Gen AI evaluation or custom evaluators; track quality metrics over time, not just latency.

Optimize training cost with preemptible/spot VMs for fault-tolerant jobs, early stopping in hyperparameter tuning, and right-sized accelerators.

Optimize inference cost with batch prediction, smaller distilled models, quantization, and autoscaling with realistic min replicas.

Retraining triggers: drift threshold breach, scheduled cadence, or performance degradation on a labeled holdout; automate via pipelines.

Use Vertex AI Model Registry aliases and versioning to roll back a bad deployment by reassigning the serving alias to a previous version.

Audit and governance: enable Cloud Audit Logs for Vertex AI, track model lineage in ML Metadata, and document data and model cards for compliance.