Build and Implement CI/CD Pipelines, Including Continuous Testing
Pipeline anatomy
A pipeline is trigger → build and test → artifact → deploy → verify. The Google Cloud primitives are Cloud Build (triggers on Cloud Source Repositories, GitHub, GitLab, or Bitbucket; build steps as containers; private pools for VPC access; substitutions and Secret Manager for secrets - never bake credentials into build config), Artifact Registry (container images, language packages, and ML artifacts, with vulnerability scanning and cleanup policies), and Cloud Deploy (delivery pipelines → targets → releases → rollouts → promotions, with approval gates and canary or blue/green strategies driven by Skaffold render and manifest steps).
Continuous testing spans layers: unit and integration tests as build steps, container and dependency scanning when an artifact is pushed, policy-as-code validation for infrastructure (terraform validate and plan, terraform vet or OPA) before apply, and post-deploy smoke tests plus SLO-based analysis during a canary.
Keep infrastructure pipelines separate from application pipelines: plan on pull request, apply on merge to main, state in Cloud Storage, and one dedicated deploy service account per environment.
ML workloads add data validation (schema and statistics checks), training pipeline orchestration (Vertex AI Pipelines or Kubeflow), evaluation gates against a baseline model, registry versioning, and continuous-training triggers. Promoting a model is a deployment like any other: shadow or canary endpoint first, then a traffic split.
Trap: Cloud Deploy has no in-place rollback - you roll back by promoting a previous release to the target. And a trigger's service account, not the human who pushed, determines what the build is allowed to do.
Rule of thumb: every artifact reaching production must be identifiable by immutable digest and traceable to a commit SHA.