I85PREP
⌘ K
CGOA study materials
CGOA study resources

Auditing Four Principles Against a "GitOps" Estate That Pushes from CI

CGOA · Put your knowledge to work in context.

All case studies

Auditing Four Principles Against a "GitOps" Estate That Pushes from CI

Helio Health says it "does GitOps" for 60 microservices. GitHub holds Helm charts and values; a GitHub Actions workflow on merge to main runs helm upgrade --install against 5 clusters using a long-lived kubeconfig stored as a repository secret. Infrastructure (VPCs, IAM, RDS) is created by Terraform Cloud workspaces triggered from the same repo, with state in Terraform Cloud. Namespace quotas and network policies were applied once by a consultant with kubectl apply and exist nowhere in Git. Secrets are created by a script that reads from 1Password and runs kubectl create secret during each deploy. An auditor asked which of the four OpenGitOps principles the team satisfies; the team answered "all of them, everything is in Git" and was challenged. Last quarter a manual quota change in production survived three months and was discovered only when a rollout failed. The team wants a defensible gap list and a minimal remediation sequence.

1. Evaluate the estate against each of the four OpenGitOps principles, citing concrete evidence from the setup and marking each as satisfied, partial, or violated.

Read the worked answer

1. Declarative - partial. Helm charts/values and Terraform describe intent declaratively, which is good. But the quota and network-policy objects were applied imperatively once and are absent from Git, so a meaningful slice of live configuration has no declarative representation anywhere. Anything not declared cannot be reconciled.

2. Versioned and immutable - partial. Manifests are in Git with history, satisfying "versioned." Immutability is weaker: the deploy path resolves mutable tags rather than digests, so two deploys of the same commit can produce different running artifacts, and the deployed revision is not uniquely recoverable. Terraform state lives outside Git, which is normal but means the record of what was applied is not fully in Git.

3. Pulled automatically - violated. The direction of control is push: GitHub Actions authenticates into each cluster with a static kubeconfig and applies. No agent inside the cluster observes Git and decides to update itself. This is the clearest and most consequential violation - it is what makes the estate CI/CD-with-IaC rather than GitOps.

4. Continuously reconciled - violated. Nothing continuously compares live to desired. The three-month manual quota change is the proof by experiment: no process detected it, and no process would have corrected it. Terraform Cloud runs when triggered, not on a loop.

Summary: 0 fully satisfied, 2 partial, 2 violated. The team's claim collapses on the two principles that distinguish GitOps from a deployment pipeline: pull and continuous reconciliation. Note that declarative + versioned are necessary but nowhere near sufficient; a large fraction of push-based pipelines pass those two and fail the other two.

2. The team proposes adding a nightly scheduled `terraform plan` job that opens a PR when drift is found. Does that satisfy continuous reconciliation? Reason about detection versus correction, and what would close the remaining gap for cloud resources.

Read the worked answer

No - it is periodic drift detection, not continuous reconciliation. Three gaps:

  1. Cadence. Nightly is sampling, not continuous; a change made at 00:05 survives the whole day and may be deliberately timed to avoid detection. "Continuous" in the GitOps sense means an agent loops and retries indefinitely, not a cron job.
  2. No converging actor. The output is a PR that a human must merge and then a separate run must apply. Reconciliation includes the action that moves live state back to desired state. A PR is a notification with a to-do attached.
  3. Blast radius of the plan. terraform plan reflects the state file and the credentials' view; drift in resources not in state, or in resources the plan is configured to ignore, is invisible.

What would close the gap depends on how far you want to extend the boundary:

  • Narrow boundary (pragmatic): declare GitOps for cluster workloads - put an in-cluster agent (Flux or Argo CD) in front of Helm/Kustomize so pull and continuous reconciliation hold - and explicitly document cloud infra as "IaC with drift alerting" rather than claiming it is GitOps. Honesty here is what the audit actually needs.
  • Wide boundary: run a controller that continuously reconciles cloud resources from Git (Crossplane, or a Terraform/OpenTofu controller that runs plan/apply on a loop with approval gates for destructive changes). This gives real convergence but raises the stakes: an auto-applying controller on a database or IAM is a much sharper tool than one applying Deployments, so pair it with policies, manual gates on destructive plan diffs, and state locking.

Also fix the manual quota objects: either import them into the declarative source or delete and recreate them from Git, otherwise they remain permanently unreconcilable.

3. Does the 1Password-plus-`kubectl create secret` script violate the GitOps model, and what pattern would preserve both the principle and the security requirement?

Read the worked answer

Yes, it is an imperative side channel. Every deploy writes live state that is not derived from Git. Consequences: the secret has no version history, a cluster restored from Git has no secrets, you cannot diff desired versus live, and re-running the pipeline with a rotated 1Password item silently changes production with no commit and no review. The deploy also needs broad write access at exactly the moment it runs.

Patterns that keep the principle intact, with tradeoffs:

  • External Secrets Operator (or Secrets Store CSI) referencing Vault/1Password. Git holds an ExternalSecret declaring which secret maps to which Kubernetes key plus refresh interval; the operator pulls the value at runtime and re-syncs on rotation. This is the strongest fit for a bank already running Vault: the reference is declarative and reconciled, values are never in Git, and Vault's short-lived Kubernetes-auth tokens replace static credentials. Tradeoff: a second source of truth for values, and the cluster now needs a live dependency on Vault - decide behaviour when Vault is unreachable (cached value vs fail closed).
  • SOPS/age (or Sealed Secrets) with encrypted values committed to Git. Everything is in Git and fully reconcilable, works offline, no runtime dependency. Tradeoff: key distribution and rotation become the hard problem, and anyone with the cluster decrypt key can read every secret in that repo - awkward for 14 teams sharing a cluster unless key scoping is per-environment.
  • Never keep the current shape (script + long-lived cluster credentials); at minimum move to short-lived, namespace-scoped credentials, but note that this alone fixes blast radius, not the GitOps gap.

Recommendation: External Secrets Operator with Vault for production and per-cluster scoping; if any edge site cannot reach Vault, use SOPS for that subset and document the exception.