CGOA study materials
CGOA study resources

"Terraform in Git, Applied by Jenkins" - Is It GitOps?

CGOA · Put your knowledge to work in context.

All case studies

"Terraform in Git, Applied by Jenkins" - Is It GitOps?

Juniper Logistics has run Jenkins for six years. On every merge to main, Jenkins tests, builds images, runs terraform apply for cloud resources, then runs kubectl apply -f rendered/ against 9 clusters. Manifests and Terraform both live in Git with mandatory PR review. A nightly Jenkins job runs terraform plan and posts "no changes" to Slack. Leadership has announced a "GitOps migration" and asked for a scorecard of the current state. Two engineers disagree: one says "Terraform in Git plus apply on merge is GitOps"; the other says "nothing here is GitOps because there is no in-cluster agent." A security review found the Jenkins controller holds cluster-admin credentials for all 9 clusters, and that the nightly plan job cannot revert anything. The team also asks whether approving deploys in Slack (ChatOps) counts as GitOps.

1. Distinguish CI/CD, IaC and GitOps for this estate, and evaluate both engineers' claims against the four principles.

Read the worked answer

The three practices are not the same axis. CI/CD is the automate-build-test-deliver pipeline; it answers "how does an artifact reach an environment?" IaC is managing infrastructure through declarative definitions in version control; it answers "how is infrastructure described?" GitOps is an operating model with four specific properties: declarative, versioned and immutable, pulled automatically by software agents, continuously reconciled. GitOps is not a superset of CI/CD - a healthy GitOps setup usually keeps CI for building and testing and replaces the delivery step.

Engineer A is wrong. "Terraform in Git and apply on merge" satisfies declarative and versioned; it fails pulled-automatically (Jenkins pushes with cluster credentials) and fails continuous reconciliation (nothing loops; the nightly plan is a report). Two out of four is not GitOps, it is push-based CI/CD with IaC - a perfectly reasonable practice that should be named accurately.

Engineer B is directionally right but overstated. "No in-cluster agent" is a good proxy for the pull principle, and the estate indeed fails it. But the desired state is declarative and is versioned in Git, and that is real, bankable progress - the migration is mainly a delivery-mechanism change, not a rewrite. Saying "nothing here is GitOps" discards two principles already satisfied and misjudges the effort.

What the migration actually requires: install an agent in each cluster that pulls from Git and reconciles on a loop; strip cluster credentials from Jenkins; keep Jenkins (or move to Actions) for build/test/image publishing only. The scorecard should list the four principles, the evidence per principle, and the owner of each gap.

2. What does the nightly `terraform plan` provide and not provide? How would you move it closer to GitOps, and what gap remains for resources Terraform manages?

Read the worked answer

What it provides: periodic detection that live cloud state differs from the state file plus configuration, at 24-hour granularity, with a human-readable diff. That is genuinely useful - it is how many teams first discover console changes. It also gives a written record that a check ran.

What it does not provide:

  • Convergence. No actor applies the fix; a human reads Slack and decides. Detection without correction is monitoring.
  • Continuity. Anything changed at 00:05 is invisible for a day; drift that is reverted before the next run is never seen.
  • Coverage. Resources created outside Terraform (console, other tooling) are not in state and may not appear; ignored attributes are invisible by design.
  • Credibility of "no changes". A green Slack message trains people to ignore the channel; when the plan does show a diff there is no defined response path.

Moving closer: run plan/apply from a controller that reconciles continuously with a defined approval policy for destructive diffs - e.g. a Terraform/OpenTofu controller (Flux's tofu-controller, Atlantis with scheduled auto-plan plus gated apply, or Crossplane if you are willing to model resources as Kubernetes objects). Pair it with state locking, policy checks (OPA/conftest) on the plan, and alerting on unmanaged resources.

Residual gap even then: Terraform's model is plan-apply against a state file with real-world ordering and destructive operations; a controller that auto-applies a plan which proposes replacing a database is dangerous. Most mature teams therefore draw an explicit boundary - workloads and cluster-scoped config are continuously reconciled from Git; cloud data stores keep plan-review-apply with drift alerting and a documented exception. State that boundary in the audit rather than pretending it does not exist.

3. Why does the pull model reduce blast radius here, and does Slack-based approval (ChatOps) change the GitOps classification?

Read the worked answer

Blast radius. Today one credential set on the Jenkins controller is cluster-admin on 9 clusters: compromise Jenkins (a plugin, a malicious PR that reaches the credential, a compromised dependency) and the attacker can do anything in any cluster, and there is no per-team boundary. The pipeline also deploys whatever it is told, from any branch, to any cluster, because nothing constrains destination.

In the pull model the credentials invert: each cluster holds a read-only credential to the Git repo (deploy key or app credentials) and a scoped identity for its own namespace set. A CI compromise yields at most poisoned images in the registry - serious, but detected by image signing and admission policy rather than total cluster control. Compromising one cluster does not yield the others. Combine with per-team AppProjects/RBAC so a team cannot name another team's namespace or repo as a destination.

Interim fixes if pull takes months: one credential per cluster per team, short-lived OIDC-federated tokens instead of static kubeconfigs, namespaced roles instead of cluster-admin, and an audit log shipped off the controller. These reduce risk; they do not make the model pull-based.

ChatOps. Approval in Slack is a human gate on a workflow - orthogonal to GitOps. If the merge to Git remains the trigger and Slack merely notifies or gates a merge, you are still consistent with GitOps. If a Slack button triggers a push-based deploy that bypasses the merge, it is a push pipeline with a chat front-end. The test is always: what does the agent in the cluster observe, and what caused the change? If the answer is "a human click, then Jenkins applied it," it is not GitOps regardless of where the click happened.