CI Service Account with cluster-admin and No Default Deny
At 220 namespaces, a platform team uses one CI service account, deploy-bot, bound to cluster-admin because the pipeline creates namespaces and applies manifests. Its token is stored as a long-lived Secret and mounted into every build pod. A penetration test compromised a build pod through a malicious dependency, read the token, and listed Secrets across all namespaces. The cluster has NetworkPolicy in only 12 namespaces, no default-deny, and many apps read credentials from environment variables. Pod Security Admission is not enforced; several builds run privileged to use a legacy image builder. The team wants to keep self-service deploys but reduce blast radius. They can spend two sprints, have no external secrets manager yet, and must not break 40 deploys per day. The CISO wants measurable risk reduction, not a tool purchase. Propose a staged redesign.
1. Redesign `deploy-bot`'s RBAC and token lifecycle to preserve self-service deploys while removing cluster-admin.
Read the worked answer
Split deploy-bot by scope. A namespace-scoped deployer Role in each team namespace should allow create and update on Deployments, Services, ConfigMaps, and only the Secrets it owns. A separate namespace-provisioner account should have create on Namespaces and RBAC objects only, not cluster-admin. Bind these with RoleBindings per namespace, and let a GitOps controller reconcile from a protected repository.
Remove the cluster-admin binding and test with kubectl auth can-i --list in a build. For 40 deploys per day, the controller needs a token that can patch only its managed resources; human break-glass access stays separate. Add admission policy that rejects workloads using cluster-admin tokens.
The key reasoning: cluster-admin was used for two different jobs, namespace creation and workload deployment. Separating those jobs removes most of the blast radius without slowing deploys.
2. What should replace long-lived tokens mounted into build pods, given no external secrets manager yet?
Read the worked answer
Stop mounting long-lived Secret tokens. Use projected service-account tokens with a short expiry and a specific audience, and disable automount where not needed. Without an external secrets manager, a bootstrap controller can request tokens via the TokenRequest API for the pipeline, and each CI job should receive a token scoped to one namespace and one run. Store the CI runner identity in the cloud provider's workload identity if available, so the cluster token is not the primary credential.
Rotate all existing tokens and delete the old Secret. In parallel, move application credentials to encrypted Secrets with KMS and restrict get and list. The reasoning is that a stolen short-lived, audience-bound token has limited use and expires, while a long-lived mounted token is a portable cluster credential. This does not solve secret storage completely, but it reduces the value of the most exposed credential.
3. Prioritize network policy and Pod Security Admission changes over two sprints; what would you do first and what would you defer?
Read the worked answer
First sprint: default-deny in the 12 namespaces that already have policy, then in all namespaces with a monitored dry-run before enforcement; block ingress to payments and from build namespaces. Enforce PSA baseline cluster-wide in audit mode, then warn, then enforce restricted for new namespaces. Exempt the legacy image builder with a named, time-bound exception and a migration ticket.
Second sprint: enforce restricted for high-risk namespaces, add egress controls for build pods, and remove privileged builds. Defer broad egress allowlists until you have flow logs, because a bad deny can break deploys.
The reasoning is to reduce the easiest lateral movement first and use audit and warn modes to avoid breaking 40 daily deploys. Measure success by the number of namespaces with default-deny and the number of privileged pods removed, not by policy objects created.