Segmentation for a Shared Kubernetes Platform
A retail company runs a shared Amazon EKS cluster in one VPC with 12 namespaces owned by six product teams. The VPC CIDR is 10.20.0.0/16. The node security group allows 0.0.0.0/0 on TCP 443 and allows SSH from the corporate range. Pods can reach each other across namespaces and reach the internet directly through a NAT gateway. A PCI assessment requires the cardholder-data namespace to be isolated from non-CDE workloads, and all egress to SaaS providers must go through an inspecting proxy for logging and allowlisting. DNS queries must be filtered. The platform team proposes Kubernetes NetworkPolicy as the only control. The security team is concerned that node-level sharing and flat networking may still permit lateral movement. There are 40 microservices; 6 handle card data. The cluster uses the AWS VPC CNI and Calico is available but not enabled for policy enforcement.
1. Design a layered segmentation model for the EKS cluster that isolates the CDE namespace from non-CDE workloads. Address pod-to-pod, node-level, and control-plane paths.
Read the worked answer
NetworkPolicy alone is insufficient on shared nodes. Use layers. First, enable Calico or the VPC CNI network policy engine and apply default-deny ingress and egress in every namespace, then allow only explicit application flows. Label namespaces and use selectors, not IP addresses. Second, isolate the CDE at the node level: dedicated node group, taints and tolerations, separate subnets, and security groups that allow only the required ports from the CDE namespace. Third, use VPC CNI security groups for pods or a service mesh with mTLS so identity is tied to the workload, not the node. Fourth, restrict the control plane: no developer cluster-admin, tightly scoped RBAC, and aws-auth changes through review. If the PCI scope allows, a separate cluster is stronger; if cost prevents it, document the residual risk of shared kernel and kubelet. The key judgement is that pod-level policy does not fix shared-node lateral movement.
2. How would you enforce and log egress to SaaS and DNS filtering without breaking 40 microservices?
Read the worked answer
Discover first. Enable VPC Flow Logs and DNS query logs in monitor mode, then map each microservice's external dependencies. Build a centralized egress VPC with AWS Network Firewall or an explicit proxy, and force traffic through it with route tables. For Kubernetes, use a dedicated egress gateway with a static IP and a NetworkPolicy that permits egress only to the proxy. Allowlist SaaS domains and required IP ranges; use PrivateLink for providers that support it. For DNS, use Route 53 Resolver DNS Firewall rules to block known malicious domains and allow only approved categories.
Roll out in phases: monitor-only, then alert on unapproved destinations, then enforce per namespace. Give teams a self-service allowlist request with an owner and expiry. Log firewall, proxy, and DNS events to the central SIEM. This avoids a big-bang breakage while producing the evidence the PCI assessor needs.
3. What validation would convince an auditor that segmentation is effective?
Read the worked answer
Show both configuration and behavior. Configuration evidence: NetworkPolicy YAML, namespace labels, node taints, security group rules, RBAC bindings, and the egress route tables. Behavior evidence: connectivity tests from a CDE pod to a non-CDE pod that must fail, from non-CDE to CDE that must fail, and approved paths that must succeed. Run a controlled attack simulation from a compromised non-CDE pod and show that it cannot reach the CDE or the metadata service beyond what is required. Include VPC Flow Logs and firewall logs showing denied traffic, plus DNS Firewall logs. Finally, show change control: who can modify policies, how changes are reviewed, and when the tests were last repeated. An auditor will accept a failed connection test more readily than a diagram.