Cluster Mesh and Multi-Cluster Networking
Cluster Mesh joins independent Kubernetes clusters into one Cilium-managed network so pods reach each other across clusters and policy can name remote workloads directly. The exam focuses on the prerequisites that make it fail and on global service semantics.
Architecture and prerequisites
Each cluster runs a clustermesh-apiserver backed by a kvstore, exposing that cluster's state; peers connect to it and replicate endpoints, identities and services. The hard requirements:
- Non-overlapping pod CIDRs and node IPs across all clusters. Overlap cannot be repaired at runtime.
- A unique
cluster.nameandcluster.id(1–255). Duplicates break identity replication. - Node-to-node connectivity between clusters on the mesh ports.
- A consistent routing design. Native routing with BGP is the common production choice because remote tunnel endpoints must be reachable.
Global services
A Service annotated service.cilium.io/global: "true" becomes global: backends from every cluster that has a Service of the same name and namespace are programmed into the local datapath. service.cilium.io/affinity sets preference - local prefers same-cluster backends with failover to remote, remote prefers the other side, none spreads across all. Without the annotation, each cluster balances only over its own endpoints.
Policy across clusters
Identities replicate, so a policy in one cluster can select workloads in another when it is cluster-wide or when it targets the remote namespace using io.cilium.k8s.policy.cluster. Namespaced CiliumNetworkPolicy objects stay namespaced; a CiliumClusterwideNetworkPolicy can span the mesh.
Worked example
Two clusters are connected and a pod in the west cluster cannot reach the api service in the east cluster, though cilium clustermesh status shows the remote as connected. Inspecting the Service in east shows service.cilium.io/global: "true"; the same Service in west lacks the annotation. Global service matching is by name and namespace across clusters, so the service must be annotated on both sides to be treated as one workload set. Add it in both clusters; endpoints from both sides then appear.
Traps
Attempting to mesh clusters with overlapping CIDRs; annotating the service in only one cluster; assuming affinity: local forbids remote backends, when it is a preference with failover; mixing native routing and tunnel mode across clusters and expecting pod IPs to route; and assuming mesh traffic is encrypted by default.