Cilium Certified Associate cheat sheet

Cilium runs as a privileged DaemonSet (cilium-agent) on every node plus a central cilium-operator Deployment; each agent programs eBPF into the kernel and the operator handles cluster-wide tasks like IPAM allocation and garbage collection.

The datapath is eBPF programs attached to TC ingress/egress hooks on network interfaces and to socket-level hooks (sock_ops, sk_msg); iptables is bypassed for pod traffic when kube-proxy replacement is enabled.

The agent exposes a local API on a Unix socket and a health endpoint; cilium-dbg (formerly cilium) talks to the local agent, while cilium CLI with a kubeconfig talks to the Kubernetes API.

Key CRDs: CiliumNetworkPolicy, CiliumClusterwideNetworkPolicy, CiliumEndpoint, CiliumEndpointSlice, CiliumNode, CiliumIdentity, CiliumBGPPeeringPolicy, CiliumLoadBalancerIPPool, CiliumGatewayClassConfig, CiliumLocalRedirectPolicy.

Identity-based security: each unique set of pod labels is assigned a numeric security identity; policy is evaluated against identities, not IP addresses, so it survives scaling and IP churn.

The kvstore (etcd) is optional for identity and IP allocation; when omitted, the operator uses Kubernetes CRDs (CiliumIdentity, CiliumNode) for the same state.

Hubble is the observability layer: hubble-server embedded in the agent streams flow events over gRPC; hubble-relay aggregates across nodes; hubble-ui is a web frontend.

eBPF programs are compiled and loaded at agent start; kernel requirements are typically >= 4.19 (5.4+ recommended for full feature set, 5.10+ for advanced features like BGPv2 or BIG TCP).

CiliumEndpointSlice (CES) is enabled by default in recent versions to reduce the load of per-endpoint CiliumEndpoint objects on the API server and agents.

Primary install path is Helm: helm install cilium cilium/cilium --namespace kube-system --set ...; the official values.yaml documents every option and helm show values cilium/cilium prints it.

cilium install (Cilium CLI) is a wrapper that renders Helm with auto-detected settings; cilium status --wait and cilium connectivity test validate the deployment.

kubeProxyReplacement=true removes kube-proxy; kube-proxy must then be uninstalled or never installed, and the API server host/port must be reachable (KUBERNETES_SERVICE_HOST/PORT or explicit k8sServiceHost/k8sServicePort).

IPAM mode is set with ipam.mode: cluster-pool (default, per-node CIDRs from CiliumNode), kubernetes (host-scope or multi-pool), eni (AWS), azure, alibabacloud, or delegated-plugin (e.g., to a CNI plugin).

Tunneling is selected with routingMode (tunnel/native) and tunnelProtocol (vxlan or geneve); native routing requires the pod CIDR to be routable on the underlying network or BGP to advertise it.

bpf.masquerade=true enables eBPF-based masquerading instead of iptables; it requires kube-proxy replacement or at least BPF NodePort, and it is incompatible with some iptables-based setups.

operator.replicas defaults to 2; hubble.enabled=true, hubble.relay.enabled=true, hubble.ui.enabled=true enable observability components.

cilium upgrade / helm upgrade performs rolling upgrades; cilium-dbg status and cilium-dbg endpoint list verify health, and cilium-dbg bpf ... inspects maps.

Configuration can be supplied via Helm values, the CiliumConfig CRD (when using the Cilium Helm chart's configmap mode), or agent ConfigMap keys under data; precedence is agent flags > ConfigMap > defaults.

Cilium installs a CNI configuration (default /etc/cni/net.d/05-cilium.conflist) and a chained binary; pod sandbox creation calls the CNI plugin, which asks the agent for an IP and programs the endpoint.

cluster-pool IPAM: the operator carves per-node PodCIDRs from ipam.operator.clusterPoolIPv4PodCIDRList and records them in CiliumNode; the agent allocates individual IPs from its own CIDR.

kubernetes IPAM uses the node's spec.podCIDR (host-scope) or CiliumPodIPPool objects (multi-pool) instead of Cilium-managed CIDRs.

Tunnel mode (VXLAN default, port 8472; Geneve port 6081) encapsulates pod-to-pod traffic between nodes; native routing sends packets directly and requires routable pod CIDRs.

Direct routing device (directRoutingDevice) and autoDirectNodeRoutes=true add per-node routes for the remote pod CIDRs when native routing is used.

enableIPv4Masquerade/enableIPv6Masquerade control SNAT for traffic leaving the cluster; egressMasqueradeInterfaces restricts which interfaces masquerade.

Dual-stack requires ipv4.enabled=true and ipv6.enabled=true plus an IPv6 pod CIDR; the cluster must also be dual-stack at the Kubernetes level.

cilium-dbg status --all-addresses and cilium-dbg ip list show allocated IPs; cilium-dbg node list shows per-node health and the tunnel/routing mode in use.

Bandwidth manager (bandwidthManager.enabled) uses eBPF EDT to enforce pod kubernetes.io/ingress/egress-bandwidth annotations without tc qdiscs.

CiliumNetworkPolicy (namespaced) and CiliumClusterwideNetworkPolicy (cluster-scoped) extend Kubernetes NetworkPolicy with L7 rules, DNS-based egress rules, and identity selectors.

endpointSelector picks the pods the policy applies to; ingress/egress rules match fromEndpoints/toEndpoints (label selectors), toFQDNs, toCIDR, toEntities (world, cluster, host, kube-apiserver, remote-node, all).

L7 rules use http, kafka, dns, or l7 fields and require a Layer 7 proxy (Envoy) to be embedded; they are evaluated per-request rather than per-connection.

Policy enforcement modes per endpoint: default (allow all), always (deny by default), never (audit only), disabled; set via policyEnforcementMode or the endpoint's --policy annotation.

enableDefaultDeny in the policy spec controls whether the policy switches the selected endpoints to default-deny for the covered direction; policies are additive (union of allows).

DNS-based egress: toFQDNs with matchName or matchPattern; requires dnsproxy/DNS interception enabled and works with toEndpoints for the DNS server.

Cilium enforces policy in eBPF at the endpoint's TC hooks; L3/L4 policy is enforced without a proxy, only L7 policy traffic is redirected to Envoy.

cilium-dbg policy get shows the resolved policy for an endpoint; cilium-dbg monitor shows verdicts (ALLOWED/DENIED) with the policy that matched.

Encryption: WireGuard (encryption.enabled=true, encryption.type=wireguard) or IPsec (encryption.type=ipsec) protects node-to-node traffic; both can coexist with policy.

cilium status --wait reports agent, operator, Hubble, and kube-proxy replacement health; cilium-dbg status --verbose adds controller and map details on a node.

cilium-dbg monitor streams datapath events; flags like -t drop show only drops, -t trace shows policy trace, --from/--to filter by endpoint or identity.

Hubble flow fields: verdict (FORWARDED/DROPPED/ERROR), source/destination identity and labels, L4 ports, L7 protocol and latency, node name, and the drop reason.

hubble observe --verdict DROPPED --namespace X filters dropped flows; hubble observe --follow tails live; hubble status shows relay connectivity.

Drop reasons are named (e.g., Policy denied, CT: no matching entry, Invalid source identity); cilium-dbg monitor -t drop prints them with the endpoint and direction.

Endpoint state machine: cilium-dbg endpoint list shows ready, waiting-for-identity, not-ready, or disconnected; cilium-dbg endpoint get <id> shows the policy revision and identity.

cilium-dbg bpf ct list and cilium-dbg bpf nat list inspect conntrack and NAT maps; cilium-dbg bpf policy get <id> shows the policy map for an endpoint.

cilium connectivity test deploys a test namespace and exercises pod-to-pod, pod-to-service, DNS, and policy scenarios; it is the fastest end-to-end validation.

Common failure sources: MTU mismatch (tunnel overhead 50 bytes for VXLAN), kube-proxy still running with replacement enabled, missing toEntities: kube-apiserver policy for the API server, and CNI config conflicts.

Cilium Service Mesh uses Envoy embedded in the agent as a per-node proxy; it can be enabled per-pod via annotations (ingress.cilium.io/...) or cluster-wide, avoiding a sidecar per pod.

CiliumEnvoyConfig and CiliumClusterwideEnvoyConfig CRDs inject Envoy listeners and routes; they are the building blocks for custom L7 handling and Ingress/Gateway.

kube-proxy replacement implements ClusterIP, NodePort, LoadBalancer, and ExternalIPs with eBPF; kubeProxyReplacement=true and bpf.masquerade are the key settings.

NodePort with nodePort.mode=dsr (Direct Server Return) or hybrid reduces the return path; nodePort.acceleration=native uses XDP for NodePort handling.

Maglev (loadBalancer.algorithm=maglev) provides consistent hashing for backend selection; default is random.

Cilium Gateway API: install Gateway API CRDs, enable gatewayAPI.enabled=true, create a GatewayClass referencing io.cilium/gateway-controller, then a Gateway and HTTPRoute.

Gateway API supports HTTPRoute, GRPCRoute, TLSRoute, and TCPRoute depending on the Cilium version and installed CRDs; the Gateway listener defines protocol and port.

Ingress support is via ingressController.enabled=true; the IngressClass name is cilium and it uses the same Envoy data plane.

CiliumBGPPeeringPolicy and CiliumLoadBalancerIPPool advertise LoadBalancer IPs via BGP; bpg control plane requires bgpControlPlane.enabled=true (v1.16+).

Cluster Mesh connects clusters by sharing identities and endpoints across a common kvstore or via the clustermesh-apiserver; each cluster keeps its own control plane.

Requirements: unique cluster names, non-overlapping pod CIDRs and node IPs, and connectivity between clustermesh-apiserver (port 2379) and agents on port 4240.

Enable with cluster.name=<name>, cluster.id=<n>, and clustermesh.useAPIServer=true; the clustermesh-apiserver runs in each cluster and is exposed via a Service.

cilium clustermesh connect --context <ctx1> --destination-context <ctx2> wires the clusters; cilium clustermesh status verifies connectivity.

Global services: annotate a Service with service.cilium.io/global: "true" to load-balance across clusters; service.cilium.io/shared: "false" keeps backends local.

service.cilium.io/affinity: local|remote|none controls backend preference; service.cilium.io/affinity.timeout sets how long local preference is kept.

CiliumNetworkPolicy fromEndpoints matches remote pods by labels; remote identities are prefixed with the cluster name in the identity store, and io.cilium.k8s.policy.cluster selects the cluster.

Cross-cluster traffic uses the same eBPF datapath; the tunnel or native routing must be able to reach the remote pod CIDR (typically via the clustermesh-apiserver's advertised routes or a shared network).

cilium-dbg troubleshoot clustermesh and cilium-dbg troubleshoot kvstore diagnose connectivity and identity sync problems between clusters.