Nutanix NCP-CN cheat sheet

NKP separates a management plane (NKP management cluster) from workload clusters; the management cluster hosts the lifecycle and addon controllers and is created first, then used to provision and upgrade workload clusters.

A workload cluster's control plane runs on the bootstrapped Kubernetes distribution (kubeadm-based) with stacked etcd on control-plane nodes by default; external etcd topology is optional and requires an odd number of etcd members for quorum.

Cluster creation is declarative: an inventory/configuration object (cluster definition with node pools, network, and addon settings) is applied to the management cluster, which reconciles the desired state onto the target infrastructure provider.

Node pools group nodes with identical role and resource profile; scaling a node pool adds or removes worker nodes without recreating the control plane, and control-plane node counts should stay odd for etcd quorum.

Supported infrastructure providers include Nutanix AHV (via the Nutanix CAPI/Prism Central integration), plus other providers such as bare metal and public clouds; the provider is chosen in the cluster definition, not at install time of the management plane.

Upgrades are performed by upgrading the management plane first, then workload clusters; Kubernetes version upgrades roll nodes in a controlled sequence honoring pod disruption budgets and node drain.

The NKP CLI/API exposes lifecycle operations (create, scale, upgrade, delete, kubeconfig retrieval); kubeconfig for a workload cluster is fetched from the management cluster rather than generated locally.

Addons (CNI, CSI, load balancer, observability, policy) are installed and versioned by the management plane per cluster; addon versions are tied to the NKP release, so upgrading NKP can change addon versions.

Cluster deletion removes provider-side resources (VMs, disks) associated with the cluster definition; orphaned resources indicate provider credentials or finalizer issues during teardown.

The CNI plugin provides pod IPAM and pod-to-pod routing; NKP clusters commonly use Calico or Cilium, and the CNI choice determines whether NetworkPolicy, BGP peering, or eBPF-based load balancing features are available.

Service type LoadBalancer requires a load balancer implementation; on Nutanix AHV the common choice is a MetalLB-style or Nutanix-integrated LB that allocates addresses from a configured pool in the cluster's VLAN/subnet.

Ingress controllers (for example NGINX or a gateway implementation) terminate L7 traffic and route by host/path; the ingress Service is exposed as type LoadBalancer or NodePort depending on the environment.

Pod and Service CIDRs must not overlap with the underlying AHV subnet ranges or with peered networks; overlapping CIDRs break routing and are a common cause of unreachable services after cluster creation.

Kubernetes DNS (CoreDNS) resolves Service names to ClusterIP; headless Services return pod IPs directly, which matters for StatefulSet peer discovery and database clusters.

NetworkPolicy enforcement depends on the CNI; if the CNI does not enforce policy, applying NetworkPolicy objects has no effect on traffic.

For hybrid connectivity, VPN or direct interconnect must carry pod and service CIDRs if cross-cluster pod communication is required; otherwise expose services through ingress or LoadBalancer VIPs only.

Node-to-node traffic on AHV uses the cluster's virtual network; MTU mismatches between overlay encapsulation and the physical network cause intermittent large-packet failures (for example TLS handshakes succeeding but bulk transfers stalling).

The Nutanix CSI driver runs as a controller Deployment plus node DaemonSet and provisions Nutanix Volumes or Nutanix Files-backed PVs through StorageClasses that reference the Prism Central/Prism Element endpoint and a secret.

StorageClass parameters select the storage backend and container (for example a Volumes storage container) and the filesystem type; the reclaimPolicy determines whether the backing Nutanix disk is deleted when the PVC is deleted.

Access modes map to backend capability: ReadWriteOnce volumes are block devices attached to a single node, while ReadWriteMany requires a shared filesystem such as Nutanix Files.

Volume expansion requires allowVolumeExpansion: true on the StorageClass; the CSI driver then grows the Nutanix volume and the filesystem online where supported.

Snapshots use VolumeSnapshotClass and a CSI snapshot controller; restoring creates a new PVC from the snapshot, which is the basis for application-consistent data protection workflows.

StatefulSets use volumeClaimTemplates so each replica gets its own PVC; deleting the StatefulSet does not delete PVCs unless explicitly removed, so orphaned volumes must be reclaimed deliberately.

Nutanix Data Services for Kubernetes (NDK) adds application-aware protection for Kubernetes workloads, including consistent snapshots and clones of namespaces or application groups across clusters.

CSI topology and node affinity ensure volumes are created in a storage container reachable from the scheduled node; a pod stuck in ContainerCreating with attach errors usually indicates node-to-storage connectivity or iSCSI session issues.

The CSI driver requires iSCSI (or NFS for Files) connectivity from worker nodes to the Nutanix storage network; CHAP credentials are configured in the CSI secret.

NDB provisions and manages database engines (for example PostgreSQL, MySQL, SQL Server, Oracle, MongoDB) on Nutanix infrastructure, with clone, snapshot, patching, and lifecycle operations driven by profiles rather than manual scripts.

Software profiles define the database engine version and binary; compute profiles define vCPU/RAM; network profiles define VLAN/IP allocation; database parameter profiles set engine configuration.

Time Machine policies define snapshot and log-capture schedules plus retention; they are the basis for point-in-time recovery and for clone refreshes used in dev/test.

NDB clones are space-efficient copies created from a snapshot or Time Machine; they are the standard mechanism for provisioning test/dev databases without copying full data.

NDB can register and manage existing databases (for example a PostgreSQL instance running in Kubernetes) to bring them under snapshot, clone, and patching workflows.

Era/version upgrades and OS patching are performed as NDB operations with pre-checks; failed pre-checks block the operation and must be resolved before retrying.

NDB integrates with Nutanix AHV and can target registered clusters; the NDB server itself runs as an appliance VM and requires network access to Prism and to the database VLANs.

Database access control in NDB is role-based (admin, operator, user roles) with entity-level permissions; audit logs record operations for compliance.

NC2 extends the Nutanix AHV and Prism stack into public cloud (AWS and Azure) as a managed cluster, giving the same Prism Central, AHV, and NKP experience on cloud infrastructure.

An NC2 cluster runs on cloud bare-metal or supported instance types with dedicated networking (VPC/VNet); connectivity back to on-premises uses VPN or dedicated interconnect.

NC2 supports Prism Central registration across clusters, enabling a single pane for multiple on-prem and cloud clusters; however, network latency and CIDR planning must be validated before peering.

Workload mobility between on-prem and NC2 uses Nutanix Move or replication-based migration; the target cluster must have compatible AHV version and sufficient capacity.

NC2 clusters are managed through the NC2 console for lifecycle (create, scale, suspend/resume, delete); suspending a cluster reduces cost but retains storage.

Licensing on NC2 is consumption-based for the cloud infrastructure plus Nutanix licensing; cost planning must include cloud egress and interconnect charges.

Multicloud operations require consistent identity (directory integration), consistent policy (NKP policy addons), and consistent observability endpoints across clusters.

CIDR overlap between on-prem and NC2 VPC/VNet is the most common blocker for hybrid connectivity; plan non-overlapping pod, service, and node networks before deployment.

Observability for NKP clusters combines Kubernetes metrics/logs (metrics-server, Prometheus-compatible scraping, log forwarding) with Nutanix Prism monitoring for the underlying AHV VMs and storage.

Security baseline: enable RBAC least privilege, admission policy (Pod Security Standards or policy engine), image scanning, secrets encryption at rest, and network policy enforcement via the CNI.

Nutanix Flow provides microsegmentation for AHV VMs, including Kubernetes worker and control-plane VMs; policy is applied at the VM NIC level and is independent of Kubernetes NetworkPolicy.

Backup of Kubernetes workloads uses two layers: etcd/cluster state backup and application data backup (PVC snapshots via CSI or NDK). A PVC snapshot alone does not capture Kubernetes objects.

Velero-style backup captures Kubernetes API objects and optionally PV data; restoring requires matching StorageClasses and CRDs in the target cluster.

DR for cloud-native workloads uses replication of the underlying Nutanix storage (protection domains / async replication) plus re-registration or restore of Kubernetes clusters at the DR site.

Recovery objectives: RPO is set by replication/snapshot frequency, RTO by restore or failover automation; both must be tested with a real restore, not assumed from configuration.

Audit and compliance: Prism Central audit logs, Kubernetes audit logs, and NDB audit logs are separate sources and must be aggregated for a complete picture.

Secrets management: Kubernetes Secrets are base64-encoded by default; enable encryption at rest via encryption providers and integrate an external secrets store where required.