NCS-Core cheat sheet

A Nutanix cluster is a set of nodes running AOS; each node hosts a CVM (Controller VM) that serves all storage I/O for that node's local disks via the distributed storage fabric (DSF).

The hypervisor runs VMs; the CVM is itself a VM with direct PCI passthrough to local SSDs/HDDs and is excluded from normal HA/DRS actions.

Stargate is the DSF data I/O manager on each CVM; Curator handles disk balancing/scrubbing; Cassandra stores cluster metadata; Zookeeper provides distributed coordination and leader election.

Genesis runs on each node and bootstraps/configures the CVM and cluster services; Prism runs the UI/API layer, with Prism Element scoped per cluster and Prism Central spanning clusters.

AHV is Nutanix's native KVM-based hypervisor; ESXi and Hyper-V are also supported, but some features (e.g., certain data protection/DR workflows) differ by hypervisor.

Data locality: VMs read/write from the CVM on their own node when possible; remote reads occur via inter-CVM communication over the internal network.

The metadata ring and extent store split: metadata (Cassandra) tracks vDisk/extent mappings, while Stargate serves the actual extent groups on physical media.

Cluster services require the CVM to have a static IP on the same subnet as the hypervisor host; the CVM uses the hypervisor's management network for external traffic.

Foundation (or Foundation Central/Foundation VM) images nodes and creates the cluster; it requires IPMI/out-of-band access plus a network with DHCP or static IP planning.

Pre-install checklist: unique CVM/host IPs, gateway, DNS/NTP reachable, correct VLAN IDs, IPMI credentials, and matched AOS/hypervisor versions in the imaging bundle.

Cluster creation requires an odd number of nodes for metadata quorum (minimum 3 nodes for a resilient cluster; single-node supported for ROBO with caveats).

The default internal cluster network is 192.168.5.0/24 on eth0/vswitch0 for CVM-to-CVM traffic; it must not overlap external subnets.

After imaging, run NCC (Nutanix Cluster Check) health checks and verify Prism Element shows all nodes 'Up' with no critical alerts before adding workloads.

Cluster configuration includes timezone/NTP, DNS servers, SMTP alert settings, and syslog forwarding - set these before production cutover.

Nutanix recommends the CVM and hypervisor management on the same VLAN; separating them adds routing complexity and can break features relying on host-CVM adjacency.

AOS and hypervisor upgrade bundles must be staged; Foundation only images, while LCM or Prism handles subsequent AOS/firmware upgrades.

AHV uses Open vSwitch (OVS) with a default bridge br0; VM NICs attach to br0 or user-created bridges; VLAN tagging is configured per-network in Prism.

The backplane/internal network carries CVM replication and storage traffic; a separate 10/25GbE link is recommended for storage/replication versus management.

Storage containers define replication factor (RF2 or RF3), compression, deduplication, and erasure coding settings; RF2 tolerates one node failure, RF3 two.

Nutanix Volumes exposes block storage (vDisk) to external hosts via iSCSI; Nutanix Files provides SMB/NFS shares via File Server VMs.

Protection domains group VMs for replication; async replication uses snapshots shipped to a remote cluster, sync replication requires low-latency links and RF constraints.

Snapshots are crash-consistent by default; use VSS/application-consistent agents for databases requiring quiescing.

Erasure coding (EC-X) reduces capacity overhead but requires RF2/RF3 and sufficient nodes; it is not recommended for latency-sensitive or small-cluster workloads.

Nutanix DR (Leap) uses protection policies with RPO/RTO targets and supports recovery plans for orchestrated failover.

Prism Element manages a single cluster; Prism Central aggregates multiple clusters and enables categories, policies, and multi-cluster LCM.

Prism Central requires a scale-out deployment (small/large) sized by VM count; it must be registered to clusters before centralized features work.

LCM (Lifecycle Manager) inventories firmware, AOS, hypervisor, and Prism Central versions, then stages and applies upgrades with pre-checks.

Categories are key-value tags used by policies (e.g., protection, placement); they replace per-VM manual configuration at scale.

Prism APIs (v2/v3) are RESTful; v3 uses POST with JSON bodies and task UUIDs for async operations, polled via /PrismGateway/services/rest/v3/... endpoints.

Reports and analysis in Prism Central provide capacity, performance, and compliance dashboards; alerts route via SMTP/syslog/SNMP.

VM operations (clone, migrate, snapshot, power) are available through Prism UI, CLI (acli/ncli), and REST; acli targets Prism, ncli targets cluster services.

Role-based access in Prism uses local users, AD/LDAP integration, and roles (e.g., Prism Admin, Viewer, VM Admin) scoped to clusters or projects.

NCC (Nutanix Cluster Check) runs health checks; schedule it and review failures before opening support cases.

Key logs: /home/nutanix/data/logs on the CVM for Stargate/Cassandra; /var/log for hypervisor; Prism alerts show severity and affected entity.

Common checks: ncli cluster info, ncli host list, allssh 'cluster status', and ncc health_checks run_all from the CVM.

Stargate/Cassandra issues often present as slow I/O or metadata errors; check CVM disk health with ncli disk list and SMART data.

CVM down or unreachable: verify IPMI console, CVM network, and that the CVM VM is powered on; a CVM reboot does not lose data.

Collect logs via Prism (Support > Log Collector) or ncc log_collect; include the case number and a recent timeline when escalating.

Escalation path: Prism support portal → create case with severity (S1-S4) → attach NCC output and logs → engage account team for S1 outages.

Alert policies in Prism Central can suppress noise; never suppress critical alerts without documenting the reason.

Upgrade order: firmware → AOS → hypervisor (or per LCM plan); check compatibility matrix for AOS/hypervisor/Prism Central versions.

AOS upgrades are rolling: one CVM at a time; the cluster remains available but performance may degrade during rebalancing.

Hypervisor upgrades use LCM/one-click or manual; AHV upgrades require the cluster to be healthy and NCC clean.

Adding nodes: image the new node with Foundation, then expand the cluster via Prism; match AOS/hypervisor versions to the existing cluster.

Node removal requires data evacuation first (remove from cluster gracefully) so DSF re-replicates data; never pull disks from a live node.

Before maintenance, verify RF and free capacity; with RF2, losing a node during maintenance leaves no redundancy.

Prism Central upgrades are separate from AOS and must be done before or after per compatibility; PC scale-out adds CVMs for capacity.

Post-upgrade: rerun NCC, confirm all services up, and validate that LCM shows no pending critical updates.

Prism authentication supports local users, AD/LDAP, and SAML/SSO; enforce MFA where available and rotate local admin credentials.

Role-based access control: assign least-privilege roles (Viewer, VM Admin, Prism Admin) scoped to clusters/projects; avoid shared admin accounts.

Cluster lockdown mode restricts CVM/host access; enable only after validating management access paths.

Data-at-rest encryption (SEDs or software encryption) requires key management; validate compliance needs (e.g., FIPS) against supported configurations.

Audit logs in Prism record user actions; forward to SIEM via syslog for retention and compliance evidence.

Network security: segment management, storage, and VM traffic; use VLANs and firewall rules between Prism Central and clusters.

CIS/STIG hardening guidance exists for AHV/AOS; apply host hardening and disable unused services.

Certificate management: replace default Prism/SSL certificates and track expiration to avoid outages.

Prism REST API v3 base path: https:///PrismGateway/services/rest/v3/; authenticate via basic auth or API key, then use task UUIDs to poll async operations.

v2 API uses GET/POST with JSON; v3 is preferred for new automation; both are documented via the API explorer in Prism.

Python SDK (nutanix) and PowerShell cmdlets (e.g., Nutanix Prism) wrap the REST API for scripting VM lifecycle and reporting.

Calm (Prism Central) provides blueprints and runbooks for self-service provisioning; blueprints define multi-tier app deployment with variables.

Categories + policies enable automated placement, protection, and reporting without per-VM scripting.

Nutanix CLI: acli (Prism/VM ops), ncli (cluster services), and psql/nosql for diagnostics - use in scripts with error handling.

Idempotency: check for existing resources before create calls; use unique names and tags to avoid duplicates in automation.

Webhooks and event-driven automation can trigger scripts on alerts or lifecycle events; validate payloads and secure endpoints.