Nutanix NCP-AI cheat sheet

Nutanix AI stack layers: Prism Central management plane, AHV hypervisor with GPU passthrough/vGPU, AOS distributed storage (DSF), Nutanix Kubernetes Platform (NKP), and GPT-in-a-Box reference blueprint on top.

AOS is a distributed storage fabric: every CVM participates in a shared pool; data is striped and replicated (RF2 or RF3) across nodes, so AI datasets and model files survive node loss without external SAN.

Prism Central is the single pane for multi-cluster management; Prism Element manages one cluster. AI inventory (GPU, storage, VMs, NKP clusters) is aggregated in Prism Central.

Nutanix Cloud Infrastructure (NCI) is the current product name for the core HCI platform formerly called AOS/AHV bundle; NCM (Cloud Manager) handles multi-cloud/hybrid extension.

GPT-in-a-Box is a validated reference architecture, not a separate SKU: it combines NCI, NKP, GPU nodes, and a curated MLOps/inference stack (NVIDIA NIM, KServe, MLflow-style tooling).

Nutanix Objects provides S3-compatible object storage on the same cluster, used for datasets, checkpoints, and model artifacts consumed by training jobs.

Nutanix Database Service (NDB) provisions and clones Postgres, MySQL, MongoDB, and SQL Server for AI metadata, feature stores, and vector-adjacent app data.

Flow Network Security (Flow) and Flow Virtual Networking provide microsegmentation and VPC-style isolation for AI tenants without external firewalls.

GPU passthrough: the whole physical GPU is assigned to one VM; lowest overhead and required for many multi-GPU NVLink/NVSwitch training topologies.

NVIDIA vGPU: a physical GPU is time-sliced into vGPU profiles (e.g., 1g.10gb, 2g.20gb on A100/H100-class cards); used for inference density and dev workstations, not for NVLink-scale training.

NVIDIA MIG (Multi-Instance GPU) partitions an A100/H100 into isolated slices with dedicated memory and SMs; each MIG instance can be passed to a separate VM for hard multi-tenant isolation.

AHV supports GPU passthrough and vGPU; NVIDIA drivers and the vGPU manager must match the hypervisor version and the guest driver branch - mismatches are the most common failure.

GPU capacity planning for LLM inference: VRAM must hold model weights plus KV cache; weights ≈ params × bytes-per-param (FP16 ≈ 2 bytes, INT8 ≈ 1, INT4 ≈ 0.5).

Tensor parallelism splits a model across GPUs within a node (NVLink); pipeline parallelism splits across nodes (network-bound). NVLink domains set the practical limit for large-model training.

Nutanix supports NVIDIA H100/A100/L40S and similar data-center GPUs; consumer GPUs are not validated for GPT-in-a-Box.

Storage for training must sustain high sequential read of many small files; AOS read caching and NVMe tiers matter more than raw capacity for dataloader throughput.

GPT-in-a-Box deliverables: validated hardware BOM, NCI + NKP install, GPU enablement, and a curated inference/MLOps stack deployed via Helm charts and manifests.

Deployment sequence: install NCI cluster → enable GPU passthrough/vGPU on hosts → deploy NKP → create GPU-enabled node pools → install the AI stack (NVIDIA GPU Operator, NIM/KServe, model store on Objects/NFS).

NVIDIA GPU Operator automates driver, container runtime, device plugin, and DCGM exporter installation inside NKP - do not hand-install drivers in the guest OS on GPU nodes.

KServe (or the included inference server) provides model serving with autoscaling, canary rollout, and a standard InferenceService CRD; models are pulled from S3 (Objects) or PVCs.

NVIDIA NIM microservices package optimized inference containers for popular models; they are deployed as pods and exposed via KServe or an ingress.

GPT-in-a-Box is sized by model class: small (7B–13B) fits single GPU; 70B needs multi-GPU tensor parallelism; larger needs multi-node.

Model artifacts should live in Nutanix Objects (S3) or a Files share, not baked into container images, so they can be versioned and shared across clusters.

Air-gapped deployments require a local registry mirror and pre-pulled images; plan for it in the BOM because GPT-in-a-Box pulls many large images.

AOS storage containers define replication factor (RF2/RF3), compression, dedupe, and erasure coding; enable dedupe on model/dataset stores where data is repetitive, but not on already-compressed checkpoints.

Nutanix Files provides SMB/NFS shares for shared datasets and home directories; NFS is the common mount for training dataloaders in NKP via CSI.

Nutanix Objects is S3-compatible, bucket-based, and versioned; use it for datasets, model registry artifacts, and checkpoints consumed by KServe/NIM.

Nutanix Objects supports bucket policies, object versioning, lifecycle/expiration rules, and WORM for compliance - relevant to model provenance and audit.

Nutanix Database Service (NDB) provides clone-based provisioning: a thin clone of a Postgres/MySQL/MongoDB instance takes seconds, ideal for per-experiment metadata DBs.

Nutanix Data Lens (or equivalent) provides analytics and ransomware/anomaly detection on file and object data; use it to detect mass-delete or encryption events on dataset shares.

Nutanix CSI driver exposes AOS volumes as Kubernetes PVCs; use ReadWriteMany via Files for shared datasets, ReadWriteOnce for per-pod scratch.

Snapshot and replication policies (Protection Domains, async replication) protect model stores across sites; RPO is set by replication schedule.

NKP (formerly Nutanix Kubernetes Engine / NKE) is Nutanix's CNCF-conformant Kubernetes distribution; it runs on AHV and can also manage clusters on other infrastructure.

NKP management cluster hosts the control plane; workload clusters run GPU node pools. Node pools are the unit of GPU type and count.

GPU scheduling in NKP: NVIDIA device plugin advertises nvidia.com/gpu as a resource; pods request it in resources.limits. Requests without limits are rejected for extended resources.

NVIDIA GPU Operator + Node Feature Discovery labels nodes with GPU model and count; use nodeSelector/affinity to pin workloads to the right GPU class.

NKP supports cluster autoscaling and node pool scaling; GPU nodes are expensive, so scale-to-zero for inference endpoints is a common pattern.

KServe InferenceService handles scale-to-zero (with Knative) and canary traffic splitting; raw Deployments do not.

Persistent storage for pods comes from the Nutanix CSI; StatefulSets for vector DBs (Milvus/pgvector) need stable PVCs and ReadWriteOnce volumes.

NKP integrates with Prism Central for cluster lifecycle (create, upgrade, scale) and with Flow for pod-level network policy.

Prism Central monitors cluster health, capacity, and alerts; GPU utilization and temperature come from DCGM exporter scraped by Prometheus, visualized in Grafana.

Key AI SLOs to monitor: GPU utilization, GPU memory used, inference latency (p50/p95/p99), tokens/sec, queue depth, and error rate per model endpoint.

NCI upgrades are LCM-driven (Life Cycle Manager) for firmware and AOS/AHV; GPU driver and vGPU manager versions must be checked against the NCI compatibility matrix before upgrade.

NKP cluster upgrades are rolling; GPU node pools should be drained and cordoned before host maintenance so training jobs checkpoint and reschedule.

Nutanix Calm (or NKP-native GitOps) automates blueprint deployment of AI stacks; GitOps (Argo CD/Flux) is preferred for reproducible model-serving configs.

Capacity forecasting: GPU is the binding constraint, not CPU/storage; track GPU-hours consumed per team to right-size node pools.

Checkpointing long training jobs to Objects/NFS at intervals lets jobs survive node reboots and LCM maintenance windows.

Alert on ECC errors, XID errors, and thermal throttling on GPUs - these precede silent training corruption.

Prism Central RBAC: roles (e.g., Prism Admin, Prism Viewer, Project Admin) scoped to Projects; Projects isolate VMs, networks, and images per tenant.

Flow Network Security provides microsegmentation with policy applied per VM/category; use categories (e.g., tenant=research) to build policy without IP bookkeeping.

NKP multitenancy: namespaces + RBAC + ResourceQuota + NetworkPolicy + LimitRange isolate teams; GPU quotas via ResourceQuota on nvidia.com/gpu.

Nutanix Objects supports bucket policies, IAM-style access keys, and pre-signed URLs for time-limited model artifact access.

Data-at-rest encryption: self-encrypting drives (SED) with external key management, or software encryption via the cluster; key manager choice affects compliance posture.

Model governance: version every model artifact and dataset in Objects, record lineage, and gate promotion to production through a review step in GitOps.

Prompt/response logging and PII redaction belong at the inference gateway, not in the model; Flow can restrict egress from inference pods to approved endpoints.

Nutanix Data Lens detects anomalous access and mass deletion; pair with WORM buckets for immutable audit and training-data retention.