Nutanix GPT-in-a-Box and AI-Ready Deployment
Nutanix GPT-in-a-Box and AI-Ready Deployment
GPT-in-a-Box is Nutanix's validated, on-premises AI-ready blueprint: NCI/AHV compute, NVIDIA GPUs, NKP Kubernetes, GPU Operator, and model-serving/MLOps tooling. The exam tests deployment prerequisites and the distinction between a blueprint and a product.
Core rules
- Validate hardware against Nutanix and NVIDIA compatibility matrices before deployment.
- Install NCI with Foundation/Prism, then use Prism Central for centralized management.
- Deploy NKP clusters with GPU-enabled node pools; install the NVIDIA GPU Operator to expose
nvidia.com/gpu. - Treat model serving as a workload: namespace, resource requests, storage class, ingress, and observability.
- Keep model artifacts in Objects or Files, not baked into container images.
Worked example
A team wants a self-hosted 70B-parameter model. Estimate GPU memory from weights plus KV cache and overhead; choose multiple high-memory GPUs or quantized weights; provision an NKP GPU node pool sized to that memory; attach an S3-compatible Objects bucket for weights; expose the inference service through a Kubernetes Service/Ingress. GPT-in-a-Box supplies the validated platform pattern, not the model weights or fine-tuning data.
Traps
- Describing GPT-in-a-Box as a SaaS LLM or a specific model.
- Sizing only by parameter count and ignoring KV cache and context length.
- Skipping GPU Operator, then wondering why pods cannot request GPUs.
- Storing large checkpoints on container-local disk.