AI Workload Infrastructure and GPU Virtualization
AI Workload Infrastructure and GPU Virtualization
This domain tests GPU consumption models on AHV and how to match training versus inference requirements to host and VM design.
GPU modes in AHV
- Passthrough: the entire physical GPU is assigned to one VM; simple and often fastest, but no sharing and limited mobility.
- NVIDIA vGPU: the physical GPU is partitioned into virtual GPUs; profiles define memory and compute slices; useful for inference density and tenant isolation. Requires a compatible GPU, host driver, vGPU software, and guest driver.
- MIG: hardware partitioning on supported GPUs provides isolated GPU instances; verify AHV and guest support first.
Size by GPU memory first, then compute, then CPU/RAM. Training usually needs large GPU memory, high-bandwidth interconnect, and few large VMs. Inference favors many right-sized profiles and latency control. Do not oversubscribe GPU memory; CPU overcommit does not create GPU capacity.
Worked example
An inference service needs 8 GB GPU memory per replica and six replicas on one 48 GB GPU. A 1x8 GB vGPU profile gives six isolated replicas totaling 48 GB; passthrough would waste the device on one VM. Verify that profile memory times count fits physical GPU memory and that licensing covers each vGPU.
Traps
- Assuming vGPU always supports live migration; check profile and platform support.
- Forgetting NVIDIA vGPU licensing and guest driver compatibility.
- Assigning GPUs without host BIOS/IOMMU preparation.
- Treating GPU memory as overcommittable like vCPU.