Cloud Services for Accelerated AI
Cloud Services covers consuming NVIDIA-accelerated AI in public or hybrid clouds: GPU instance selection, images and containers, orchestration, storage, networking, identity, and cost. The exam tests deployment choices rather than provider-specific console clicks.
Core rules
- Match GPU memory and interconnect to the workload. Single-GPU inference can use a smaller GPU; multi-node training needs high-bandwidth RDMA and a topology that keeps collectives local.
- Use NVIDIA AI Enterprise or NGC containers and Helm charts where available to get validated frameworks, drivers, and security patches. Pin driver, CUDA, and framework versions together.
- In Kubernetes, the GPU Operator manages drivers, the device plugin, container runtime, and monitoring. MIG partitions a GPU for isolation, but it is not a substitute for full-GPU training.
- Separate storage traffic from collective traffic. Object storage is fine for datasets and checkpoints; parallel file systems or local NVMe are better for training hot data.
- Budget for egress, idle GPU time, snapshots, and support. Autoscaling helps inference but can hurt distributed training because workers must start together.
Worked example
A 14GB FP16 model needs about 4GB of KV cache at target concurrency plus 2GB runtime overhead: 20GB. A 24GB GPU fits but leaves little headroom; a 40GB or 80GB GPU supports more concurrent sequences or a larger context. For eight-way tensor parallel training, choose an instance with NVLink/NVSwitch inside the node and RDMA between nodes, not eight isolated PCIe GPUs.
Traps
Assuming the cloud automatically provides lossless GPU fabric; selecting by GPU count rather than memory and interconnect; ignoring egress and storage throughput; and treating MIG slices as interchangeable with full GPUs for large training.