Accelerated Computing and GPU Operations
Accelerated Computing and GPU Operations
This domain tests GPU architecture and day-two operations: SM and Tensor Core utilization, memory bandwidth, MIG, NVLink topology, persistence mode, clocks, ECC, XID events, and DCGM metrics.
Core rules and concepts
- Match the workload to the bottleneck: Tensor Core math, memory bandwidth, or interconnect. A GPU at 100% utilization can still be memory-bound.
- MIG partitions A100/H100 GPUs into isolated slices with dedicated memory and SMs; it is not a time-slicing quota.
- Use nvidia-smi topo -m to confirm NVLink versus PCIe paths before multi-GPU jobs.
- Enable persistence mode where supported to cut initialization latency; watch clock throttle reasons.
- Treat uncorrectable ECC and XID 79 as node-level failures; drain and replace.
Worked example
An A100 80GB host must serve six isolated inference tenants, each needing no more than 8GB and no cross-tenant sharing. Choose six 1g.10gb MIG slices: each gets one GPC-equivalent compute slice and 10GB, and tenants cannot see each other. The reasoning is that 1g.10gb is the smallest profile that meets 8GB, and six slices fit within the seven-slice maximum. Do not use 2g.20gb, which halves tenant density.
Traps that cost marks
Assuming MIG adds aggregate memory; expecting full GPU memory bandwidth per slice; confusing utilization with efficiency; and ignoring PCIe topology when NVLink is absent.