AI Infrastructure and Data Center Operations
AI Infrastructure and Data Center Operations
This domain tests the physical and operational layer of an NVIDIA AI cluster: rack power, cooling, out-of-band management, GPU health telemetry, and fault escalation. Questions often ask you to choose a rack design or interpret a sensor or XID reading.
Core rules and concepts
- Design to maximum facility power, not typical workload power. A DGX H100 node has a roughly 10.2 kW maximum; four nodes exceed a 40 kW rack before switches and cooling.
- Keep A/B power feeds independent and do not count redundant PSU capacity as usable capacity.
- Use BMC/Redfish for inventory, firmware, and power control; use DCGM for GPU ECC, XID, thermal, power, and utilization telemetry.
- Monitor coolant flow, pressure, and leak sensors on liquid-cooled racks, plus inlet temperature and GPU throttle reasons.
- Treat XID 79 (GPU fallen off the bus) and uncorrectable ECC as page-worthy; correctable ECC and thermal slowdowns are trends to track.
Worked example
Eight DGX H100 nodes at 10.2 kW each total 81.6 kW. A 30 kW air-cooled rack cannot host them. Two nodes per rack gives 20.4 kW, leaving headroom for switches; alternatively use a liquid-cooled rack with a CDU rated above 81.6 kW plus margin. The reasoning is that facility load includes PSU losses and cooling, so sizing to exactly 100% leaves no failover capacity.
Traps that cost marks
Confusing nameplate power with measured power; ignoring PSU redundancy; treating every XID as fatal; and cooling only the GPUs while forgetting leaf switches and storage.