Networking for AI Fabrics
Networking is the domain where the exam tests how you design, tune, and troubleshoot the fabric that carries distributed AI traffic. Expect Ethernet and InfiniBand terminology, RoCEv2 behavior, congestion control, topology, and NCCL/GPUDirect paths.
Core rules
- AI training is often more sensitive to tail latency and packet loss than to raw bisection bandwidth. A single slow link can stall a collective.
- Use rail-optimized, non-blocking leaf-spine designs for large GPU clusters. Each GPU/NIC maps to a rail; same-rail hosts share a leaf, and spine uplinks provide cross-rail paths.
- On Ethernet, RoCEv2 needs lossless or near-lossless treatment: PFC for hop-level backpressure, ECN marking, and DCQCN or a vendor equivalent for end-to-end congestion control. PFC alone is not a congestion-control strategy.
- Match MTU end to end; jumbo frames reduce per-packet overhead but do not fix congestion. Check FEC and optics compatibility per link.
- GPUDirect RDMA lets NICs move data directly to and from GPU memory; NCCL chooses rings or trees and the transport. Fabric errors show up as slow collectives, not always as link-down alarms.
Worked example
A leaf has 32 x 400G ports. Four 8-GPU servers each present 8 x 400G NICs, so each server needs 3.2Tb/s and the leaf can attach four servers at 12.8Tb/s. If the leaf has only 16 x 400G uplinks to spines, uplink capacity is 6.4Tb/s against 12.8Tb/s downlink, a 2:1 oversubscription. For all-to-all training, that can bottleneck; for storage or inference traffic it may be acceptable.
Traps
Assuming TCP congestion control is sufficient for RoCE; ignoring PFC storms; mixing FEC modes; designing 3:1 oversubscription for synchronous training; and measuring only link up/down instead of ECN/PFC counters and NCCL bandwidth.