I85PREP
⌘ K
NVIDIA NCP-AIN study materials
NVIDIA NCP-AIN study resources

NVIDIA NCP-AIN practice questions

One question at a time. Build your NVIDIA NCP-AIN understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Networking

In an NVIDIA AI cluster using Spectrum-X, what primary role does RoCEv2 play in the network fabric?

  1. It replaces PCIe for intra-node GPU-to-GPU communication
  2. It encrypts GPU memory transfers between hosts using IPsec
  3. It carries RDMA traffic over routable UDP/IP packets between GPU nodes Correct answer.
  4. It provides lossless Fibre Channel transport for storage arrays

Explanation

RoCEv2 encapsulates RDMA transport over UDP/IP, which makes RDMA routable across a Layer 3 fabric and is the basis for Spectrum-X Ethernet AI networking. Intra-node GPU communication uses NVLink, not the network fabric.

Question 2Networking

Which technology provides direct high-bandwidth GPU-to-GPU communication inside a single server node?

  1. InfiniBand HDR
  2. NVLink Correct answer.
  3. RoCEv2
  4. PCIe Gen3 x4

Explanation

NVLink is NVIDIA's high-speed interconnect for GPU-to-GPU communication within a node, often paired with NVSwitch. Network transports such as RoCEv2 and InfiniBand connect nodes rather than GPUs inside one chassis.

Question 3Networking

What is the purpose of Priority Flow Control (PFC) in a lossless Ethernet fabric for AI workloads?

  1. It compresses packets to reduce bandwidth consumption
  2. It prioritizes CPU control traffic over GPU data traffic at Layer 2
  3. It load-balances flows across multiple uplinks by hashing
  4. It pauses specific traffic classes to prevent buffer overflow and packet drops Correct answer.

Explanation

PFC generates per-priority pause frames so a congested switch can temporarily halt a specific traffic class instead of dropping frames, which is essential for lossless RDMA. Load balancing and compression are separate functions.

Question 4Networking

In an InfiniBand fabric, which component is responsible for routing packets between subnets?

  1. The router Correct answer.
  2. The switch
  3. The channel adapter
  4. The subnet manager

Explanation

InfiniBand routers forward packets between separate subnets, while the subnet manager configures and manages a single subnet. Channel adapters are endpoints and switches forward within a subnet.

Question 5Networking

Which NVIDIA networking technology combines Ethernet with adaptive routing and congestion control to deliver deterministic performance for AI workloads?

  1. NVLink-C2C
  2. BlueField DPU
  3. Spectrum-X Correct answer.
  4. GPUDirect Storage

Explanation

Spectrum-X is NVIDIA's Ethernet networking platform that adds adaptive routing and congestion control for AI performance. NVLink-C2C is a chip-to-chip interconnect, GPUDirect Storage is a data path, and BlueField is a DPU.

Question 6Networking

What does the term 'rail-optimized' topology describe in a GPU cluster network design?

  1. The management network is duplicated for redundancy
  2. All GPUs connect to a single top-of-rack switch for simplicity
  3. Each GPU connects to a dedicated leaf switch so same-index GPUs share a rail Correct answer.
  4. Storage traffic is isolated on a separate physical rail

Explanation

In a rail-optimized design, corresponding GPUs across nodes (for example GPU 0 of every server) connect to the same leaf switch, forming a rail. This keeps collective communication traffic on a single hop for many patterns.

Question 7Networking

Which protocol does GPUDirect RDMA rely on to move data directly between GPU memory and a network adapter?

  1. Peer-to-peer memory access over PCIe Correct answer.
  2. NVLink fabric routing
  3. TCP segmentation offload
  4. RDMA over Converged Ethernet encapsulation

Explanation

GPUDirect RDMA uses peer-to-peer memory access over PCIe so the NIC can read or write GPU memory directly, bypassing host memory copies. The other options describe unrelated offload or transport functions.

Question 8Networking

What is the function of ECMP in a leaf-spine network carrying AI training traffic?

  1. It guarantees in-order delivery of every packet
  2. It distributes flows across multiple equal-cost paths to use available bandwidth Correct answer.
  3. It assigns priorities to different traffic classes
  4. It encrypts traffic between leaf and spine switches

Explanation

Equal-Cost Multi-Path spreads flows across multiple equally costed paths, increasing fabric utilization. It does not guarantee in-order delivery, set priorities, or encrypt traffic.

Question 9Networking

A cluster shows elevated RDMA retransmissions on a RoCEv2 fabric during large collective operations. Which configuration issue should be investigated first?

  1. Incorrect NTP server assignment on the management network
  2. Excessive NVLink bandwidth within each server
  3. Mismatched PFC priority configuration between hosts and switches Correct answer.
  4. Overprovisioned storage RAID parity

Explanation

Lossless RoCEv2 depends on consistent PFC priority mapping across hosts and switches; a mismatch lets RDMA traffic be dropped under congestion, causing retransmissions. The other items do not cause RDMA packet loss on the fabric.

Question 10Networking

When designing a leaf-spine fabric for GPU training, why is a non-blocking or low-oversubscription ratio preferred?

  1. Collective operations create bursty all-to-all traffic that saturates uplinks Correct answer.
  2. It reduces the number of required switch ports
  3. It allows the use of unmanaged switches
  4. It eliminates the need for a subnet manager

Explanation

Collective operations such as all-reduce generate synchronized bursts that can saturate oversubscribed uplinks and stall training. A non-blocking fabric preserves full bisection bandwidth under these patterns.

Question 11Networking

What does adaptive routing do in a Spectrum-X fabric under congestion?

  1. It dynamically selects less-congested paths per packet to reduce latency Correct answer.
  2. It increases the MTU on congested ports
  3. It converts RoCEv2 traffic to TCP
  4. It permanently disables congested links

Explanation

Adaptive routing lets switches choose among multiple viable paths based on current congestion, spreading load and reducing tail latency. It does not disable links, change MTU, or alter the transport protocol.

Question 12Networking

Which congestion control mechanism does NVIDIA recommend enabling to complement PFC and reduce congestion spreading in RoCEv2 fabrics?

  1. Explicit Congestion Notification with DCQCN Correct answer.
  2. Address Resolution Protocol
  3. Spanning Tree Protocol
  4. Link Aggregation Control Protocol

Explanation

ECN marks packets before buffers overflow, and DCQCN uses those marks to rate-limit senders, reducing congestion spreading. STP, LACP, and ARP are Layer 2 mechanisms unrelated to congestion control.

Question 13Networking

Which NVIDIA NIC feature allows a single physical port to present multiple PCIe functions to the host for multi-tenant isolation?

  1. VLAN trunking
  2. SR-IOV virtual functions Correct answer.
  3. Jumbo frames
  4. Flow control

Explanation

SR-IOV exposes multiple virtual functions from one physical device, letting tenants or VMs get isolated hardware paths. Jumbo frames, VLAN trunking, and flow control are traffic handling features, not device partitioning.

Question 14Networking

In an InfiniBand fabric, what does the subnet manager compute and distribute to switches?

  1. MAC address tables for VLANs
  2. DNS records for hostnames
  3. IP routing tables for OSPF
  4. Linear forwarding tables that map destinations to output ports Correct answer.

Explanation

The subnet manager discovers the topology, assigns LIDs, and computes linear forwarding tables pushed to switches so they can forward within the subnet. IP routing, MAC tables, and DNS are Ethernet/IP concepts.

Question 15Networking

A team wants collective communication libraries to use RDMA over the GPU cluster fabric. Which combination must be correctly configured?

  1. Docker with the correct image tag
  2. NCCL with the correct network interface and RoCE or InfiniBand HCA selection Correct answer.
  3. CUDA cores with the correct clock frequency
  4. NFS with the correct mount options

Explanation

NCCL selects the network transport and interface for collectives; it must be pointed at the correct RoCE interface or InfiniBand HCA to use RDMA. Clock frequency, container tags, and NFS mounts do not determine the fabric transport.

Question 16Networking

What is the benefit of using jumbo frames on a storage or data network in an AI cluster?

  1. It enables lossless RDMA automatically
  2. It eliminates the need for flow control
  3. It reduces per-packet header overhead and CPU processing for large transfers Correct answer.
  4. It increases the number of available IP addresses

Explanation

Larger frames carry more payload per header, lowering overhead and per-packet processing for bulk transfers. Jumbo frames do not expand address space, enable lossless behavior by themselves, or remove the need for flow control.

Question 17Networking

Which statement best describes the role of a BlueField DPU in an AI cluster network?

  1. It acts as a top-of-rack switch
  2. It provides the subnet manager for InfiniBand
  3. It offloads networking, storage, and security processing from the host CPU Correct answer.
  4. It replaces the GPU for model training

Explanation

A BlueField DPU offloads infrastructure functions such as networking, storage, and security from the host CPU, freeing compute for AI work. It does not train models, switch fabric traffic, or manage InfiniBand subnets.

Question 18Networking

What does a non-blocking fabric with a 1:1 oversubscription ratio guarantee?

  1. All flows use the same path
  2. All packets arrive in the order they were sent
  3. No switch will ever drop a packet
  4. Full bisection bandwidth is available when all ports transmit at line rate Correct answer.

Explanation

A 1:1 ratio means uplink capacity matches downlink capacity, so full bisection bandwidth is available under worst-case traffic. It does not guarantee ordering, zero drops, or single-path routing.

Question 19Networking

Why is consistent MTU configuration across all fabric devices important for RoCEv2 traffic?

  1. It changes the RDMA queue pair numbering
  2. Fragmentation or drops occur when a larger frame reaches a device with a smaller MTU Correct answer.
  3. It enables PFC automatically
  4. It sets the InfiniBand LID range

Explanation

If a frame exceeds a downstream device's MTU, it may be fragmented or dropped, harming RDMA performance. MTU settings do not affect queue pair numbering, enable PFC, or assign LIDs.

Question 20Networking

Which tool or command is commonly used on a Linux host to verify that the RDMA stack and RoCE devices are operational?

  1. tcpdump
  2. ip route
  3. nvidia-smi
  4. ibv_devinfo Correct answer.

Explanation

ibv_devinfo queries RDMA devices and reports port state and capabilities for RoCE and InfiniBand HCAs. ip route shows IP routes, nvidia-smi reports GPU status, and tcpdump captures packets.

Question 21Networking

In a rail-optimized cluster, how does the design typically reduce latency for all-to-all collective operations?

  1. Each GPU uses a unique path through the management switch
  2. Same-index GPUs communicate over a single leaf hop on their shared rail Correct answer.
  3. All GPUs share one spine switch for every exchange
  4. Traffic is routed through the storage network first

Explanation

When corresponding GPUs across nodes attach to the same leaf, many collective exchanges stay within one hop on that rail, reducing latency and spine load. Sharing a single spine or detouring through storage/management networks would add hops and contention.

Question 22Networking

An engineer observes that RDMA performance degrades only when many flows hash to the same spine uplink. Which mechanism most directly addresses this flow-level imbalance?

  1. Disabling ECN marking
  2. Flowlet-based or adaptive load balancing across equal-cost paths Correct answer.
  3. Reducing the MTU on all ports
  4. Increasing PFC pause quanta

Explanation

Static ECMP hashing can pin multiple large flows to one path; flowlet or adaptive balancing redistributes them based on path congestion. Adjusting PFC, disabling ECN, or lowering MTU does not solve hash-based imbalance.

Question 23Networking

Why can PFC deadlock occur in a multi-hop lossless fabric, and what mitigates it?

  1. RDMA queue pairs expire; restarting the subnet manager mitigates it
  2. Cyclic pause dependencies can form; watchdog timers and congestion control mitigate it Correct answer.
  3. PFC frames are always dropped; increasing buffer size mitigates it
  4. ECN marks are lost; disabling adaptive routing mitigates it

Explanation

In a looped or multi-hop lossless fabric, pause frames can create cyclic dependencies where all involved ports stall, a PFC deadlock. PFC watchdog timers detect stuck pauses and congestion control reduces pressure to mitigate it.

Question 24Networking

A cluster uses GPUDirect RDMA with RoCEv2, and throughput is far below line rate despite no packet drops. Which factor is most likely limiting performance?

  1. The subnet manager is assigning duplicate LIDs
  2. The management VLAN is misconfigured
  3. PCIe peer-to-peer access is not enabled or the NIC is not in the GPU's PCIe root complex path Correct answer.
  4. The NTP offset exceeds one second

Explanation

GPUDirect RDMA requires peer-to-peer PCIe access between the NIC and GPU; if it is unsupported or the devices sit behind different root complexes, traffic falls back to host-memory copies and throughput drops. LIDs, management VLANs, and NTP do not limit RDMA throughput.

Question 25Networking

When comparing InfiniBand and Ethernet for an AI fabric, which statement reflects a key architectural difference relevant to lossless operation?

  1. InfiniBand relies on TCP for reliable delivery, while Ethernet uses UDP
  2. InfiniBand provides credit-based link-level flow control by design, while Ethernet requires PFC configuration Correct answer.
  3. Ethernet cannot carry RDMA traffic under any configuration
  4. InfiniBand requires ECN marking for every packet

Explanation

InfiniBand uses credit-based flow control inherently, making links lossless by design, whereas Ethernet needs PFC and congestion control configured to approximate lossless behavior for RoCEv2. InfiniBand does not use TCP, Ethernet can carry RoCEv2, and ECN is not mandatory in InfiniBand.

Question 26Networking

In an InfiniBand fabric, which mechanism lets a host discover the local adapter's LID and the fabric's SM LID without manual configuration?

  1. The subnet manager pushes them via an in-band MAD to the host channel adapter Correct answer.
  2. The adapter derives them from its PCIe bus address during enumeration
  3. The host administrator assigns them statically in the driver configuration file
  4. The adapter obtains them from a DHCP server on the management network

Explanation

InfiniBand management datagrams (MADs) travel in-band, and the subnet manager assigns LIDs and reports fabric parameters to each HCA. The host learns its LID and the SM LID through these MAD exchanges rather than through DHCP or static host files. This is why an unmanaged IB subnet leaves ports without usable LIDs.

Question 27Networking

Which address type in an InfiniBand fabric is a 128-bit identifier assigned to a port and usable for routing without a subnet manager lookup?

  1. GID Correct answer.
  2. LID
  3. SLID
  4. P_Key

Explanation

The GID is a 128-bit address formed from a subnet prefix plus a GUID, analogous to an IPv6 address, and global routing can use it directly. The LID is the smaller local identifier used inside a single subnet. P_Key is a partition membership key, not an address.

Question 28Networking

What is the purpose of an LAG (link aggregation group) on a pair of NVIDIA Spectrum switch ports connecting to a GPU server?

  1. To assign distinct IP subnets to each member link automatically
  2. To convert RoCEv2 traffic into native InfiniBand frames at the switch boundary
  3. To encrypt all east-west traffic between the server and the leaf switch
  4. To bundle member links into one logical interface for higher bandwidth and link redundancy Correct answer.

Explanation

A LAG presents multiple physical links as a single logical port, adding bandwidth and allowing traffic to continue if one member fails. Hashing distributes flows across members, so per-flow bandwidth still depends on hash distribution. LAG does not encrypt, translate protocols, or create subnets.

Question 29Networking

Which VLAN configuration is required so that RoCEv2 traffic from a GPU host reaches a leaf switch on a dedicated lossless segment?

  1. The switch port must be configured as a routed port with no VLAN membership at all
  2. The host NIC and the switch port must be members of the same VLAN with PFC configured on that VLAN Correct answer.
  3. The host NIC must be placed in VLAN 1 while the switch port uses an arbitrary access VLAN
  4. The host must use untagged frames and rely on the switch default VLAN for lossless behavior

Explanation

Lossless behavior is configured per priority on a VLAN, so the host interface and switch port must agree on VLAN membership and the PFC priority mapping. Mismatched VLANs drop or misclassify the traffic before PFC can apply. A routed port or default-VLAN mismatch breaks the intended isolation.

Question 30Networking

A GPU node sends 9000-byte jumbo frames while one leaf switch uplink still has an MTU of 1500. What is the most likely symptom on that path?

  1. The switch rewrites the packet size to 1500 without any performance impact
  2. The link automatically negotiates a larger MTU and the mismatch is harmless
  3. Large packets are dropped or fragmented, causing stalls in bulk RDMA transfers Correct answer.
  4. PFC pauses the entire fabric until the MTU is corrected

Explanation

A device that receives a frame larger than its configured MTU cannot forward it intact and drops or fragments it, which stalls large message transfers. MTU is not auto-negotiated on Ethernet links. Neither PFC nor silent rewriting resolves a size mismatch.

Question 31Networking

Which statement describes the role of a P_Key in an InfiniBand fabric shared by multiple tenants?

  1. It sets the service level used for arbitration between virtual lanes
  2. It defines the maximum transmission unit for the port
  3. It records the physical cable length for routing decisions
  4. It restricts which ports may communicate, providing partition-level isolation Correct answer.

Explanation

P_Keys partition an InfiniBand subnet so that only ports holding a matching key can exchange packets, which gives tenant or job isolation. MTU, service level, and cable length are separate attributes. Partition enforcement is checked at the HCA and switch.

Question 32Networking

What does the service level (SL) to virtual lane (VL) mapping accomplish in an InfiniBand switch?

  1. It encrypts payloads between HCAs using the management key
  2. It assigns LIDs to newly connected nodes during subnet discovery
  3. It separates traffic classes into distinct buffers so one class cannot head-of-line block another Correct answer.
  4. It selects the physical lane of the cable used for transmission

Explanation

The SL field in a packet is mapped to a VL, and each VL has its own buffers and arbitration, so high-priority traffic is isolated from bulk traffic. This is the basis for QoS in IB fabrics. LID assignment, encryption, and physical lane selection are unrelated functions.

Question 33Networking

Which counter on an NVIDIA ConnectX adapter should an engineer check first to confirm that RoCEv2 packets are being received on the intended priority?

  1. Per-priority pause and discard counters in the RoCE statistics Correct answer.
  2. The PCIe correctable error counter
  3. The adapter's firmware version string
  4. The link speed reported by ethtool

Explanation

RoCE statistics expose per-priority pause frames and discards, which reveal whether lossless treatment is actually applied to the traffic class in use. PCIe errors, firmware version, and link speed do not show priority-level behavior. This makes the RoCE counters the direct diagnostic.

Question 34Networking

In a leaf-spine fabric, what does the term 'spine' refer to?

  1. The storage switches that front the parallel file system
  2. The top-of-rack switches that directly attach servers
  3. The upper tier of switches that interconnect all leaf switches and provide path diversity Correct answer.
  4. The management switches dedicated to out-of-band console access

Explanation

Spines form the upper tier and connect to every leaf, giving any leaf a path to any other leaf through one spine hop. Leaves are the top-of-rack devices that attach servers. Management and storage networks are separate concerns from the leaf-spine data fabric.

Question 35Networking

Which design choice most directly limits the blast radius of a single spine switch failure in a leaf-spine AI fabric?

  1. Configuring a single spine with the highest available port density
  2. Disabling ECMP so that traffic follows one deterministic path
  3. Using a star topology with one central switch
  4. Providing at least two spine switches with equal-cost paths from every leaf Correct answer.

Explanation

With multiple spines and ECMP, a failed spine removes only part of the capacity and the remaining spines carry the flows. A single spine or a star topology creates a single point of failure. Disabling ECMP removes the ability to use alternate paths.

Question 36Networking

What is the function of a gateway (default router) address on a GPU server's RoCE interface in a routed leaf-spine fabric?

  1. It identifies the next-hop switch interface to which off-subnet traffic is sent Correct answer.
  2. It is the address of the PFC pause frame generator
  3. It is the multicast group address used for collective operations
  4. It is the address the server uses to register with the subnet manager

Explanation

When the destination is outside the host's subnet, the host sends frames to the gateway's MAC address so the router can forward them. This is standard IP forwarding behavior applied to RoCE interfaces. SM registration, multicast groups, and PFC generation are separate mechanisms.

Question 37Networking

An engineer needs to verify that a leaf switch will forward RoCEv2 frames using the DSCP value the hosts mark. Which configuration element must be checked?

  1. The trust mode on the ingress port and the DSCP-to-traffic-class mapping table Correct answer.
  2. The spanning-tree priority of the leaf switch
  3. The NTP server configured on the switch
  4. The syslog severity level for interface events

Explanation

If the port does not trust DSCP, the switch may overwrite the marking, and the DSCP-to-traffic-class map determines which internal queue and priority the frame receives. Both must align with the host marking for lossless treatment. Spanning tree, NTP, and syslog do not affect QoS classification.

Question 38Networking

Why is it recommended to separate RoCEv2 storage traffic and GPU collective traffic into different traffic classes on the same physical fabric?

  1. Each class gets independent buffering and pause behavior so one workload cannot starve the other Correct answer.
  2. Traffic classes are required for the switch to learn MAC addresses
  3. Separate classes double the available link bandwidth for each workload
  4. Class separation removes the need for any congestion control mechanism

Explanation

Different traffic classes map to different priorities and buffers, so pause or congestion in one class does not block the other. Classes do not create additional physical bandwidth. MAC learning and congestion control are independent of class separation.

Question 39Networking

Which statement best explains why a larger MTU reduces CPU overhead for TCP-based storage traffic on an AI cluster network?

  1. Fewer packets are needed per byte transferred, so fewer interrupts and headers are processed Correct answer.
  2. Larger frames increase the number of packets per second the host must handle
  3. Larger frames allow the NIC to skip checksum computation
  4. Larger frames eliminate the need for TCP acknowledgments entirely

Explanation

With a larger MTU each packet carries more payload, so a given transfer requires fewer packets, headers, and interrupts. TCP still uses acknowledgments and checksums regardless of MTU. More packets per second would increase, not reduce, overhead.

Question 40Networking

What does an oversubscription ratio of 2:1 on a leaf's uplinks mean for traffic heading toward the spine?

  1. The fabric provides twice the bisection bandwidth of a non-blocking design
  2. Uplinks automatically double their line rate when congested
  3. Each host is guaranteed twice its access-link bandwidth toward the spine
  4. Host-facing demand can be twice the available uplink capacity, so congestion is possible Correct answer.

Explanation

A 2:1 ratio means the sum of downlink capacity is twice the uplink capacity, so simultaneous full-rate flows can oversubscribe the uplinks. It is not a guarantee of extra bandwidth and it reduces, not increases, bisection bandwidth relative to 1:1. Line rate does not change under load.

Question 41Networking

Which practice helps prevent a single large flow from being pinned to one uplink in a leaf-spine fabric?

  1. Rely on a single uplink per leaf to simplify troubleshooting
  2. Configure static routes for each server to a fixed spine
  3. Disable all hashing so traffic uses the first available uplink
  4. Enable flow-based hashing that includes source and destination IP plus L4 ports Correct answer.

Explanation

Hash fields that include L3 and L4 information spread many flows across uplinks, so no single flow monopolizes one link. Disabling hashing or using one uplink concentrates traffic. Static per-server routes to a fixed spine defeat load spreading.

Question 42Networking

A network team must confirm that a newly cabled leaf-spine fabric passes traffic at the expected rate between two GPU hosts. Which approach is most appropriate?

  1. Run a bandwidth test between the hosts and compare the result with the expected link rate Correct answer.
  2. Review the switch uptime counter to infer throughput
  3. Ping the management interfaces and treat successful replies as proof of line rate
  4. Check the switch fan status and assume the fabric is healthy

Explanation

An end-to-end bandwidth test measures the actual achievable throughput across the path, which validates cabling, MTU, and hashing together. Fan status, ping reachability, and uptime say nothing about forwarding rate. Only a throughput measurement confirms the expected rate.

Question 43Networking

What is the primary reason to use a dedicated out-of-band management network for switches and BMCs in an AI cluster?

  1. It carries the RoCEv2 traffic for GPU collective operations
  2. It provides the low-latency path used by GPUDirect RDMA
  3. It keeps management access available even when the data fabric is misconfigured or down Correct answer.
  4. It replaces the need for a subnet manager in InfiniBand

Explanation

An out-of-band network gives administrators a path to devices that is independent of the data fabric, so recovery is possible when the fabric fails. It does not carry GPU collective or RDMA data traffic. It also does not replace the subnet manager.

Question 44Networking

How does NTP synchronization across switches and hosts support troubleshooting of an AI fabric incident?

  1. It lets logs and counters from different devices be correlated on a common timeline Correct answer.
  2. It determines which spine is elected as the ECMP root
  3. It guarantees that PFC pause frames arrive in the correct order
  4. It sets the DSCP value used for RoCEv2 traffic

Explanation

Accurate, common time lets engineers align events from hosts, leaves, and spines to reconstruct an incident sequence. NTP does not influence pause frame ordering, DSCP marking, or ECMP path selection. Without synchronization, timestamps from different devices cannot be compared reliably.

Question 45Networking

A RoCEv2 host shows a large count of packets discarded at the ingress queue for its lossless priority. Which condition most likely explains this?

  1. The MTU is larger than the standard value
  2. The host has an incorrect default gateway address
  3. The switch is running an older firmware version
  4. The pause threshold is misconfigured or PFC is not enabled on that priority Correct answer.

Explanation

Discards on a lossless priority indicate that the buffer filled and pause was not effective, typically because PFC is disabled or thresholds are wrong. Gateway errors and firmware age do not directly cause ingress queue discards. A larger MTU does not by itself cause this symptom.

Question 46Networking

An engineer observes that a RoCEv2 flow stalls whenever it traverses a particular spine, while the same flow succeeds through other spines. Which investigation is most targeted?

  1. Increase the MTU on all hosts to the maximum supported value
  2. Reinstall the collective communication library on every node
  3. Compare the QoS and PFC configuration on the suspect spine against the working spines Correct answer.
  4. Replace all host adapters in the affected rack

Explanation

Because the failure follows a specific spine, the difference is most likely in that device's configuration, such as QoS mapping or PFC settings. Host adapters, libraries, and MTU are common across paths and would not explain path-specific behavior. Comparing the suspect spine with its peers isolates the cause.

Question 47Networking

In an InfiniBand fabric, what is the consequence of having two subnet managers active with the same priority on one subnet?

  1. All ports are assigned two LIDs and use both simultaneously
  2. They can conflict over LID assignment, so priority and state must be set to elect a single master Correct answer.
  3. The switches fall back to Ethernet forwarding
  4. The fabric automatically doubles its bisection bandwidth

Explanation

Only one SM should be master; otherwise competing LID assignments can disrupt the fabric. SM priority and state determine which instance becomes master, with the other standing by. Multiple SMs do not add bandwidth, dual LIDs, or Ethernet fallback.

Question 48Networking

Why does a leaf-spine fabric generally scale better than a three-tier core-aggregation-access design for east-west GPU traffic?

  1. Every leaf is a constant number of hops from every other leaf, which bounds and equalizes latency Correct answer.
  2. It requires fewer physical cables than any other topology
  3. It removes the need for Layer 3 routing between racks
  4. It allows all servers to share a single broadcast domain without limits

Explanation

In a leaf-spine design, any leaf reaches any other leaf through exactly one spine hop, so latency is bounded and uniform as the fabric grows. Cable count is not necessarily lower, Layer 3 routing is still used, and large flat broadcast domains are avoided rather than encouraged.

Question 49Networking

A GPU cluster's all-reduce throughput drops sharply when job placement changes so that communicating ranks span multiple leaves. Which factor best explains this?

  1. Cross-leaf traffic must traverse spine uplinks, where oversubscription and shared queues add contention Correct answer.
  2. Cross-leaf traffic is automatically routed over the management network
  3. All-reduce requires every rank to be on the same leaf to function at all
  4. Spine switches disable RoCEv2 for traffic arriving from other leaves

Explanation

When ranks spread across leaves, their traffic uses spine uplinks, which may be oversubscribed and shared with other flows, increasing contention and latency. All-reduce does not require same-leaf placement. Traffic stays on the data fabric, and spines forward RoCEv2 normally.

Question 50Networking

An engineer is planning an NVIDIA Quantum-2 InfiniBand fabric for a GPU cluster and wants each host port to use the highest port speed supported by that switch platform. Which per-port signaling rate should the design target?

  1. 100 Gb/s per port using four 25 Gb/s NRZ lanes
  2. 400 Gb/s per port using four 100 Gb/s PAM4 lanes Correct answer.
  3. 200 Gb/s per port using two 100 Gb/s NRZ lanes
  4. 56 Gb/s per port using four 14 Gb/s NRZ lanes

Explanation

Quantum-2 (NDR) InfiniBand delivers 400 Gb/s per port, achieved by bonding four 100 Gb/s PAM4 lanes into a single port. Earlier generations scale down from there: HDR is 200 Gb/s, EDR is 100 Gb/s, and FDR is 56 Gb/s. Choosing the correct per-port rate matters because leaf-spine uplink and oversubscription calculations for GPU fabrics are based on line rate per port.