vMotion Moves a Latency-Sensitive SR-IOV VM onto a Contended NUMA Host
An enterprise runs VMware vSphere 8 with four ESXi hosts in one cluster. Each host has two 25 GbE NICs on a vSphere Distributed Switch with LACP, NIOC, and SR-IOV enabled. A latency-sensitive VM has 16 vCPUs, 64 GB RAM, two SR-IOV VFs, and a 1 GB hugepage reservation. It runs on esx-01 with CPU affinity to physical cores 0 through 15. After a vMotion to esx-02, the VM's packet rate drops 40% and %RDY rises from 1% to 12%. esx-02 has hyperthreading enabled; cores 0 through 15 are shared with vMotion and vSAN traffic. The vDS NIOC shares are vMotion 25, vSAN 50, and VM 100. The VM's VFs are on NUMA node 0, but its vCPUs are spread across both NUMA nodes. The VM has no NIOC reservation. The team wants consistent packet rate after future vMotions without dedicating an entire host.
1. Which two factors most likely explain the packet-rate drop and the rise in %RDY?
Read the worked answer
Two factors dominate. First, CPU contention: %RDY rising from 1% to 12% means the VM is waiting for pCPUs. The affinity still points at cores 0-15, which on esx-02 are shared with vMotion and vSAN, so the VM is competing for the same physical cores. Second, NUMA mismatch: the SR-IOV VFs are on node 0, but vCPUs are spread across both nodes, so some packet processing and memory accesses cross NUMA. That raises latency and reduces packet rate. Packet rate is more sensitive to scheduling jitter and remote memory than to raw link bandwidth.
2. Is increasing the VM's NIOC shares or changing CPU affinity the better fix? Justify your answer.
Read the worked answer
Increasing VM NIOC shares alone is not the right first fix. Shares only arbitrate under contention; they do not remove vMotion/vSAN from the same cores or fix NUMA placement. Changing affinity alone can help, but only if the new pCPUs are on NUMA node 0 with the VFs and are not shared with vMotion/vSAN. The better fix is combined: pin vCPUs to pCPUs on node 0, reserve separate cores for vMotion and vSAN, and use NIOC reservations or limits rather than relying on shares. Then measure %RDY, packet rate, and NUMA locality.
3. Recommend a placement and resource design that keeps the VM mobile and consistent across future vMotions.
Read the worked answer
Use a VM-host affinity rule that keeps the VM on hosts where NUMA node 0 has free capacity, and configure NUMA node affinity so vCPUs and VFs stay on node 0. Reserve pCPUs for vMotion and vSAN, or cap those workloads with NIOC limits so they cannot steal the latency-sensitive cores. Disable hyperthreading for this VM or pin to full physical cores to avoid sibling contention. Add a post-vMotion validation that checks %RDY, per-vCPU wait, and packet rate. This preserves host mobility without dedicating an entire host.