The Noisy Neighbor on Shared Compute
A product team runs a batch image-resizing service on ten cloud VMs behind a load balancer. Each VM has 4 vCPUs and 16 GB RAM; average CPU sits at 30%, but p95 latency spikes to 8 seconds every weekday between 09:00 and 10:00, and the autoscaler adds four VMs that sit idle. During those spikes, host-level CPU steal time reaches 25% on three of the ten VMs, while the other seven show steal under 2%. The team's on-call engineer proposes doubling vCPUs on all ten VMs. The finance owner notes the workload is already over budget and asks for evidence before approving any size change. Application logs show no errors, and queue depth on the resize workers grows during the same window.
1. Diagnose the most likely root cause of the 09:00 latency spikes and explain why the proposed fix may not help.
Read the worked answer
The uneven steal time (25% on three VMs, under 2% on seven) points to noisy-neighbor contention on shared physical hosts, not to insufficient per-VM CPU. If the workload were simply under-provisioned, all replicas would show similar pressure and the autoscaler's new VMs would absorb load; instead four new VMs sit idle while queue depth grows, which suggests the bottleneck is on specific hosts rather than aggregate capacity. Doubling vCPUs on all ten VMs is a blunt fix: it raises cost, may not move the affected VMs off contended hosts, and leaves the seven healthy VMs over-provisioned. The evidence to gather first is per-VM steal time correlated with request routing and host placement, plus confirmation that the load balancer is not pinning traffic to the three degraded hosts. A targeted response-spreading instances across host groups or moving to dedicated/guaranteed-capacity instances for the affected replicas-addresses the actual contention at lower cost than a fleet-wide resize.
2. What evidence would you collect to distinguish host contention from an application-level bottleneck before spending budget?
Read the worked answer
Correlate three signals over the same time window: (1) per-instance CPU steal and host placement, to confirm that slow requests land disproportionately on the three high-steal VMs; (2) per-instance request distribution from the load balancer, to rule out uneven routing that would make a few VMs look overloaded; and (3) worker queue depth and processing time per request, to see whether the application itself slows down regardless of host. If slow requests track high-steal hosts even when request volume is equal, host contention is supported. If queue depth rises uniformly across all instances, the application or downstream dependency is the cause. Also check whether the autoscaler's new VMs receive traffic at all-idle new capacity during a spike often indicates a warm-up, routing, or health-check problem rather than a raw CPU shortage.