Month-End Batch: A 24 vCPU Database VM That Never Gets Migrated

What the numbers are telling you

• CPU ready is measured per vCPU and expresses wait time. 19% on a 24 vCPU VM is roughly 4.5 vCPUs permanently waiting - real loss, not noise.
• Each host has 2 sockets x 12 cores = 24 physical cores. A 24 vCPU VM cannot fit inside one 12-core NUMA node, so it always spans both nodes. Add hyper-threading: the host has 48 logical CPUs but only 24 physical cores, so at 24 vCPU the VM is co-scheduled with HT siblings and every stall shows up as ready plus co-stop. Co-stop at 8% confirms the scheduler is waiting for the widest member, not the busiest.
• The 52% is a host average across 48 logical CPUs. It hides fully saturated pCores.

Measure before changing

1. esxtop per-PCPU utilisation (%CSTP, %USED) rather than host average.
2. Per-vCPU %RDY to see whether wait is spread evenly or concentrated.
3. The VM's configured cores-per-socket; if it is one socket of 24 cores, vNUMA exposes a single virtual node wider than the physical node, guaranteeing remote memory access.
4. Whether the batch is CPU-bound at all: check guest processor queue length and datastore latency (DAVG/cmd) before attributing everything to CPU.

Why DRS does nothing

DRS balances host-level load across the cluster. All three hosts are identical, there is no imbalance, and DRS does not migrate a VM because its own guest scheduler is starved. Moving the VM cannot help: it has the same NUMA shape everywhere.

A fourth host does not help.

Ready time here is caused by the width of the VM relative to a physical NUMA node, plus HT sharing. Adding hosts increases cluster capacity; it does not make a 24 vCPU VM fit into 12 cores per node. The VM would still span nodes and still co-schedule against HT siblings on whichever host it lands.

The change to make

Reduce the VM to 16 vCPU (or 12), which fits within a single NUMA node on this hardware, and set cores-per-socket so each virtual node maps cleanly onto a physical node if you must stay wider. Validate with the guest's own CPU demand: if the batch peaks at 10-12 vCPU of actual work, 24 vCPU was never justified and only added co-scheduling cost.

SIOC

• SIOC is enabled per datastore and only acts when multiple VMs contend for the same datastore. If the ERP VM's I/O is the only outstanding load, SIOC changes nothing.
• Its threshold is a latency tolerance (default 30 ms). On all-flash, normal latency may never reach it, so SIOC stays idle. Lowering it blindly would throttle everyone else.
• The honest answer: fix vCPU sizing first, because that is the measured constraint. Then collect datastore latency during the next batch run and enable SIOC with shares only if the array, not the scheduler, turns out to be the limiter.

Validate without waiting for month-end

• Reproduce the contention: run the batch's heaviest known query set against a restored copy of the database on an isolated host or a maintenance-window slot, and compare %RDY, co-stop and elapsed runtime at 24 vCPU versus 16 vCPU.
• Otherwise, use the historical data: pull per-vCPU %RDY and guest CPU utilisation for the last three month-end windows from your monitoring platform. If peak guest demand never exceeded roughly 60% of 24 vCPU while ready time was high, the reduction is low risk and you can say so with evidence.
• Agree a rollback: vCPU count can be restored with a power cycle during the next window, so the change is reversible.

Risk framing for the DBA

• Fewer vCPU cannot reduce a workload that was not using them; it reduces scheduling overhead and removes co-stop.
• The genuine risk is the power cycle required to change vCPU, not the count.
• If the batch proves to need more than 16 vCPU, the correct answer is not 24 vCPU on 12-core sockets but a host model with larger NUMA nodes, or splitting the batch into parallel workers across VMs. Make that the fallback so the conversation is about the right fix, not about refusing to change anything.