Prefetcher Enabled, Throughput Down 18%: Bandwidth or NUMA?
A team is tuning an in-memory hash-join service on a two-socket server: 8 cores per socket at 2.4 GHz, 256 KB private L2 per core, 24 MB LLC per socket, four DDR4-2400 channels per socket, 512 GB total. Before the change, the 16-thread benchmark reached IPC 1.42 and 62% of peak DRAM bandwidth. After enabling the vendor's "aggressive streaming prefetcher" in BIOS, throughput falls 18%: IPC 1.15, LLC misses per kilo-instruction rise from 4.1 to 7.8, L2 MPKI from 12 to 19, DRAM bandwidth sits at 88%, and front-end stalls barely move. Uncore counters show 41% of memory traffic is remote-socket. A manager proposes buying faster DIMMs and widening LLC way allocation; an engineer proposes reverting the prefetcher and binding threads to NUMA nodes. The benchmark input is a fixed 400 GB table, and repeated runs currently vary by plus or minus 3%.
1. Using the counters, state where the bottleneck is and whether faster DIMMs would help.
Read the worked answer
The signature is bandwidth saturation, not latency or front-end limitation. Three counters point the same way: DRAM at 88% of peak sits at the practical ceiling (sustained streams rarely exceed roughly 85-90% because of refresh, bank conflicts and read/write turnaround); IPC fell while MPKI rose, meaning more work is being spent fetching and less is retiring; front-end stalls did not move, so instruction supply is not the problem. Faster DIMMs raise peak bandwidth, so they help only if the added bytes are useful. Here the doubled LLC MPKI suggests the prefetcher is pulling lines that are evicted before use, so extra bandwidth would mostly raise the ceiling on wasted traffic. The 41% remote figure is a second, independent loss: remote traffic crosses the inter-socket link, whose bandwidth is far below local DRAM, so a faster local DIMM may simply move the queue to that link. Correct order of work is to reduce traffic first, re-measure, then decide whether capacity or speed is still short. Buying DIMMs now risks spending capital on a symptom.
2. Design an experiment set that separates prefetcher pollution from NUMA imbalance, and say what result would falsify each hypothesis.
Read the worked answer
Run a paired A/B/A sequence on identical input, five runs per cell, pinning the CPU governor to performance and reporting medians with confidence intervals, because 3% noise cannot resolve an 18% effect with single runs.
Cell 1, baseline (prefetcher on, current affinity): record DRAM bytes/s per socket, remote-traffic share, LLC MPKI, IPC. Cell 2, prefetcher off, same affinity: if MPKI and DRAM bytes fall while IPC climbs, the prefetcher was polluting and issuing useless traffic. Falsified if MPKI is unchanged and only noise moves. Cell 3, prefetcher on, first-touch placement plus node binding: if remote share drops toward 10% and IPC climbs, NUMA placement dominates. Falsified if binding changes nothing, which would mean the allocator already places pages locally or the table does not fit one node. Cell 4, prefetcher off plus binding: the interaction test. If the gain is additive, both matter; if cell 4 equals cell 3, prefetching is irrelevant once locality is fixed.
Add a per-socket uncore byte counter split rather than trusting the aggregate, and record migration counts to prove binding was honoured. Treat any result that only appears in one run as noise.
3. What would you change first, and what constraints make that change risky on this specific machine?
Read the worked answer
Change configuration, not hardware: disable the aggressive prefetcher and add explicit NUMA placement. Both are free, reversible within a reboot, and testable inside a maintenance window; DIMMs are capital expense and cannot fix wasted traffic.
The risk specific to this host is the 400 GB table against 512 GB of RAM split into 256 GB per socket. Strict binding to one node guarantees exactly what the benchmark cannot afford: the table cannot be held locally, so either the allocator spills or the process fails. Options are interleaving for the shared scan part, or partitioning the probe side per node and replicating the smaller build side, which trades memory for locality. Measure before committing.
Second risk is collateral damage: BIOS prefetcher settings are host-wide and may help other workloads on the same machine, so the change needs a per-workload regression, not a single benchmark. Third is measurement discipline: with plus or minus 3% run-to-run variance and neighbour noise, a 5-run paired comparison and a fixed input file are the minimum to claim a win. Finally, do not widen LLC ways as a first move; with 24 MB and high associativity, conflict misses are an unlikely explanation, and the eviction counters should confirm or reject that idea before any microarchitecture change is proposed.