Shift-Change Latency Across Fourteen Distribution Centers
TransCargo operates 14 distribution centers running a 12-year-old warehouse management system hosted in a single data center in Columbus. Each site connects over a 20 Mbps MPLS hub-and-spoke circuit; all handheld scanning and label-printing traffic round-trips to Columbus. At shift change, when roughly 400 associates log in and begin picking simultaneously, average scan response time climbs from 0.4 s to 6-9 s and pick rates drop about 18%. Measured round-trip latency from a Denver handheld to the WMS database is 180 ms. Off-shift, response times are normal.
Last quarter a four-hour WMS outage in Columbus stopped outbound shipments at all 14 sites; operations estimates the cost at $310,000. The application vendor now offers a multi-tenant cloud WMS, and the network team proposes replacing MPLS with internet-based SD-WAN and local internet breakout at every site. Finance has approved funding for exactly one initiative this fiscal year.
1. What evidence would you gather to establish where the latency actually originates, and why does the answer change the investment decision?
Read the worked answer
The 180 ms round trip is real but cannot by itself produce 6-9 seconds. At 180 ms, a 6-second response implies roughly 30 sequential round trips per scan, or a server-side or client-side bottleneck. Do not spend against a hypothesis until the three layers are separated.
Network: per-site MPLS utilization during the shift-change window (a 20 Mbps circuit can saturate on login storms and label spooling), jitter and loss, and a packet capture at one site showing the request-response sequence for a single scan. Application: distributed traces or logs showing how many round trips one scan generates and where the time is spent. Server: database wait statistics, connection-pool exhaustion, and lock contention during mass login. Client: Wi-Fi roaming behavior on handhelds and whether provisioning or configuration download happens at login.
Also establish a baseline from a site with low volume and from off-shift hours, so the shift-change spike can be attributed to concurrency rather than to distance.
The reason this matters: if the cause is chattiness or connection-pool exhaustion, bandwidth and SD-WAN purchase latency improvements that the application will consume 30 times per scan, and the fix is protocol batching, local label printing, pooled sessions, and staggered authentication. If the cause is circuit saturation, the WAN is the right target. Buying the wrong one leaves the symptom in place and consumes the single funded initiative.
2. Compare keeping the centralized architecture with targeted remediation against moving to the cloud WMS with SD-WAN and local breakout, and recommend a course of action with conditions.
Read the worked answer
Centralized plus remediation keeps one source of truth and no data-synchronization problem, and it is fast: local label printing, staggered shift-change authentication, connection pooling, and batched scan transactions can be delivered in weeks at low cost. It does not, however, remove the Columbus single point of failure, which is the risk that actually cost $310,000.
Cloud WMS with breakout removes tromboning (site-to-cloud round trips typically drop to tens of milliseconds) and distributes the failure domain, but it imports new obligations: dual-carrier diversity per site, an edge security stack, local print and label services that must survive a WAN outage, multi-tenant customization limits, and exit costs from MPLS contracts that can run a year or more.
Recommendation: sequence rather than choose. First, deliver the low-cost remediation and instrumentation, which is justified under either option and will produce the evidence the larger decision needs. In parallel, pilot the cloud WMS with SD-WAN at two sites chosen for representative volume and worst RF conditions. Treat the Columbus single point of failure as a separate problem with its own budget case, because 12 of 14 sites would remain on the current platform during any migration and are still exposed.
Conditions: no MPLS termination before pilot evidence; contract language permitting staged site cutover and data export; and an explicit decision gate at the end of the pilot.
3. Define the pilot's success criteria and the conditions under which you would stop and revert.
Read the worked answer
Success criteria should be operational, measured against the current baseline, and agreed before the pilot starts. Suggested set: 95th-percentile scan response under 1 second during a simulated shift change with the site's full associate count; pick rate at or above the pre-migration baseline within two weeks of cutover; label printing continues during a simulated WAN failure via local print services; and a circuit-pull test demonstrates failover with no lost scans (queued and replayed, not dropped).
Also test the things pilots routinely skip: handheld roaming across the full floor, month-end and peak-season volume, integration with the host ERP, and the time to onboard a new site.
Stop-and-revert triggers: p95 latency above 1 second after remediation, more than two hard cutovers in a month, any dependency of label printing on the WAN, or vendor inability to demonstrate a data-export and exit path. Define the revert cost in advance, including the MPLS reinstatement terms and the effort to resynchronize inventory transactions that occurred in the cloud instance.
Finally, validate the commercial assumptions in the pilot, not after it: per-site circuit costs from at least two carriers, the actual customization gap between multi-tenant and the current system, and the internal staffing needed to run the edge. A technically successful pilot with a broken business case is still a failed pilot.