71% Utilization, 34% Retries, and an ISP That Blames the Firewall
Corvus Fulfillment's 210,000 sq ft warehouse has 38 MR46 access points, all left on auto channel and auto power with 20 MHz channels on 5 GHz. Between 06:00 and 09:00, roughly 240 handheld barcode scanners report scan-to-screen latency above three seconds. Dashboard wireless health shows 71% average channel utilization on 5 GHz and a 34% retry rate. A site walk shows -48 dBm at the receiving dock and -74 dBm on the mezzanine. Separately, the MX250's WAN 2 has flapped fourteen times in 24 hours, because a brief fiber micro-outage each morning causes the LTE backup to be preferred and then handed back. The operations manager wants more access points. The ISP says the firewall is dropping packets.
1. Separate the two problems and state what evidence you would pull for each. Why is it important that you do this before changing anything?
Read the worked answer
They are unrelated, and treating them as one symptom is how you end up changing wireless settings to fix a carrier problem.
Wireless evidence: channel utilization per AP over the 06:00–09:00 window, not the daily average - an average of 71% hides the peak, and the incident is a peak phenomenon. Client count per AP to find the hot spots. Retry and SNR per client for a sample of the scanners. The scanner model's supported bands and roaming standards (802.11k/v/r) - many handheld scanners are 2.4 GHz-only or have aggressive roaming behavior, which changes the diagnosis completely.
WAN evidence: the MX uplink event log for the fourteen flaps with timestamps, whether they correlate with the morning fiber outages, the uplink configuration (which link is primary and how failover is configured), and live tools from the MX - ping and traceroute to the ISP's first hop during a flap. Packet loss on a WAN link does not need a firewall explanation; a flapping LTE handoff is a sufficient one.
Doing this before changing anything matters because both problems produce the same user complaint ('it is slow in the morning') and the fixes are in different devices. Also, once you change wireless settings, you lose your baseline and cannot prove anything later.
2. Do you add access points? Give your recommended change order and the reasoning behind each step.
Read the worked answer
No - not first, and probably not at all. 38 APs in 210,000 sq ft is already dense. The signal readings say the problem is not coverage: -48 dBm at the dock is hot, and even -74 dBm on the mezzanine is workable for a scanner. The symptom set - high utilization plus high retry - is the signature of co-channel interference and sticky clients, and adding APs to a high-interference environment makes it worse by adding more transmitters competing for the same airtime.
Recommended order, cheapest and most reversible first:
- Verify the peak, not the average. Pull per-AP utilization for the 06:00–09:00 window and identify the specific overloaded APs. You may find six APs carrying most of the load rather than a uniform problem.
- Build a real channel plan. Auto channel with this many APs in an open warehouse results in overlapping assignments. Assign non-overlapping 5 GHz channels manually across the floor plan, keeping neighbors apart, and cap 5 GHz width at 20 MHz - wider channels consume more airtime per transmission.
- Reduce and flatten transmit power. The -48 dBm reading means APs are shouting. Lower max power so clients associate with the nearest AP and roam, rather than clinging to a distant one.
- Handle sticky clients. Raise the minimum bitrate so distant, slow-talking clients are encouraged to roam, and check whether the scanners support band steering before enabling it.
- Check the scanner side. Confirm what the devices actually support; if they are 2.4 GHz-only, your 5 GHz tuning will not touch them.
Only if, after steps 1–5, specific zones still show saturation with a healthy channel plan do you add APs - and then in those zones, not uniformly.
3. How do you prove improvement to the operations manager and rebut the ISP's claim at the same time?
Read the worked answer
Both questions are answered with the same discipline: fix the measurement before the fix, then show the same measurement after.
For wireless: capture the 06:00–09:00 baseline now - per-AP utilization, retry rate, and a latency sample from a handful of named scanners during the peak. After the changes, re-capture the same window on the same APs. Report retry rate and peak utilization, not the daily average, because the daily average is what hid the problem in the first place. If the scanners have their own diagnostics or logs, get a before/after from them too - the ops manager cares about scan-to-screen time, not about airtime.
For the ISP: the uplink event log with timestamps, plus an MX live-tools ping to the ISP's first hop run during a flap, is the rebuttal. It shows the link going down from the MX's side while the LAN is fine. Correlate the flap timestamps with the fiber outage narrative the ISP themselves gave you. Then fix the preference: make the fiber the primary path with the LTE strictly as failover, so a micro-outage does not trigger a handoff and hand-back cycle for every session in the building.
Present the two as separate findings with separate evidence. The operations manager will accept 'the wireless is fixed and here is the peak-hour number' far more readily than a narrative about airtime.