Stalls, Thirty-Minute Disconnects and a Saturated Location
Wingtip has 25 remote networks and 5,000 mobile users served from two compute locations. Over three weeks the help desk has collected four complaints. (1) Sites in Brazil report file transfers to the data center that consistently stall around the 2 MB mark and then resume, every time. (2) One office disconnects and reconnects roughly every 30 minutes, always close to :00 and :30 past the hour. (3) Prisma Access Insights shows the Brazil compute location at 92% of licensed bandwidth between 09:00 and 11:00 local. (4) The tunnel to the Brazil site shows four flaps in 24 hours while other tunnels show zero. The team wants to open a TAC case but has only a screenshot of a bandwidth graph and no timestamps.
1. How would you triage these four complaints, and what evidence would you gather before opening a case?
Read the worked answer
Separate the complaints before treating them as one outage, because the data suggests at least two unrelated causes.
Group A - throughput. The 92% peak-bandwidth reading at the Brazil location is a capacity signal, and it can explain slow transfers generally, but not a stall at a consistent 2 MB. Consistent size-based failure is a packet-size problem, not a bandwidth problem: large packets are being dropped and retransmitted while small ones pass, which users experience as stall-then-resume.
Group B - the 30-minute reconnect. That regularity is a timer, not congestion. Check the CPE's IKE and IPSec lifetimes and dead peer detection settings, the ISP's NAT translation timeout (a common source of periodic tunnel renegotiation behind carrier NAT), and whether the reconnects correlate with a DHCP lease or a periodic circuit event. Compare against the default lifetimes on the Prisma Access side - if you find a mismatch, you have your timer.
Group C - the four flaps. Pull the exact flap timestamps from Panorama and check them against the reconnects in Group B and against any change windows. Flaps at the same times as the reconnects make them one problem.
Evidence to collect: Prisma Access Insights bandwidth trend for the location with timestamps; per-tunnel status history with timestamps; IKE and IPSec logs from the CPE and from Prisma Access; the remote network's routing and MTU configuration; the exact source, destination and timestamp of a failing transfer; and a test showing the largest packet size that passes. A screenshot without timestamps will get sent back.
2. A Brazil user can browse fine but a 40 MB file copy to the data center stalls at roughly 2 MB each time. What is your diagnosis and fix?
Read the worked answer
This is the classic signature of a path MTU black hole. Small packets - browsing, DNS, TCP handshakes - succeed. Once the transfer ramps up and full-size segments are sent, they are dropped somewhere on the tunnel path, and either the ICMP 'fragmentation needed' message is filtered or the tunnel does not honour it. TCP backs off, retransmits with a smaller segment, gets through, ramps up again, and stalls - repeatedly, at approximately the same point in the transfer.
Confirm: test with the DF bit set from a client behind the tunnel, stepping the payload size down until replies arrive. If the largest working payload plus headers is below the expected path MTU, you have found it. Check both directions, and check the CPE's interface MTU and TCP MSS settings.
Fix: clamp TCP MSS on the tunnel so endpoints negotiate a segment size that fits inside IPSec overhead, and adjust the tunnel MTU to what the Prisma Access tunnel expects - commonly around 1400 bytes, but verify for your deployment rather than assuming. Apply the change on both the CPE side and, where applicable, in the remote network configuration, then repeat the same file copy as the acceptance test.
Also consider: the 92% bandwidth peak makes the transfer slow and can mask or worsen the symptom, but it will not produce a repeatable stall at the same byte offset. Fix MTU first because it is deterministic and cheap to test; treat the capacity question separately.
3. The Brazil compute location runs at 92% of licensed bandwidth every morning. What would you recommend, and how do you avoid this being blamed for the other complaints?
Read the worked answer
Treat it as a capacity decision with its own evidence trail, and keep it separate from the tunnel and MTU work so neither masks the other.
Quantify first. Pull the peak and 95th-percentile bandwidth for the location over four to six weeks, broken down by top applications and top source remote networks or user groups. Identify whether the peak is business traffic, backup replication, OS or application updates, video, or personal streaming. Check whether the peak is genuinely users or a scheduled job that could move out of the 09:00–11:00 window.
Then choose, in cost order:
- Move or throttle non-business and deferrable traffic with QoS or application-based bandwidth limits.
- Shift traffic to the other compute location or to local internet breakout for trusted SaaS categories, if latency permits - verify with RTT measurements rather than assuming.
- Re-architect paths for the heaviest flows, such as a service connection that shortens a data-center-bound path.
- Add licensed bandwidth, which is the right answer only if 1–3 are exhausted and the growth trend supports the spend.
Avoiding misattribution: state explicitly in the incident record that sustained 92% at peak degrades performance but does not explain a transfer stalling at a fixed byte count, nor a 30-minute reconnect timer. Keep separate tickets for the MTU issue, the periodic disconnect and the capacity item, each with its own timestamps and acceptance test. If you merge them into one 'Brazil is slow' ticket, you will close it when bandwidth is added and the stalls will still be there on Monday.