IPsec MTU Black Hole on a 2 GB File Transfer
A user at branch 10.8.4.0/24 reports that a 2 GB file copy to a server at 10.8.9.20 stalls at about 40 MB and then resumes after a minute. A packet capture from a SPAN port on the branch uplink shows the client negotiating a TCP maximum segment size of 1,460 bytes, then sending 1,460-byte TCP segments in 1,500-byte IP packets with the DF bit set. The capture contains multiple ICMP destination-unreachable messages, code 4, fragmentation needed, with next-hop MTU 1,400, but the client continues to retransmit the same full-size segments. The branch uses an IPsec tunnel over a 200 Mbps link; the tunnel adds 60 bytes of overhead. The firewall between sites drops all ICMP. Web browsing and email work normally. The workstation has a local firewall and a VPN client that is not connected.
1. Identify the likely fault and the packet-level evidence that confirms it.
Read the worked answer
This is a Path MTU Discovery black hole. The IPsec tunnel adds 60 bytes, so the real path MTU is about 1,400 bytes, but the client still advertises MSS 1,460 and sends 1,500-byte IP packets with DF set. The router cannot fragment them and emits ICMP type 3 code 4 with next-hop MTU 1,400. The firewall drops all ICMP, so the client never learns to reduce its segment size. It retransmits the same full-size segment, TCP backs off, and the transfer stalls until a smaller packet or a cache entry lets it resume. Web and email work because their packets are small.
Evidence to confirm:
- DF bit set on the large segments and TCP MSS option of 1,460, not clamped for the tunnel.
- Repeated retransmissions of the same sequence number around the stall.
- ICMP type 3 code 4 present at the branch uplink capture but not reaching the client, or absent because the firewall drops it before the SPAN point.
- Tunnel interface MTU and MSS clamping configuration showing no adjustment.
A duplex mismatch would show CRC/FCS errors and late collisions; random loss would affect small packets too.
2. What configuration change would you make, and what are the risks?
Read the worked answer
Clamp TCP MSS on the tunnel interface to 1,360 bytes (1,400 minus 20 bytes IP and 20 bytes TCP) or slightly lower, such as 1,340, to leave margin. Also allow ICMP type 3 code 4 through the firewall with rate limiting, because PMTUD depends on it. Set the tunnel interface MTU to 1,400 and verify that the physical interface is 1,500. These changes address both TCP and non-TCP traffic: MSS clamping helps TCP, while ICMP delivery lets UDP and other protocols react.
Risks and trade-offs:
- Allowing ICMP can expose reconnaissance or ICMP-based DoS; permit only the needed type/code, rate-limit, and log.
- MSS clamping reduces the maximum segment size, slightly lowering efficiency on large transfers, but prevents black holes.
- MSS clamping does not fix UDP; applications that set DF and rely on PMTUD still need the ICMP path.
- A wrong clamp value can cause fragmentation or poor performance, so test with captures and a large file transfer.
Do not solve this by disabling DF or fragmenting at the firewall; both create other problems.
3. How would you distinguish this from a duplex mismatch or generic packet loss before changing configuration?
Read the worked answer
Check interface counters first. A duplex mismatch produces late collisions, FCS/CRC errors, and alignment errors on one side; generic congestion shows discards and random retransmissions across all packet sizes. Here the stall is size-dependent: small web and email flows work, while the 2 GB copy fails only when full-size segments are sent.
Run a ping sweep with DF set at 1,400 and 1,500 bytes. If 1,500-byte pings fail but 1,400-byte pings pass, the path MTU is the issue. Capture at both ends of the tunnel and compare: if the ICMP fragmentation-needed message arrives at the branch but the client keeps retransmitting, the client is not receiving it. Check the firewall ICMP policy and the tunnel MTU. Finally, look for a repeated sequence number and exponential backoff in the capture; that is TCP's reaction to a black hole, not to random loss. Only after these checks would I change MSS or ICMP rules.