Prague Branch: Login Works, Large Transfers Hang

The pattern is a Path MTU Discovery black hole. The IPsec tunnel carries only 1400 bytes, but the branch hosts and the HQ servers still believe the path supports 1500. Large packets are sent with the Don't Fragment bit set; the tunnel cannot forward them, and if the ICMP fragmentation-needed message is lost or filtered, the sender never learns to reduce its segment size. Small packets pass, so the TCP handshake and authentication complete, then the connection stalls when a large response or a directory listing is transferred.

Supporting evidence: small curl succeeds; SSH authenticates but ls -l hangs; ping -M do -s 1400 fails while -s 1300 succeeds; HQ LAN users are fine; the tunnel is not saturated; no MSS clamping is configured.

Ruling out alternatives:

• DNS: the test uses an IP address and succeeds, and the login page resolves.
• Packet loss: ping shows 0% loss at small sizes, and failure tracks packet size, not time or load.
• Firewall: the firewall permits the subnet and the TCP handshake completes; a block would fail earlier and symmetrically.
• Application: HQ users use the same CRM without problems, and the failure follows large payloads across different applications, HTTPS and SSH.

A useful confirmation is tcpdump on both sides showing retransmissions of full-size segments and no ICMP type 3 code 4 returning.

Clamp TCP MSS on the tunnel path so endpoints never send segments larger than the tunnel can carry. On the branch router, apply an MSS clamp of 1360 on the IPsec tunnel interface for traffic in both directions: 1400 minus 20 bytes IPv4 header and 20 bytes TCP header. If the tunnel interface MTU can be set, set it to 1400 as well, but MSS clamping is the part that fixes remote senders. Also confirm that ICMP type 3 code 4 is allowed through the tunnel and HQ firewall, so PMTUD can work as a fallback.

Verification, from a branch client rather than the router:

• ping -M do -s 1372 10.10.20.15 should succeed (1372 + 28 = 1400), while -s 1400 may still fail because it exceeds the tunnel MTU; that is expected.
• curl -o /dev/null a large file from the CRM and confirm it completes.
• SSH to the HQ server and run ls -l in the large directory; the session should no longer hang.
• git clone a repository of at least a few hundred MB.
• tcpdump -i <tunnel> 'tcp[tcpflags] & tcp-syn != 0' should show SYN segments with MSS 1360.

Repeat the tests at peak time and from more than one branch host, then record the before and after timings.

Lowering the branch LAN MTU helps only traffic originated by branch hosts, and only if every host picks up the change. It does nothing for packets sent from HQ servers to branch clients: those servers still have a 1500-byte MTU and will send full-size segments with DF set. If the ICMP fragmentation-needed message is dropped, the HQ side still experiences a black hole. TCP MSS clamping works differently: the router rewrites the MSS option in the SYN as it crosses the tunnel, so both endpoints advertise a segment size that fits the path regardless of their local MTU. That is why clamping at the tunnel is the right control, with a lowered MTU as a secondary measure.

Documentation should include the tunnel topology and endpoint addresses; the tunnel MTU and the MSS value chosen, with the header arithmetic; the exact router rule and interface; the requirement that ICMP type 3 code 4 must not be filtered; a repeatable test procedure with the ping sizes and expected results; known-good throughput and latency; and a note that overhead differs for IPv6 and for additional encapsulation. Add the change to the change-management record and to the branch onboarding checklist so a future tunnel is built with clamping from day one.