Intermittent HTTPS timeouts caused by return-path asymmetry
An R81.20 gateway on an open server has two external interfaces: eth1 (ISP-A, default route) and eth2 (ISP-B, backup). The LAN is 10.50.0.0/16. Users report that HTTPS sessions to 203.0.113.50:443 sometimes complete and sometimes time out after about 30 seconds; HTTP to the same host is also affected. A capture with fw monitor -e 'host(203.0.113.50) and port(443);' shows the client SYN leaving via eth1, but the SYN-ACK arriving on eth2. tcpdump -ni eth2 confirms the SYN-ACK on eth2. fwaccel stats -s shows a high count under TCP violations, while fw ctl zdebug drop run for 10 seconds shows no matching drop. The routing table has only one default route via eth1; ISP-B is reachable only through a specific route for 198.51.100.0/24. No NAT change was made. The customer wants a fix that does not disable SecureXL.
1. Why does SecureXL see the SYN-ACK as invalid, and what evidence distinguishes a SecureXL problem from a policy drop?
Read the worked answer
SecureXL maintains a connection state that expects the reply on the same path or interface as the outbound SYN. The firewall sees the SYN leave on eth1 and the SYN-ACK arrive on eth2, so the packet does not match the accelerated connection and is counted as a TCP violation. A policy drop would usually appear in fw ctl zdebug drop or in SmartLog with a rule number; here the short zdebug capture shows no matching drop while fwaccel stats -s shows violations, which points to SecureXL and stateful path handling rather than the rulebase. fwaccel conns can show the connection template and interface. fw monitor capture points (i, I, o, O) confirm direction, while tcpdump -ni eth2 proves the return packet's ingress. The intermittent success is explained by flows that happen to return on eth1.
2. Give a test plan to prove the path asymmetry, and one temporary mitigation.
Read the worked answer
Test plan: 1) Resolve the server IP and run ip route get 203.0.113.50 to document the chosen egress. 2) Run simultaneous tcpdump -ni eth1 and tcpdump -ni eth2 filtered on the server and port 443. 3) Run fw monitor -e 'host(203.0.113.50) and port(443);' and record the capture points for SYN and SYN-ACK. 4) Compare the ingress interface of the SYN-ACK with the egress interface of the SYN. 5) Check fwaccel stats -s before and after a failing connection. Temporary mitigation: disable SecureXL with fwaccel off, retest, then re-enable with fwaccel on; if the failures disappear, path asymmetry is confirmed. Do not leave SecureXL off. Alternatively, add a temporary /32 route for the server via eth2 if the return path is consistently eth2.
3. What durable fix and validation would you recommend?
Read the worked answer
The durable fix is symmetric routing. Either make ISP-A the only path for this destination and ensure the return traffic also uses ISP-A, or route 203.0.113.50/32 (or the affected SaaS range) via eth2 and ensure outbound also uses eth2. If both ISPs must be used, configure policy-based routing or dynamic routing with return-path awareness, and verify that NAT and the ISP's reverse path are consistent. Validate with repeated curl or openssl connections, fw monitor showing SYN and SYN-ACK on the same interface pair, fwaccel stats -s showing no new TCP violations, and SmartLog showing accepts rather than drops. Keep SecureXL enabled; disabling it only masks the routing problem and reduces performance.