Asymmetric Return Path After Adding a Second ISP
River Logistics has a two-node NGFW cluster at its data center. ISP A provides a /29 public block and is the default route; ISP B was added for guest Wi-Fi capacity. The internal web server 10.10.20.15 is statically NATed to 203.0.113.50 (ISP A) for remote HTTPS. After adding ISP B, the routing table has two equal-cost default routes. Remote users report intermittent HTTPS timeouts. Packet captures show the client's SYN arriving on ISP A, the server's SYN-ACK leaving via ISP B with source 198.51.100.60 (ISP B's NAT address), and the client never completing the handshake. The NGFW connection log shows the flow as 'established' on the inside but no return packets. ISP A applies uRPF. The business wants guest traffic on ISP B and server traffic stable on ISP A.
1. Diagnose the failure. Why does the NGFW show the flow as established while the client never completes the handshake?
Read the worked answer
The two equal-cost default routes allow new outbound flows, including the web server's return traffic, to select either ISP. The server's SYN-ACK must leave through ISP A with the published address 203.0.113.50, but the capture shows it leaving ISP B with 198.51.100.60. The remote client sent its SYN to the ISP A address and will not accept a response from a different address, so the handshake fails. The NGFW can still show the flow as established because it has created state for the inbound connection; state creation is not proof that the remote client accepted the return path. Asymmetric routing and mismatched NAT are the core problems. ISP A's uRPF can also drop traffic with an unexpected source if the path changes again. This is a routing/NAT determinism issue, not an access-rule denial.
2. Design a routing and NAT fix that keeps guest traffic on ISP B and the published server on ISP A.
Read the worked answer
Remove the equal-cost default routes. Set a single default route through ISP A, then use policy-based routing or Multi-Link outbound policies to send the guest subnet, for example 192.168.200.0/24, through ISP B. For the server, either leave it on the default ISP A path or add an explicit PBR rule for source 10.10.20.15 through ISP A's gateway. Ensure the NAT rules match the chosen egress: the server's static NAT must use 203.0.113.50 on ISP A, and guest NAT must use an ISP B address. The goal is deterministic egress per source or service so return traffic uses the same ISP and public address as the inbound flow. Verify that health checks and failover policies do not reintroduce per-flow load balancing for NATed services.
3. How would you verify the fix and monitor for recurrence?
Read the worked answer
Capture traffic on both external interfaces while testing HTTPS from an external client to 203.0.113.50. Confirm the SYN arrives on ISP A, the SYN-ACK leaves on ISP A with source 203.0.113.50, and the handshake completes. Check the connection log for the egress interface and NAT source address, and review the routing table to confirm there is no equal-cost default. Test guest traffic separately and confirm it uses ISP B with the correct NAT address. Add monitoring for link health, route changes, and asymmetric-flow symptoms, and document the PBR and NAT pairings so future ISP changes do not silently break the return path.