Dual-ISP ECMP Causes Asymmetric Return Traffic
An FTD 2130 pair in HA at a distribution site has two outside interfaces: ISP-A 198.51.100.2/24 and ISP-B 203.0.113.2/24. Both providers advertise a default route, and the FTD has two equal-cost static defaults with the same administrative distance. Inside users are on 10.90.20.0/24. After adding the second ISP, HTTPS sessions to several SaaS providers fail intermittently; some users can log in but large downloads stall. Packet captures show the client SYN leaving ISP-A, while the SYN-ACK arrives on ISP-B. The FTD logs show no matching connection and TCP packet outside window drops. NAT is configured as interface PAT on both outside interfaces; the ACP is Allow with IPS. The customer wants ISP redundancy but not intermittent application failures. No dynamic routing is configured.
1. Explain the failure mechanism using the captures and drop counters.
Read the worked answer
The FTD is stateless at the routing boundary but stateful for inspection. With two equal-cost defaults, ISP-A and ISP-B are both selected per flow by ECMP, but the return path from the SaaS provider is chosen by the Internet, not by the FTD. A client SYN can leave via ISP-A and create a connection with NAT on ISP-A, while the SYN-ACK arrives on ISP-B. The FTD has no matching flow on ISP-B, and the packet is dropped as no matching connection; later packets may also fail TCP window checks. The captures showing SYN on A and SYN-ACK on B are the direct evidence. This is not an IPS false positive or an ACP block. Interface PAT on both ISPs makes it worse because the translation is tied to the egress interface. The intermittent pattern fits flows that hashed to one ISP while return traffic preferred the other; small requests may succeed while large downloads stall.
2. What change would provide ISP redundancy while avoiding asymmetric flows?
Read the worked answer
Remove ECMP for outbound traffic. Use a primary default route via ISP-A with an IP SLA or tracking object, and a floating static default via ISP-B with a higher administrative distance. That gives redundancy without per-flow load sharing. If outbound load sharing is required, use policy-based routing to assign specific inside subnets or applications to a specific ISP, and NAT those subnets to that ISP's address; do not mix ECMP and interface PAT. Ensure return traffic for each NAT address comes back to the same ISP. For inbound services, use separate public addresses and provider-independent policy. Test the design by shutting ISP-A and confirming that new flows use ISP-B while existing flows fail and recover as expected. The goal is deterministic path selection per flow, not equal-cost multipath.
3. How would you verify the fix and monitor it after deployment?
Read the worked answer
Verify with FMC packet tracer and simultaneous captures on ISP-A and ISP-B while reproducing a large download. Check show route for a single active default plus the floating static, show asp drop for no-matching-connection and window drops, and show failover for HA health. Use synthetic probes through each ISP and confirm the source address and return path. Monitor tracked route state and syslog or SNMP alerts for ISP-A failure. Test failover during a maintenance window: shut ISP-A, confirm the tracked route withdraws, new connections use ISP-B, and NAT uses the correct interface address. After restoring ISP-A, confirm the primary returns without breaking existing sessions. Keep the captures and drop-counter baselines so a future intermittent complaint can be compared against a known-good asymmetric-free state.