CCSE (EC-Council Certifications): ISP Redundancy Load Sharing Causes Out-of-State TCP

The symptom is asymmetric routing caused by per-connection link selection. Load sharing can choose the backup link for a new egress flow even though the inbound SYN arrived on the primary link. The return packet leaves the wrong interface, the firewall sees a state mismatch, and the client or server reports out-of-state TCP. The equal-metric default routes make the behavior less predictable.

For stable inbound mail and VPN, High Availability mode is the safer recommendation. It keeps one active link and makes the return path consistent. The business can still use both links by putting non-critical outbound browsing in a separate design, but the cluster should not load-share the addresses that terminate inbound services and VPN.

If load sharing must remain, it has to be paired with sticky connection handling and explicit link selection, not just equal default routes. Even then, inbound services are harder to support because each public address must be consistently reachable. High Availability removes that ambiguity and is easier to troubleshoot under incident pressure.

Separate the traffic classes. For outbound browsing, keep load sharing only if sticky connections are enabled for the services that need session persistence, and confirm that return traffic follows the same link. For inbound mail, publish a single stable address on the active link, or use DNS with a health check that removes a failed link; do not rely on clients randomly choosing between two MX addresses. Ensure the static NAT rule for 10.0.0.25 matches the address actually used and that the corresponding security rule is installed on the cluster.

For VPN, set Link Selection on the cluster to use the primary public address as the main address and the backup as the backup address. Bind the VPN community to that selection so IKE and IPsec use the stable VIP rather than a physical member address. Enable dead peer detection and confirm the remote-access clients use the DNS name, not a hard-coded IP.

Verify the routing table before and after changes. If policy-based routing is used, make sure it does not override the link chosen by ISP Redundancy. Test inbound mail and VPN from outside the network after each change; a successful outbound browse does not prove inbound symmetry.

Use a maintenance window and a canary group. Create a small VPN test group with a few users on a separate client profile, and keep the majority on the production profile. Identify a low-traffic period and notify the service desk. Test inbound mail with an external probe that checks SMTP banner and TLS certificate, not just port 25 reachability.

Before failover, record the active link, VPN session count, and a baseline fw monitor capture for one test user and one mail flow. Disable the primary link or remove its route in a controlled way, then measure how long the canary VPN user takes to reconnect and whether mail continues on the backup. Check cphaprob and the cluster state throughout. Do not reboot both members or change both links at once.

After the test, restore the primary link and verify that new sessions use the intended path. Review logs for out-of-state errors and IKE retransmits. If the canary group is stable, expand the test to a slightly larger group before the next full failover. Keep a rollback step that restores the previous ISP Redundancy mode and routing if the test degrades production.