POS Timeouts After SD-WAN Cutover on a Dual-ISP Retail Estate

There are three interacting causes and they must be separated, because fixing only the obvious one leaves the failures in place.

1. Steering rule is wrong for the workload. 'Maximize bandwidth' with source-IP hashing does not evaluate link quality; it distributes sessions and can place a card authorization on a link already at 96% with 8% loss. POS is a latency- and loss-sensitive transactional workload, not a bandwidth-hungry one, so the rule is optimising the wrong variable.
2. Sessions are being moved or answered asymmetrically mid-flow. The handshake completes and the reply never arrives, while the DC sees return traffic on the opposite ISP. Verify whether fail-forward is enabled on the rule - that setting permits existing sessions to be re-routed when a link degrades, which breaks stateful inspection and produces exactly this half-open symptom.
3. The DC drops the asymmetric return. A reverse-path-forwarding check on the 601E is correct hygiene in general but fatal here, because the store's forward path and the DC's chosen return path disagree.

Order:

• First, stop the bleeding at the store edge: give POS its own best-quality rule with explicit SLA targets on loss and jitter, pinned to ISP1 as preferred, so no new authorization lands on a degraded link.
• Second, confirm and correct session behaviour so in-flight authorizations are never re-routed.
• Third, fix path symmetry at the DC.
• Fourth, absorb the general traffic the store still needs.
• Do all of it through FortiManager templates pushed to device groups, since no engineer is visiting a site.

Separate rules by business requirement, not by department. The estate has three genuinely different traffic classes and one that should be deprioritised.

• POS / card authorization: best-quality rule with health-check SLA targets matching what the payment processor tolerates (for example loss under 1%, jitter under 30 ms, latency under 80 ms), ISP1 preferred, ISP2 as fallback only when ISP1 breaches SLA. No load balancing across links ever, because the transaction is short, low-volume, and the cost of a retry at the till is a customer-visible failure. Tag the traffic with a DSCP value and enforce it at the DC head-end too, or the priority is lost in the tunnel.
• VoIP to store support: same best-quality logic but with tighter jitter and latency thresholds and a shaping guarantee, since one-way audio indicates the media path is competing with bulk traffic. Do not let VoIP and POS share a single rule with a shared SLA - you lose the ability to prove which one is breaching.
• General store traffic (back-office, updates, guest Wi-Fi): load-balanced across both ISPs, which is where 'maximize bandwidth' is actually appropriate. Cap guest Wi-Fi with a traffic shaper so it can never saturate ISP2 and push transactional traffic into breach.
• Bulk and backup: lowest priority, ISP2 only, shaped, scheduled outside the 11:00-14:00 peak.

Also shorten health-check intervals on the POS-relevant checks so a degrading link is detected within a couple of probe cycles rather than after several lost transactions. If checks are too slow, the rule cannot react before the till has already failed. Push it as a FortiManager template and roll it to two pilot stores outside peak hours before the estate-wide push.

The DC is where the fix can quietly create a security regression, so treat it carefully.

Check first: whether both ISP tunnels land on the active 601E, whether the HA pair is synchronising sessions, and whether the return path is chosen by a static route or by SD-WAN at the head-end. Many estates fix the branch and forget that the DC has its own SD-WAN rules, which can select a return path inconsistent with the branch's forward path. Aligning both ends is the clean fix and costs no security posture.

Options, with risk:

• Align SD-WAN policies at both ends so return traffic follows the same link as forward traffic. Preferred. No weakening of the firewall's antispoofing logic; requires change control and testing during the 11:00-14:00 window you are trying to fix, so use a canary store.
• Tune or disable reverse-path-forwarding on the affected interface only. Fast, but it removes a genuine antispoofing control on a PCI-relevant path. If you do it, scope it to the specific SD-WAN tunnel interfaces, log it, and put a compensating control in place - otherwise you have traded a timeout problem for a spoofing exposure that the next assessor will find.
• Force all POS to a single ISP at the branch and a single return path at the DC. Simplest and safest for the transaction flow, but removes redundancy for that traffic class unless the fallback is tuned and tested.

Given PCI scope, take the first option, and only accept the second as a documented, time-boxed mitigation with an expiry date. Report one-way audio separately - it likely shares the same root cause but needs its own SLA evidence.