HA Checksum Mismatch, Slow Failover, and Voice Session Loss
A financial customer runs two FortiGate 1800F in active-passive HA, FortiOS 7.2.8, 40 VDOMs and 1,200 firewall policies. Traffic peaks at 8 Gbps and 25,000 concurrent sessions. After a policy change on the primary, the secondary reports out of sync; diagnose sys ha checksum cluster shows mismatches in several VDOMs. During a planned link-monitor test, failover takes 25 seconds and 30% of VoIP calls drop even though session pickup is enabled. The secondary also logs repeated HA link flapping on port1. The HA heartbeat and data traffic share port1 on some VDOMs; there is no dedicated heartbeat link. The customer has a change freeze in 48 hours and cannot accept a full outage. The team wants to avoid a full configuration sync during business hours and needs to reduce failover impact.
1. How would you diagnose the HA checksum mismatch and safely restore synchronization without causing a production outage?
Read the worked answer
First, do not run execute ha synchronize start blindly; it can push a large configuration while traffic is live and cause a longer outage. Identify the scope: diagnose sys ha checksum cluster shows which VDOM and table differ; diagnose sys ha checksum show gives details. Then compare the primary and secondary with get system status, diagnose sys ha status, and the relevant show commands. A mismatch after a policy change often means the secondary did not receive the full configuration because HA sync was interrupted, a VDOM was locked, or an object referenced a per-device value that cannot sync. Check diagnose sys ha status for the last sync result and whether the secondary is in standby or working state. If the mismatch is limited and understood, re-save the changed policy on the primary and watch the checksum recalculate. If the secondary is truly stale, schedule a maintenance window and force a sync, then verify checksums and run a failover test. In 40 VDOMs, also check for configuration intentionally unsupported in HA, such as per-device interface IPs in a VDOM that is not part of an HA virtual cluster. Fix the root cause; repeated forced syncs hide a broken sync path.
2. Why are VoIP calls still dropping despite session pickup, and what would you change in HA and session settings?
Read the worked answer
session-pickup alone is not enough for voice. It synchronizes TCP sessions; RTP is UDP, so enable session-pickup-connectionless as well. For SIP expectations, enable session-pickup-expectation; if NAT is in the path, session-pickup-nat may be needed. But 30% call loss after a 25-second failover is more likely a failover-time problem than a session-table problem: calls that are mid-setup, or whose media traverses a different path, cannot be saved. Reduce failover time with consistent heartbeat/hello timers and link-monitor settings; avoid a link monitor that requires several external probes before declaring failure. Check whether voice traffic is offloaded to NP7; not all offloaded sessions behave the same during failover, so test with the actual codec and call model. Align the branch and SIP provider's session timers with HA. Use a maintenance window to measure: start a call, fail the primary, capture diagnose sys session and diagnose sniffer packet on the secondary. If calls still drop, the design may need active-active voice gateways or a SIP load balancer rather than relying on stateful HA for RTP.
3. What is causing the link-monitor and HA link flapping, and how should the monitor and heartbeat be reconfigured?
Read the worked answer
Link-monitor flapping is a design signal: a single monitored target is too fragile. Configure the monitor with multiple servers, a source IP in the monitored subnet, and sane interval, failtime, and recoverytime; do not set failtime to 1. If you monitor the internet, monitor the ISP gateway plus two reliable anycast targets, and require more than one failure. Ensure the monitored interface is the one that actually carries the traffic and that the route to the target does not itself depend on the link being up. For HA, check ha-priority and override; if the secondary has a higher priority and override is enabled, it can preempt. Disable override unless priority-based preemption is truly wanted. Also check port1 flapping: a bad SFP, cable, or speed/duplex mismatch can cause HA heartbeat loss, which is worse than a link-monitor failure. Use a dedicated HA heartbeat interface and verify diagnose hardware deviceinfo nic port1 for CRC/errors. Then test failover with a controlled link-monitor failure, not by pulling the heartbeat. Keep the monitor conservative enough to avoid false positives but fast enough for the business SLA.