Stateful Firewall Insertion with Asymmetric Return Traffic in ACI
A media company runs a Cisco ACI fabric with three spines and six leaves. A three-tier web application is being migrated from a legacy VLAN design. Security requires all client-to-web traffic to pass through a pair of stateful firewalls, and all web-to-database traffic to pass through the same firewalls. The firewalls are deployed in routed mode with two interfaces each and do not support PBR. The application uses a load balancer VIP in the DMZ EPG. The current design uses a service graph with PBR and one-arm firewall, but about 40% of TCP sessions reset after 10–30 seconds. The firewall logs show out-of-state drops. The team wants to keep the firewalls in routed mode and avoid any traffic tromboning between sites.
1. Explain why the TCP sessions reset and recommend a service-graph and contract design that preserves stateful inspection without firewall PBR.
Read the worked answer
The resets and out-of-state drops indicate asymmetric routing. In a one-arm PBR service graph, ACI redirects client-to-web traffic to the firewall, but the firewall's return path may be routed directly to the client or to the web EPG without traversing the same service node. The firewall sees only one direction and drops the flow as out of state. Do not try to fix this with firewall PBR; the firewalls do not support it. Use a two-arm service graph with distinct inside and outside interfaces and bridge domains, and attach contracts so the firewall is the only path between consumer and provider. Redirect traffic at the ACI leaf to the firewall inside IP, and make sure the web-to-database contract also traverses the same service graph. If the firewall pair is active/standby, use a service-node anycast address or a symmetric PBR hash so both directions land on the same firewall. Validate with contract statistics, interface counters on both firewall sides, and a packet capture that shows both directions of the TCP handshake.
2. How would you verify symmetry and prevent traffic from tromboning between sites after the redesign?
Read the worked answer
Use show service graph and endpoint tables to confirm that the consumer and provider endpoints resolve through the intended service node, and check that the PBR policy is applied on the leaf closest to the consumer. Capture counters on both firewall interfaces; both directions of a flow should increment on the same firewall. Run a traceroute or use ACI contract statistics to confirm the path does not leave the site and return. Keep the firewall and load balancer in the same site as the web EPG, and avoid stretching the web or database bridge domain across sites. If disaster recovery requires a second site, use a separate service graph there rather than backhauling traffic through the primary firewall pair. Finally, test with long-lived TCP sessions and a firewall failover to confirm that state synchronization and anycast service-node behavior keep the flow symmetric.