VXLAN BUM RPF Failure After Adding a Spine
An 18-building campus has 18 leaf switches, 3 border leaves, and 2 spines. VXLAN BGP EVPN serves 2,200 access points and 60 wireless VLANs. The underlay is OSPF, and VXLAN BUM traffic uses PIM-SM with an anycast RP configured on border-01 and border-02. After a third spine is added for ECMP, wireless clients in VLAN 210 on building-7 cannot obtain DHCP addresses during the morning join burst. Unicast traffic to the DHCP server is fine. On the building-7 leaf, display pim rpf shows the RPF neighbor for the anycast RP address as the new spine, while the RP set is still on the border leaves. The (*,G) entry exists but no (S,G) is built, and BUM counters show drops. The team proposes moving the RP to the new spine and disabling PIM on the old path to simplify the topology.
1. Why does the new spine break DHCP for VLAN 210 even though unicast to the DHCP server still works?
Read the worked answer
Unicast works because the route to the DHCP server follows the normal IGP path. DHCP discover is broadcast and is carried as VXLAN BUM traffic, which depends on multicast replication in the underlay. PIM-SM builds the shared tree toward the anycast RP and performs an RPF check on the incoming interface. Adding the third spine changed the IGP cost to the anycast RP loopback, so building-7 leaf now resolves the RPF neighbor as the new spine. If that spine is not a proper anycast RP member, lacks the MSDP session, or has no (,G) state, the RPF check fails and the BUM packet is dropped. The (,G) join can still exist while no (S,G) is built because the source registration path is broken. The proposed change to move the RP to the new spine and disable PIM on old paths would make the failure worse by creating a second RP inconsistency. The fix is to restore consistent anycast RP reachability and MSDP mesh, or move BUM to ingress replication if the fabric scale permits it. Check display pim rpf, display msdp peer-status, and display multicast routing-table.
2. Design a resilient VXLAN BUM and anycast RP design, and list the verification steps you would use.
Read the worked answer
Make the anycast RP address a stable loopback reachable with equal cost from every leaf, and run the same RP address on at least two border leaves with a full MSDP mesh between them. Keep PIM-SM enabled on every spine that can be an RPF next hop; do not disable PIM on a spine just because it is new. For VXLAN BUM, verify that the VNI uses the intended multicast group or ingress replication, and that the group range is permitted by PIM. If the campus is below the scale where multicast is needed, ingress replication removes the RPF dependency but raises head-end replication load. Verification: on building-7 leaf check display pim rpf for the RP and group, display multicast routing-table for (*,G) and (S,G), display vxlan vni BUM mode, and display msdp peer-status on both RPs. Trigger a DHCP renew from one client and confirm BUM counters increment and the address is assigned.