BGP Route Present but Not Installed After a Static Route Removal
AS 65001 is a single-homed enterprise with two border routers, BR-1 and BR-2, each running eBGP to a different ISP. The core uses two route reflectors, RR-1 and RR-2, in the same cluster, with all iBGP sessions over Loopback0. A data-centre router, R5, is a client of both RRs and has no external links.
Last week BR-1's 10GE uplink was replaced. During the window the engineer removed an old static route on R5: ip route-static 203.0.113.0 255.255.255.252 10.255.0.1, where 10.255.0.1 is BR-1's loopback.
Since then, traffic from R5 to the partner prefix 198.51.100.0/24 fails, although the same prefix works from R6, another client. On R5, display bgp routing-table 198.51.100.0 shows the path learned from BR-1 with next hop 203.0.113.1, but the prefix is not installed in the IP routing table. R5's default route points to the core firewall, and there is no other route toward 203.0.113.0/30.
1. Why is the route in the BGP table but not in the RIB, and why did the removed static route hide the defect?
Read the worked answer
BR-1 learned 198.51.100.0/24 from its ISP over eBGP, where the next hop is the ISP peer address 203.0.113.1. When BR-1 advertises the prefix to its iBGP peers, the eBGP next hop is preserved by default; a route reflector also reflects the route without rewriting the next hop. R5 therefore receives a valid BGP path whose next hop is 203.0.113.1.
Before the route can be installed, R5 must resolve 203.0.113.1 recursively through the IGP or a static route. That address sits on BR-1's external /30 and is deliberately not carried in the IGP. The deleted static route to 203.0.113.0/30 via BR-1's loopback was the only resolution path, which is why the design appeared to work. Once it was removed, recursion failed, the path became invalid and VRP did not install it in the RIB. R6 still works because it retains a resolution path of its own.
The lesson is that a prefix in the BGP table is only a candidate. Next-hop reachability is a separate check, and a static route used as a next-hop crutch masks a missing peer next-hop-local on the border router.
2. Where exactly should next-hop-self be configured, and why is it wrong to configure it on the route reflectors?
Read the worked answer
Configure it on BR-1 toward both reflectors, inside the IPv4 unicast address family: peer 10.255.0.101 next-hop-local and peer 10.255.0.102 next-hop-local, assuming those are the RR loopbacks. BR-1 is the egress point and its loopback is reachable throughout the IGP, so R5 can resolve the next hop and install the route. Apply the same on BR-2 for its own ISP-learned routes.
Do not configure it on RR-1 or RR-2. The RR does not forward the traffic and may not even sit on the forwarding path. Rewriting the next hop to the RR would attract traffic for every reflected route to the control-plane device, creating a blackhole or a severe detour, and it changes the next hop for all clients rather than for the routes that actually leave through a border router. The RR's job is to reflect the originator's next hop unchanged.
Check the result from R5: display bgp routing-table 198.51.100.0 should now show next hop 10.255.0.1, and display ip routing-table 10.255.0.1 should resolve via OSPF. If your VRP release allows next-hop recursion through a default route, the prefix could be installed and silently blackholed at the firewall instead, so verify the recursion target explicitly.
3. What operational checks would you add to prevent a repeat, and which quick alternative is acceptable or not?
Read the worked answer
Add a verification step to the change template for any border-router maintenance:
- After the window, run
display bgp routing-table <critical-prefix>and confirm the route is marked as best and installed, not merely present. - Run
display ip routing-table <next-hop>for each external next hop to prove recursion succeeds. - Check both border routers, not only the one being worked on, because the RR can reflect the surviving path while a client still holds a stale or unresolvable one.
- Alert on prefixes whose best path has an unreachable next hop, and review any static route whose only purpose is to make an external next hop resolvable. Those are configuration smells that should become
peer next-hop-local.
Acceptable quick mitigation: temporarily restore the static route to the external /30 to restore service while the proper change is prepared. Not acceptable as a permanent fix: redistributing the external /30 into OSPF. It works until the link or addressing changes, and it can blackhole traffic by making a dead next hop look reachable. Also avoid relying on the default route for recursion; confirm the platform behavior rather than assuming it.