Three Paths for 10.40.0.0/16: Local Preference Beats a Better IGP Cost
AS 64500 runs SR OS PEs with a pair of iBGP route reflectors. At 09:12, PE-2 (system address 10.0.0.2) holds three paths for 10.40.0.0/16: (1) eBGP from AS 64510 at 10.1.1.1 - local preference 100, AS path '64510 64520', MED 50, next hop reachable at IGP cost 10; (2) iBGP from RR-1 - local preference 150, AS path '64520 64530', MED 100, next hop 10.0.0.9 at IGP cost 30; (3) iBGP from PE-7 - local preference 150, AS path '64520 64530', MED 100, next hop 10.0.0.7 at IGP cost 120.
Design intent was that the eBGP path be primary and the two iBGP paths backups. In practice PE-2 forwards toward 10.0.0.9 while the customer's return traffic arrives on the 10.1.1.1 link, so a stateful firewall between PE-2 and the customer drops sessions at random. Two engineers disagree: one insists the shorter AS path from PE-7 should have won, the other insists MED 50 on the eBGP path should have won. Neither explanation survives the algorithm.
1. Which path does PE-2 install, and at which decision step is each losing path eliminated? Explain why both engineers' arguments are wrong.
Read the worked answer
Installed path: the one from RR-1, next hop 10.0.0.9.
Walking the order once the path is valid (next hop resolvable):
- Highest local preference. eBGP is 100, both iBGP paths are 150. The eBGP path is eliminated here. Everything after this step is irrelevant to it - its better IGP cost (10) and lower MED (50) never get consulted, which is why the MED argument is wrong. MED sits below AS-path length and below local preference in the chain.
- Shortest AS path. The two surviving paths are identical ('64520 64530', length 2). No decision. Note that the AS path from PE-7 is not shorter - the engineer misread the path, or assumed the peer that is 'closer' in the topology must have the shorter path.
- Lowest origin. Both iBGP paths are IGP-origin. Tie.
- Lowest MED. Both carry MED 100, and both were learned from neighbor AS 64520, so the comparison is permitted (MED is only compared among paths from the same neighbor AS unless always-compare-med is on). Tie.
- eBGP over iBGP. Both are iBGP. Not applicable.
- Lowest IGP cost to next hop. 30 versus 120. RR-1 wins and the route is installed.
The lesson: a two-line policy that sets local preference 150 on iBGP routes silently overrides every IGP-cost and MED argument in the network, and it is invisible to anyone reading only show router bgp routes output on the border router.
2. A change is wanted so that the eBGP path becomes primary at PE-2 without destroying the intent of the existing iBGP policy. Compare the realistic options and recommend one.
Read the worked answer
Options:
Raise local preference on the eBGP path. Local preference is only meaningful on the router that holds the competing paths, and it is not transitive over iBGP. If the policy is applied only on PE-2, PE-2 switches to the eBGP path but the rest of the AS may still prefer something else, and the asymmetry simply moves. The robust version: at the ingress PE (PE-2), attach a community such as 64500:100 to routes learned from AS 64510, then apply an import policy on every PE that matches that community from iBGP and sets local preference 200. Now every router makes the same decision for the same reason.
Lower the iBGP local preference from 150 to 100. Check what the policy was for. If it exists to prefer reflector-learned paths for a specific set of prefixes, scope it with a prefix list rather than deleting it; otherwise you will fix this prefix and break a dozen others that relied on the 150.
Per-neighbor preference override. SR OS allows a preference value per neighbor, but it is a blunt instrument: it applies to every prefix from that peer, in every address family, and it interacts with the protocol-preference comparison rather than the local-preference step. Easy to get wrong and hard to audit later.
Recommendation: community-mark at ingress plus a consistent local-preference import policy everywhere. It is auditable, survives new PEs, and keeps the decision in one place.
3. At 14:40 the IGP loses its route to 10.0.0.9. What does PE-2 do, what does it advertise to its iBGP peers, and what could still black-hole traffic?
Read the worked answer
PE-2 re-runs the decision. The RR-1 path is no longer valid because its next hop cannot be resolved, so it is dropped from consideration before the comparison even starts. PE-2 falls to the PE-7 path (local preference 150, next hop 10.0.0.7, IGP cost 120) and installs it. Forwarding shifts to PE-7, which must itself still have a valid route for 10.40.0.0/16 - if PE-7's own path also depended on 10.0.0.9, the two routers can end up disagreeing about who owns the traffic, and the packets loop between them until TTL expires. Check both routers' installed path, not just PE-2's.
The black-hole risk sits in how the eBGP path is advertised. If PE-2 ever becomes the best exit for the prefix (for example after both iBGP paths fail), it advertises the route over iBGP with the next hop unchanged at 10.1.1.1. That address is on an external point-to-point link attached to PE-2 and is normally not in the IGP, so every other PE receives the route, marks it invalid, and installs nothing - or worse, installs it if a leaked connected route exists and forwards to an address that only PE-2 can reach. Either redistribute the eBGP next-hop address into the IGP, or configure next-hop-self on the iBGP sessions so the next hop is PE-2's own system address. Verify with show router bgp routes 10.40.0.0/16 detail on several PEs and compare the next-hop column; a route present on the border but absent on the far side is this failure, not a policy problem.