BGP Path Selection Across Mixed Transports
A financial services firm runs an SD-WAN overlay over three underlays: MPLS (20 Mbps, 15ms RTT), business broadband (500 Mbps, 30ms RTT), and 5G (300 Mbps, 40ms RTT, variable). The overlay runs eBGP between edges and a regional route reflector. A critical trading application must prefer MPLS for latency, but a bulk data replication job must prefer broadband for capacity. Today both applications hash across all three transports because the BGP best-path selection sees equal MED and equal local preference on all three tunnels. Ops reports the trading app occasionally takes 5G and sees 200ms+ spikes, and replication saturates the 20 Mbps MPLS circuit weekly.
1. Explain why standard BGP best-path selection is not steering these applications correctly, and describe two overlay-native mechanisms that would.
Read the worked answer
BGP best-path selection is per-prefix, not per-application. All three tunnels advertise the same prefix with equal attributes, so the router installs one best path (or ECMP if multipath is enabled). With ECMP, all traffic - trading and replication - hashes across all transports. BGP has no concept of "this flow is latency-sensitive."
Two overlay-native mechanisms:
- Application-aware routing (AAR) / SLA-based path selection: define per-application SLA profiles (e.g., trading: latency < 20ms, loss < 0.1%; replication: throughput priority) and let the overlay continuously probe each transport and steer flows to the transport meeting the SLA. This is dynamic and reacts to 5G jitter.
- Policy-based traffic steering with per-app path preference: classify traffic (by app signature, DSCP, or 5-tuple) and bind each class to an ordered transport list (trading: MPLS → broadband → 5G; replication: broadband → 5G → MPLS). This is deterministic and complements AAR.
A third option: per-tunnel BGP attributes (local preference or MED) to bias best-path, but this is static and cannot react to 5G variability - it would fix the symptom for trading but not the dynamic spikes.
2. Design the policy that satisfies both applications and explain how you prevent replication from saturating MPLS even during a failover.
Read the worked answer
Trading app policy:
- Classify by app signature / DSCP EF.
- AAR profile: latency ≤ 20ms, loss ≤ 0.1%, jitter ≤ 5ms.
- Preferred order: MPLS → broadband → 5G.
- Hard exclusion of 5G unless both MPLS and broadband fail - 5G's variable latency disqualifies it for this app.
- If no transport meets SLA, alert rather than silently degrading.
Replication policy:
- Preferred order: broadband → 5G → MPLS.
- Rate-limit on MPLS to a small fraction (e.g., 2 Mbps of the 20 Mbps) so even if it fails over, replication cannot starve trading or control traffic.
- Schedule large jobs outside market hours where possible.
Preventing MPLS saturation:
- Per-tunnel shaper on MPLS reserving bandwidth for trading + control (e.g., 8 Mbps reserved, 12 Mbps shared).
- QoS queues: EF for trading, AF for control, BE for replication with a hard cap.
- AAR re-evaluates every probe interval; if MPLS latency rises due to queueing, trading can shift to broadband automatically - but only if broadband meets the 20ms SLA, which at 30ms RTT it may not. This is why the reservation matters more than the failover path.
3. The 5G transport shows 40ms average RTT but 200ms p99 spikes. How should the SLA probe be configured so the trading app is not fooled by the average?
Read the worked answer
Averaging hides tail latency. Configure the probe to evaluate p99 (or max) latency and jitter, not mean, over a rolling window (e.g., 30–60s). Set the trading SLA on the worst-case metric: latency p99 ≤ 20ms and jitter p99 ≤ 5ms. 5G at 40ms mean fails the p99 ≤ 20ms test outright, so it is excluded regardless of spikes.
Also:
- Use short probe intervals (e.g., 1s) with enough samples to make p99 meaningful.
- Include loss and MOS in the SLA, not just latency.
- Set hysteresis / dampening so a single good probe window does not flip the path back to 5G.
- Log probe history so post-incident review can show why a path was or was not selected.