Dual-Homed Enterprise: MED, Prepending and the Policy Applied in the Wrong Direction

MED is a hint from one AS to its neighbouring AS about which of its entry points to use. It has two hard limits that decide this case.

First, MED is not transitive. The value the CE sets toward ISP-B stops at ISP-B. It is never carried to ISP-A, never carried to any tier-1 above ISP-B, and never reaches the networks whose traffic the enterprise wants to steer. The engineer's mental model - 'a lower number means the internet prefers me here' - does not correspond to any field in any BGP update that the distant ASes will examine.

Second, MED is only compared among paths learned from the same neighbouring AS (unless always-compare-med is enabled, which is not interoperable practice). Toward ISP-B's routers there is exactly one path to the farm, the one advertised by the CE. There is nothing else to compare, so the value 100 is inert - ISP-B's internal decision is made by its own IGP cost, its own local preference and hot-potato routing. The MED only becomes meaningful when the same provider has two entry points to choose between, which is not the case here.

What actually decides inbound is AS-path length as seen by each receiver, and local preference inside the receiving AS - and local preference sits above AS-path length. ISP-B's edge routers almost certainly set a high local preference for routes from their own access customers, so those customers' traffic egresses via their direct ISP-B attachment regardless of what the enterprise prepends. That is normal, defensible provider policy, not a failure of the customer's configuration. MED cannot override it; the only lever that can is an ISP-B policy change.

The policy was attached as an import policy on the ISP-A session, and import governs what the CE accepts and prefers, not what it sends. Nothing the CE advertises changed, so inbound was unaffected. What did change is the CE's own outbound decision: routes learned from ISP-A arrived with three additional AS-path hops, so the ISP-B route - previously maybe a tie or a close second - became the best path. Outbound moved to the ISP-B link, which then ran at 95%. The engineer's diagnosis and the observed symptom are consistent with each other: prepending on an import policy is an outbound traffic-engineering tool applied by accident.

The rule: an export policy attached to a peer controls what you advertise to that peer, which steers traffic toward you - inbound. An import policy controls what you accept and how you rank it, which steers traffic away from you - outbound.

Correct application in SR OS: on the CE, attach an export policy to the eBGP session toward ISP-B, matching the farm's prefix(es) and applying an as-path prepend of 65001 three times. Equivalent result, for ISP-B's session specifically. Keep the match tight - a prefix list of the advertised aggregates, not 'everything' - because an export policy on a session with two providers is one typo away from advertising a provider's routes back into the other provider, which is the classic route-leak incident. Also confirm the prepend is long enough to matter at the observation points you care about and short enough not to look like a hijack attempt; three to five hops is conventional.

If the intent is ISP-B as strict standby, consider not advertising the farm to ISP-B at all, or advertising a summary only - far more deterministic than any amount of prepending, and it removes the 60% inbound from ISP-B's customers entirely.

Inbound (what the CE advertises to each ISP)

• Toward ISP-A: advertise the farm prefix with no prepend, no MED games.
• Toward ISP-B: either do not advertise the farm, or advertise it with three to five prepends for standby use. Remove the MED values entirely; they add nothing between two different providers and create a false sense of control.
• Do not advertise anything learned from either provider to the other. Explicit export policy, deny by default.

Outbound (what the CE accepts and prefers)

• Import policy on the ISP-A session: local preference 200 on ISP-A's routes; on ISP-B's session, local preference 100. Local preference is the correct tool because it is local to the CE and affects the CE's choice directly, unlike MED which is a hint to someone else.
• Consider preferring ISP-A for the farm's return-path symmetry: if outbound leaves via ISP-B while inbound arrives via ISP-A, a stateful firewall will drop flows. Local preference is again the mechanism, plus a check on whether the provider's own return path agrees.
• Keep a default route from both, or accept full tables if the hardware supports it - but set import limits and max-prefix on both sessions.

What to ask the ISPs

• Ask ISP-A for a customer community that sets a high local preference for the farm prefix across its network. This is the only mechanism that reliably steers ISP-A's own customers and its downstreams, and it is a contract/policy matter, not a configuration one.
• Treat ISP-B's offer to 'set a lower local preference' with care: ask exactly what it means, on which routers, for which prefixes. If it is a lower preference for the customer route inside ISP-B's AS, it does help inbound from ISP-B's customers - but it is a manual intervention that is undocumented in the customer's own configuration and will not survive a provider migration.
• Ask both for BFD support on the eBGP sessions and for a documented maximum-prefix policy on the customer side.

The judgement: MED does not cross AS boundaries, local preference is not transitive, and prepending is a blunt but honest tool. Design with the tool that operates at the layer where the decision is actually made.