MC-LAG ICL Failure Leaves Orphaned Server Stranded
Two QFX5120-48Y switches, leaf-01 and leaf-02, provide MC-LAG to 48 bare-metal servers in a legacy rack. Each server dual-homes with LACP active-active. The leaves are joined by a 2x100G ICL configured as ae99, and ICCP runs over the management network. leaf-03 is single-homed to leaf-01 and hosts the rack's out-of-band management. During a maintenance window, both ICL members are removed to replace optics. The MC-LAG LAGs remain up, but servers attached to leaf-02 lose all connectivity, including to their default gateway, which is an anycast IRB configured on both leaves. leaf-01 continues forwarding. ICCP logs show peer state transitions. No EVPN is deployed in this rack.
1. What is the most likely cause of the leaf-02 server outage, and which commands would confirm it?
Read the worked answer
MC-LAG provides Layer 2 access redundancy, not a routed northbound path. If leaf-02's only route to the rest of the data center is through the ICL, removing ae99 isolates leaf-02's IRB even though the access LAGs stay up. Servers on leaf-02 still ARP for the anycast gateway and get a local MAC, so they believe the gateway is reachable; leaf-02 then has no path to the destination and drops the traffic. Confirm with show iccp, show interfaces ae99, show mc-lag interfaces, show route 0.0.0.0/0, and show route forwarding-table on leaf-02. You should see the ICL down and the default or spine routes pointing to ae99 or missing. Also check show log iccp for peer state and split-detection messages. Do not confuse this with LACP failure: if the access LAG were down, the server would lose link, but here the server link is up and only the northbound path is gone.
2. Would configuring no-split-detection help? Explain the effect on the data path and on loop prevention.
Read the worked answer
no-split-detection would not restore the northbound path. It tells the MC-LAG peers not to disable LAG links when the ICL fails, which can keep both peers active and create a split-brain risk: duplicate anycast gateway behavior, stale MAC learning, and possible loops if there is any other L2 path. It might make the server's LACP links stay up, but leaf-02 would still have no route to the destination, so the outage remains. Split detection is a loop-prevention mechanism, not a redundancy mechanism for routing. Use it only when the peer keepalive path is more reliable than the ICL and the design intentionally keeps both peers active; it does not replace an independent uplink. The correct fix is to give leaf-02 its own Layer 3 uplink to the spine, or move the rack to an EVPN ESI design where each leaf has an overlay uplink and the anycast gateway is backed by EVPN Type-2 and Type-5 routes.
3. What migration plan to EVPN ESI would you propose, and what must be validated before cutover?
Read the worked answer
First validate that the underlay can support EVPN: BGP reachability between VTEP loopbacks, MTU for VXLAN, and a plan for the anycast gateway. Pre-stage EVPN on leaf-01 and leaf-02 with a VLAN-aware bundle, the same ESI on the server LAG, and the same LACP system ID. Do not run MC-LAG and ESI on the same LAG as a long-term state; if both are active, the leaves may disagree about DF election and MAC ownership. Migrate one server or one LAG first: shut the MC-LAG member on one leaf, convert the LAG to ESI, bring up EVPN, and verify Type-2 and Type-4 routes, DF election, and traffic. Then repeat. Give leaf-02 a real uplink before cutover; otherwise the new design repeats the same single-path failure. Validate with show evpn instance, show evpn database, show route table evpn, and a bidirectional ping and traceroute from the server. Keep a rollback that restores the MC-LAG config and the ICL.