I85PREP
⌘ K
CNPA study materials
CNPA study resources

The Redundant Fibre That Took Down VLAN 20

CNPA · Put your knowledge to work in context.

All case studies

The Redundant Fibre That Took Down VLAN 20

Northgate Manufacturing runs three switches: SW1 at the distribution layer (root bridge, priority 4096) and SW2 and SW3 in the wiring closets. Every inter-switch link is an 802.1Q trunk carrying VLAN 10 (voice), VLAN 20 (office) and VLAN 99 (management). Last Thursday a contractor added a fibre trunk between SW2 and SW3 'for redundancy and faster convergence', then issued spanning-tree portfast default and no spanning-tree vlan 20 on SW3 because users had complained about a 30-second outage during earlier cabling work. Within two minutes VLAN 20 collapsed. SW3 CPU held at 97%, its MAC address table showed 340 entries for the same MAC alternating between two uplinks, and SW1 logged continuous topology-change messages. The VLAN 10 phones stayed registered throughout. The new fibre is cabled and cannot be pulled during business hours, and management wants VLAN 20 usable tonight without a maintenance window.

1. Explain the mechanism that took VLAN 20 down and why VLAN 10 was unaffected.

Read the worked answer

Three changes combined into a loop.

  1. The new SW2-SW3 fibre closed a physical triangle (SW1-SW2-SW3-SW1).
  2. no spanning-tree vlan 20 removed the only process that blocks a redundant path for that VLAN, so broadcast, unknown-unicast and multicast frames are flooded in both directions indefinitely.
  3. The classic signature follows: one MAC alternately learned on two ports, CPU consumed by forwarding and topology-change processing, and users failing at DHCP/ARP rather than seeing a clean link-down.

VLAN 10 survived because its spanning-tree instance was never disabled - the per-VLAN STP instance still had a port in blocking state, so the triangle was logically a tree. VLAN 20 had no instance, so both SW2 and SW3 forwarded on all ports.

spanning-tree portfast default made things worse but is not the root cause. PortFast only skips listening/learning on edge ports; applied globally it also lets trunk ports forward immediately after a link event, shortening the window before the storm saturates the CPU. The root cause is the missing STP instance, not PortFast.

2. Give the recovery sequence you would run tonight, the risk of each step, and the verification that the loop is gone.

Read the worked answer

Work from SW3's console - in-band Telnet/SSH is likely unusable at 97% CPU.

  1. Evidence first: show spanning-tree vlan 20 and show spanning-tree summary to confirm the VLAN is disabled. This proves what changed before you change it back.
  2. configure terminal, spanning-tree vlan 20, end. Risk: STP elects a root and blocks a port within seconds, so expect roughly 30-50 seconds of VLAN 20 outage. Announce it to the floor; it is far shorter than the ongoing storm.
  3. no spanning-tree portfast default, then apply spanning-tree portfast and spanning-tree bpduguard enable only on access ports facing PCs and handsets.
  4. Verify: show spanning-tree vlan 20 shows one root port and at least one Alternate/Blocking port; trunk interface rate counters drop to normal; show mac address-table | include <mac> returns a single port; SW3 CPU falls below 20%.
  5. Confirm SW1 still holds root priority 4096 and that show spanning-tree inconsistentports is empty.

3. Twelve months later, what design and monitoring controls would prevent this class of failure recurring without anyone noticing?

Read the worked answer
  • Root guard on distribution-facing ports: an access switch that starts advertising superior BPDUs is put in root-inconsistent state instead of stealing the root role.
  • Loop guard on non-designated and root ports: catches unidirectional fibre faults that would otherwise let a port age out its BPDUs and start forwarding.
  • BPDU guard plus PortFast on every edge port: a switch or rogue device plugged into a wall port is err-disabled immediately.
  • Storm control on access ports (for example 1% broadcast) as a last-resort circuit breaker.
  • Migrate to Rapid PVST+ for sub-second convergence; keep the same VLAN-to-instance discipline the standard template enforces.
  • Prune allowed VLANs on trunks so no VLAN rides a link it does not need.
  • Configuration control: forbid no spanning-tree vlan in the template, back up configs daily, and alert on config diffs.
  • Monitoring: SNMP trap and syslog alert on topology-change, root-guard and BPDU-guard events, plus CPU and broadcast-rate thresholds. A silent storm means the alerting failed, not just the switching.