Slow-Drain and ISL Oversubscription in a Core-Edge Fibre Channel SAN
A financial services firm runs two independent SAN fabrics, A and B. Each fabric has two Cisco MDS 9396S core switches and four edge switches. Each edge switch connects 12 hosts with dual 32-Gb FC HBAs; each host zones one HBA to Fabric A and one HBA to Fabric B. The core connects to four all-flash arrays, each with eight 32-Gb ports. Each edge-to-core link is a 4-port 32-Gb port channel, and each core-to-array link is a 4-port 32-Gb port channel. Monitoring shows transmit wait time and credit-zero counts rising on edge ports connected to backup hosts, and ISL utilization spikes to 85% during backup windows. Zoning currently places all initiators in one zone per array, with all targets. The team wants to add eight more backup hosts and is considering replacing the 4-port ISLs with 8-port ISLs.
1. Diagnose the likely cause of the rising transmit wait and credit-zero counters, and recommend remediation before adding more hosts.
Read the worked answer
The pattern points to slow-drain behavior from a backup host, not simply a lack of ISL bandwidth. When a host stops returning buffer credits, the edge port's transmit queue fills, and the congestion can propagate over the ISL to unrelated hosts. Verify with show interface fc1/1, show port-monitor, and credit-loss counters, and identify which host is holding frames. Replace the all-initiator, all-target zone with single-initiator, single-target or small single-initiator zones to reduce RSCN scope and unnecessary traffic. Isolate backup hosts on dedicated edge switches or a separate VSAN, and consider QoS or rate limiting for backup traffic. Check the HBA queue depth and NPIV queue depth on the backup hosts; an undersized queue can create the same symptom. Do not start by adding ISL ports only; if the slow-drain device is not fixed, more ISL bandwidth just spreads the congestion.
2. Evaluate the proposed 8-port ISL upgrade and state whether it is sufficient for the added backup hosts.
Read the worked answer
Current edge oversubscription is 12 hosts times 32 Gb, or 384 Gb, against a 4-port 32-Gb ISL of 128 Gb, or 3:1. Replacing the 4-port ISL with an 8-port ISL doubles capacity to 256 Gb and reduces oversubscription to 1.5:1. That is a reasonable target for backup bursts, but only if the port channel is built within the correct port-group or ASIC limits on the MDS 9396S and if the core-to-array side can absorb the traffic. The arrays have eight 32-Gb ports each, but the current 4-port core-to-array port channel limits each array to 128 Gb; consider 8-port array connections before blaming the edge. Keep Fabrics A and B separate, confirm that the added hosts are dual-homed, and validate with port-monitor and ISL utilization after a pilot backup window.