Networking and Connectivity
Networking and Connectivity tests the data-centre underlay and overlay: VLANs and trunks, link aggregation, MTU, spine-leaf oversubscription, storage transports such as FC, iSCSI and NVMe-oF, and the places where latency rather than bandwidth is the real constraint.
Core rules
- Jumbo frames (MTU 9000) must match end to end: host NIC, virtual switch, every physical hop, and the storage target. One 1500-byte hop silently fragments or drops, and the symptom is slow iSCSI rather than a clean failure.
- Link aggregation (LACP) needs matching configuration at both ends and, on stacked or MLAG pairs, a correct multi-chassis group. Dual-homing a host without MLAG or stacking creates a spanning-tree loop.
- Leaf-spine gives predictable east-west latency: every leaf is one hop from every other leaf, and oversubscription is set by the leaf uplink count.
- Fibre Channel zoning: single-initiator, single-target zones are the defensible default; zone by WWPN rather than physical port so a controller replacement does not force re-zoning.
- iSCSI: use CHAP, a dedicated VLAN or separate physical fabric, and MPIO with per-path configuration instead of relying on a single session.
Oversubscription arithmetic
Oversubscription = host-facing (south) bandwidth / uplink (north) bandwidth. A 48-port leaf with 24 x 25 GbE host ports and 4 x 100 GbE uplinks gives 600 Gbps south and 400 Gbps north, i.e. 1.5:1. Lower is better for storage-heavy east-west traffic; 3:1 is a common general-purpose ceiling.
Worked example
A design must support 24 hosts at 25 GbE per leaf with no more than 2:1 oversubscription. South bandwidth = 600 Gbps. At 2:1, north must be at least 300 Gbps, so three 100 GbE uplinks satisfy it and four deliver 1.5:1. The reasoning to show: convert ports to bandwidth, divide south by north, and state the ratio in the direction the design constraint uses.
Traps
- Quoting line rate without accounting for the ports actually available after uplinks are cabled.
- Enabling jumbo frames on the storage VLAN only.
- Counting a second NIC cable as redundancy when both land on the same switch.
- Zoning by switch port instead of by WWPN.