Data Center Network Architecture and Design
Why spine–leaf
Every candidate should be able to explain why a two-tier CLOS (spine–leaf) fabric replaced three-tier designs in the data center: predictable any-to-any latency of two hops, scale-out by adding leaves or spines, and ECMP everywhere instead of spanning tree. Leaves connect servers (top-of-rack) and never connect to each other; spines connect only to leaves. Know the vocabulary: oversubscription (downlink bandwidth ÷ uplink bandwidth), east–west vs north–south traffic, and the L2/L3 boundary - modern designs keep Layer 2 inside the rack or extend it only through an overlay.
Design decisions that show up on the exam
- Rack vs end-of-row: ToR gives short copper runs and per-rack fault domains; EoR/MoR saves ports and optics but adds cabling bulk and a shared failure domain.
- Cabling and optics: DAC/AOC for short in-rack runs, multimode or single-mode fiber for leaf uplinks; know reach and media trade-offs.
- Physical redundancy: dual-homed servers (VSX), A/B power feeds, N+1 or N+N power and cooling, and separate out-of-band management.
- MTU: the underlay must carry the largest frame on any link, including VXLAN overhead. Use jumbo frames (commonly 9000, minimum ~1600) end to end.
Traps
Memorising a product name instead of a role; forgetting that an advertised oversubscription ratio is per layer, not end-to-end; assuming the overlay fixes an MTU or routing problem; designing L2 domains across racks without a control plane.
Rule of thumb
Compute oversubscription with a worked example: a leaf with 48 × 25 GbE server ports = 1200 Gb/s downlink and 6 × 100 GbE uplinks = 600 Gb/s gives 2:1. Halve the uplinks and it becomes 4:1 - that is a bandwidth decision, not a protocol decision.