Data Center Spine-Leaf Migration with Legacy Core
A financial services company is migrating its data center from a traditional three-tier architecture (core, aggregation, access) to a spine-leaf fabric. The current data center has 200 physical servers, 40 racks, and uses 10GbE access switches connected to aggregation switches with 40GbE uplinks. The core consists of two large chassis switches with high port density. The company wants to adopt a spine-leaf architecture with 25GbE to servers and 100GbE spine links to support future growth and reduce latency. However, they have a significant investment in the existing core switches, which are only 3 years old and support 40GbE. The applications running include high-frequency trading (HFT) with strict latency requirements (<10 microseconds), a large Oracle database cluster, and a VMware virtualization environment. The company also has a disaster recovery (DR) site 50 km away, connected via dark fiber. The IT team is concerned about the cost of replacing the core and the complexity of integrating the new fabric with the existing WAN edge. They are considering a hybrid approach where the spine-leaf fabric is deployed for new servers while the legacy core remains for existing servers, connected via a border leaf. The company also needs to maintain compliance with PCI DSS and SOX, requiring network segmentation and auditing. The data center has a mix of East-West and North-South traffic, with HFT generating significant East-West traffic.
1. Assess the hybrid approach of deploying spine-leaf alongside the legacy core. What are the technical challenges and how would you address them?
Read the worked answer
The hybrid approach can be viable but introduces challenges: 1) Spanning Tree Protocol (STP) domains: the legacy core likely uses STP, while the spine-leaf fabric uses Layer 3 routing (e.g., BGP). Integrating them requires careful design to avoid loops, possibly using a border leaf that connects to the core via routed links, and disabling STP on the fabric. 2) Latency: HFT requires <10 microseconds; the hybrid path may introduce additional hops. Ensure that HFT servers are placed in the new fabric and use direct spine-leaf paths, while legacy servers remain on the core. 3) Segmentation: PCI DSS and SOX require isolation. Use VLANs/VRFs on the border leaf to extend segmentation across both environments. 4) Management: two separate management domains; consider a unified management platform or orchestration tool. 5) Capacity: the legacy core may become a bottleneck for North-South traffic; ensure border leaf has sufficient bandwidth. 6) Migration path: plan to eventually migrate all servers to the fabric and retire the core, but in the interim, use the border leaf as a gateway. Address by designing a clear migration plan, using L3 routing between core and fabric, and implementing micro-segmentation in the fabric for compliance.
2. How would you design the spine-leaf fabric to meet HFT latency requirements and support future growth?
Read the worked answer
For HFT, latency is critical. Design considerations: 1) Use a fixed-form-factor spine-leaf with low-latency ASICs; avoid chassis-based switches that add latency. 2) Ensure all links are 100GbE with low-latency optics. 3) Use equal-cost multi-path (ECMP) routing with BGP to distribute traffic; configure BGP for fast convergence. 4) Place HFT servers on leaf switches that are directly connected to multiple spines, minimizing hops. 5) Use cut-through switching instead of store-and-forward. 6) For future growth, design a non-blocking fabric with oversubscription ratio of 1:1 or close to it. Start with 4 spines and 16 leaves, with capacity to add more. 7) Consider using 25GbE to servers for HFT to reduce serialization delay. 8) Implement QoS to prioritize HFT traffic. 9) For the DR site, use dark fiber with DWDM to extend the fabric or use EVPN-VXLAN for Layer 2 extension, but be mindful of latency. 10) Ensure the fabric supports network automation for rapid provisioning.