AI Infrastructure Architecture and Design
This domain tests whether you can turn an AI workload profile - model size, parallelism strategy, dataset scale, job duration, inference latency target - into a physical and logical Cisco design: rack layout, power and cooling envelope, compute/storage/management planes, and the management stack that ties them together.
Design pillars
- Performance: AI traffic is east-west and many-to-many. Training clusters are typically designed 1:1 non-blocking; inference tiers can tolerate 3:1 or higher.
- Scale: design for the next cluster size. Reserve spine ports and addressing (an IPv4 /16 or an IPv6 /48 per POD) to allow a doubling without re-IP.
- Resilience: N+1 power per rack, dual-homed servers, redundant super-spines, graceful job restart from checkpoint.
- Manageability: keep out-of-band management, in-band management, and data planes in separate VRFs and separate cabling.
Reference building blocks
Cisco Validated Designs for AI/ML split a cluster into GPU compute PODs, a high-speed AI fabric (RoCEv2 or InfiniBand), a storage fabric (NFS, S3, NVMe over Fabrics), a management plane (Cisco Intersight, Nexus Dashboard), and the data center underlay (Nexus 9000 spine-leaf). The management plane is not optional infrastructure - firmware policy, telemetry, and automation all depend on it, so it must be designed and sized in the first revision.
Worked example: rail-optimized sizing
Requirement: 32 GPU servers, 8 GPUs each, one 400G NIC per GPU, non-blocking.
- Rail-optimized means NIC n on every server connects to leaf n, so same-index GPU traffic stays one hop.
- 8 rails x 32 servers = 256 server-facing 400G ports.
- Each rail uses two leaf switches with 16 downlinks each, giving 16 leaves.
- Each leaf uplinks to every spine. With 8 spines and 8 uplinks per leaf, leaf oversubscription is 2:1; to reach 1:1 either double the spines to 16 or double the uplinks per leaf. The design trap is quoting 1:1 while the uplink budget says 2:1.
Traps that cost marks
- Omitting the out-of-band network and storage fabric from the port budget.
- Sizing racks at a traditional 5 to 8 kW when GPU nodes draw well over 10 kW; power, not ports, becomes the binding constraint.
- Assuming InfiniBand is mandatory. 400G RoCEv2 on Nexus 9000 is a supported Cisco AI fabric design.
- Collapsing tenants, compute, and storage into one flat VRF.
- Treating inference and training as the same design; their oversubscription, latency, and buffering needs differ sharply.