DCNP cheat sheet
DCNP · Keep the key ideas within reach.
Data Center Networking Fundamentals and Architectures
Three-tier design: core (Layer 3, routed), aggregation/distribution (policy, default gateway), access (Layer 2 to hosts). North-south traffic crosses tiers; east-west traffic stays within access/aggregation.
Spine-leaf: every leaf connects to every spine (full mesh); leaves never connect to leaves and spines never connect to spines. Path length is always leaf-spine-leaf, so inter-leaf latency is predictable (equal-cost multipath friendly).
Oversubscription ratio = southbound (host-facing) bandwidth / northbound (uplink) bandwidth. 3:1 means hosts can demand three times the uplink capacity; common designs target 3:1 or lower at the access layer.
AS number planning for eBGP underlay: use a private 4-byte ASN per device (or per tier) so each leaf/spine is a distinct BGP speaker; RFC 7938 describes single-ASN-per-switch eBGP with maximum-paths for ECMP.
Layer 2 domain size rule: keep VLANs small and confine them to a leaf pair or single rack; large flat Layer 2 domains increase broadcast, STP convergence, and failure blast radius.
Rack unit math: a 42U rack with 1U switches and 2U servers; power and cooling per rack (commonly 5-15 kW) often constrain density before port count does.
ToR (top-of-rack) switching places the access switch in the rack; EoR/MoR places it at the row or middle of row. ToR shortens cable runs and simplifies rack-level moves but multiplies managed devices.
Fat-tree k-ary: k pods, (k/2)^2 core switches, k^2/2 aggregation and k^2/2 edge switches, supporting k^3/4 hosts with full bisection bandwidth.
Anycast/gateway placement: distributed anycast gateways put the default gateway on every leaf with the same IP/MAC, avoiding hairpinning to a central aggregation pair.
Switching, Routing, and Overlay Technologies
VLAN tagging: 802.1Q inserts a 4-byte tag (12-bit VLAN ID, 4094 usable); native VLAN traffic is untagged on trunk ports, so native VLAN mismatches merge broadcast domains silently.
STP variants: 802.1D (legacy), 802.1w RSTP (fast convergence), 802.1s MSTP (per-instance trees). Port roles: root, designated, alternate, backup. Lower bridge ID wins root election (priority then MAC).
LACP: use active/active or active/passive with matching speed/duplex; a port-channel hashes per flow, so a single flow never exceeds one member link.
OSPF in the DC: area 0 backbone, point-to-point links with /31 or /32 loopbacks; default reference bandwidth 100 Mbps mis-sets cost on 10G+ links, so set auto-cost reference-bandwidth to 40000 or higher.
BGP best-path order (memorize the first steps): highest weight, highest local preference, locally originated, shortest AS-path, lowest origin, lowest MED, eBGP over iBGP, lowest IGP metric to next hop, lowest router ID.
ECMP: install multiple equal-cost next hops; hash on 5-tuple (src/dst IP, src/dst port, protocol) so flows stay ordered. Unequal-cost multipath requires vendor-specific configuration.
VXLAN: 24-bit VNI (about 16 million segments), 8-byte header, 50-byte overhead over IPv4 (outer IP+UDP+VXLAN+inner Ethernet). UDP destination port 4789.
VXLAN VTEP: encapsulates at the source leaf, decapsulates at the destination leaf; the underlay only needs IP reachability between VTEP loopbacks.
EVPN route types: Type 1 Ethernet auto-discovery, Type 2 MAC/IP advertisement, Type 3 inclusive multicast, Type 4 Ethernet segment, Type 5 IP prefix. Type 2 carries MAC and optionally IP for ARP suppression.
Underlay vs overlay separation: the underlay is a routed, ECMP fabric (often eBGP or OSPF) that carries only VTEP-to-VTEP traffic; the overlay is the EVPN/VXLAN control and data plane.
Network Virtualization and Software-Defined Networking
SDN separation: control plane computes forwarding state, data plane forwards packets, management plane configures devices. Centralized control enables global policy but creates a controller availability dependency.
Southbound protocols: OpenFlow (flow-match/action), NETCONF (XML, RFC 6241), RESTCONF (HTTP/JSON, RFC 8040), gNMI/gRPC (streaming telemetry). Northbound APIs are typically REST/JSON for applications.
YANG models define configuration and state schema; NETCONF uses YANG datastores (running, candidate, startup) with commit, confirm-commit, and rollback capabilities.
Overlay virtualization: network virtualization decouples virtual topology (VLAN/VNI, IP subnet) from physical topology, allowing overlapping tenant IP space isolated by VNI.
NFV: virtual network functions (firewall, load balancer, router) run as VMs/containers on commodity hosts; service chaining steers traffic through an ordered sequence of VNFs.
Controller-based fabric (e.g., SD-Access, NSX-style) uses a central control plane to distribute endpoint location and policy; data plane forwarding can be distributed while control is centralized.
Intent-based networking: declarative intent (desired end state) is compiled into device configuration, with closed-loop verification comparing actual state to intent.
VXLAN vs VLAN scaling: 12-bit VLAN (4094) vs 24-bit VNI (about 16.7 million); overlay removes VLAN exhaustion as a tenant-scaling limit.
Telemetry vs polling: SNMP polling is interval-based and coarse; streaming telemetry (gNMI/gRPC) pushes structured data at sub-second intervals for high-fidelity monitoring.
Network Security and Segmentation
Segmentation methods: VLANs (Layer 2), VRFs (separate routing tables), VXLAN VNIs (overlay), and security groups/firewall zones. VRFs provide control-plane isolation; ACLs enforce data-plane filtering.
Stateful firewall tracks connection state (SYN/ESTABLISHED) and permits return traffic automatically; stateless ACLs must permit both directions explicitly.
Zero-trust principle: authenticate and authorize every session regardless of network location; microsegmentation enforces per-workload policy rather than perimeter-only defense.
802.1X: supplicant (host), authenticator (switch), authentication server (RADIUS). EAP methods (EAP-TLS certificate-based, PEAP/MSCHAPv2 password-based) determine credential type.
MACsec (802.1AE) encrypts Layer 2 hop-by-hop with GCM-AES; IPsec encrypts Layer 3 end-to-end (ESP for confidentiality, AH for integrity only).
Private VLANs: isolated ports cannot talk to each other, community ports talk within a community, promiscuous ports talk to all - used for DMZ and service-provider separation.
First-hop security: DHCP snooping builds a trusted binding table; dynamic ARP inspection validates ARP against that table; IP Source Guard filters spoofed source IPs.
Control-plane protection: CoPP (control plane policing) rate-limits traffic destined to the CPU; uRPF (strict/loose) blocks spoofed source addresses at ingress.
Encryption in transit: TLS 1.3 for management/API (replaces TLS 1.0/1.1); SSHv2 replaces Telnet; SNMPv3 with authPriv replaces v1/v2c community strings.
Automation, Orchestration, and Programmability
Idempotency: applying the same configuration repeatedly yields the same result. Declarative tools (Ansible modules, Terraform resources) are idempotent; imperative scripts are not by default.
Ansible: agentless, SSH-based, YAML playbooks, push model, inventory-driven. Terraform: state file tracks resources, plan/apply workflow, provider-based, pull model for cloud/API resources.
Data formats: JSON (RFC 8259, strict quoting, no comments), YAML (indentation-sensitive, comments allowed), XML (verbose, schema-validated). JSON is the default for REST APIs.
REST constraints: stateless request/response, resource URIs, HTTP verbs (GET read, POST create, PUT replace, PATCH partial update, DELETE remove), status codes (200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 404 Not Found, 500 Server Error).
Python essentials for network automation: requests for HTTP, json for parsing, paramiko/netmiko for SSH, jinja2 for templating configs, re for regex parsing of show output.
CI/CD for network changes: version control (Git) → lint/validate (syntax, YANG, intent) → stage/lab test → deploy → post-change verification → rollback plan.
Orchestration vs automation: automation executes a single task; orchestration coordinates multiple automated tasks across systems into an end-to-end workflow.
Configuration drift: compare running config/state to the source of truth; detect with periodic collection or streaming telemetry, remediate by re-applying declarative intent.
Monitoring, Troubleshooting, and Operations
Interface counters: runts (<64 bytes), giants (>MTU), CRC/input errors (physical/cabling), late collisions (duplex mismatch), discards (QoS or buffer). Rising CRC errors point to Layer 1.
MTU and fragmentation: VXLAN adds 50 bytes over IPv4; set underlay MTU to at least 1550 (commonly 9000 jumbo) or large frames fragment/drop. DF bit set means no fragmentation.
Ping/traceroute limits: ICMP may be filtered; traceroute relies on TTL expiry and may show * for filtered hops. Use TCP-based probes when ICMP is blocked.
Flow analysis: NetFlow/IPFIX/sFlow export flow records (5-tuple, bytes, packets, timestamps); use for top-talkers, capacity, and security anomaly detection.
Syslog severity 0-7: 0 emergency, 1 alert, 2 critical, 3 error, 4 warning, 5 notice, 6 informational, 7 debug. Lower number = higher severity.
Mean time metrics: MTTR (mean time to repair) and MTBF (mean time between failures). Availability = MTBF / (MTBF + MTTR).
Change management: pre-change backup, documented rollback, maintenance window, post-change verification. Most outages trace to unverified configuration change.
BGP troubleshooting order: check session state (Idle, Connect, Active, OpenSent, OpenConfirm, Established), then prefix receipt, then best-path selection, then RIB/FIB installation.
Oversubscription and buffer monitoring: track egress queue drops and buffer utilization; microbursts can drop traffic even when average utilization looks low.
Operations and Lifecycle Management (cross-domain operations)
Operational runbooks: standardized procedures for common failures (link down, BGP flap, power loss) reduce MTTR; each step should have a verification command and expected output.
Backup strategy: automated config backup on change plus scheduled interval; store off-device; test restore. Configuration is the fastest recovery artifact after hardware replacement.
Capacity planning inputs: port utilization trends, oversubscription ratio, growth rate, and link upgrade lead time. Trigger upgrade when sustained utilization exceeds the design threshold (commonly 70%).
Documentation of record: physical topology (rack elevations, cabling), logical topology (VLANs, VRFs, subnets), IP address management (IPAM), and circuit inventory.
Incident severity levels: define impact and response times; severity 1 = total outage, severity 2 = degraded service, severity 3 = minor, severity 4 = request.
Firmware lifecycle: track vendor advisories (PSIRT), test in lab, schedule upgrades in maintenance windows, verify post-upgrade health, and keep a known-good image for rollback.
Vendor support and RMA: maintain support contracts, serial number inventory, and spare parts for critical components (optics, power supplies, supervisors).
SLA/SLO definitions: availability target (e.g., 99.9% = about 8.77 hours downtime/year), latency budget, and change success rate.
Post-incident review: blameless root-cause analysis, timeline reconstruction, contributing factors, and tracked corrective actions with owners and due dates.