Firmware Upgrade Config Zone Blast Radius
The fabric has three APICs, two spines (201, 202) and six leaves in three vPC pairs: 101/102, 103/104, 105/106. Current release is 5.2(8) and the target is 6.0(5). Config zones are CZ-APIC (APIC-1, APIC-2, APIC-3), CZ-Spine (201, 202), CZ-Leaf-A (101, 102), and one zone for each remaining leaf. During the previous upgrade, every host on the 101/102 vPC pair lost connectivity for about 12 minutes, while hosts on 103/104 and 105/106 saw only a brief failover. Validation passed before the window, APIC cluster health was 100%, and no faults were open. After the window, Leaf-101 runs 6.0(5) and Leaf-102 still runs 5.2(8); the vPC between them is up. The next maintenance window is four hours, and the business requires no more than 30 seconds of loss for any vPC-attached host.
1. What caused the 12-minute outage, and how should the config zones be redesigned?
Read the worked answer
Leaf-101 and Leaf-102 are in the same config zone, so the upgrade restarted both vPC peers in the same step. A vPC protects against a single member failure; when both members reload, the host's dual uplinks are both gone, and the 12-minute outage is the combined boot and re-convergence time. The other pairs had one leaf per zone, so only one side restarted and the peer kept forwarding, which is why those hosts saw only a brief failover. The design rule is one vPC member per config zone. A workable layout is CZ-A = 101, 103, 105 and CZ-B = 102, 104, 106, with the two spines in separate zones and each APIC in its own zone so the cluster never loses quorum during the controller upgrade.
2. Why did validation pass despite this design, and what should you check before the next window?
Read the worked answer
Validation checks readiness, not blast radius. It verifies image integrity, compatibility, APIC cluster health, disk space and open faults, but it does not know that two vPC peers share a maintenance group. The config zone layout is an operator decision, and nothing in the image metadata flags it. Before the next window, check the target release against the exact leaf and spine PIDs in the compatibility matrix, confirm APIC disk usage and cluster health, verify the image is in the repository and show firmware reports it, review open faults, and confirm the vPC pairs are split. Also check the maintenance policy window against the four-hour limit, and make sure the upgrade is scheduled so that one member of each pair is always up.
3. Leaf-101 and Leaf-102 are now on different releases. What is your immediate next step, and what upgrade order do you follow?
Read the worked answer
Finish the incomplete pair first: upgrade Leaf-102 in a zone by itself after verifying Leaf-101 is healthy, its vPC is up, and the hosts have re-converged. Mixed releases in a vPC pair should be treated as transient and completed as soon as possible, not left as a steady state. After that, follow the standard order: APICs one at a time with cluster health checks between them, then spines one at a time, then leaves grouped so that no vPC pair shares a zone. With four hours, you can complete the pair and continue with the remaining leaves only if each step is validated; if time runs short, stop at a consistent release rather than leaving a mixed pair.