High Availability and Upgrade Design for a 9800 Campus and Branch
A headquarters campus has 1,100 APs and 18,000 daily clients. A single Cisco 9800-80 controller is installed in the data center, with a second 9800-80 still in the box. The branch office has 60 APs and a single 9800-CL on VMware; the WAN RTT is 40 ms. Last month a controller reboot caused a 45-minute outage, and a failed upgrade stranded 200 APs. The business requires less than five minutes of control-plane downtime per year and no more than one minute of data-plane loss during software upgrades. The data center has dual power feeds and two separate switches, but the controller is connected to only one switch. You are asked to design high availability and an upgrade plan.
1. Design the high-availability architecture for headquarters and the branch. Justify each choice.
Read the worked answer
At headquarters, put the two 9800-80s into HA SSO with a direct redundancy port connection and an optional EtherChannel for state synchronization. Connect each controller to a different switch, use LAG or dual links, dual power feeds, and verify that the switches are not a single stack member failure domain. HA SSO gives a hot standby with synchronized CAPWAP and client state, which addresses the 45-minute reboot outage. At the branch, do not try to stretch 9800-CL HA SSO across a 40 ms WAN; the RP and state synchronization require low latency and reliable L2 adjacency. Instead, use an embedded wireless controller on a C9300 stack if the branch can standardize on it, because it gives stack-level redundancy and local survivability for 60 APs. If central policy is mandatory, keep the 9800-CL but accept that WAN loss isolates the branch; that conflicts with the business requirement. For 60 APs, local survivability is the stronger design. Document AP fallback behavior and power redundancy for both sites.
2. What upgrade strategy meets the one-minute data-plane loss requirement?
Read the worked answer
First verify that the chosen 9800 software release supports the required in-service upgrade behavior for HA SSO; not every release and feature combination qualifies. Pre-download AP images to the APs during a low-impact window, then upgrade the controller pair using HA SSO switchover so the standby takes over while the active is upgraded. Use rolling AP upgrade to limit the number of APs rebooting at once. Schedule a maintenance window for the controller image anyway, because the one-minute requirement is a target, not a guarantee; test the exact release in a lab. For the branch EWC, use a stack upgrade process that keeps at least one member up, or schedule a short window if the platform cannot do hitless upgrades. Back up configurations, verify Smart Licensing registration, and confirm that the standby is synchronized before starting. The upgrade plan must include a rollback image on both controllers and a validation checklist: AP join count, client count, RP status, and EC status.
3. How would you prove the design works before relying on it?
Read the worked answer
Run a controlled failover test in a maintenance window. Confirm the standby is in hot standby, RP is up, and the EC is healthy. Force a switchover and measure AP CAPWAP rejoin or state preservation, client session continuity, and time to restore forwarding. Do not just ping the management address; test a real client application and a voice call across the failover. Test a single power supply failure, a single switch failure, and a single uplink failure separately, because redundant components can share a hidden dependency. For the branch, disconnect the WAN and verify local survivability if EWC is used, or document the outage if the 9800-CL is retained. Repeat the test after every software upgrade, because HA behavior is release-dependent.