Splitting a Three-Node Manager Cluster Across Two Sites
Northwind runs two data centers 6 ms apart with a stretched management VLAN. The NSX 4.1 deployment has a three-node NSX Manager cluster, all three nodes currently sitting in DC-A, and 60 host transport nodes split 40 in DC-A and 20 in DC-B. Business continuity requires that if DC-A is lost entirely, engineers must still be able to change distributed firewall policy and create new segments within 30 minutes. A consultant proposes splitting the manager cluster 2/1 across the two sites to meet the requirement. No external backup target is configured today and there is no tested restore procedure.
1. Evaluate the 2/1 manager placement against the stated requirement. Does it deliver, and what else must be true for it to work?
Read the worked answer
What a 3-node cluster tolerates. NSX Manager uses a majority quorum: two of three nodes must be reachable for the cluster to serve the UI and API. A 2/1 split therefore survives the loss of the site holding the single node, and nothing else. Losing the site with two nodes leaves one node, no quorum, and no management plane.
Direction matters. The requirement is specifically "survive DC-A loss," so the split must be 2 in DC-B, 1 in DC-A. The consultant's proposal is only correct if the pairing is stated that way; the same 2/1 topology placed 2 in DC-A fails the requirement outright.
Latency is not the constraint here. 6 ms RTT is comfortably inside the supported limit for inter-manager communication and for host-to-manager control-plane traffic, so stretching is viable.
What must also be true:
- The management network, DNS, and NTP that the managers depend on must be reachable from DC-B and must not transit DC-A.
- All 60 host transport nodes must be able to reach the manager cluster IPs over a path that survives DC-A loss; otherwise the 20 DC-B hosts cannot receive policy.
- Capacity in DC-B must exist to run the second manager node.
A stretched quorum is a protection against site loss, not against configuration error or corruption, so backups are still mandatory.
2. During a full DC-A outage, which NSX functions keep running on the 20 surviving DC-B hosts and which stop? What is the practical impact on the 30-minute requirement?
Read the worked answer
Keeps running:
- The data plane. Each host holds a realized copy of its segments, transport zones, and DFW policy and continues to forward and filter using the last policy it received.
- Existing established sessions, including their stateful firewall verdicts.
- Edge services already realized on DC-B edge nodes: routing, NAT state, and BGP/BFD sessions to local peers.
- Logging from hosts to the configured remote log server, if that server is reachable.
Stops:
- Any configuration change - new segments, new DFW rules, group membership updates, host preparation. The control plane runs on the manager cluster, and the surviving quorum cannot serve it because DC-A took the majority.
- Realization of pending changes to hosts and edges.
- Traceflow, NSX Intelligence, alarms, and the UI/API.
Impact on the requirement: the 30-minute target is not met by the 2/1 split as designed - the surviving node has no quorum. Meeting it requires the majority to be in DC-B (2 managers in B, 1 in A), plus verified network reachability from the DC-B hosts. Even then, the first hours after failover should be treated as read-only until the cluster is confirmed healthy.
3. Regardless of the placement decision, what protection should be added before this design is signed off, and what does a configuration backup not cover?
Read the worked answer
Add before sign-off:
- A scheduled configuration backup to an external SFTP server located outside both data centers. Backups are taken from the manager cluster and should be scheduled at a frequency that matches the change rate - hourly during project work, daily in steady state.
- A written, rehearsed restore procedure with a target recovery time, because the restore flow includes deploying a replacement manager, restoring the backup, and re-pointing transport nodes.
- Confirmation that the backup credential material and the SFTP endpoint are reachable when one site is down.
What the backup does not give you:
- It is a configuration backup. It does not preserve in-flight stateful sessions, NAT translations, or routing adjacencies held by an edge node that no longer exists.
- It does not restore the realized runtime state on hosts; hosts re-sync from a restored manager, which takes time proportional to inventory size.
- It does not substitute for redundancy of the components that carry traffic. Losing all edge nodes in a site is a data-plane outage that no manager backup addresses.
Treat the backup as the recovery path for corruption and operator error, and the quorum placement as the recovery path for site loss. The two are not interchangeable.