Volume Shadowing Member Resynchronization After Site Power Loss

A merge state means the shadowing driver has detected that a former member's contents diverged from the current shadow set (typically because writes occurred while that member was unavailable) and it must reconcile the differences before the member can rejoin.

Why the proposed plan is dangerous:

• MOUNT/BIND= is for adding a brand-new or known-good member to an existing shadow set. It does not perform merge reconciliation. Binding a stale member that contains old data can make the shadow set serve stale blocks to applications, silently corrupting files.
• Rebooting NODE1 to "clear the state" does not resolve divergence; it just restarts the node with a stale member still in the set.
• The application owner's concern is correct: writes on NODE2 while NODE1 was down are the authoritative data. Any member that missed those writes must be merged, not bound.

Correct handling: let the shadowing driver perform a merge (or, if merge is not possible, copy the member) under the control of the shadowing software. MOUNT with the appropriate merge/copy options, or allowing the automatic merge on mount, is the supported path. Never force a stale member into service.

Pre-checks

• SHOW DEVICE/FULL DSA1 and each member to confirm which member is in merge/copy state and which are current.
• Confirm the shadow set is mounted on NODE2 and serving normally; do not disturb the serving node.
• Verify free space and I/O headroom on NODE2, since merge/copy generates heavy I/O.

Restore the member

1. If the member is in merge state, allow the merge to complete. Merges are typically automatic on mount; monitor with SHOW DEVICE DSA1 until the member shows as a normal member (no merge flag).
2. If merge cannot complete (e.g., the member is too divergent or the merge was interrupted), the supported recovery is to remove the stale member, then add it back as a copy target so the driver performs a full copy from a current member. Use MOUNT/SYSTEM with the copy option appropriate to your OpenVMS version, or SET DEVICE/MOUNT per documented shadowing procedure - never BIND a divergent member.
3. Do the copy/merge during a low-I/O window; it reads from a current member and writes the target.

Verification

• SHOW DEVICE DSA1 shows all three members present with no merge/copy state.
• ANALYZE/DISK_STRUCTURE DSA1 (or /REPAIR only if needed and after backup) reports no errors.
• Confirm application I/O on NODE2 is unaffected and that NODE1 can mount the set read/write.
• Record the event and confirm the member's write bitmap is cleared.

Several layered improvements:

Shadowing configuration

• Ensure write bitmaps are enabled for members (the default on modern OpenVMS). Bitmaps let the driver perform a fast merge instead of a full copy after a member is offline, dramatically reducing divergence and recovery time.
• Keep members on independent controllers/paths so a single failure does not take out multiple members.

Cluster and boot configuration

• Put both nodes on UPS-backed power so a node does not drop mid-write. The site event took NODE1 down abruptly; UPS coverage prevents that class of divergence.
• Ensure the cluster is configured so that a node losing power does not leave the shadow set in an ambiguous state; verify expected votes and quorum behavior.

Operational

• After any unplanned node loss, check SHOW DEVICE for merge/copy states before allowing the node to rejoin production I/O.
• Document the recovery runbook so no one reaches for MOUNT/BIND on a divergent member.
• Periodically verify ANALYZE/DISK_STRUCTURE cleanliness so a merge is not compounded by pre-existing corruption.