High Availability Cluster Management
High Availability Cluster Management
This domain tests the cluster control plane: Corosync membership and quorum, Pacemaker scheduling from the CIB, resource agents, constraint scoring, fencing, and operations with pcs and crm_mon.
Core rules
- Corosync carries membership and messaging; Pacemaker places resources. Quorum is policy: a two-node cluster needs explicit handling such as Corosync
two_node: 1withwait_for_all: 1, or a quorum device, to fail over safely. - The CIB is desired state. Change it with
pcsorcrm configure, not by hand; inspect withcibadmin --query. - Prefer OCF agents. Give every long-running resource a monitor operation with realistic
intervalandtimeoutso a hung service is detected and fenced. - Constraints encode dependencies:
orderfor sequence,colocationfor togetherness,locationfor preference.INFINITYis mandatory, finite scores are preferences,-INFINITYis a ban. - STONITH is mandatory for shared data. If the cluster cannot prove a node is dead, it must fence before starting the resource elsewhere.
Worked example
A two-node web cluster needs a floating address before httpd. Create WebIP as ocf:heartbeat:IPaddr2 and WebServer as systemd:httpd, then:
pcs constraint order WebIP then WebServer
pcs constraint colocation add WebServer with WebIP INFINITY
pcs constraint location WebServer prefers node1=100
Reasoning: ordering starts the address first and stops httpd first; colocation prevents httpd from running without its address; the finite location score biases placement without blocking failover. Configure and test STONITH first.
Traps
pcs resource move leaves a temporary location constraint; clear it with pcs resource clear or the resource never fails back. Do not leave stonith-enabled=false in production. Default quorum can stop resources after one node of two dies. Check pcs status --full, pcs constraint --full, and journalctl -u pacemaker before changing scores.