CCSE (CrowdStrike Certifications): Management HA Out of Sync with an Over-Privileged API Account

Why not promote mgmt-b. It has been out of sync for 41 days, while policy was installed 12 days ago and the API account has been making daily changes. mgmt-b therefore holds a stale database. Promoting it would roll back roughly a month of object and rule changes and would leave mgmt-a as the stale node. The clean-looking option is the destructive one here.

Safe sequence in the window.

1. Verify mgmt-a is healthy: cphaprob stat and cpstat mg, disk space on $FWDIR and /var/log, and no pending verification warnings.
2. Take a verified backup and an export of mgmt-a, and test that the export completes; keep both off-box.
3. Confirm mgmt-b's version and jumbo level match mgmt-a. If they diverge, align them first – a version mismatch is a common root cause of the 41-day gap.
4. Rebuild mgmt-b from the mgmt-a export (or re-initialize it as a secondary via cpconfig), then re-establish SIC and let replication catch up.
5. Check HA state again and run a test policy install to a non-critical target before touching the branch gateway.

Watch-outs. Do not restore mgmt-b from its own old backup; do not run an export while a publish is in flight; and keep mgmt-a running throughout, since re-initializing the secondary does not require stopping the primary.

Replace Superuser with a purpose-built profile.

• For the automation account: a permission profile limited to the branch domain, allowing object and rule editing plus policy install, and read-only everywhere else. Remove "Read/Write all". If the automation only needs to publish at a fixed time, consider dropping install rights entirely and letting a human install.
• Scope the administrator to the relevant domain so a mistake cannot reach the other tenant.
• Use key or certificate-based authentication with a defined rotation, not a shared password, and give the account an owner in the object comment.
• For the contractor: a read-only profile that includes log and object read (Monitoring plus read-only policy view), assigned to the single domain. Do not give Superuser, and do not rely on the account being "trusted" – API calls do not bypass permissions, so a correct profile is real protection.

Verify before handover. Log in as each account and attempt a forbidden action (publish a rule, edit an object) to confirm the denial; check that the contractor cannot see the other domain; confirm the audit log records the API actions with the account name; and confirm session behaviour – the "database is locked" error is the automation and the GUI publishing at the same time, so stagger the automation's publish to a scheduled window and make it the only publisher at that moment.

For the HA drift.

• Alert on HA state: a daily check of cphaprob stat/cpstat mg on both nodes, with the result shipped to the monitoring system, would have caught the state change within a day instead of 41.
• Make jumbo hotfix installation a paired change: patch the secondary first, verify replication, then the primary, with the HA check in the change's exit criteria.
• Track replication lag as a metric, and treat any lag older than 24 hours as an incident, not a backlog item.

For the API over-privilege.

• Default to least privilege: no new automation account gets Read/Write all; a permission profile is part of the request ticket.
• Correlate every policy install with a change record. If an install arrives with no matching change, it raises an alert – this is exactly the gap that let changes appear "without a recorded change request".
• Review admin accounts and their profiles quarterly, including API keys, with an explicit expiry.
• Keep publish as a controlled step: one publisher at a time, and the "database is locked" error treated as a signal that two actors are racing, not as an inconvenience to click past.

Report to the auditors. State the detection time before and after each control. "We added an alert" is weak; "drift is now detected within one day and paged to the on-call engineer" is auditable.