PSAA study materials
PSAA study resources

Two Night Analysts, 350 Alerts, and a Noisy EDR Rule

PSAA · Put your knowledge to work in context.

All case studies

Two Night Analysts, 350 Alerts, and a Noisy EDR Rule

Halcyon Financial's SOC ingests nine sources at a 14,000 EPS peak and receives about 350 alerts a day, staffed by two analysts on nights. One EDR rule produces 68% of all alerts and is triggered mostly by an authenticated vulnerability scanner that runs nightly at 01:00; the same rule caught two genuine ransomware precursors last year. Median time to detect is 9 hours and median time to respond is 4 days. A SOAR platform is licensed but only 12 playbooks are enabled, all of them notification-only. Leadership has set a 90-day target: median time to detect under one hour for the top five use cases, with no additional headcount. Asset criticality data exists in a CMDB but is not joined to alerts.

1. What is the first change you make, and how do you avoid losing the true positives the noisy rule currently catches?

Read the worked answer

The first change is to remove the scanner's noise from the alert queue without removing the behaviour from detection.

The rule is not wrong, it is under-conditioned: it fires on a behaviour that a known, authenticated scanner legitimately performs every night. Exclude by the scanner's immutable identity and host tags, not by the process name or by time of day, because both are trivially spoofed and both would mask a real attacker who simply runs at 01:05. Keep a low-severity informational event for scanner matches so the behaviour is still recorded.

Before enabling the exclusion, backtest it: replay 12 months of historical alert data through the modified logic and confirm both prior true positives would still have fired. If either would be suppressed, the exclusion is too broad. That test is the entire justification, and it is what you show the CISO when someone asks whether the SOC turned off a working detection.

Then add a canary: alert if the same behaviour appears from a host that is not the scanner, or from the scanner's identity against unexpected targets. That converts the exclusion from a blind spot into a narrow one.

The expected effect is roughly 240 fewer alerts a day, which is the precondition for every other improvement - with two night analysts, almost nothing else matters until the queue is survivable.

2. How would you choose the top five use cases, and how would you actually prove the sub-one-hour detection target?

Read the worked answer

Choose by consequence and by whether the telemetry already exists. Candidates for a financial institution: credential theft and impossible travel, ransomware precursor behaviour, data exfiltration to unsanctioned SaaS, privileged cloud or identity configuration change, and endpoint isolation evasion. Each must map to a source already ingested, otherwise the 90-day clock is unrealistic.

For each use case, define the detection start event precisely - the log line that represents the adversary's first observable action, not the ticket creation time. Current MTTD of 9 hours is almost certainly measured from ticket creation, which flatters the number and hides the real gap; fix the measurement before trying to improve it.

Prove it with repeatable adversary emulation: run one atomic test per use case per month and record the wall-clock gap between the test's action and the alert appearing in the queue. That gives a defensible per-use-case number and exposes detection gaps that dashboards hide.

Add the CMDB join, because an alert on a domain controller and an alert on a lab VM currently cost the same analyst time and should not. Criticality-driven prioritisation is how two night analysts get to the highest-impact alert first without hiring.

3. What should SOAR do in the first 90 days, given that all 12 existing playbooks only notify?

Read the worked answer

Move from notification to enrichment before moving to containment. Analysts lose the most time assembling context, not clicking buttons.

Phase one: enrichment and deduplication. Every alert gets asset criticality from the CMDB, identity risk for the involved account, threat-intel reputation for any indicator, and a link to prior related alerts. Merge duplicates so one incident equals one ticket. This is low-risk, improves MTTD because the analyst starts with the answer, and can be measured as handle-time reduction.

Phase two: containment in approval mode. Session revocation, mailbox rule removal, and endpoint isolation proposed by the playbook with a one-click approver. Approval-mode automation keeps a human accountable while cutting the four-day response median, which is dominated by waiting for someone to decide, not by the action itself.

Phase three, only after the first two are stable: auto-contain for a narrow, high-confidence signature with an explicit reversal path.

Guardrails: no playbook may close an incident without human review in the first 90 days; every automated action is logged with the run ID and the rule that triggered it; and measure automation by analyst minutes saved per incident rather than by playbook count, because 12 notification playbooks already demonstrate that count is a vanity metric.