Alert Flood During an EDR Policy Rollout
At 02:47 UTC on a Tuesday, a SOC's primary EDR console begins emitting 'Suspicious PowerShell execution' alerts after a policy update lowers the detection threshold on 4,800 endpoints. By 03:27, 6,200 alerts are queued. Of these, 1,180 come from a single hourly inventory script that runs Invoke-Command against server names, and the rest are duplicates of the same script under different parent processes. The P1 SLA is 15 minutes; three analysts are scheduled, but one is at lunch and one is troubleshooting a SIEM ingest lag. At 03:05, a single alert fires: 'LSASS memory access' on FIN-WS-214, a finance workstation, attributed to a signed binary running from C:\Users\Public\Tools. The ticketing system deduplicates by alert title only, so the LSASS alert is grouped into the PowerShell flood and auto-closed as a duplicate. The SOC manager asks for a queue-health decision by 08:00 and whether to declare a major incident.
1. At 03:30, what do you do first, and in what order? State the decision criteria you use to separate the LSASS alert from the PowerShell noise.
Read the worked answer
The deduplication rule is wrong: title-only matching conflates an inventory script with a credential-access alert. The LSASS alert has higher fidelity because the process is in a user-writable path and is accessing LSASS memory, which is a credential-dumping precursor regardless of signature status.
Order of work:
- Reopen the LSASS alert, check the process hash, signer, parent process, command line, and user context.
- Contain FIN-WS-214 with EDR network isolation, which preserves agent communication and volatile evidence.
- Collect a triage package: process tree, LSASS access rights, 4624/4625/4688 events, SMB connections, scheduled tasks, and new services.
- Only after validation, bulk-close the PowerShell alerts as a known script and open a tuning ticket.
Decision criteria for major incident: multiple hosts, use of privileged credentials, outbound C2, or evidence of data access. A single contained workstation does not meet that bar yet, but if the binary is unsigned or has read access to LSASS, escalate to the IR on-call immediately. Do not let the 6,200-alert backlog dictate priority; the one rare, high-impact event does.
2. What will you document and communicate to the SOC manager and the finance business owner before 08:00?
Read the worked answer
Document the alert IDs, timestamps, host, user, process path and hash, the containment action, who approved it, and the rationale for not declaring a major incident. Record the SLA impact separately: the bulk PowerShell queue breached the 15-minute P1 SLA, while the LSASS alert was triaged on its own timeline.
For the SOC manager, report queue health in concrete terms: 6,200 alerts, 1,180 from one validated script, percentage auto-closed after validation, true positives, mean time to triage, and the deduplication defect. State the decision: one potential credential-theft event contained on FIN-WS-214, no evidence of spread yet, no major incident declared, next update at 06:00.
For the finance business owner, avoid alarm and avoid false reassurance: a finance workstation is isolated, payroll data access has not been confirmed, the user may need a password reset and a replacement device, and the team will confirm by 10:00 whether any file share was touched. Give the owner a single point of contact and a time for the next update. Communication is part of containment because the business may otherwise disable controls to restore work.
3. What detection-engineering change would you propose so the same flood does not hide a high-fidelity alert again? Include the risk of the change.
Read the worked answer
First, change the deduplication key from alert title to a composite of title, host, user, process hash, and parent process. That alone prevents the LSASS alert from being collapsed into script noise.
Second, create a scoped suppression for the inventory script: signer, file hash, parent process, command-line pattern, and user context, with a 30-day expiry and an owner. Do not suppress by alert title. Keep a low-volume count alert if the script's hash changes or if it starts contacting new hosts.
Third, add a priority override for LSASS access, credential-dumping behavior, and security-tool tampering so these alerts cannot be auto-closed by bulk rules. Route them to a separate queue with a 5-minute SLA.
The risk is over-suppression: an attacker can reuse the same script path or signed binary. Mitigate with hash pinning, a change alert on the suppressed indicator, and a weekly review of suppression hit counts. Also test the new rule against the last 30 days of data before enabling it, and document the rollback. This is a detection-engineering change, not a threshold change, and it should go through peer review.