SOC Alert Noise and a Missed LOLBin Intrusion
Metro Health Partners runs a 24x7 SOC with 11 analysts. The SIEM ingests 190 sources and generates about 2,600 alerts per day. The top rule, 'possible credential dumping,' fires 900 times daily and is closed as a false positive 99.7% of the time; analysts have started bulk-closing it. EDR covers 8,200 of 8,340 endpoints; 140 legacy clinical workstations have no EDR. Over 30 days the SOC's mean time to acknowledge is 19 minutes, but mean time to detect for a confirmed intrusion was 6 days. The intrusion used certutil and PowerShell to stage data and lateral movement over SMB; no SIEM rule covered that sequence. The CISO wants a 90-day plan to reduce noise without increasing blind spots, and asks whether to buy more analysts or invest in detection engineering.
1. What should be done with the top credential-dumping rule, and how can tuning avoid missing true positives?
Read the worked answer
Do not bulk-close the credential-dumping rule. At 900 alerts/day and 99.7% false positives, the rule is still catching about 2.7 true positives per day, and bulk closure trains analysts to ignore it. First, sample 50 closed alerts and verify the disposition. Then split the rule by asset criticality and log source: keep it high-fidelity on domain controllers and servers, suppress known benign admin tooling on workstations with a tight allowlist, and add correlation with EDR process trees. Add risk scoring so an alert on a finance server outranks one on a kiosk.
Measure precision and recall before and after tuning. If the rule cannot be tuned, replace it with behavior-based detection rather than deleting coverage. The goal is to reduce noise while preserving true positives, not to hit an arbitrary alert target.
2. Given the confirmed intrusion chain, what detection-engineering priorities and coverage gaps should the 90-day plan address first?
Read the worked answer
The confirmed intrusion used certutil and PowerShell for staging and SMB for lateral movement, and 140 clinical workstations lack EDR. Prioritize detection engineering for that exact chain: PowerShell script-block logging with encoded-command and download-cradle analytics, certutil decode and urlcache detection, SMB copy and admin-share anomaly rules, and data-staging thresholds. Add honeytokens on file shares.
For the 140 endpoints, either deploy EDR or isolate them into a restricted clinical VLAN with limited SMB and no internet; do not leave them as an unmonitored blind spot. Extend log retention beyond 30 days because a six-day dwell time and legal hold needs at least 90 to 365 days. Build a detection-as-code pipeline with tests so new rules are validated against atomic tests before production.
3. How would you measure success and decide between hiring two analysts and hiring two detection engineers?
Read the worked answer
Decide with metrics, not headcount alone. Track alert-to-incident conversion, precision by rule, MTTD and MTTA, dwell time, coverage of MITRE techniques, and analyst backlog. If false positives dominate and MTTD is six days because no rule exists, two detection engineers will reduce risk more than two analysts. If alerts are valid but unworked, add triage capacity.
In practice, fund one detection engineer and one automation or EDR engineer first, then reassess triage backlog in 90 days. Report to the CISO: noise reduced, precision improved, new coverage for LOLBins and SMB, 140 endpoints closed or isolated, and retention increased. Do not claim success from alert volume alone; a quieter SOC that still misses the same intrusion is not an improvement.