Six analysts, 2,300 alerts a day, and one beacon that waited 52 hours
Meridian Logistics' SOC has six analysts across two shifts plus a manager, a SIEM ingesting 4,100 sources, and roughly 2,300 alerts per day. Only 38% are triaged; the rest auto-close after 24 hours. EDR covers 9,600 of an estimated 11,000 endpoints. Mean time to triage is 19 hours, mean time to contain 4 days. Last month a real intrusion - a phishing payload that became Cobalt Strike beaconing - sat as a medium-severity "unusual parent process" alert for 52 hours before an analyst noticed it. The board has approved budget for either two additional analysts or an automation platform, and the CISO must choose one. Current detection content carries a 61% false-positive rate, and the top three rules generate 44% of all alerts. Threat intelligence says the sector is being targeted by initial-access brokers using commodity loaders.
1. Analysts or automation - what do you recommend, and how do you justify it with the numbers in front of you?
Read the worked answer
Neither option survives alone; the real question is which one changes the number that matters first. The number is 38% triaged, not headcount.
Rough arithmetic: 2,300 alerts/day, 61% false positive, and 3 rules producing ~1,012 alerts/day. Two analysts add maybe 60-80 triage-hours a week - real, but spent largely on those 1,012 low-value alerts. Automation applied to the same rules removes volume before it reaches a human, and that gain compounds daily.
So: automation first, headcount decision deferred by 90 days with a measurement gate.
Priority order for automation:
- The top three rules - tune or enrich-to-close them, whether by automating investigation (pull process tree, hash reputation, user context, sign-in history) or by writing a deterministic disposition for the benign pattern. Target: eliminate half of their 1,012 daily alerts.
- Phishing triage - the highest-volume, most repetitive, most pattern-driven workflow; commodity loaders arrive this way, so it is also the workflow where the 52-hour failure occurred.
- Enrichment-only automation - do not auto-close on a heuristic; auto-gather so the analyst's first 30 seconds is decision-making, not clicking.
Why not analysts first: hiring takes 8-12 weeks, then ramp time, and the new hires would be consumed by the same noise, so MTTD barely moves. Why not pure automation either: a 61% false-positive rate means automation must not close on ambiguity, and someone must own the detection content.
State the gate: after 90 days, if triage rate is not above 70% and the queue is still dominated by the same rules, the automation thesis failed and the headcount request is justified by evidence rather than by hope.
2. At 02:00 Sunday, one analyst is on shift. Design the severity model and autonomy rules so a Cobalt Strike-class event cannot hold for 52 hours again.
Read the worked answer
The failure was not analyst diligence - it was a queue design that let a high-consequence alert inherit a medium severity and wait behind noise. Fix the routing, not the analyst's attention.
Create a "never queue" class. Certain alert types skip the queue and page immediately because their consequence is extreme and their false-positive rate is manageable:
- EDR high-confidence beaconing or C2 signatures, and credential-dumping behavior on any host.
- Identity events: successful MFA push bombing followed by a login, impossible-travel to a new country for a privileged account, mass sign-in failures.
- Tampering with logging or the SIEM forwarder - this is nearly always malicious.
- Guardrail events from cloud control planes (new IAM user, audit trail modification).
Reclassify by consequence, not by rule name. "Unusual parent process" sounds medium; "unusual parent process on a host that later made outbound connections to a newly registered domain" is high. Correlation raises severity automatically - do not rely on the analyst to connect the dots at 02:00.
Autonomy rules for a single-shift operation:
- Auto-isolate (with a 15-minute human veto window) on EDR high-confidence C2 detection on a laptop or workstation.
- Never auto-isolate production servers or domain controllers; instead page and pre-stage the isolation action so one click executes it.
- Auto-enrich, auto-open a ticket, auto-post to the incident channel for everything else in the never-queue class.
Escalation path: a documented on-call rota reaching a senior analyst and the IR lead inside 30 minutes, with a written runbook step that says "if beaconing is confirmed, engage IR, do not wait for business hours."
Finally, add an aging alarm: any alert above medium that ages past 4 hours pages automatically. The 52-hour dwell should have been structurally impossible, not merely unfortunate.
3. What metrics would you put in front of the board to show the SOC is actually improving, given that MTTR and alert-closure metrics are both easy to game?
Read the worked answer
Any metric can be gamed, so the design principle is to pair a speed metric with a quality metric and a coverage metric so that gaming one is visible in the others.
1. Queue health, not closure volume. Report P90 alert age at each severity, and percentage of high-severity alerts acknowledged within 15 minutes. Closing alerts faster by triaging less is not an improvement; P90 age exposes it.
2. False positive rate measured against a labeled sample. Take a random sample of 200 closed alerts monthly and have a second analyst label them. This yields an honest FP rate rather than one derived from analyst dispositions, which analysts optimize.
3. Detection coverage validated adversarially. Run quarterly purple-team exercises replicating the threat model (commodity loaders and initial-access brokers, per threat intel). Report: techniques attempted, techniques detected, median detection time. This is the only metric that measures the thing the board actually cares about - whether the SOC would catch the next Cobalt Strike.
4. Containment capability. Time from confirmed-beacon detection to host isolation, measured in exercises; and percentage of endpoint estate instrumented (currently 9,600 of ~11,000 - an honest 87%).
5. Analyst load. Alerts triaged per analyst per shift, plus time spent in enrichment versus decision-making. Rising load with flat coverage predicts attrition and missed detections six months out.
What not to report: total alerts closed, or MTTR averaged across severities. Both reward closing easy things and hide a medium alert sitting for 52 hours. Present MTTD and queue age by severity, and show the adversarial exercise result alongside, so speed gains cannot be claimed without demonstrated detection.