Alert Fatigue at a Two-Analyst Security Operation
A managed security provider sends about 5,000 alerts per day to an internal team of two analysts. Roughly 40% are duplicates from a recurring vulnerability scanner, and the critical-asset inventory is incomplete. EDR, firewall, and proxy logs are collected, but there is no formal triage runbook. Analysts currently investigate alerts in arrival order and close most as false positives after 10-15 minutes each. Last month a real intrusion involving a compromised service account was closed as a scanner false positive. Management wants a dashboard showing alert volume and closure rate. The team has no budget for additional staff this quarter.
1. How should the team redesign triage to reduce the chance of missing a real intrusion?
Read the worked answer
Move from arrival-order triage to risk-based triage. Define a small set of high-signal detections tied to identity, privileged access, and critical assets, and route those to a priority queue regardless of arrival time. Suppress or deduplicate scanner traffic at the source by tuning the scanner's known source ranges and scheduled windows, not by ignoring the rule. Build a runbook with decision points: what makes an alert a true positive, when to escalate, and what evidence to collect. Require peer review or a second look before closing alerts involving service accounts or privileged identities. Use a simple severity matrix based on asset criticality and identity sensitivity. Volume and closure rate are not sufficient success measures; add true-positive rate, time to acknowledge high-severity alerts, and repeat-alert suppression rate.
2. How should the incomplete critical-asset inventory be handled in the short term?
Read the worked answer
Create a provisional tiering using what is known: domain controllers, financial systems, backup infrastructure, and service accounts with broad rights. Assign owners and confirm the list with IT and business leads within two weeks. Until the inventory is complete, treat alerts on unknown hosts conservatively if they involve privileged credentials or outbound connections to new destinations. Tag assets in the SIEM as they are confirmed. This is a judgement call: a provisional list with explicit assumptions is better than waiting for a perfect inventory, but analysts must know which assets are still unclassified so they do not over-trust the tiering.
3. What should the management dashboard show instead of only alert volume and closure rate?
Read the worked answer
Show leading indicators of detection quality: percentage of alerts auto-suppressed as known duplicates, high-severity alerts acknowledged within target, percentage of alerts closed with documented reason codes, number of alerts requiring escalation, and aging of open investigations. Include coverage indicators such as whether critical assets and privileged accounts are logging. A closure-rate-only dashboard rewards fast closing, which contributed to the missed intrusion. Pair volume metrics with a sample audit of closed alerts and a trend of repeat false positives by rule, so tuning decisions are visible.