Five Telemetry Sources, Three Alerts, Zero Correlation

The failure is architectural, not analytical. Every detection is written against one source, so the platform only correlates when two events happen to carry the same string in the same field. CORP\jsmith, [email protected] and arn:aws:sts::…:assumed-role/AWSReservedSSO_Developer_9f2c/jsmith are three representations of one person, and exact-match resolution treats them as three unrelated entities. The three alerts therefore land in three different entity timelines and never accumulate risk against the same object.

Concrete fixes, in order:

1. Canonical identity schema. Normalize every source to a single user_id (the Okta immutable ID) plus typed aliases: sAMAccountName, UPN, mail, role-session name, employee ID. Map on ingest, not at query time.
2. Session stitching. The SSO session identifier links the Okta authentication, the email click, and the CloudTrail role session name. Without stitching, a single authenticated session looks like three independent actors.
3. Entity graph, not entity column. Model user→device→IP→cloud-role as edges with timestamps and confidence, so an alert on the laptop raises the risk score of the user and of the role she assumed minutes later.
4. Entity-risk scoring with decay. Low findings should never page on their own but should push an entity over a threshold when they co-occur in a short window.
5. A normalized event schema (OCSF or ECS) so new sources do not each require bespoke field mapping.

One test of the fix: replay the reconstructed intrusion and confirm it produces one incident with three contributing findings, not three tickets.

Start with capacity arithmetic rather than opinion. Seven analysts, realistically 4–5 hours of triage each per day after other duties, is roughly 30 analyst-hours per day; at 15 minutes per alert that is about 120 alerts per day, or 3,600 per month, with no slack for investigations or hunting. A 4,100-alert month is therefore already over capacity, which is why mean time to triage is three days while detection is measured in minutes.

The redesign:

• Collapse alerts into entity-level incidents before the queue. Five endpoint alerts about one host is one investigation. This is the single largest volume reduction and costs nothing in coverage.
• Rank by risk, not by rule severity as written. Score on asset criticality, identity privilege, data sensitivity, detection confidence, and whether the finding is corroborated by a second source. A medium finding on a domain controller with a cloud data role outranks a high on an unmanaged kiosk.
• Fix the top 5 rules by volume first. If 92% of endpoint alerts close benign, a handful of rules is producing most of the queue. Measure false-positive rate per rule monthly and rewrite or constrain the worst offenders.
• Deprioritize, do not delete. Move noisy rules to a low-cost tier that feeds entity risk and a weekly review, rather than suppressing them. Blanket suppression is how the next intrusion gets missed; a risk-contributing tier keeps the signal.
• Add compensating detections for anything suppressed, and require an owner and an expiry date on every tuning change.

Report backlog age as a first-class metric, not just alert counts; a growing backlog is a coverage failure even when detection works.

It is almost certainly measuring the wrong thing and should be treated as a marketing number until it is decomposed. Four minutes is presumably the interval between the first ingested event and the alert generation, which says nothing about whether an alert was generated at all.

Better measures:

• Detection latency per technique in scope, not per alert. For the reconstructed intrusion, the meaningful numbers are: how long after the phishing email did any detection fire, how long after the encoded PowerShell, and how long after the AssumeRole. Our case shows 0 minutes for PowerShell because it did alert - and effectively never for the chain, because the chain was never assembled.
• Coverage-based MTTD. Track against a known adversary technique list and report the share of techniques where a telemetry-to-detection path exists and has been tested. Untested detections should be counted as absent.
• Detection-to-incident latency, which is where the three-day triage number belongs.
• Silent-source monitoring. Ingest volume per source per hour; if CloudTrail or Okta stops flowing, MTTD looks excellent precisely because nothing is being evaluated. Alert on ingest gaps.

Finally, validate by purple-team replay: run a scripted chain quarterly and measure end-to-end. If the exercise produces three unrelated tickets, the four-minute figure is accurate and irrelevant.