Noisy Sign-in Alerts After Conditional Access Rollout
Contoso's SOC operates a single Microsoft Sentinel workspace in the East US, ingesting Microsoft Entra ID sign-in logs and Defender XDR incidents through the connector. Three weeks ago the identity team enforced a Conditional Access policy requiring compliant devices for all cloud apps. Since then, the 'Sign-in from unfamiliar location' analytics rule fires 400-600 times daily against a baseline of about 20. Roughly 70% of the alerts involve the same 40 traveling sales staff, all of whom now authenticate through a VPN concentrator whose egress IPs are not in the workspace's allowlist. Analysts triaged 1,200 alerts last week and closed 1,150 as benign. The SOC manager wants to cut volume without losing detection of genuine impossible-travel attacks, and has no budget for new tooling. Two analysts are on shift per 12-hour period.
1. Recommend how to reduce this alert volume while preserving detection of real impossible-travel attempts. Justify each change against the stated constraints.
Read the worked answer
The root cause is not the rule's logic but a legitimate change in authentication topology (VPN egress) plus a policy change that altered sign-in patterns. Treat it as tuning, not suppression.
Enrich the rule, don't disable it. Modify the analytics rule query to join sign-in events against a watchlist of approved VPN egress IP ranges, and exclude only those events from the unfamiliar-location branch. Because the exclusion is IP-scoped, a genuine sign-in from an unlisted foreign IP still fires.
Add entity mapping and grouping. Map the Account and IP entities and configure alert grouping (e.g., 4-hour window) so one traveler's burst of sign-ins becomes one incident rather than dozens. This directly attacks the 400-600/day figure without touching detection logic.
Keep a compensating control. Retain impossible-travel detection based on Entra ID risk detections (which already factor in the VPN if the identity team flags the IPs as trusted) and add a lower-severity rule for successful sign-ins from non-allowlisted countries so the exclusion cannot become a blind spot.
Validate before deploying. Replay the last 30 days of data against the modified query and confirm the 1,150 benign closures drop while the 50 true positives still fire. No new tooling is required - watchlists, grouping, and KQL edits are native.
Given two analysts per shift, the goal is to move from ~500 alerts to tens of grouped incidents, which is operationally sustainable.
2. A week after your changes, an analyst notices a real attack - a successful sign-in from an unlisted country followed by mailbox rule creation - was never raised as an incident. Diagnose the most likely cause and describe how to confirm it.
Read the worked answer
The most likely cause is over-broad exclusion in the tuned query. Common failure modes: the VPN watchlist contains a /8 or a public cloud range that happens to include the attacker's IP; or the allowlist was built from 'all IPs our staff used in 30 days,' which silently captured attacker infrastructure during the original noisy period.
Confirmation steps:
- Run the current rule query in Log Analytics against the attack timestamp and inspect whether the sign-in was filtered by the IP exclusion. Use
SigninLogs | where TimeGenerated between (...) | where IPAddress == '<attacker IP>'and then apply the exclusion clause manually to see if it drops the row. - Check the watchlist contents for overly broad CIDRs and for IPs added in the last 30 days.
- Review the rule's incident grouping window: if the mailbox-rule event was folded into a grouped incident that was closed, the signal was buried rather than missed.
Remediation: tighten the watchlist to specific /32 or small CIDRs owned by the VPN provider, add a scheduled analytics rule that fires on mailbox rule creation from a new country regardless of IP allowlist, and add a regression test query run weekly that asserts known-bad IPs are not excluded.