The synthetic monitor is green and 18% of users are waiting four seconds
An e-commerce web front end is watched by a single-location browser monitor that clicks search → product → cart every 15 minutes. It has been green for weeks at roughly 480 ms for the click path. Front-end RUM on the same application shows the 'Load' user action exceeding 4 s for 18% of real users, concentrated in one country, and Apdex has fallen from 0.94 to 0.81. Two weeks ago the marketing team added a tag manager container that pulls in four third-party scripts. Backend service response times are flat. The business asks why monitoring says the site is fine.
1. Why doesn't the green synthetic monitor contradict the RUM data, and what would you change about it?
Read the worked answer
They measure different things over different populations.
- The monitor runs from one location on one network path with one browser, a warm cache, and no ad-blockers, extensions or device variation. If the slow behaviour is geographically concentrated - which the RUM data already says it is - that location cannot see it.
- It samples one run every 15 minutes. That is a point measurement of a single journey, not a distribution over a user population. It can be green while 18% of sessions are slow, because 18% is invisible in one sample.
- It may not even execute the slow path. Third-party scripts are frequently geo-targeted or consent-gated; a monitor with no consent cookie may never load the four scripts that the affected users do.
Changes worth making: add browser monitors from locations inside the affected country/region; keep a journey that includes consent handling so the same third-party tags load; and check whether any existing monitor is HTTP-only, in which case it executes no JavaScript at all and cannot ever see this class of problem. But be honest about the ceiling: synthetic tells you is it functional from here, RUM tells you is it good for users. For this question, RUM is the source of truth and synthetic is a supplement.
2. With the data you have, how do you isolate the cause?
Read the worked answer
Work down from the population to the session:
- Filter the user action by geolocation to confirm the concentration and get the shape - is it the whole country or a subset of ISPs? Then split by ISP/network and by browser/OS/device.
- Split the slow population by whether the third-party resources loaded. If the slow sessions consistently load all four tag-manager scripts and the fast ones don't (consent, blocker, geo-gate), that is the correlation you need.
- Open the resource waterfall for a handful of slow sessions. You are looking for a third-party host with long connection or TTFB times, a script that blocks rendering, or a synchronous tag that serialises everything behind it. On the front end you can see third-party timing that the backend never sees, which is precisely why backend response times are flat.
- Check the timeline. RUM retention should cover the two weeks - plot the share of slow 'Load' actions by day and look for the step change coinciding with the tag manager. A step change at a deploy is far stronger evidence than a correlation across a noisy month.
The flat backend response time plus a browser-only symptom plus geographic concentration leaves two live candidates: a third-party whose servers are far from or slow for that country (invisible from your monitoring location), or a CDN/geo-DNS path problem into that country. The waterfall separates them - the first shows the delay inside a third-party host, the second shows it in connection setup to your own hosts.
3. What do you report to the business, and which SLO do you attach?
Read the worked answer
Report the tail and the population, never the mean. The mean load time is dominated by the 82% who are fine and would have made this look like a non-event - which is exactly how it got missed for two weeks.
What to show:
- % of user actions exceeding 4 s, by country, with the trend line - the headline number is "18% of users in X wait over four seconds", not "average load time is 1.1 s".
- Apdex and its movement (0.94 → 0.81) as the single satisfaction trend, since it compresses the distribution into something the business can track.
- Estimated affected sessions, so the cost is legible.
- A clear split between "is it up" (synthetic availability) and "is it good" (real-user Apdex). The confusion in the room came from treating one as a proxy for the other.
SLO: real-user Apdex on the 'Load' action, or the percentage of user actions under a 4 s threshold, evaluated per geography where the business cares about that market. Do not SLO on backend response time for this - it is flat and would have scored 100% throughout the incident.