I85PREP
⌘ K
Akamai Web Performance study materials
Akamai Web Performance study resources

Intermittent 5xx Spikes Traced to a Single POP

Akamai Web Performance · Put your knowledge to work in context.

All case studies

Intermittent 5xx Spikes Traced to a Single POP

An e-commerce site sees intermittent 5xx errors during peak hours. Overall error rate is 0.8%, but spikes to 6% for 2-4 minutes several times an hour. Users in one region report failures; other regions are fine. The CDN reports high hit rate overall. Origin logs show occasional timeouts under load. The property has no stale-if-error, a 60s TTL on API responses, and origin connect timeouts at 1s. Error spikes correlate with traffic bursts in the affected region. The team wants to know whether to blame the CDN or the origin.

1. How would you localize the failures and decide whether the CDN or origin is at fault?

Read the worked answer

Localize by dimension: break 5xx down by POP/region, edge vs. origin-generated status, and time. If spikes concentrate in one region and correspond to origin timeouts, the edge is likely returning 5xx because the origin is slow under regional bursts - the CDN is the messenger, not the cause. Check whether the 5xx are edge-generated (e.g., 504 from origin timeout) or passed through from origin.

The 1s connect timeout is aggressive; under burst, connections queue and time out, producing 5xx. High overall hit rate can mask the problem because API responses (60s TTL) are a small share of requests but a large share of failures. Conclusion: origin capacity/latency under regional bursts plus a tight timeout, not a CDN defect - but confirm with edge logs before assigning blame.

2. What configuration changes reduce user-visible errors during bursts?

Read the worked answer
  1. Enable stale-if-error on API responses so the edge serves the last good response when origin fails, converting 5xx into slightly stale 200s. Bound the stale window.
  2. Increase origin connect/read timeouts modestly (e.g., 3-5s) so transient burst latency doesn't immediately become a 5xx, while keeping a ceiling.
  3. Enable request collapsing so burst duplicates for the same API object don't multiply origin load.
  4. Add retries with backoff at the edge for idempotent GETs, capped to avoid amplification.
  5. Consider origin shielding/tiered distribution to absorb regional bursts.

Trade-off: stale-if-error may serve stale prices briefly; pair with short TTLs and purge on critical updates. The goal is graceful degradation instead of hard errors.

3. What monitoring would you put in place to catch this earlier?

Read the worked answer

Alert on 5xx rate by POP and by edge-vs-origin source, with short-window anomaly detection rather than daily averages. Track origin latency p95/p99 and timeout counts, plus hit rate split by content type (API vs. static) so a high aggregate hit rate can't hide API misses. Add synthetic checks from the affected region during peak. Correlate error spikes with traffic bursts and origin saturation metrics. Dashboards should show the 2-4 minute spikes that averages hide.