Bursty Video Manifest Traffic Overwhelms Origin
A streaming service launches a live event. Playback manifests are small JSON files fetched every 2 seconds per viewer. At peak, 250,000 concurrent viewers generate ~125,000 manifest requests/s. The CDN property caches manifests with a 2s TTL and 'Honor origin TTL' on. The origin emits Cache-Control: no-cache on manifests so it can update them. During the event, origin CPU saturates, manifest fetch latency rises, and rebuffering spikes. The origin team proposes adding more origin servers. The CDN has request collapsing and stale-while-revalidate available but disabled.
1. Explain the root cause and why adding origin capacity is a poor first fix.
Read the worked answer
The origin sends no-cache, which requires revalidation before reuse. With 'Honor origin TTL' on, the edge must validate each manifest. At 125,000 req/s, even with collapsing, the origin faces enormous validation load. Adding origin servers scales a fundamentally misconfigured caching pattern: the traffic is periodic and highly collapsible, so the right fix is at the edge, not more origin hardware.
Also, a 2s TTL is shorter than the 2s polling interval for many viewers, so many requests arrive just as objects become stale, producing synchronized revalidation storms (thundering herd). Origin capacity treats the symptom; it doesn't remove the per-request validation requirement or the herd.
2. Recommend an edge configuration to absorb the burst, naming specific behaviors and their effect.
Read the worked answer
Use the CDN's request-coalescing and freshness tools:
- Enable request collapsing so concurrent requests for the same manifest result in one origin fetch; the rest wait and share the response. This is the single biggest lever for 125k req/s of identical objects.
- Replace
no-cachewith a short explicit TTL (e.g., 2-4s) so the edge can serve fresh without revalidating on every request. - Enable
stale-while-revalidateso the edge serves slightly stale manifests while refreshing in the background - decoupling user latency from origin latency. - Enable
stale-if-errorso origin trouble degrades to slightly stale manifests rather than errors/rebuffering. - Consider tiered distribution so parent tiers absorb collapsing before hitting origin.
Effect: origin sees a small fraction of requests, latency at the edge stays low, and rebuffering drops. The trade-off is up to a few seconds of manifest staleness, acceptable for live playback.
3. What risk does stale-while-revalidate introduce for live manifests, and how would you bound it?
Read the worked answer
For live content, serving stale manifests can point players at segments that are no longer available or skip a discontinuity, causing playback errors. Bound it: cap the stale window tightly (e.g., 2-4s), pair with stale-if-error only for short windows, and ensure the player tolerates brief staleness via its buffer. Monitor origin fetch success and manifest age; if origin latency exceeds the stale window, fall back to error rather than serving very old manifests. The goal is to absorb bursts, not to mask a persistent origin outage.