Cloudflare Application Security study materials
Cloudflare Application Security study resources

Cache-Busting HTTP Flood on a Black Friday Search Endpoint

Cloudflare Application Security · Put your knowledge to work in context.

All case studies

Cache-Busting HTTP Flood on a Black Friday Search Endpoint

At 12:04 UTC on Black Friday, an online retailer's storefront, proxied by Cloudflare, drops from an 88% cache hit ratio to 6%. The attack targets GET /api/v2/search?q=<random 12 chars>&_cb=<random> at roughly 480,000 requests per second. Requests arrive from about 2.4 million distinct IPs, 60% of them in residential-proxy networks across the US and EU - the same countries as 95% of real customers. Each request carries a plausible Chrome user agent and a full header set over HTTP/2 with a consistent fingerprint. Origin nginx hosts sit at load average 190 on 8 vCPUs with p99 latency of 34 seconds, and the database connection pool is exhausted. The retailer has WAF, Bot Management and Advanced DDoS Protection. A ransom email arrives demanding payment before Cyber Monday. The on-call engineer wants to enable 'I'm Under Attack' mode globally and add a country block on two EU countries.

1. Rank the first three mitigations you would deploy, and explain why 'I'm Under Attack' mode and the country block are poor first moves here.

Read the worked answer
  1. Fix the cache key. The attack works because _cb and the random q make every request unique. A cache rule that ignores _cb and normalises or allowlists q pushes most of the 480k rps back to the edge; moving 6% to 80%+ hit ratio cuts origin load by more than an order of magnitude while blocking zero real customers. Highest leverage, fully reversible, do it first.
  2. Floor the residual with a targeted rate limiting rule on /api/v2/search (see the next task). This handles traffic that is genuinely uncacheable.
  3. Enable attack-specific L7 protection for the hostname through Cloudflare support/adaptive rules, and protect the origin while it recovers: connection limits, origin shield, and a check that autoscaling is not thrashing the database further.

'I'm Under Attack' is a poor first move because it serves a JavaScript interstitial to every HTML navigation. That converts the checkout funnel into a challenge wall, breaks the mobile app's API calls, and does nothing to the cached GET traffic doing the damage - the flood is not a browser-navigation pattern. The country block is worse: 60% of attack IPs sit in exactly the countries where 95% of customers live, so it would suppress more revenue than attack traffic, and residential proxies are not geo-honest anyway. Neither move touches the amplification vector, which is uncacheable, cheap-to-generate GETs.

2. Design the rate limiting rules: what you key on, thresholds, action, and how the response differs for verified bots and for the retailer's own mobile app.

Read the worked answer

Key on what the attacker must vary, and pick an action that protects conversion.

  • Rule A - per-client search limit. Expression: http.request.uri.path eq "/api/v2/search" and method GET. Characteristic: IP, or IP plus the session cookie when one exists. Threshold: something a human cannot exceed - roughly 40 requests per 60 seconds. Action: managed challenge, not block. A shopper typing queries cannot reach 40/min; a script at 30/second trips it in under two seconds.
  • Rule B - anonymous search limit. Same path, counting on IP, but with the added condition that no session cookie and no document Referer are present: 10 per 60 seconds, action block. Scripts rarely carry either.
  • Verified bots: add a skip rule matching cf.verified_bot_category so Googlebot is not caught during catalogue indexing. Do not exempt on a UA substring - that is trivially spoofed.
  • Mobile app: it should not be hammering an anonymous search endpoint at script rates. If it does, authenticate it - mTLS or a signed header the WAF validates - and skip Rule A only for requests that present that credential. Never exempt on UA alone, or the attacker simply copies the app's UA string.

Because most source IPs make only a few requests each, IP keys alone will miss a slice of the flood. That slice is what the cache fix covers; the rate limits are for the uncacheable remainder.

3. What would you change about the caching layer, and what residual risk remains after the attack is absorbed?

Read the worked answer

Cache changes: (a) a cache rule for /api/v2/search that strips or ignores _cb before building the key; (b) normalise q - trim, lowercase, cap length at around 64 characters, reject longer values before they reach the cache; (c) tiered cache or cache reserve so the upper tier is not re-fetching for every edge location; (d) cache the negative/empty result path too, with a short TTL of 1–5 minutes, because the long tail of random queries is otherwise inherently uncacheable.

Residual risks:

  • Caching search means a mis-keyed entry could serve one customer's query context to another. Confirm the response carries no per-user data before raising TTLs, and check Vary and Cache-Control handling on the origin.
  • The attacker will adapt to a genuinely uncacheable endpoint - cart, per-SKU inventory, checkout - so pre-write rate limiting rules for those paths now, while nobody is panicking.
  • Rate limits written during an incident tend to be too tight. Keep them after Monday, review them weekly against false-positive counts, and move thresholds upward as the attack subsides rather than deleting the rules.
  • The ransom note suggests a repeat attempt. Retention matters: keep Logpush output for the attack window so the next attempt can be compared against the first, and add an alert on cache hit ratio dropping more than 20 points in 10 minutes - that signal fired too late here.