NodeBalancer Flapping and Global Latency
A SaaS vendor runs a public web application on six Compute Instances in a single region behind a NodeBalancer that terminates TLS. The backend health check is an HTTP GET to /health on port 3000, interval 3 s, timeout 2 s, two retries before a node is marked down. /health executes a small query against the Managed Database, which sits on a VLAN shared with the app servers and has no public interface. During weekday peaks the NodeBalancer briefly marks one or two backends down in 30–90 second bursts and users see 502s. CPU on the affected hosts is around 55%. Separately, users in Singapore and Frankfurt report 350–500 ms time-to-first-byte and slow image loads; images are served by the app servers from local disk.
1. Diagnose the flapping using the evidence given, and specify changes to the health check and to the application.
Read the worked answer
Host CPU at 55% argues against saturation, but the health check itself is the suspect: /health performs a database query, so the NodeBalancer is judging application health on database latency. A brief slow query or connection-pool stall pushes the check past its 2 s timeout, two retries fail, the node is marked down, in-flight requests are disrupted (502s), and the node returns 30–90 seconds later when the query completes. The pattern matches the burst duration exactly.
Fix the check first:
- Make /health a local, dependency-free check (process alive, listener accepting).
- Expose a separate readiness or deep check that includes the database, used for alerting rather than load-balancer membership.
- Raise the timeout to 5 s, retries to 3, and lengthen the interval to reduce probe load.
- Enable connection draining so removal does not drop requests.
Then fix the app: connection pooling and bounded query timeouts against the Managed Database, plus p95 database latency alerting.
2. Design the delivery path for images and dynamic responses for global users, naming what each component removes and why.
Read the worked answer
The application servers should serve dynamic HTML and API only; static assets should never touch them.
Path: browser -> CDN edge -> object origin for images; browser -> NodeBalancer -> app servers for dynamic requests.
- CDN in front of Object Storage removes image bytes from the app servers and terminates long-haul TCP/TLS close to the user, which is the main contributor to the 350–500 ms TTFB in Singapore and Frankfurt. Repeated thumbnails come from edge cache, so origin egress falls well below one request per view.
- NodeBalancer in one region keeps app-server membership and TLS handling; it does not fix geographic distance.
- For TTFB on dynamic pages, options are a second region with DNS or GTM steering, or edge-side caching of cacheable API responses with short TTLs.
The judgment: do not put images behind the NodeBalancer, and do not expect a CDN to fix uncacheable dynamic latency in a single-region deployment.
3. The vendor plans a Frankfurt region next quarter. What constraints shape that plan?
Read the worked answer
VLANs and managed services are region-scoped, so nothing here stretches.
- VLANs do not span regions. The database VLAN in the current region has no equivalent reach into Frankfurt; each region needs its own private network and its own database endpoint.
- The Managed Database does not move with the app tier. Choose between a Frankfurt database with cross-region replication or a primary/replica split, and decide which writes must be local. Replication over the public internet needs TLS and firewall rules; there is no shared L2 segment to lean on.
- Load balancing is per region. A second NodeBalancer is required, with traffic direction handled at DNS or a global traffic steering layer, including health-based failover.
- Data gravity. A 45 TB object corpus is not copied per region; keep images in one origin behind the CDN and let edge absorb the distance, while moving only latency-critical compute.
Draft the network diagram before pricing: the database topology is the expensive decision.