GSLB Keeps Sending Traffic to a Site With a Failed Application
An active-active GSLB deployment spans London and Frankfurt for gslb.example.com. DNS delegates the zone to the ADC GSLB vserver, which returns both site VIPs in round robin with a 300-second TTL. Each site has a local LB vserver and three services. GSLB services use TCP monitors against each site's public VIP. After a failed database maintenance, London's application returns HTTP 503 for all requests, but the VIP and TCP port remain up. GSLB still returns London about 50% of the time, so users see intermittent 503s and retry. Site persistence is source-IP with a 10-minute timeout. MEP is enabled between sites and metrics look normal. The business wants automatic failover within 60 seconds and no manual DNS changes. You must preserve site affinity for returning users when both sites are healthy.
1. Why did GSLB not fail over, and what is the limitation of the current monitor?
Read the worked answer
GSLB did not fail over because the GSLB service is monitored with TCP, which only proves the public VIP accepts a connection. London's ADC can accept TCP and return HTTP 503 from the application, so the GSLB service stays UP. GSLB selects sites based on GSLB service state and metrics; if the local LB vserver also lacks an application monitor, the VIP remains UP and MEP has no reason to mark London down. The limitation is Layer 4 versus Layer 7 health. A TCP monitor cannot detect a database read-only state, an HTTP 503, or an application dependency failure. MEP looking normal is expected; it exchanges metrics, not application payload health.
2. Design the monitor and GSLB service changes to meet the 60-second failover goal.
Read the worked answer
Bind an HTTP-ECV monitor to each GSLB service and to the local LB service group: GET /health, receive string OK, interval 10s, timeout 5s, retries 3. That yields worst-case detection around 30-40 seconds, under the 60-second goal. Reduce the GSLB DNS TTL from 300s to 30s; otherwise clients and resolvers can keep the failed site for up to five minutes regardless of GSLB state. Keep MEP enabled and verify metrics exchange. Use a site monitor only if you need to remove an entire site; here the application monitor is the correct control. Test by forcing London's /health to return 503 and confirm GSLB stops returning the London VIP. Also verify the local LB vserver marks its services down so the site does not advertise a VIP that returns 503.
3. How do site persistence and DNS TTL interact with failover and recovery?
Read the worked answer
Site persistence can override health if configured poorly. Source-IP persistence for 10 minutes may keep a user pinned to London even after GSLB marks it down, depending on the persistence method and whether the GSLB vserver honors it during failover. Use HTTP cookie-based site persistence for web traffic, with a short timeout, and test that a downed site is not selected. Existing DNS caches are the other issue: with TTL 300s, some users will still go to London after failover. TTL 30s limits that. When London recovers, do not immediately return 50% of traffic; use slow start or a gradual weight increase, and keep the HTTP monitor active so a flapping application does not cause repeated failovers.