Checkout Cart Loss After a Pool Member Flaps
An online retailer's checkout VIP uses cookie persistence with a 30-minute timeout and a pool of six members: two 16-core servers and four 4-core servers. The load-balancing method is Least Connections (member). The HTTP health monitor sends GET /health every 5 seconds with a 16-second timeout and expects OK. During a nightly batch job, one 4-core member begins responding in about 2.1 seconds and its monitor alternates between three failures and two successes. Customers on that member see an empty cart because cart state is stored in memory on each server, not in a shared store. OneConnect is enabled with a 10-minute idle timeout. The site carries 40,000 concurrent users, and the application team cannot deploy a shared session store for at least two weeks. Business wants to stop cart losses tonight without permanently removing the member from service.
1. Explain the sequence that causes cart loss when the member flaps, including the roles of cookie persistence and OneConnect.
Read the worked answer
The 2.1-second response is within the 16-second timeout, so the monitor is failing because some probes exceed 16 seconds or return an empty response; three failures at a 5-second interval marks the member down. When it goes down, BIG-IP stops sending new connections there, but the persistence table still contains cookies pointing to that member.
On the next client request, the cookie persistence lookup finds a down member. BIG-IP falls back to load balancing, selects a different member, and rewrites the cookie. That new member has no in-memory cart state, so the cart appears empty. OneConnect does not preserve application state; it only reuses idle server-side TCP connections, and those connections may be reset when the member goes down. The result is a user-visible session loss that looks like a persistence failure but is really a state-locality problem.
2. What configuration changes would you make tonight to reduce cart loss without a code change?
Read the worked answer
There is no BIG-IP setting that copies in-memory cart state between members. The immediate goal is to stop unnecessary flapping and avoid moving users who are still healthy.
- Make the monitor less sensitive to transient batch latency: increase the interval to 10 seconds, raise the timeout to 31 seconds, and require four failures before marking down. This does not hide a hard failure.
- Set Slow Ramp Time on the pool to 300 seconds so a recovered member is not flooded and immediately fails again.
- Apply a connection limit on the four 4-core members (for example 4,000 connections) so Least Connections does not overload them.
- If the business accepts reduced capacity, place the 4-core members in a lower priority group so they receive overflow only; most sessions stay on the stable 16-core members.
Do not disable the member permanently without capacity review, and do not add source-IP persistence as a fallback, because it would still point at the same down member.
3. State the long-term fix and how you would validate it before the next peak event.
Read the worked answer
Move cart state to a shared session store such as Redis or a database, so any pool member can serve any request. Until then, use graceful drain: before maintenance, disable the member with connection drain so existing connections finish and no new persistence entries are created. After the shared store is in place, cookie persistence can still improve locality, but it is no longer required for correctness.
Validate with a load test that includes a deliberately slow member and a mid-test failure. Confirm that carts survive a member down event, that the monitor transitions are visible in logs, and that Slow Ramp and connection limits behave as expected. Check tmsh show ltm pool and persistence statistics to confirm no unexpected fallbacks occur.