Intermittent logouts after adding a second front-end
A retail customer runs PROD with two front-end servers behind an F5 load balancer and a shared SQL Server database. Before the change, PROD had one front-end. To handle growth, the team cloned the front-end and added it to the pool. The load balancer was configured with round-robin and no persistence because 'the app is stateless'. Within a day, 12% of users report being logged out mid-checkout, and some screens show stale data after a user updates a record. Service Center logs show no errors. The platform is OutSystems 11; sessions are configured in-memory and cache invalidation is enabled. Both front-ends share the same database and the same application version. The team wants to keep two front-ends. Diagnose the symptom and propose fixes and validation.
1. Explain the most likely cause of the logouts and the stale data.
Read the worked answer
The symptoms point to missing session affinity and incomplete cache invalidation across the two front-ends. With sessions stored in memory, a user's session lives on the front-end that authenticated them. Round-robin without persistence sends the next request to the other node, which has no session, so the user is treated as logged out. The stale data is a related but separate issue: OutSystems cache invalidation must reach every front-end; if the second node was cloned without being correctly registered, or a firewall rule blocks the invalidation traffic, one node keeps serving cached data after the other updates it. The fact that Service Center shows no errors is consistent: this is a routing and cache-coherence problem, not an application exception. The 'stateless' assumption was wrong for this deployment because in-memory sessions are node-local state. The fix must address both session routing and cache coherence; enabling only one will leave either logouts or stale reads.
2. What configuration changes would you make, and in what order?
Read the worked answer
First, stop the bleeding: enable cookie-based persistence on the F5 for the application virtual server, or configure OutSystems to store sessions in the database so any front-end can serve them. For a quick, low-risk change, enable persistence first, then plan the session-database change in a maintenance window. Second, verify the cache invalidation configuration: both front-ends must be registered in the environment, the cache invalidation service must be running on each, and the required ports must be open between nodes. Third, remove any clone-specific machine identifiers or keys copied from the first front-end. Fourth, restart the second front-end in a controlled way and watch for session and cache errors. Validate with a scripted test: log in, force requests to alternate nodes, update a record, and read it back from both nodes. Only then re-enable full round-robin without persistence if that is the target design.
3. How would you prove the fix is complete?
Read the worked answer
Run a 24-hour validation with three evidence streams. Functional: synthetic users log in, complete checkout, update a record, and read it back while the load balancer alternates nodes; zero unexpected logouts and zero stale reads. Resilience: drain and kill each front-end in turn during active sessions; users should either continue or re-authenticate cleanly, and no data should be lost. Operational: Service Center and load-balancer dashboards show no session-store errors, cache-invalidation failures, or unusual 401/500 rates; compare the error rate to the pre-change baseline. Add a canary group of internal users for the first business day. Document the configuration as code and add a deployment checklist item: any new front-end must be added to the cache invalidation topology, and session mode must match the load-balancer persistence policy. The reasoning: absence of errors in logs does not prove correctness for a distributed session problem; you need user-visible and failure-mode tests.