502s From a Keep-Alive Race and Three Cookie Findings
A SaaS team runs a stateless Python API behind a cloud load balancer that terminates TLS and forwards plain HTTP to three application instances. After a traffic increase, users report intermittent 502 responses - roughly one every 90 seconds per instance - always on connections that have been idle for about a minute, and the application logs show no corresponding request at all. Configuration: the load balancer's idle timeout is 60 s, the application server's keep-alive timeout is 50 s, and the load balancer health-checks /healthz every 10 s; that endpoint returns 200 without touching the session or the database. Separately, a customer's security review reports three findings: the session cookie is issued without the Secure and HttpOnly attributes; the cookie value is byte-identical before and after login; and no SameSite attribute is set, while the API also accepts state-changing GET requests such as /api/orders/cancel?id=.... The team has one sprint before the audit deadline.
1. Diagnose the 502s and give the configuration change, explaining why the health checks are a red herring.
Read the worked answer
Root cause: the backend closes idle connections before the load balancer does. The app's keep-alive timeout (50 s) is shorter than the load balancer's idle timeout (60 s). At 50 s the app sends FIN and discards the connection, but the load balancer still holds that socket in its pool as usable for another 10 s. If a request is dispatched in that window, it goes onto a connection the backend has already torn down; the backend answers with RST or nothing at all, so the load balancer has no upstream response to relay and returns 502 - with no application log line, exactly as observed. The ~90 s cadence reflects how often a pooled connection happens to be reused right at the boundary under current traffic.
Health checks are irrelevant. They open their own connections to /healthz every 10 s; they do not keep client connections warm and they do not reset the idle timers on the pooled sockets in question.
Fix. Make the app's keep-alive timeout strictly greater than the load balancer's idle timeout - 75 s against 60 s - so the load balancer always initiates the close and knows the connection is gone. Confirm by capturing FIN direction on an idle connection and by replaying requests at 45/55/65 s after idleness. Also distinguish 502 from 504 in dashboards, and retry idempotent requests at the load balancer as a safety net.
2. Address the three cookie findings, including why Secure and a TLS-terminating load balancer interact.
Read the worked answer
Secure is missing because the app sees HTTP. TLS terminates at the load balancer, so the app receives http:// and any framework that sets Secure conditionally on the request scheme will omit it. Set Secure unconditionally in application config. If instead you honour X-Forwarded-Proto, trust it only from the load balancer's address range - a client that can inject that header can otherwise influence scheme-dependent behaviour (and, in some misconfigurations, suppress Secure or cause redirect loops).
HttpOnly blocks JavaScript access to the cookie, so an XSS payload cannot read and exfiltrate the session. There is no cost for a server-side session cookie the SPA does not need to read.
Byte-identical cookie after login is session fixation. The pre-authentication ID must be discarded: generate a new session ID on privilege change, migrate the user's state to it, and invalidate the old ID server-side so a cookie planted before login cannot be used after it.
SameSite plus the state-changing GETs. SameSite=Lax (or Strict if the flow allows) stops the cookie riding along on cross-site subrequests, but Lax still sends it on top-level GET navigations - so an attacker-supplied link or a prefetcher can trigger /api/orders/cancel?id=.... Fix the verbs: state-changing operations must be POST/DELETE, protected by a CSRF token or a server-side Origin/Sec-Fetch-Site check. SameSite alone is not CSRF defence, especially across sibling subdomains.
3. You have one sprint. Sequence the four pieces of work and justify the order.
Read the worked answer
1. Session fixation first. It is the only finding that enables direct account takeover, it is the cheapest to fix (regenerate the ID at login and on privilege change, destroy the old one), and it carries almost no regression risk. A short-lived window of exposure here is worse than every other item combined.
2. Cookie attributes second. Secure + HttpOnly + SameSite is configuration and a couple of lines; it can ship in the same deploy as (1) and be verified in one browser session. Removing HttpOnly later is trivial; retrofitting it after an XSS incident is not.
3. Keep-alive timeout third. A one-line, reversible config change with a clear verification method. It is not a security finding, but it is customer-visible availability and will keep paging on-call, and it touches proxy configuration, so review it alongside the X-Forwarded-Proto trust decision from (2) - both are about how much the app believes the proxy.
4. GET-to-POST migration last. This is an API contract change: existing clients, mobile builds, and any partner integrations must move. Rushing it breaks users. In the interim, SameSite and a CSRF/Origin check reduce the residual risk, and the audit can be answered with the migration scheduled and the interim control in place.
The ordering principle: irreversible or externally-visible changes go last; cheap, high-severity, low-risk changes go first.