Java EE 7 Application Developer: Thread Pool Exhaustion on a Blocking Search Servlet

Thread Pool Exhaustion on a Blocking Search Servlet

An online ticketing site runs on a Java EE 7 full-profile server. The search page is a Servlet 3.1 endpoint mapped to /search that, on every request, calls a partner inventory service over HTTPS with a 1.8-second round trip. Load tests show throughput collapsing at about 120 concurrent users: the container's 200 request worker threads are all blocked in socket reads, requests queue, and users receive 503 after 30 seconds, while CPU sits at 8 percent. The team raised maxThreads to 800; latency grew further and large response writes started failing with OutOfMemoryError.

Two more symptoms appeared in the same release. A logging filter declared with @WebFilter("/*") sometimes writes its entry line after the authentication filter that is declared in web.xml, so unauthenticated requests occasionally reach the search servlet. And a HashMap stored in the HttpSession under the key "cart" silently loses line items when a user opens two tabs and both submit at the same time.

1. The architect proposes converting /search to an asynchronous servlet using startAsync(). Explain what that changes about thread usage, what must be configured for it to work at all, and what can still go wrong under sustained overload.

Read the worked answer

What async does and does not buy. The servlet container's request thread is released back to the pool while the partner call is in flight; the work continues on another thread (a client library callback or a managed executor). That converts a thread-per-request architecture bounded by maxThreads into a concurrency-per-connection architecture bounded by the connector, the HTTP client's connection pool, and the executor size. It does not make the partner faster and it does not create capacity by itself.

Prerequisites. asyncSupported defaults to false. It must be true on the servlet and on every filter and servlet in the dispatch chain (@WebFilter(asyncSupported = true) or <async-supported>true</async-supported>), otherwise startAsync() throws IllegalStateException. Also required:

AsyncContext.setTimeout(...); the default is container-specific and the client will otherwise hang.
An AsyncListener for onTimeout, onError, and onComplete; on timeout you must write a response or dispatch, or the container completes with an error.
complete() (or dispatch()) in every path, including exception paths, or the request leaks.

What still breaks. If the partner takes 1.8 s and you allow 5,000 concurrent async requests, you simply move the queue to the HTTP client pool and to the partner, which will throttle or fail. Async needs backpressure: a bounded work queue with a fast 503/429 rejection, a circuit breaker, and cached or pre-computed inventory. Raising maxThreads failed because each thread holds a stack and buffers while doing nothing - memory grew with concurrency and nothing got faster. Async removes that per-request memory cost, but only admission control protects the downstream system.

2. Why is the @WebFilter logging filter sometimes ordered after the web.xml authentication filter, and how do you make the order deterministic? Does the async change affect those filters?

Read the worked answer

Ordering rule. For @WebFilter, the servlet specification states that the order in which filters are invoked is not defined when they are declared by annotation - the container is free to pick any order. Only <filter-mapping> elements in the deployment descriptor establish a deterministic chain, and the order is the order of the <filter-mapping> entries, not the order of the <filter> declarations.

Fix options.

Declare both filters (and both mappings) in web.xml and order the mappings: authentication first, logging second. This is the only fully portable fix.
Keep the annotations but add <filter-mapping> entries in web.xml for both classes. Be careful: adding a mapping for an annotated filter can cause the filter to be mapped twice unless you set <filter><filter-name>...</filter-name><filter-class>...</filter-class><async-supported>true</async-supported></filter> metadata-complete, so the cleanest answer is option 1.
Merge the two concerns into a single filter whose internal order is explicit.

A filter that performs authentication must be the first mapping, because a filter can only short-circuit requests that have not yet passed further down the chain.

Async interaction. If the servlet calls startAsync() and any filter in the chain does not support async, the request is rejected with IllegalStateException at startAsync(). The container does not silently skip such a filter. So both the authentication filter and the logging filter need asyncSupported=true (or <async-supported>true</async-supported>), and any state they store in ThreadLocal will not be visible after the async dispatch unless it is re-established in an AsyncListener or the request attributes are used instead.

3. Explain the lost cart line items and describe a fix that survives both concurrent requests and a two-node cluster.

Read the worked answer

Cause. HttpSession attributes are plain object references. The container guarantees that the session object can be looked up, not that attribute objects are thread-safe, and it will happily run two requests for the same session on two threads at once (two browser tabs, or one page firing several XHRs). Two threads calling map.put on a HashMap race: entries are lost, and during a resize a get can loop or return null. synchronized(session) helps only in one JVM and is discouraged because the container may wrap the session in a facade whose identity changes across requests.

Fixes, in order of preference.

Do not keep mutable shared state in the session. Store the cart in the database keyed by a cart id, load it inside a transaction, and let JPA optimistic locking (@Version) arbitrate concurrent updates - the second tab gets an OptimisticLockException and the UI can merge or refresh. This also survives node failure.
If the cart must stay in memory, use a ConcurrentHashMap or a session-scoped CDI bean whose mutating methods are synchronized on a dedicated private lock object. Correct within one JVM.
For a cluster, in-memory approaches still break: most containers replicate at request end and last-write-wins, so the second node's copy overwrites the first. Sticky sessions reduce but do not remove the window (failover, redeploy). Replication also requires every attribute value to be Serializable, and mutations made without a subsequent setAttribute may not be detected - many containers only replicate on setAttribute.

Guard the checkout path with a server-side cart version and reject stale submissions, so a lost update cannot turn into a double charge.