Virtual Threads That Stop Scaling
A document service blocks about 200 ms per request on a remote parser. With a fixed pool of 200 platform threads and 5,000 concurrent requests, p95 latency is 4 s. The team moves to Executors.newVirtualThreadPerTaskExecutor() on a 2-core container and throughput improves tenfold. Then the persistence layer lands: every task runs its JDBC call inside synchronized (lock) { ... }. Throughput collapses to roughly the old baseline and some requests hang past the 30 s client timeout, although Thread.currentThread().isVirtual() returns true inside them. A ThreadLocal<SimpleDateFormat> is still cached per thread, and a shared int activeCount++ executed inside the tasks ends up lower than the number of completed requests.
1. Explain precisely why `synchronized` around the blocking call destroys the gains, and what the replacement is.
Read the worked answer
A virtual thread unmounts from its carrier thread when it blocks on I/O - that is the whole reason 5,000 virtual threads can run on 2 cores. But in Java 21, a virtual thread that blocks while holding a synchronized monitor cannot unmount: it pins its carrier. The carrier pool is the ForkJoinPool scheduler with parallelism equal to Runtime.availableProcessors() - here 2. So at most two conversions can be inside the critical section (or even waiting to enter a monitor held across blocking I/O), and the server behaves like a 2-thread server regardless of how many virtual threads exist.
isVirtual() returning true is expected and proves nothing; use jcmd <pid> Thread.dump_to_file -format=json and look for pinned to confirm.
Replacement:
lock.lock();
try { jdbcCall(); } finally { lock.unlock(); }
ReentrantLock blocks by unmounting, so carriers stay free. Also keep monitors short and never hold one across a network or disk call.
2. The `ThreadLocal<SimpleDateFormat>` and the `activeCount++` counter each cause a distinct problem. Identify both and fix them.
Read the worked answer
ThreadLocal: the old advice existed because platform threads are pooled. Virtual threads are created per task - potentially millions - so a per-thread cache multiplies into a memory leak rather than a cache, and a mutable SimpleDateFormat shared any other way is not thread-safe at all. DateTimeFormatter is immutable and thread-safe: make it a static final and drop the ThreadLocal. Any genuinely per-request state should be a parameter or record field; scoped values are still a preview API in 21, so do not rely on them for production code here.
Counter: activeCount++ is a read-modify-write on a shared field with no atomicity and no happens-before edge, so concurrent increments are lost and reads may be stale - a final value lower than the number of completed requests is the textbook symptom. Use AtomicInteger.incrementAndGet() / decrementAndGet(), or LongAdder under heavy contention. If each task returns its own result, sum from the Futures instead and share nothing. volatile alone does not fix a compound action.
3. A colleague proposes `Executors.newFixedThreadPool(1000, Thread.ofVirtual().factory())` to "bound concurrency". Evaluate that and describe how to actually protect the database.
Read the worked answer
Pooling virtual threads is the wrong tool: they are cheap and disposable, and a pool reintroduces exactly the queueing you were trying to escape. A fixed pool of 1000 virtual threads caps concurrency at 1000 for no benefit and adds hand-off overhead; newVirtualThreadPerTaskExecutor() is the intended shape (one virtual thread per task, use try-with-resources since ExecutorService is AutoCloseable in 21).
What actually needs bounding is the scarce resource - the JDBC connection pool. Bound that with a Semaphore sized to the pool (and to the database's realistic capacity), acquired around the database section:
semaphore.acquire();
try { lock.lock(); try { jdbcCall(); } finally { lock.unlock(); } }
finally { semaphore.release(); }
Also set pool and query timeouts so a stuck call fails fast instead of queueing virtual threads behind it. The general rule: virtual threads make waiting cheap, so bound the scarce resource, not the threads.