Image Upload Service Blocking the Event Loop and Retrying
PixelServe accepts image uploads through Express with multer memory storage and a 25 MB limit. It then runs sharp to resize, calls fs.writeFileSync to disk, and returns a base64 thumbnail in JSON. The container has 512 MB RAM and 1 vCPU. Under 30 concurrent uploads, event loop lag reaches 4.2 seconds, health checks time out, and the process runs out of memory. Logs show MaxListenersExceededWarning and ENOSPC on /tmp. A client retry storm sends the same 25 MB file three times because the client times out at 10 seconds. The team wants to keep synchronous code for readability and asks whether cluster will fix the CPU spikes. nginx has client_max_body_size 30m and proxy_read_timeout 10s. There is no queue and no object storage; it is a single host.
1. Separate the event-loop, memory, and CPU problems in this service. Explain why `cluster` is not the right first fix, and identify what `MaxListenersExceededWarning` and `ENOSPC` each indicate.
Read the worked answer
fs.writeFileSync blocks the event loop for the duration of each disk write, and sharp resize is CPU-bound native work that also occupies the main thread unless dispatched to a worker. Memory pressure comes from buffering entire 25 MB files in multer memory storage, then creating base64 strings, which add roughly 33% overhead, and from concurrent requests each holding buffers. The 512 MB limit is exhausted before the 1 vCPU is fully used. cluster forks multiple processes; each process has its own memory, so it does not reduce per-request buffering and can make OOM worse on a 512 MB host. It also does not make synchronous disk I/O non-blocking. MaxListenersExceededWarning usually means listeners or event emitters are being added per request without removal, for example stream or sharp listeners accumulating under concurrency. ENOSPC means the filesystem or inode table is full, likely because failed or retried uploads leave partial files in /tmp. First fix the request path: stream to disk or object storage, bound concurrency, and clean up partial files.
2. Design a streaming upload path with backpressure, bounded concurrency, and explicit limits. Include the status codes and error handling you would return.
Read the worked answer
Stop using memory storage. Accept the upload as a stream and pipe it to a temporary file with a byte counter that aborts and deletes the partial file once the limit is exceeded. Use stream.pipeline so errors propagate and destinations are destroyed; honor backpressure by awaiting the pipeline rather than calling write in a loop. Enforce Content-Length when present and also count bytes, because the header can lie. Set nginx client_max_body_size and Express limits consistently, and return 413 Payload Too Large for oversize bodies. Dispatch sharp work to a worker thread pool with a small concurrency limit, for example two workers on one vCPU, and reject or queue beyond that with 503 Service Unavailable and Retry-After. Write the final file atomically by renaming from a temporary name in the same filesystem. Return 400 for unsupported image types, 422 for a file that claims to be an image but fails decoding, and 500 only for unexpected failures. Ensure every error path unlinks the temporary file.
3. The retry storm sends the same file three times. Propose idempotency and timeout handling that aligns nginx, the client, and the Node process without adding a full queue.
Read the worked answer
Require an Idempotency-Key header per upload attempt. Store a short-lived record keyed by that value with states such as in_progress, completed, and failed, plus the resulting file identifier. On duplicate requests, return the existing result or 409 Conflict while the first attempt is still running, rather than starting a second resize. Align timeouts: nginx proxy_read_timeout of 10 seconds is shorter than the work the service is doing, so the client sees a timeout and retries while Node continues. Either raise the proxy timeout to a realistic value for the bounded operation, or change the contract to return 202 Accepted with a job id and let the client poll. The latter is better for large files and avoids holding the connection. Add a request-level deadline in Node and abort the pipeline when it expires. Do not rely on cluster or on retries alone; the key must make the second and third deliveries harmless. Clean up idempotency records and temporary files with a TTL.