Choosing a Caching Layer Under Deadline

The proposal is reasonable in principle because the workload is read-heavy and the database has little headroom, but the details matter. A 60-second TTL matches the editors' stated expectation of roughly one minute, so the freshness requirement is not obviously violated. The bigger risks are consistency and operational behaviour. With 20 instances each holding its own cache, two users hitting different instances can see different catalogue versions for up to 60 seconds, and during a deployment the restarted instances will start cold and hit the database simultaneously. That cold-start stampede is the risk most likely to cause an incident, because it concentrates load on a database that is already at 70 percent CPU.

A second risk is that the 60-second TTL is an average, not a guarantee. An item written just before its entry expires can be served stale for nearly the full TTL, and an item written just after a refresh can be stale for almost zero seconds. If editors publish a price change, some users may see the old price for up to a minute, which may be acceptable for descriptions but not for prices. I would check whether the catalogue contains fields with different freshness needs and consider splitting them. A third risk is memory: 20 instances each caching the catalogue multiplies memory use, and a large catalogue could cause eviction churn that makes the cache ineffective. Finally, there is no shared invalidation, so an emergency takedown of a product cannot be propagated quickly. These risks are manageable but must be designed for, not discovered in production.

I would keep the in-process cache because it is the fastest option to build and requires no new infrastructure, but I would change three things. First, I would split the cache by freshness class: static descriptive fields get a longer TTL, while price and availability get a short TTL or are read through. Second, I would add a shared invalidation channel, for example a lightweight pub/sub message that the editorial tool publishes on every change, so instances can drop the affected entry immediately rather than waiting for expiry. That closes the emergency-takedown gap at modest cost. Third, I would protect the database from cold-start stampedes by warming the cache on startup with jittered timing and by adding a short request-coalescing layer so concurrent misses for the same key result in one database read.

The trade-offs are explicit. We accept up to the TTL of staleness for descriptive fields in exchange for a large reduction in database reads. We accept the operational complexity of a pub/sub dependency; if the channel fails, the system degrades to TTL-based freshness rather than breaking, which is an acceptable failure mode. We do not build a shared distributed cache in four weeks, because that adds a new system to operate and a new failure mode, and the team does not have time to do it safely. We also accept that memory use per instance grows, so we would cap the cache size and monitor eviction rate. I would document these trade-offs and set a review point after launch to decide whether a shared cache is warranted once traffic doubles.