Rendition Pipeline Overload After a Bulk Upload
A retail client uploads 60,000 product images in a single overnight batch ahead of a seasonal launch. Their Widen configuration generates five renditions per image (thumbnail, web, print, zoom tile set, and a watermarked preview). Within hours, users report that new uploads sit in "processing" for hours, the asset detail pages are slow, and the launch microsite's zoom viewer is serving broken tiles. The batch included 12,000 images that are actually duplicates of existing assets and 4,000 that are corrupt or zero-byte. The launch is in three days. The team has one administrator and no maintenance window approved yet.
1. Diagnose the likely cause of the slowdown and prioritize what to fix first given the three-day deadline.
Read the worked answer
Diagnosis: the rendition pipeline is saturated. 60,000 uploads × 5 renditions = 300,000 rendition jobs queued at once. Processing is FIFO-ish and single-threaded per queue, so everything behind the batch waits. The zoom tile set is the most expensive job, which is why the zoom viewer specifically shows broken tiles: tiles for partially processed assets are missing or stale. The 4,000 corrupt/zero-byte files also waste processing slots and can throw errors that clog the queue.
Priority order (deadline-driven):
- Stop feeding the fire. Pause or throttle further bulk uploads so the queue can drain.
- Purge the junk. Identify and remove the 4,000 corrupt/zero-byte assets so they stop consuming rendition attempts.
- Handle the 12,000 duplicates. Deduplicate so you are not generating renditions twice for the same image.
- Protect the launch path. The microsite's zoom viewer is customer-facing; make sure the launch assets' tile sets are prioritized/complete before nice-to-haves like watermarked previews.
- Defer non-critical renditions. If the platform allows, disable or delay the print and watermarked renditions for the batch so the queue drains faster on what matters.
Communicate a realistic ETA to stakeholders rather than promising all 60,000 by launch.
2. What would you change about the ingestion process to prevent a repeat, and how do you justify the change to a client who sees it as added friction?
Read the worked answer
Process changes:
- Pre-ingestion validation: reject zero-byte/corrupt files and flag duplicates before they enter the DAM, so the pipeline only ever sees clean, unique assets.
- Staged/batched uploads: cap batch size (e.g., a few thousand at a time) with pauses, so rendition queues drain between waves instead of stacking 300,000 jobs.
- Rendition policy by asset type: not every product image needs all five renditions. Generate print and watermarked previews only when needed, or on a schedule, rather than eagerly on upload.
- Deduplication at the door: hash or match on filename+dimensions to catch the 12,000 duplicates automatically.
Justifying it: frame it as protecting the launch, not slowing the team. The friction they fear (validation, batching) is far cheaper than a three-day outage where the customer-facing zoom viewer is broken. Quantify it: this incident cost hours of processing backlog and a degraded microsite during a seasonal launch. A validation step that takes minutes on upload prevents that. Offer to build the checks into their existing upload workflow so it is invisible to end users - the goal is fewer rejected assets later, not more steps now.