45 TB of Images Outgrowing Block Storage
A photo-print service stores about 45 TB of customer images - originals plus generated previews - and grows roughly 2 TB per month. Six Block Storage volumes of about 7.5 TB each are attached to two application servers, and a nightly rsync job copies them between hosts for redundancy. The catalog lives in a Managed Database in the same region. Roughly 90% of reads happen in the first 30 days after upload, and the service pushes about 12 TB of image egress per month. Pages and thumbnails are served by the application servers themselves, peaking at 900 requests per second during holiday campaigns. The volumes are approaching the per-volume size limit, the rsync window now runs five hours, and the team wants lower cost and simpler scaling without rewriting the app's file paths.
1. Recommend a storage architecture and compare monthly cost for the bulk image data only. Use Block Storage at $0.10/GB-month and Object Storage at $5 per TB-month, which includes 1 TB of storage and 1 TB of egress per TB stored.
Read the worked answer
Block Storage today: 45,000 GB x $0.10 = $4,500/month, plus 12 TB of egress that the app servers must push, plus a five-hour rsync window that grows every month.
Object Storage: 45 TB x $5 = $225/month. The plan bundles 1 TB of egress for every TB stored, so 45 TB of included egress covers the current 12 TB with room to grow; extra egress would be billed per GB above that allowance.
Savings are roughly $4,275/month, and growth costs $10/month per added TB instead of $100.
There is a second, non-cost argument: the 7.5 TB volumes are near the per-volume ceiling and rsync is a manual replication strategy. Object Storage removes both constraints and gives an S3-compatible API the application can write to directly. Keep a small Block Storage volume for working files, temp renders, and anything needing POSIX semantics, but move the image corpus.
2. The lead engineer wants to serve every thumbnail directly from Object Storage to browsers. Where do you agree, and where do you push back?
Read the worked answer
Agree on the direction. Application servers should stop being the file servers; offloading object reads removes the majority of their disk I/O and lets them scale with request count rather than stored bytes.
Push back on three points:
- Authorization. Customer photos are private. Raw object URLs must not be guessable or permanent; issue short-lived pre-signed URLs from the app after it authenticates the request, and keep the bucket non-public.
- Latency and caching. Origin reads at 900 requests/second during campaigns still cost latency and egress. Put a CDN in front of origin so repeat thumbnails are served from edge, and set cache headers per object class.
- Thumbnail generation. Do not resize per request. Generate the sizes you sell at upload time and store them as separate objects; per-request resizing reintroduces the compute load you just removed.
The judgment is that direct object access is correct only when it is signed, cached, and pre-rendered.
3. What must change in the application, and what should stay on Block Storage?
Read the worked answer
Application changes:
- Swap filesystem paths for an S3-compatible client behind a thin storage interface, so the code can still be tested locally.
- Define an object key convention (customer/date/asset-id/size) and store keys in the Managed Database rather than deriving them from local directory structure.
- Make uploads write to Object Storage first, then record metadata; add a reconciliation job for orphans.
- Generate signed URL endpoints and set cache-control metadata at write time.
Keep on Block Storage: render scratch space, the print pipeline's intermediate files, and any component that needs a real filesystem with locking or mmap. These are high-churn and low-durability-value.
Retire: the rsync replication job. Durability now comes from the object store; validate that assumption with a quarterly restore test rather than a nightly copy.