Designing Storage Tiers for a Growing Photo Archive
A photography SaaS stores customer image libraries. Total volume is 400 TB and growing ~15 TB/month. Access patterns are sharply skewed: about 3% of objects (recent uploads and frequently viewed galleries) account for 92% of reads; the remaining 97% are viewed rarely, typically once or twice in their first month and then almost never. The product must serve the hot 3% with low latency (under 100 ms) and must retain cold objects for at least 7 years for contractual reasons. Compliance requires that deleted objects be recoverable for 30 days. The finance team is pushing hard on storage cost because cold data dominates the bill. The engineering team is small and does not want to build custom tiering logic.
1. How should the team structure storage tiers, and what is the risk of simply moving everything to the cheapest tier?
Read the worked answer
Structure around the access skew, not the total volume:
- Hot tier (standard object storage) for the ~3% actively read. This is small in bytes relative to 400 TB but must stay in a low-latency class to meet the <100 ms requirement.
- Infrequent-access tier for objects past their early-life reads (say, older than 30 days) but still plausibly retrievable.
- Archive/cold tier for the long-tail objects that must be held 7 years.
The risk of dumping everything into the cheapest archive tier: retrieval cost and latency explode. Archive classes typically charge for retrieval and impose restore delays (minutes to hours). If even a small fraction of the 97% gets requested - a customer revisiting an old gallery, a support ticket - those retrieval fees and delays can erase the storage savings and break the latency promise. The skew means the hot 3% must never be demoted prematurely.
The judgement: match tier to access probability and retrieval tolerance, and keep a low-latency floor for anything a user might touch interactively.
2. The team does not want to write custom tiering code. What managed capability should they use, and how should lifecycle rules be designed?
Read the worked answer
Use the object store's native lifecycle management with automatic tiering/transition rules, so demotion happens on a policy rather than bespoke code:
- Transition rule: move objects from standard to infrequent-access after ~30 days of no access (aligns with the observed first-month read pattern), then to archive after a longer idle period (e.g., 180 days).
- Retention rule: keep archive objects for 7 years to satisfy the contract, then expire - but only after legal sign-off.
- Deletion recovery: configure versioning plus a 30-day noncurrent-version retention / soft-delete window so deleted objects remain recoverable, meeting the compliance requirement without a separate backup system.
- Guardrails: exclude a small "pinned" prefix or tag for galleries customers mark as favorites, so an automated rule never demotes something in active use.
Key point: the skew is time-based as well as object-based (hot when new, cold later), so age + last-access rules capture most of the benefit. Review the rules quarterly against actual access logs to tune thresholds.
3. How should the team verify the cost savings are real and not undermined by hidden charges?
Read the worked answer
Model the full cost of ownership per tier, including operations, not just storage per GB:
- Storage cost by tier: compute GB-months per tier under the proposed rules using a sample of real access logs.
- Request and retrieval costs: cold tiers often bill per retrieval and per GB retrieved. Estimate expected retrievals from historical access; a small retrieval rate on 400 TB can dominate.
- Minimum storage duration charges: many cold tiers bill a minimum retention period (e.g., 30/90/180 days). Objects deleted or moved early incur an early-deletion fee. Check whether churny objects would be penalized.
- Transition costs: each lifecycle transition may have a per-request charge. At 15 TB/month of new objects, transition volume matters.
- Latency/compliance cost: confirm the <100 ms promise still holds for anything a user can reach interactively; if not, the savings are illusory because you will re-promote data.
Run the numbers on a 30-day pilot of one customer segment before company-wide rollout. The judgement: cheap-per-GB is not cheap-in-practice when retrieval, minimum-duration, and transition fees are included.