Tiering 80 TB of media masters and proxies
A media production company stores 80 TB of video masters and 12 TB of low-resolution proxies in NIFCLOUD object storage. Proxies are read daily by editors and served through a CDN; masters are requested about 1% of the time per month, usually with 24 hours' notice, but the legal team requires any master to be retrievable within 4 hours during an active dispute. The company must retain masters for 10 years and has 3 TB under legal hold. Storage cost has grown 35% year over year, and the CFO has asked whether everything can go to archive storage. The team has no lifecycle rules today; all objects are in one bucket, and deletion is allowed by anyone with the bucket key. A test restore from archive took 6 hours for a 500 GB master.
1. Recommend a storage tiering and lifecycle design for masters, proxies, and legal-hold data. Justify each tier choice against the access and recovery requirements.
Read the worked answer
The requirements pull in different directions, so a single tier is not appropriate.
- Proxies (12 TB): Keep in standard object storage. They are read daily, served through a CDN, and need low-latency access. Moving them to archive would add retrieval delays and likely increase cost through repeated restores.
- Masters (80 TB): The normal access rate is low, about 1% per month, but the 4-hour retrieval requirement during an active dispute is the deciding constraint. Archive storage is attractive on cost, but the observed 6-hour restore for a 500 GB master already exceeds the 4-hour objective. A safer design is to keep masters in object storage with lifecycle rules that move only older, non-disputed objects to archive, while maintaining a small set of recently requested or high-risk masters in a faster tier. Alternatively, use archive only after confirming that the provider's retrieval option can meet 4 hours for the largest expected master.
- Legal hold (3 TB): Separate bucket or prefix with an explicit retention policy. Legal-hold data must not be deleted by lifecycle rules or by ordinary bucket-key holders. Keep it in a tier that satisfies the dispute retrieval requirement; do not archive it just because it is old.
The key judgement is that archive storage is a cost tool, not a compliance tool. The 4-hour recovery objective is a hard constraint that overrides the CFO's blanket 'everything to archive' proposal.
2. Evaluate the CFO's proposal to move all objects to archive storage. What risks and costs would you present?
Read the worked answer
The proposal reduces the storage line item but can increase total cost and operational risk.
- Retrieval time risk: A 500 GB master took 6 hours to restore in a test, and the legal requirement is 4 hours. If a dispute requires a larger master, the gap worsens. Missed legal deadlines are a business risk, not just an IT inconvenience.
- Retrieval cost: Archive storage usually charges for retrieval and for the number of requests. The 1% monthly access rate sounds small, but 1% of 80 TB is about 800 GB per month, and repeated restores during a dispute can multiply. Model retrieval charges for the largest expected file, not just the average.
- Proxy impact: Moving proxies to archive would break daily editor workflows and CDN caching. Every proxy request could become a restore operation. This alone makes 'all objects' unreasonable.
- Deletion and legal hold: A single bucket with broad delete rights is dangerous. If archive lifecycle rules are added without separating legal-hold data, a retention rule could conflict with a legal obligation.
- Minimum storage duration: Archive tiers often have minimum-duration charges. Objects deleted early may still be billed, so the cost reduction may be smaller than expected.
A better presentation to the CFO is a tiered cost model with three lines: proxies in standard object storage, masters split between standard and archive based on retrieval class, and legal-hold data in a protected bucket. Show the cost of one missed 4-hour recovery alongside the storage savings.
3. The current bucket allows deletion by anyone with the bucket key. Propose controls to reduce deletion risk without blocking legitimate lifecycle management.
Read the worked answer
Deletion risk should be addressed at the identity, bucket-policy, and lifecycle layers.
- Separate human delete rights from automated lifecycle rights. Most engineers should be able to read and upload objects but not delete masters. Deletion of masters should require a separate role or a break-glass procedure.
- Use bucket policies or IAM conditions to restrict delete operations by prefix. For example, the legal-hold prefix should deny deletion to all ordinary users and allow only a compliance role. Production masters should have a similar restriction.
- Enable versioning or object locking where the service supports it. Versioning turns an accidental delete into a recoverable event; object locking or a retention setting protects legal-hold data from overwrite and deletion.
- Make lifecycle rules explicit and reviewable. Lifecycle transitions should be defined by prefix and age, and changes to lifecycle rules should be logged and require approval. A lifecycle rule that expires objects is effectively a delete permission, so it must be governed like one.
- Monitor delete events. Alert on delete operations against the masters and legal-hold prefixes, especially bulk deletes outside a change window.
- Test recovery. Periodically restore a deleted master from a version or backup and record the time. A control that has never been tested is an assumption.
The design principle is least privilege for deletion, with legal hold and production masters treated as separate protection domains rather than as ordinary bucket contents.