40 TB Media Archive: Object Storage or More NAS?
A media company stores 40 TB of video masters and thumbnails - about 5 million objects - on two NAS appliances in a single datacentre. The volumes are 85% full, protected by RAID 6, and synchronised to each other nightly with rsync. Only about 3% of objects are accessed in a typical month, but the CDN origin pulls the newest 2 TB frequently. Legal requires seven-year retention and immutability for the first 90 days after upload; deletions must be recoverable for 30 days. The team proposes buying two larger NAS units and keeping the same design. Finance wants predictable storage cost and no large capital purchase. A developer needs an S3-compatible endpoint because the encoding pipeline already writes to S3. The existing 15 VMware VMs are backed up to the same NAS with 30-day retention. The company has one 1 Gbps internet link and no second site.
1. Recommend a target architecture for the archive and CDN origin, and explain why adding NAS capacity alone is insufficient.
Read the worked answer
Use Object Storage as the system of record for the 40 TB archive and keep only the hot 2 TB working set on NAS or a fast tier. Create an S3-compatible bucket, enable versioning, and use lifecycle rules: standard for the first 30 days, an infrequent-access tier for warm objects, and an archive tier for the seven-year tail. Put a CDN in front of the hot prefix, not the whole bucket, and set cache headers. RAID 6 and rsync give availability and a copy, not durability against deletion, ransomware or site loss; rsync also propagates deletions. Adding NAS capacity repeats the single-site, capital-heavy design and gives no S3 endpoint. Plan the initial 40 TB transfer: at 1 Gbps, 40 TB is about 320 Tb, or roughly four days at line rate before overhead, so seed over a scheduled link or use an approved transfer appliance and reconcile checksums. Budget for CDN egress and request costs, and measure the 3% access pattern before choosing tiers.
2. Design the retention, immutability and deletion-recovery controls needed to satisfy legal.
Read the worked answer
Enable versioning so deletes create markers rather than destroying data. For the first 90 days, use S3 Object Lock in compliance mode if the bucket supports it; otherwise enforce a bucket policy that denies s3:DeleteObject and s3:DeleteObjectVersion to all but the lifecycle role, and keep a separate immutable copy. Lifecycle should expire noncurrent versions after 30 days, matching the recoverability requirement, but not before. For seven-year retention, transition objects to archive and keep an inventory manifest with checksums. Test recovery by restoring a sample object and a sample prefix, not just by reading the lifecycle configuration. Restrict delete permissions to a break-glass role and log all data-plane events to the audit trail. Document that legal hold overrides lifecycle expiration. Review the configuration whenever retention policy changes.
3. How should the existing VMware backups be handled? Can they share the archive design?
Read the worked answer
Do not put VM backups in the same bucket or lifecycle as the archive. Backups need short RPO, fast restores and independence from the archive's retention rules; a lifecycle mistake or compromised archive credential must not delete backups. Use a separate bucket and credential, with Veeam writing to S3-compatible object storage and immutability for at least the 30-day retention. Keep a local performance tier if restore time matters, and copy to object storage. Schedule jobs outside the CDN's peak and check that the 1 Gbps link can carry both change traffic and backups; calculate daily change and the backup window. Test a full VM restore quarterly, and monitor repository capacity and job success. The NAS can remain a landing zone, not the only copy.