Animation Studio: Choosing Between SFS Turbo, EVS and OBS for a 400 TB Archive Plus a 20 TB Render Working Set
An animation studio keeps 400 TB of source footage and rendered frames in the cloud, growing about 30 TB per month. Only around 5% of files older than 30 days are read again, but when they are, artists need them within minutes for a re-render. A separate 20 TB working set is read concurrently by 60 Linux render nodes through a POSIX filesystem; the renderer cannot be rewritten to use a REST API, and aggregate read throughput during a job peaks at 8 GB/s. One engineer proposes mounting an EVS disk on each node and rsyncing the working set nightly. Regulators require all footage to be retained for seven years and produced on request within 48 hours. The studio wants the lowest storage cost that still satisfies the re-render and rendering requirements, and asks whether one service can cover both workloads or whether two must be combined.
1. Select the storage service for the 20 TB working set and justify the choice against EVS and OBS.
Read the worked answer
Choice: SFS Turbo (high-performance tier), mounted over NFS by all 60 nodes.
- Why not EVS: an EVS disk is a block device in one AZ attached to a single ECS (only specially configured shared volumes support a few nodes). 60 nodes would need 60 copies = 1.2 PB, and the nightly rsync window would either lag behind writes or serve stale frames mid-render. Cost and consistency both fail.
- Why not OBS alone: the renderer needs POSIX semantics - random reads, file locking, directory traversal, rename-heavy temp files. OBS is an object store with a REST API; a mounted object gateway hides latency and metadata-consistency problems that break render jobs at 8 GB/s. Rewriting is explicitly off the table.
- Why SFS Turbo fits: a single namespace shared by all nodes, NFS v3 mount, throughput that scales with the capacity tier purchased - size the tier so the guaranteed bandwidth exceeds 8 GB/s, with headroom for metadata operations.
Caveats to state: SFS Turbo is AZ-scoped, so render nodes must sit in the same AZ; plan a bandwidth test with the actual frame sizes, and keep the SFS Turbo share as a cache that is periodically flushed to OBS for durability rather than as the system of record.
2. Design the lifecycle policy for the 400 TB archive, given the 'minutes for re-render' and '48 hours for regulators' requirements.
Read the worked answer
Key fact that drives the design: Infrequent Access is readable immediately - only Archive and Deep Archive require a restore operation, and that restore takes minutes to hours (Deep Archive longer; confirm current SLA).
So a bucket-wide 'move everything to Archive after 90 days' rule violates the re-render requirement.
Design by prefix/project, not bucket-wide:
| Age | Class | Rationale |
|---|---|---|
| 0–30 days | Standard | Active production |
| 30–180 days | IA | Still plausible re-render targets; immediate read, lower storage price, retrieval fee |
| 180 days–2 years | Archive | Restore in minutes/hours; acceptable for occasional, planned re-renders |
| > 2 years | Deep Archive | 48-hour regulatory delivery is met even with a long restore |
Operational rules: closed projects move to Archive on a project-close tag rather than on age alone; keep an 'active-project' prefix exempt from all transition rules; expire incomplete multipart uploads; use OBS access logging to re-tune the 180-day boundary after six months of real data.
Expected saving comes from the bulk of 400 TB sitting in IA/Archive while only the live 20–30 TB stays hot.
3. What protection features should be enabled for seven-year regulated retention, and what cost or sequencing traps exist?
Read the worked answer
- Versioning: protects against overwrite and deletion, and is a prerequisite for cross-region replication. Trap: it multiplies storage cost for frequently overwritten objects, so pair it with a lifecycle rule that expires noncurrent versions after a defined period.
- WORM / retention policy: for the regulatory subset, an object-lock style retention prevents deletion even by an administrator. Apply it only to the compliance prefix - it is irreversible.
- Cross-region replication: gives a second copy in another region for disaster recovery. Needs versioning on both source and destination; historical objects are not replicated retroactively unless a backfill rule is configured.
- Server-side encryption (SSE-KMS or SSE-OBS) for footage that may contain client IP.
- Bucket policy + IAM least privilege: only the render service agency writes; humans get read-only on specific prefixes.
Sequencing traps: enabling versioning or WORM after the fact does not retroactively protect existing objects - do it before loading new footage; and lifecycle transitions are billed as requests, so a badly staged rule on 400 TB generates a one-off cost spike. Test the rules on a 1 TB prefix first.