Storage and Data Management
Storage and Data Management tests mapping data to the correct store - object, block, file, relational, NoSQL, warehouse, cache - and then to the correct tier and protection mechanism. The decisive inputs are access frequency, retrieval latency tolerance, sharing model, and retention rules.
S3 classes and lifecycle
S3 Standard for hot data; Intelligent-Tiering for unknown or shifting patterns; Standard-IA and One Zone-IA for infrequent access (One Zone-IA survives a single AZ loss, not an AZ failure); Glacier Instant Retrieval for archive with millisecond access; Glacier Flexible Retrieval (minutes to hours); Glacier Deep Archive (12-hour standard retrieval, 48-hour bulk). Durability is eleven nines across Standard classes. Lifecycle rules transition and expire objects; versioning protects against deletion and overwrite; Object Lock supports WORM retention.
Block and file storage
EBS volumes are network-attached block storage bound to one AZ - an instance and its volume must match. gp3 is general purpose, io2 is high-IOPS, st1 is throughput-oriented HDD, sc1 is cold HDD. Snapshots are incremental and stored durably, enabling cross-AZ and cross-Region restore. Instance store is ephemeral and lost on stop. EFS is multi-AZ NFS file storage for Linux workloads; FSx for Windows File Server provides SMB; FSx for Lustre gives high-performance parallel file access for HPC.
Databases and data movement
RDS Multi-AZ uses a synchronous standby for failover; read replicas are asynchronous and offload reads. Aurora is MySQL/PostgreSQL-compatible with shared, replicated storage. DynamoDB is serverless key-value NoSQL. Redshift is a columnar warehouse for analytics. ElastiCache and MemoryDB serve caching and in-memory workloads. AWS Storage Gateway bridges on-premises systems to AWS storage; the Snow family moves large datasets physically.
Worked example
A regulator requires seven years of retention for records accessed roughly twice a year with a 48-hour retrieval tolerance. Reasoning: latency tolerance of 48 hours plus near-zero access eliminates Standard and Instant Retrieval. S3 Glacier Deep Archive holds the objects at the lowest cost, and a lifecycle rule transitions objects from Standard after 90 days and expires them at year seven, satisfying retention without manual scripts.
Traps that cost marks
- Assuming an EBS volume can attach across AZs - it cannot; snapshots are the migration path.
- Treating One Zone-IA as fault tolerant.
- Using RDS Multi-AZ standby to serve read traffic - that is what read replicas do.
- Mounting S3 as a shared file system instead of choosing EFS or FSx.