Replacing Tape for a 40 TB File Estate on a Saturated Direct Connect

Compute the pipe before choosing the product. 200 Mbps at 60% utilization leaves ~80 Mbps, or about 10 MB/s. Seeding 40 TB: 40,000,000 MB / 10 MB/s = 4,000,000 seconds = 46 days of continuous transfer. That is not a seed, it is a project. Use an AWS Snowball Edge device for the initial load and a DataSync agent on-premises afterwards.

Ongoing: 12 TB/month = 12,000,000 MB / 10 MB/s = 1,200,000 seconds = ~14 days of continuous transfer per month. That fits, but it consumes roughly 46% of your available bandwidth every month before any other traffic grows. Two consequences: schedule DataSync into a bandwidth-limited window and monitor it, and put the DX upgrade to 1 Gbps on the roadmap now rather than after the first missed backup.

The 90%-rewrite detail matters. DataSync compares modification time and size, so a rewritten file with changed content is re-transferred in full - your monthly transfer really is 12 TB, not the net byte delta. Do not plan on deduplication you do not have.

Why not File Gateway alone: S3 File Gateway would give you NFS semantics with local caching and continuous upload, which is attractive for a 1-hour RPO because it removes the nightly job entirely. But it does not solve the 40 TB seed over a saturated link any faster, and it adds an appliance to manage. A defensible split is Snowball for the seed, then File Gateway for steady-state continuous replication, with DataSync retained for the periodic bulk reconciliation.

Reject: a site-to-site VPN over the internet for the seed. 40 TB over an internet path fails the compliance requirement for the data and will not finish sooner.

Target store: S3 with versioning on, Object Lock in compliance mode for the 7-year term, and a bucket policy that denies s3:DeleteObjectVersion and s3:PutBucketVersioning to every principal including account root. Compliance mode is the correct choice for a regulator: governance mode can be bypassed by anyone holding s3:BypassGovernanceRetention, which makes it an operating control, not an immutability guarantee. Note the consequence out loud - compliance-mode objects cannot be deleted early, even in error, for 7 years.

Lifecycle: day 0 Standard, day 30 Glacier Instant Retrieval (retrieval must be minutes for the claims team), day 90 Glacier Deep Archive, expire at 2,557 days. Transitioning to Deep Archive is where the money is, but only for objects above the 128 KB lifecycle floor - scanned policy PDFs are typically 200 KB to 2 MB, so unlike a small-object archive, these will actually transition.

Growth model for Finance. Versioning plus 90% rewrites means the estate grows by the full 12 TB/month regardless of net content change: 40 TB + (12 TB x 84 months) = ~1,048 TB at year 7. At that point nearly all of it sits in Deep Archive at $0.00099/GB: 1,024,000 GB x $0.00099 = ~$1,014/month, plus a rolling ~90 days of hotter tiers (roughly 40–76 TB) in Standard and Instant Retrieval, which is the expensive part at $0.023 and $0.004 respectively.

Give Finance three numbers, not one: month 1 ($60–90, because almost nothing has accumulated), month 12, and month 84 ($1,100–1,400). The single steady-state figure will be rejected as implausible against this month's bill, and the year-7 figure alone will look alarming. Also price the retrieval side explicitly: Deep Archive standard retrieval is $0.02/GB, so a full 40 TB restore costs roughly $800 in retrieval fees alone before compute.

Name the finding first: a full restore probably cannot meet the 24-hour RTO. Deep Archive standard retrieval takes up to 12 hours and supports bulk retrieval at up to 48; add download and verification of 40 TB at your DX capacity and the current quarterly test is likely already exceeding the target. If the test cannot pass, the RTO is fiction, and that is the most valuable thing the exercise produces. Either narrow the RTO to 'documents retrievable within 24 hours' and prove it with sample restores, or budget to keep the most recent 90 days in Instant Retrieval so a full restore is a hot-tier operation.

Evidence-grade sampling design:

• Draw a stratified random sample each quarter - for example 2,000 objects spanning all 84 monthly partitions, all source systems, and both the Instant Retrieval and Deep Archive tiers.
• Restore into an isolated account (not production), verify SHA-256 checksums against a stored manifest, and confirm the restore is recorded by CloudTrail with the requester, time, and object count.
• Compare an S3 Inventory report against the source system's file manifest to prove nothing is silently missing - restores prove recoverability, inventories prove completeness, and auditors want both.
• Automate with AWS Backup restore testing plans where the resource type supports it, and attach the generated report to the compliance record.

Cost and timing of the test: 2,000 objects plus inventory scanning is a few dollars and hours, not $800 and days, so you can run it monthly instead of quarterly and build a trend line rather than a single data point.

Keep one full restore per year as the exception, scheduled with the retrieval lead time accounted for, because sampling can miss a systemic failure - a corrupted index, an expired KMS key, or a lifecycle rule that transitioned the wrong prefix will show up in a full restore and may not appear in a 2,000-object sample.