Cross-region key reuse for regulated object storage
A lender stores 4 TB of loan documents in two IBM Cloud Object Storage buckets: one in us-south and one in eu-de. Both buckets use a single Key Protect instance in us-south and one root key. The EU regulator requires customer-managed keys, dual control, and FIPS 140-2 Level 4 for the EU bucket; the US bucket requires Level 3. The team plans to rotate the root key every 90 days and has a 7-day deletion window. A recent test showed that disabling the root key made both buckets unreadable, and that the EU bucket has no regional Key Protect instance to manage its key. The team wants a corrected key topology and a safe rotation and deletion procedure that still allows recovery from a regional outage.
1. Diagnose the design flaws in the current key topology and recommend the corrected regional design.
Read the worked answer
There are three flaws. First, the EU bucket depends on a US key instance, which is a data-residency and availability problem: the EU data cannot be managed under EU control, and a US regional outage or key disablement affects EU access. Second, one root key protects both buckets, so there is no separation between jurisdictions or data classifications, and one mistaken disable or deletion has blast radius across both. Third, Key Protect does not meet the stated FIPS 140-2 Level 4 requirement for the EU bucket; that requirement points to Hyper Protect Crypto Services.
The corrected design should be regional and purpose-specific. In us-south, create a Key Protect instance and a root key dedicated to the US bucket. In eu-de, create a Hyper Protect Crypto Services instance and a separate root key dedicated to the EU bucket, with dual control configured and the customer retaining control of the master key material. Each COS bucket should be authorized to use only its regional key instance. Do not use one key ring or one root key as a convenience across regions.
If cross-region copies are needed for resilience, each copy should be encrypted under a key in its own region, not under a remote key. That may require re-encryption during replication rather than simply copying ciphertext. Document which key protects which bucket and region so that a future disablement has a known, limited scope.
2. Explain the key hierarchy and what must happen after a root key rotation so old key versions can be retired safely.
Read the worked answer
COS uses envelope encryption: COS generates data encryption keys for objects, and the root key in Key Protect or Hyper Protect Crypto Services wraps those data keys. The root key does not directly encrypt the 4 TB of object data. When the root key is rotated, a new key version is created. New data keys are wrapped with the new version, but existing objects still have their data keys wrapped by the old version. The bucket remains readable as long as the old version is available.
Rotation alone therefore does not retire the old version. To retire it, the team must rewrap or re-encrypt the affected objects so their data keys are protected by the new root key version, then verify a sample of objects can be read before disabling the old version. For 4 TB, this is a planned background job with throughput limits, not a same-day operation. The 90-day rotation schedule should include a completion check for rewrap, not just a key rotation event.
Deletion is stricter. The root key should have dual authorization, deletion should be restricted to a break-glass role, and alerts should fire on disable or schedule-deletion events. The 7-day window is a recovery aid, not a backup. Test the procedure in a non-production instance and document that deleting the root key makes the data unrecoverable even if the COS objects still exist.
3. How would you handle regional outage recovery without recreating the same cross-region key dependency?
Read the worked answer
I would design recovery around regional independence. For each region, keep the COS bucket, its key instance, and the service authorization local. For the EU data, maintain a replicated copy in a second EU region under a Hyper Protect Crypto Services instance in that region; for the US data, use a second US region with its own Key Protect instance. The replication process must produce ciphertext encrypted under the destination region's key, or the destination copy must be independently encrypted, so that loss of the primary region does not require a remote key that may also be unavailable.
Operationally, document a recovery order: confirm the key instance status, confirm the bucket and key service authorization, validate a sample object, then redirect the application. Keep break-glass procedures for key administrators in each region, with dual control and audit logging. Do not store the only copy of a root key's operational authority in one region.
Finally, test a regional failover at least annually, including a key-disable simulation in non-production. The test should prove that the EU bucket can be read using only EU-region services and that the US bucket cannot be affected by an EU key action. That is the property the current design lacks.