DP-420 study materials
DP-420 study resources

Ledger Consistency and Key Custody for a Regulated Bank

DP-420 · Put your knowledge to work in context.

All case studies

Ledger Consistency and Key Custody for a Regulated Bank

A European bank runs its ledger on Azure Cosmos DB for NoSQL with the write region in West Europe and read regions in North Europe and East US. Auditors require that any read issued after a write is acknowledged returns that write or a newer one, and that a write is never acknowledged before it is committed in a majority of replicas in the write region. The reporting service in East US frequently reads balances several seconds old even after the payments service in North Europe has committed them. Regulatory policy requires encryption keys for the account to be customer-managed in Azure Key Vault, rotated every 90 days, and revocable immediately on compromise. The analytics team must query two years of transactions but must not be able to read account-holder names or IBANs. Transactions are append-only; account balances live in a separate container.

1. Recommend a consistency level. Address whether the auditors' requirement is literally met, and what the bank gives up.

Read the worked answer

The requirement as written - any read issued after an acknowledged write returns that write or newer, with synchronous majority commit in the write region - is the definition of strong consistency. Bounded staleness, even at its minimum window (5 seconds / 100,000 operations), explicitly permits a read to return stale data inside that window, so it does not satisfy the sentence the auditors wrote. Session consistency satisfies it only for the client that performed the write.

What the bank gives up by choosing strong:

  • No multi-region writes. Strong consistency restricts the account to a single write region. If the roadmap includes active-active writes for resilience, strong and the roadmap are mutually exclusive; the bank must renegotiate the wording instead.
  • Read latency in East US. Strong reads outside the write region must be satisfied against the committed state, so remote reads inherit replication latency to West Europe. Expect tens of milliseconds, not single digits, for the reporting service.
  • Cost of being wrong at failover. A regional failover is only lossless if the target region has the committed state; the account-wide failover is a deliberate, slower operation than a single-write-region account with asynchronous replication.

Pragmatic path: propose strong for the ledger account and, if the East US read latency is unacceptable, run a formal risk acceptance for bounded staleness on the reporting path while keeping strong on the transactional path - which means splitting reporting into a separate account or a change-feed-projected container rather than weakening the account the payments service writes to. Be explicit in the design doc that 'bounded staleness with a 5-second window' does not meet the auditors' sentence.

2. Explain why the East US reporting service sees stale balances even though the payments service committed seconds ago, and give two ways to fix it that do not require strong consistency.

Read the worked answer

Both services are almost certainly on session consistency, which is the default. Session guarantees are scoped to a session token that the SDK maintains per client instance, per partition. The payments service's write advances its token; the reporting service in East US has its own token and no knowledge of that write, so it is entitled to be served from a replica that has not caught up. This is not a bug - it is the guarantee being narrower than the team assumes.

Fixes without going to strong:

  1. Propagate the session token. Read the x-ms-session-token response header from the payments write and pass it to the reporting read via ItemRequestOptions.SessionToken. The reporting read then blocks until the replica catches up. It works, but it couples the services: the token must travel through your message bus or API contract, and it must be the token for the right partition key range, not a blanket value.
  2. Stop reading the operational container for reporting. Build a projection with a change-feed processor and let reporting read that, accepting an explicit, measured lag (say under 5 seconds) that the business signs off on. This converts an implicit consistency surprise into a stated SLA.

Also worth checking before either fix: whether the reporting service's client is being re-created per request (which loses the session token entirely) and whether it reads with a query rather than a point read from a different partition key range than the write touched.

If the business genuinely cannot tolerate any lag, fix 1 plus a bounded-staleness account is a legitimate middle: bounded staleness guarantees all clients, not just the writer, are within the window, at the cost of slower writes in the write region.

3. Design key management and PII protection for this account. Where does customer-managed key encryption fail to deliver what the compliance team thinks it delivers?

Read the worked answer

Customer-managed keys, correctly implemented:

  • Create or use a user-assigned managed identity; grant it get, wrapKey, unwrapKey on the Key Vault key (RBAC or access policy).
  • Set the account's keyVaultKeyUri to an RSA 2048-bit key; the account uses it to wrap the data encryption keys that protect storage, including backups.
  • Rotation: create a new key version in Key Vault, point the account URI at the new version, and let re-wrapping happen in the background. Keep the previous version enabled until re-wrap completes - disabling it early makes existing data undecryptable.
  • Revocation is immediate and total: disabling or deleting the key (or removing the identity's unwrap permission) makes the account unavailable for reads and writes. That is the compliance control, and it is also an availability risk - rehearse it in a non-production account, because the failure mode is 'everything 403s' with no partial degradation.

Where the compliance assumption breaks:

CMK protects data at rest against someone who obtains the disks. It does not stop the analytics team from reading names and IBANs, because they hold data-plane RBAC on the account and Cosmos DB has no column-level access control or per-item masking. Anyone with Cosmos DB Built-in Data Reader can read every field of every item they can reach.

So PII separation must be done in the data model and the authorization model:

  1. Project. Run a change-feed processor that writes a de-identified analytics container (pseudonymous account id, hashed IBAN, amounts, timestamps) - no names at all.
  2. Scope RBAC. Grant the analytics team's identity data-plane RBAC on the analytics container only, ideally via a custom role definition limited to that container's id, not on the ledger container.
  3. Keep the ledger tight. The ledger container's data-plane role assignments should be limited to the two services' managed identities.

If field-level encryption is a hard regulatory must, that has to be client-side (encrypt the field before writing, decrypt in the service) - CMK will not be accepted as a substitute by an auditor who reads the control carefully.