Active-passive multi-region failover with OCI DNS and RPC
A payments platform runs active-passive across Frankfurt (primary) and Amsterdam (standby). Primary VCN 10.20.0.0/16, standby 10.21.0.0/16. Each region has a DRG and a remote peering connection. The public web entry is an OCI DNS traffic management steering policy with a failover rule, a 300-second TTL, and an HTTP health check every 30 seconds. The standby app tier is pre-provisioned but scaled to 2 instances; the database uses asynchronous replication with measured RPO 4 minutes. During a test, the primary load balancer was powered off; DNS continued returning the primary for 6 minutes, and the standby could not reach the primary database because the DRG route table had no route for 10.20.0.0/16. Security lists in Amsterdam allow only 10.21.0.0/16. The business requires RTO 15 minutes and RPO 10 minutes. The team wants to avoid a global load balancer and keep using OCI DNS.
1. Why did DNS failover take longer than the health-check interval, and how would you meet a 15-minute RTO?
Read the worked answer
The health check detects failure, but DNS clients cache the previous answer for the TTL. With a 300-second TTL, a client can keep using the primary for five minutes after the health check flips, and recursive resolvers may add their own caching. The total observed six minutes is therefore plausible. Reduce the TTL to 30-60 seconds, shorten the health-check interval to 10 seconds, and use a failure threshold of 2 rather than a long threshold. Point the health check at an application readiness endpoint, not just the load balancer port, so a half-broken primary fails over. DNS alone will not meet a 15-minute RTO if the standby has only two instances; pre-scale the standby with an instance pool and capacity reservation, automate database promotion, and include DNS change propagation and application warm-up in the runbook. Test with real client resolvers, not only the OCI console. Keep local load balancers in both regions for regional HA; avoiding a global load balancer is fine, but do not treat DNS as a proxy or connection-draining mechanism.
2. What DRG/RPC and security changes are needed for the standby to reach the primary database during a partial failover, and what would you verify?
Read the worked answer
The RPC connects the two DRGs, but it does not create VCN routes or security allowances by itself. In Amsterdam, the VCN route table needs a route for 10.20.0.0/16 to the DRG, and the Amsterdam DRG route table needs that prefix pointing to the RPC attachment. Frankfurt's DRG route table needs the return route for 10.21.0.0/16 over the RPC, and its VCN route table must send that prefix to the DRG. Security lists or NSGs on the database subnet must allow the standby CIDR on the database port, and the database listener must accept the connection. Verify each layer independently: check the DRG route table, the VCN route table, the security list or NSG, and then test with nc or a database client from the standby. Also confirm that no more-specific route sends the traffic to an Internet Gateway or NAT gateway. RPC is not transitive, so do not expect it to provide paths to a third VCN.
3. The team wants to avoid a global load balancer. Is OCI DNS sufficient? What additional mechanisms would you add for data and application consistency?
Read the worked answer
OCI DNS traffic management is sufficient for name-based failover, but it is not sufficient by itself for application consistency. DNS does not drain connections, does not understand database replication state, and cannot scale the standby. Add a regional load balancer in each region, automated health checks that cover the full stack, and a runbook that includes stopping replication, measuring lag, promoting the database, updating connection strings, and validating the standby before DNS is switched. Use pre-provisioned standby capacity and autoscaling so the two-instance standby can absorb production traffic. For data, verify that the measured 4-minute RPO is still below the 10-minute requirement during peak load, and monitor replication lag continuously. Add a canary transaction that writes and reads through the standby path after failover. OCI DNS can be the steering layer, but the RTO depends on the surrounding automation, capacity, and data promotion process.