Oracle Cloud Infrastructure Architect Associate: Twelve terabytes of Object Storage billed as internet egress

One route table is in effect. Routing is chosen per subnet, or per VNIC when a VNIC has its own route table. sn-app uses rt-app. rt-shared, where the service gateway rule lives, is not consulted by these instances at all, so the only matching rule for the Object Storage public IPs is 0.0.0.0/0 to NAT. The service gateway exists and is healthy; it simply never sees the packets. Egress therefore leaves over the internet and is billed.

Fix the routing. Add a rule to rt-app with destination all-services-in-oracle-services-network (console label "All Services In Oracle Services Network", shown as all-fra-services-in-oracle-services-network for Frankfurt) and target sgw-prod. Longest-prefix match means that rule wins over 0.0.0.0/0 for OSN destinations only, so internet egress still reaches NAT. Traffic to Object Storage inside the region is then handled on the OCI network at no egress charge.

Check the NSG now, not later. Security lists and NSGs are both evaluated and a flow is allowed if either permits it, so today the permissive security-list egress rule is what lets the packets out and nsg-app is not the blocker. That changes the moment the list is tightened. Add an explicit egress rule to nsg-app for the OSN CIDR on TCP 443 first, so the hardening work does not break Object Storage.

Common traps. Confirm no instance has a VNIC-level route table overriding the subnet's; confirm DNS still resolves the bucket hostname to the public Object Storage address (the service gateway intercepts routing, not name resolution); and confirm the service gateway was created in the same VCN, since it cannot span VCNs.

Pattern A - service gateway reached through the DRG. The on-premises host resolves the bucket to its normal Object Storage public IP; the CPE is configured to send that prefix to the DRG instead of the internet. Inside the VCN, the route table that governs traffic arriving on the DRG attachment must contain the OSN CIDR pointing at sgw-prod. That is the detail that breaks: teams add the rule to the application subnet's route table, but ingress from on-prem is evaluated against the DRG attachment's VCN route table, so the traffic bounces back out or is dropped. Security rules must also allow the on-prem CIDR to the OSN CIDR. The service gateway does not accept traffic from outside the VCN by itself - the DRG path is what makes it work.

Pattern B - Object Storage private endpoint. Create a private endpoint in a subnet of the VCN, receive a private IP, and point the analytics job at that IP (DNS override or /etc/hosts style alias). Then route that subnet from on-prem over the DRG and allow TCP 443 in the security list or NSG. This avoids routing public IP ranges internally and is easier to justify to a security reviewer; the cost is DNS bookkeeping and one more object to manage per region.

Both patterns need: a non-overlapping address plan between on-prem and the VCN, the on-prem firewall opened to the chosen destination, and confirmation that no proxy is silently sending the traffic out to the internet anyway. Pick Pattern B if policy forbids internal use of public IP space, Pattern A if you want zero client-side DNS changes. Either way, verify by watching the NAT gateway counters stay flat during the analytics run.

What the change actually does. Removing 0.0.0.0/0 from the security list only breaks flows that no NSG rule permits, because a flow is allowed if either the security list or an attached NSG allows it. So the first job is an inventory of flows per instance, not a guess.

Checklist before the change:

1. Is the NSG actually attached? An NSG only applies to a VNIC if it was specified when the VNIC was created or updated. Instances launched before nsg-app existed, or cloned from an older configuration, may have no NSG at all and are relying entirely on the security list. Enumerate VNICs per instance and confirm the NSG list is non-empty.
2. Object Storage. nsg-app must allow egress to the OSN CIDR on 443, or the bucket traffic that now flows through the service gateway stops the moment the list changes.
3. Database. The NSG allows the DB subnet, which suggests the rule is expressible as a CIDR or, better, as an NSG-to-NSG reference so the rule survives a subnet re-IP.
4. Everything unglamorous. OS patching and package mirrors, DNS to the VCN resolver, Vault endpoints, monitoring and log upload, and any third-party API the app calls. These are the flows nobody documented and the ones that page someone at 02:00.

Most likely outage. Egress to the Oracle Services Network, because it is the newest flow and the one nobody wrote down - which is exactly the flow this project is trying to make private and cheap.

Sequence it safely. Add the NSG rules first, confirm traffic is unaffected, then narrow the security list. Keep the old rule in a change record so you can restore it in one edit, and watch NAT and service-gateway counters during the change window.