Three-Tier VCN: NAT Charges, Stateful Rules and an Open SSH Port
Finbank's payments service runs in a VCN 10.20.0.0/16 in uk-london-1: a public subnet for the load balancer tier, a private subnet for application VMs, and a private subnet for an Autonomous Database. The database subnet's security list allows ingress TCP 1521 from 10.20.0.0/16 and egress to 0.0.0.0/0. Each application VM's VNIC is a member of nsg-app, which allows egress to nsg-db on 1521. The public subnet's security list allows ingress 0.0.0.0/0 on 80, 443 and 22, and Cloud Guard raised a problem for the port 22 rule. Finance queried NAT gateway processing charges of about 900 GBP per month even though the application calls only Object Storage and Vault. During a change window an engineer deleted the nsg-app egress rule, but the DBA reports the application still reaches the database. The application subnet has no route to an internet gateway.
1. Explain the NAT gateway charge and the exact change that removes it without giving the application subnet internet access.
Read the worked answer
Why the charge exists. Object Storage and Vault are reachable two ways: the public regional endpoint, or the Oracle Services Network. With no service gateway, the application VMs resolve the public endpoint, and because the subnet has no internet gateway the only path out is the NAT gateway. NAT gateway billing is per GB processed, so a chatty application pulling objects and secrets pays continuously - 900 GBP is a traffic signature, not a fixed fee.
The fix. Create a service gateway in the VCN, enable the Object Storage and Vault services on it, and add a route rule on the application subnet's route table: destination All LHR Services in Oracle Services Network, target the service gateway. Traffic to those services then stays inside the Oracle network: no NAT processing, no public endpoint, no internet path.
Two details that catch people. First, the service gateway must be in the same VCN as the subnet, and the route rule must be on the subnet's route table, not the VCN's default one if the subnets use separate tables. Second, if you later tighten the security list or NSG egress from 0.0.0.0/0 to specific destinations, add the service CIDR explicitly, or the routing exists but the packet is dropped before it leaves the VNIC.
Verification. Confirm with flow logs that destination addresses for Object Storage are the service network ranges, and confirm the NAT gateway's processed-bytes metric flattens.
2. The database is still reachable after the nsg-app egress rule was deleted. Is a control broken? Explain what is actually happening and how you would prove it.
Read the worked answer
Not necessarily broken - three explanations to test, in order.
- Statefulness. Both NSGs and security lists are stateful. Once a flow is established, the connection-tracking table permits the return and continued traffic; deleting an egress rule does not tear down live sessions. Long-lived connection pools against an Autonomous Database can persist for hours or days, so the DBA's observation may be stale evidence, not a broken filter.
- NSG membership. NSG rules apply only to VNICs that are members of that NSG. If the application VMs were re-created by an autoscaling rule or a stack redeploy, the new VNICs may not be in nsg-app at all, in which case the rule deletion changed nothing. List the NSG's VNIC members and compare against the current instance list.
- The security list still permits it. The database subnet's security list allows 1521 from the whole 10.20.0.0/16, and a subnet security list applies to every VNIC in that subnet regardless of NSGs. OCI has no deny in security rules; enforcement is the union of allows from security lists plus the NSG on the VNIC.
Proof. Open a new connection from an application VM (for example nc -zv <db-ip> 1521) and watch the result, then re-check NSG membership. Do not accept a surviving connection as evidence that the rule set is wrong, and do not re-add the rule before you have established which of the three is true.
3. Rank the Cloud Guard port 22 finding against the other work and describe the remediation, including the blast radius of the obvious fix.
Read the worked answer
Rank it high, but not first. It is low effort and it exposes management access on a subnet that should only host a load balancer - nothing legitimate needs SSH there. The NAT and service-gateway work is a cost and architecture issue; SSH from the internet is an intrusion path, so fix it in the same week.
Remediation. Remove the 0.0.0.0/0 rule for 22 from the public subnet's security list and keep 80/443. For administrative access, use the OCI Bastion service: it creates time-limited, IAM-audited sessions to targets with no public IP, so the hosts never need an internet-facing management port. If operations insists on SSH, restrict ingress to the bastion's private endpoint or a specific corporate egress range, not a /0.
Blast radius. A security list is subnet-wide: removing the rule affects every VNIC in that subnet, including the load balancer and anything else someone quietly placed there. Before you delete, confirm from flow logs and Audit that no host in that subnet has recent inbound sessions on 22, and announce the change window. Also check the same rule is not duplicated in the other subnets' lists.
Close the loop. After remediation, mark the Cloud Guard problem resolved rather than leaving it open - an unresolved, already-fixed problem trains the SOC to ignore the console, which is how the next real finding gets missed.