Overlapping CIDRs Block VPC Peering Between Two Business Units
Two business units must exchange private traffic inside one region. Unit A's VPC uses 10.0.0.0/16 with subnets 10.0.1.0/24 (web) and 10.0.2.0/24 (app). Unit B, acquired last year, also uses 10.0.0.0/16 with a single subnet 10.0.1.0/24; its order API listens on 10.0.1.20:8443. Unit A's app servers must call that API, and Unit B's servers must query Unit A's internal DNS at 10.0.2.10. Security policy forbids any inter-unit traffic crossing the public internet. Separately, 180 Linux servers in Unit A need outbound access to public package repositories for weekly patching; each currently has its own EIP, the bandwidth bill is high, and the security team objects to 180 public addresses. Unit A also runs a customer-facing website that must terminate HTTPS and scale horizontally. The network team proposes creating a VPC peering connection between the two VPCs and, separately, attaching EIPs to the new web servers.
1. Can VPC peering connect the two VPCs? Explain the constraint and rank the alternatives.
Read the worked answer
No. Peering requires the two VPC CIDR blocks to be non-overlapping. Both sides use 10.0.0.0/16 and both have 10.0.1.0/24, so a route for 10.0.1.0/24 would have two contradictory next hops. The console will reject the connection or the route will conflict; even if a connection object were created, traffic to 10.0.1.20 is ambiguous.
Options, best first:
- Re-IP Unit B (e.g. 10.1.0.0/16, subnet 10.1.1.0/24). Highest one-off effort - DNS records, app configs, certificates - but gives a clean, supported, symmetric route with security-group referencing, and unblocks future units. Preferred long-term.
- Dual-NIC relay ECS: one ENI in a Unit A subnet, one in a Unit B subnet, IP forwarding enabled, route tables pointing the remote subnet at the relay, security groups tightened to port 8443 and 53 only. Works without re-addressing, but is a bandwidth-bound single point of failure; run two relays and monitor.
- Publish the API via a VPC endpoint service if Unit B can expose the order API as an endpoint service - the consumer's endpoint gets an address from its own subnet, sidestepping the overlap for that one service, but does not give general L3 reachability or DNS.
VPN over the internet is excluded by policy; Direct Connect solves a different problem (on-premises) at far higher cost.
2. Redesign Unit A's outbound internet access for 180 servers and explain why the EIP-per-server approach fails.
Read the worked answer
Why the current design is wrong: an EIP is a public, inbound-reachable address. 180 of them means 180 potential entry points, a large attack surface to audit, and 180 separately billed bandwidth pipes with no aggregation discount. It also burns the region's EIP quota.
Design:
- Deploy a NAT Gateway per AZ (for HA) in Unit A's VPC, with a subnet dedicated to it.
- Add SNAT rules for each server subnet, and point those subnets' route tables at 0.0.0.0/0 → NAT Gateway.
- Remove the EIPs from the 180 ECS instances. Only the NAT Gateway holds an EIP, purchased with a shared bandwidth package.
- Security groups on the servers deny all inbound from 0.0.0.0/0; the NAT gateway permits only established outbound flows, so patching works while nothing is exposed.
Sizing and caveats: check the NAT gateway's SNAT connection limit and throughput against 180 concurrent package downloads; keep a second gateway for AZ redundancy; optionally put a local caching proxy behind the NAT to cut repetitive repository traffic; and confirm the EIP is bound to the NAT, not to any server.
3. Design the public web tier and explain why attaching EIPs to web servers is the wrong pattern.
Read the worked answer
Target architecture:
- Dedicated ELB in the VPC with an HTTPS listener (certificate from the certificate service), HTTP listener redirecting 80→443, backend server group spread across both AZs, health checks on the real application endpoint, and session persistence only if the app is stateful.
- EIP on the ELB (or shared bandwidth), never on the ECS instances.
- The web ECS sit in private subnets with no EIP; their security group permits application ports only from the ELB's backend subnet, not from 0.0.0.0/0.
- The Auto Scaling group attaches to the ELB's backend server group, so scaled-out instances register and pass health checks automatically.
- Put WAF in front of the ELB for the customer-facing site.
Why not EIPs on servers:
- EIPs bypass the ELB, so health checks, TLS termination and traffic distribution are meaningless - clients would have to know each address.
- Each server becomes independently internet-reachable, so a misconfigured security group exposes the host directly.
- Bandwidth is billed per EIP with no pooling, and scale-in leaves orphaned addresses that are still billable and still a target.
- Autoscaling requires a stable front-end address; a pool of EIPs is not one.