Retail Platform: Inspection Cost, Latency, and a Peering Mesh

Cost:

• Processing: 400 TB = 409,600 GB x $0.02 = $8,192/month
• Endpoints: 3 AZ x 730 h x $1.50 = $3,285/month
• Total ~$11,477/month, about $138,000/year

On the exemption, separate the two arguments that are being conflated:

Cost argument: if pricing is ~15% of egress, exempting it saves roughly 61,440 GB x $0.02 ≈ $1,229/month. That is under 11% of the inspection bill. Exempting your highest-value, internet-facing transaction path to save 11% of a security control's cost is a poor trade and would be hard to defend after an incident.

Latency argument: the 120 ms budget against 40 ms current leaves 80 ms of headroom, and a properly built inspection path - same-AZ gateway load balancer endpoints, AZ affinity preserved, no hairpin across AZs, no asymmetric return routing - typically adds single-digit milliseconds. The assumption that inspection 'costs' 80 ms has not been measured.

Recommendation: do not grant a blanket exemption. Measure first with a canary and the actual p99 through the inspection path. If the SLA is breached, the response is architectural (co-locate the endpoint in the pricing AZ, dedicated appliance capacity) before it is a policy exception. Only if measurement fails should a time-bound exception with compensating controls be requested - and it should name the owner and the review date, not be a permanent route-table entry nobody remembers creating.

The 780 number is itself a finding. Full mesh across 40 accounts x 3 VPCs = 120 VPCs would be 120 x 119 / 2 = 7,140 peering connections. At 780, this is a partial mesh - meaning some paths work and some do not, and nobody can state the connectivity model from memory. That is a correctness risk before it is a cost problem.

Peering mesh problems:

• Peering is non-transitive, so central inspection cannot be enforced without per-VPC route plumbing and repeated appliance attachments.
• Route table sprawl and account-level route limits are already being approached.
• Enforcing 'internet only via inspection' is essentially unenforceable when 120 VPCs each hold their own routes.

Transit gateway trade-off:

• Cost: roughly 120 attachments x 730 h x $0.05 ≈ $4,380/month in attachment hours, plus data processing on top of the inspection bill already computed (~$8,192/month at $0.02/GB for the egress volume). That is real money and Finance is right to flag it.
• Benefit: one routing domain, transitive connectivity, a single inspection VPC that all spokes must traverse, and segmentation by route table rather than by hope.

Judgement: the transit gateway is justified primarily on enforceability and operational correctness, with the cost argued honestly rather than hidden. Before committing, test the cheaper option - consolidating 120 VPCs into fewer, larger VPCs reduces attachments and is the highest-leverage cost lever available.

Bypass is prevented at three layers, because route tables alone are a configuration, not a control:

1. No internet gateway attached to workload VPCs. Default routes point only at the inspection VPC's attachment. A route cannot bypass an attachment that does not exist.
2. Service control policies denying ec2:CreateInternetGateway, AttachInternetGateway, and route creation to 0.0.0.0/0 outside the network account. SCPs are the enforcement layer that survives a well-meaning engineer with admin rights in their own account.
3. Detective control - AWS Config rule or equivalent alerting on any IGW attachment or default route appearing outside the approved network account, with the alert routed to the SOC.

PCI workload placement: put cardholder data in its own account, not merely its own subnet.

• Account boundary gives a hard blast-radius and a clean audit scope; subnet ACLs and security groups are stateful/stateless filters inside one trust domain and one set of IAM permissions.
• Security groups are allow-list and stateful but cannot express 'this workload may not talk to that account at all'.
• The dedicated account also lets you attach a narrower SCP, restrict who holds admin, and keep the PCI scope small enough to actually assess.

Judgement: the inspection VPC must itself be treated as Tier 0 - it sees all egress in plaintext and is the single point whose compromise silently disables every downstream control.