The Noisy Neighbor on a Single Instance
A small SaaS team runs a customer-facing API on one large virtual machine in a single cloud region. During a marketing push, one enterprise customer begins sending batched requests that saturate the VM's CPU for minutes at a time. Other tenants see 3-5 second response times and occasional timeouts. The team has a hard monthly budget that precludes re-architecting to microservices this quarter. They have metrics showing the abusive tenant is roughly 40% of total request volume but only 8% of revenue. Leadership wants the problem solved within two weeks without losing the enterprise customer. The platform uses a shared application process with no per-tenant isolation today, and the team has no experience with container orchestration.
1. What is the most defensible first move, and why is it preferable to immediately splitting the workload across many small instances?
Read the worked answer
The first move should be tenant-aware rate limiting and workload isolation at the application layer, not horizontal scaling. Reasoning:
- The root cause is a shared fate problem: one tenant's burst consumes the resource pool all tenants depend on. Adding more instances raises cost linearly and only postpones the collision; the abusive tenant can still saturate whatever pool exists.
- Rate limiting (per-tenant quotas, request queueing, or a bulkhead pattern) directly addresses the failure mode and is cheap to implement in the existing process.
- Splitting into many small instances without isolation still lets a single tenant's traffic land on shared backends, and it introduces operational complexity the team explicitly lacks.
A reasonable plan: impose a per-tenant concurrency cap, move the enterprise tenant's batch traffic to an off-peak queue or dedicated worker, and monitor tail latency per tenant. This solves the symptom within two weeks while preserving revenue. Horizontal scaling becomes a later capacity decision once isolation exists.
2. Leadership asks whether to simply move the enterprise customer to a dedicated instance at 3x their current infrastructure cost. How would you frame the trade-off?
Read the worked answer
Frame it as cost of isolation vs. cost of churn and SLA risk, not as an infrastructure line item.
- The tenant is 8% of revenue. Compute the annualized revenue at risk against the incremental infrastructure cost. If 3x current infra spend is small relative to that revenue, dedicated isolation is a rational stopgap.
- But note the asymmetry: dedicated capacity for one tenant does not fix the other tenants' exposure to future noisy neighbors. It treats one instance of a systemic problem.
- A better framing: dedicated isolation buys time, and the same two weeks should also deliver the rate-limiting controls so the next large tenant does not require another 3x spend.
Recommendation: approve temporary dedicated capacity only if paired with the isolation work, and set a review date. Otherwise the org accumulates bespoke infrastructure per customer.