API Gateway Rate Limiting with Partner Tiers
A Boomi API Management gateway fronts a customer-facing order-status API. Three partner tiers exist: Gold (10 requests/second, 99.9% availability), Silver (2 requests/second), and Bronze (0.5 requests/second). The backend is a Boomi process that calls an on-premise ERP system with a hard limit of 50 concurrent connections. Last month, a Bronze partner's misconfigured retry loop sent 40 requests/second for 20 minutes, saturating the ERP connection pool and causing Gold partners to see 503s. The gateway currently has a single global rate limit of 100 requests/second and no per-client quotas. The operations team wants to add per-tier limits but is concerned about added latency. What approach would you recommend, and how would you validate it?
1. Design a rate-limiting and isolation strategy that protects the ERP backend while preserving Gold-tier SLAs.
Read the worked answer
Apply per-client rate limits at the gateway keyed on the API key or OAuth client ID, with tier-specific thresholds (Gold 10 rps, Silver 2 rps, Bronze 0.5 rps) and burst allowances. Critically, also enforce a concurrency limit at the backend process layer, not just a request rate, because the ERP constraint is concurrent connections (50), not requests per second. Use separate connection pools or a semaphore per tier so that a Bronze flood cannot consume Gold's share. For example, reserve 30 connections for Gold, 15 for Silver, and 5 for Bronze. Add a circuit breaker that opens when ERP latency exceeds a threshold, returning 429 with Retry-After to well-behaved clients. The gateway rate limit alone is insufficient because a retry loop can still occupy connections if requests are slow; concurrency isolation is the decisive control.
2. How would you test that the new limits do not introduce unacceptable latency and actually contain a misbehaving client?
Read the worked answer
Run two test tracks. First, latency: baseline p50/p95/p99 for each tier before and after enabling limits, using a load generator that mimics each tier's allowed rate plus 10% headroom. Acceptable overhead is typically under 5 ms at the gateway; if exceeded, check whether limits are enforced in-memory at the gateway node versus a remote call. Second, containment: replay the Bronze incident (40 rps for 20 minutes) against a staging ERP stub and confirm that Gold p95 stays within SLA and that Bronze receives 429s after its allowance. Verify that the ERP connection count never exceeds 50 and that the circuit breaker opens and recovers correctly. Include a canary phase with one Gold partner before full rollout.