Rate-Limit Structure for a Multi-Tenant Payments API on API Connect

Why the proposal fails. In API Connect a plan rate limit is a shared bucket for every application subscribed to that plan. Raising it to 5,000 rps does not give each consumer 5,000 rps and does not give any consumer a guarantee; it just makes the bucket bigger. At month-end the mobile app's 1,200 rps still drains the bucket first, partners still receive 429s, and the aggregator's bursts still compete with both. This is a shared-fate design: one neighbour's burst determines another consumer's success rate, which is a contractual breach for the fintechs, not a tuning problem. A fourth gateway pod adds CPU but cannot fix a shared counter.

Structure to put in place. Separate the traffic by contract, not by convenience:

• One product and plan per enforceable limit. The 40 partners must not share a single 50 rps plan, because that plan is again one bucket. Model tiers (for example Partner-Standard, Partner-Premium) as distinct products/plans, or use per-application rate limiting if your release supports it in the plan definition, so each subscription carries its own bucket.
• A dedicated plan for the mobile app sized above its 1,200 rps peak, with spike arrest in front to absorb sudden bursts rather than letting them hit the backend.
• A separate plan, and ideally a separate catalog, for the aggregator. Its 5,000-in-90-seconds pattern is a burst profile, not a sustained rate; a plan with an appropriate burst allowance plus batch chunking on the client side keeps it inside the limit.

Then capacity. Validate the gateway sizing only after isolation exists: measure cost per call including token validation, then load test three pods against realistic assemblies. Four pods may be needed anyway, but as a capacity decision, not a fix for 429s.

Move it, but only for tokens that can be validated locally. At 1,200 rps the mobile tier alone spends roughly 48 seconds of backend time per second on token checks that never change outcome; the work is duplicated across every service and every retry, and each service needs its own identity-provider integration and its own cache. Pushing validation to the API gateway gives consistent enforcement at the edge, lets the backend accept only requests already carrying verified claims, and removes a per-call dependency between the backend and the identity provider.

Before you do it, settle these:

1. Token format. Self-contained JWTs can be verified at the gateway with the issuer's public keys fetched from the JWKS endpoint and cached, with a refresh on unknown key ID. Opaque tokens require introspection, which is a network round trip - that reintroduces the latency you are trying to remove, so cache introspection results for a short window and accept the resulting revocation lag, or move those clients to JWT.
2. Claim-to-policy mapping. Decide where authorization lives. If the backend enforces fine-grained entitlements on claims, the gateway must not strip them, and both sides must agree on the claim names - otherwise you have moved authentication and left authorization ambiguous.
3. Failure mode. Decide what the gateway does when the JWKS endpoint is unreachable: fail closed with 401 is correct for payments, but it means the identity provider's availability becomes part of your API's availability. Cache aggressively and alert on refresh failure.
4. Gateway capacity. Signature verification is CPU work on the same pods whose limits you just questioned. Re-measure cost per call before sizing the cluster.

Diagnose first: is it throttling or contention? If the aggregator's 429s come from the shared plan bucket, isolation alone may restore the deadline. If they come from backend slowdowns caused by the mobile app, isolation at the gateway only queues the problem. Compare the aggregator's request timings during a non-month-end hour against month-end before choosing.

Options, cheapest first:

• Dedicated plan and catalog. Give the aggregator its own plan sized for its burst, and its own catalog so that catalog-level limits and gateway settings do not couple it to customer traffic. This is the structural fix and should be done regardless.
• Shape the client. 5,000 requests in 90 seconds is a client-side scheduling choice. Chunking the batch, adding client backoff that respects the Retry-After header rather than hammering, and spreading the run over the hour reduces the peak without changing the contract. Cheapest option, no platform change.
• Reserve capacity rather than share it. If the platform supports it, segregate the aggregator onto its own gateway service or dedicated gateway pods so its bursts cannot consume the capacity customer traffic depends on. This costs infrastructure but gives a real guarantee.
• Do not simply raise limits. Raising the aggregator limit inside the shared plan just relocates the pain onto partners at the moment they are least able to absorb it.

Finally, confirm the 06:00 deadline is an API-throughput constraint at all. If reconciliation depends on data arriving from core systems that are themselves slow at month-end, no gateway change will fix it, and you should say so before selling an API fix.