Seat Reservation: Strong Consistency vs. 50x Peak
An online ticketing platform sells seats for events. Average load is 120 requests per second, but a popular concert produces 6,000 requests per second at onsale, with 1.2 million users and an 80,000-seat arena. The current monolith uses a relational database and a single seat-map service; p95 response time reached 9 seconds at the last onsale, and 300 seats were oversold and had to be compensated. The business requires no double-sold seat, p95 under 2 seconds during onsale, and payment through an external provider with a 3-second timeout. The team has nine developers and six months. The product owner proposes caching the seat map in Redis and accepting reservations asynchronously, reconciling later. Operations proposes scaling up the database. The lead developer proposes a full microservices rewrite.
1. Evaluate the three proposals against the quality requirements and the constraints. Which elements are acceptable and which are not, and why?
Read the worked answer
Separate the invariant from the performance problem. 'No double-sold seat' must hold before a user is told the seat is theirs; reconciliation after the fact is compensation, not consistency.
- Redis cache plus asynchronous acceptance: caching the seat map for display is sensible and reduces read load. Accepting a reservation asynchronously and reconciling later is not acceptable if it can confirm a seat before the authoritative allocation commits; it risks more overselling. If 'async' only means queueing the request before an authoritative decision, it may be acceptable, provided the user sees a pending state, not a confirmation.
- Scale up the database: may improve throughput, but a single seat-map service and one database remain capacity and failure bottlenecks. It is a valid tactical measure for the next onsale if load-tested, not a six-month architecture answer. It also does not fix the concurrency bug that allowed 300 oversales.
- Full microservices rewrite: six months and nine developers make this high-risk while the current system must survive an onsale. It also does not by itself guarantee seat consistency; a distributed allocation design still needs a single authoritative writer per seat.
The reasoning: the binding constraints are the allocation invariant, p95 under 2 s at 6,000 requests per second, and a payment timeout. Architecture choices must be judged against those, not against fashion.
2. Design a target architecture for the onsale path. Name components, data ownership, consistency mechanism and failure handling.
Read the worked answer
A workable design separates read scaling from authoritative allocation.
Components and ownership:
- A waiting room or admission control component smooths the 6,000 rps spike and protects downstream systems.
- A seat-map read service serves availability from a cache or read model, clearly labelled as indicative until reservation.
- A seat allocation service owns seat state, partitioned by event (and possibly seat block). Each seat has one authoritative writer.
- A reservation store holds short-lived holds with expiry, owner and version.
- A payment adapter calls the external provider with an idempotency key and a timeout below 3 s.
Consistency: allocate with an atomic conditional update or optimistic version check: reserve only if seat is free or an expired hold. A successful reservation creates a hold with a TTL. Payment success converts the hold to a sale; payment failure or timeout releases it. Do not confirm before the authoritative write commits.
Failure handling: idempotent requests, retries with backoff, circuit breaker around payment, outbox for events, expiry worker for abandoned holds, load shedding when queues grow.
Trade-off: a short hold may temporarily deny a seat to another user, but that is preferable to overselling. Display cache staleness is acceptable if the user is not promised the seat until allocation.
3. What would you validate before committing, and which risks remain?
Read the worked answer
Validate the design against the actual onsale conditions, not a demo.
Validation:
- Load test at 6,000 rps with realistic think time and a hot event; measure p95 end-to-end and per component.
- Concurrency test: many users attempt the same seat simultaneously; assert exactly one successful hold and no oversell.
- Payment fault injection: timeout, slow response, duplicate callback and provider outage; verify hold expiry and idempotency.
- Queue and waiting-room behaviour under overload: check admission rate, rejection behaviour and recovery.
- Operational test: deploy, roll back and observe under load; check cost and capacity headroom.
Remaining risks: a single hot event may still concentrate load on one partition; cache staleness may frustrate users who see an available seat that is already held; queueing can create a poor user experience if not communicated; the external payment provider may be the true bottleneck; and the team may underestimate the operational complexity of the new components.
Mitigations: pre-warm and shard by event, label availability as indicative, cap queue wait, negotiate provider limits and rehearse the onsale with a game-day exercise. Commit only after the invariant and latency tests pass, and keep a tested fallback to the current system for the next sale.