Flash sale on a synchronous portal
TicketWave sells concert tickets on an O11 public-facing portal. During a flash sale, 25,000 concurrent users hit the site within 90 seconds. The home and event pages each call seven server actions in their preparation; three of those actions run aggregates over the full Orders entity (1.8 million rows) and return all matching records to the client for filtering in a list widget. A timer runs every minute, loads 200,000 pending order rows into memory, and loops over them to call a payment REST API synchronously. Database CPU reaches 96% and front-end CPU 38%. The platform uses two front-end servers and one database server, with no distributed cache. The business wants zero overselling and a page render under 1.5 seconds.
1. Which three changes give the largest scalability gain, and why?
Read the worked answer
1. Push filtering and paging into the database. Three aggregates return full result sets and the list widget filters client-side, so the database does the work and the network carries rows nobody displays. Replace with parameterized aggregates filtered by status/date and paginated; add a composite index matching the filter and verify the execution plan. This attacks the dominant DB CPU cost directly.
2. Remove synchronous REST calls from the timer loop. 200,000 rows times one serial HTTP round trip is the worst possible pattern: throughput is bounded by payment-provider latency, and threads stay occupied. Publish pending orders to BPT or a queue, process in bounded batches with idempotency keys, and call the external API outside the database transaction.
3. Consolidate screen preparation and cache read-only data. Seven service actions per screen means seven round trips and repeated work. Combine into one or two composite actions and cache reference data in a distributed cache. Front-end CPU is only 38%, so the bottleneck is database and serial I/O, not rendering; these changes reduce both.
2. How would you redesign order processing to be resilient and avoid overselling under peak?
Read the worked answer
Inventory: reserve stock with a single atomic update, UPDATE ... SET available = available - :qty WHERE id = :id AND available >= :qty, inside a database transaction, and check that exactly one row was affected. Never read-then-write; that is the classic overselling race.
Payment: create an OrderPayment record and hand off to BPT or a queue. The external REST call runs outside the database transaction with a timeout, retry policy, and idempotency key so a retry cannot double-charge. Mark rows with a claimed status so a second worker cannot pick them up.
Timers: process batches of 500-1,000, cap Max Instantiations to protect the database, and record failures in a dead-letter state for replay. Do not hold row locks while waiting on the payment provider.
Under peak, accept checkout into a pending queue and return a pending status instead of blocking the user. This preserves correctness under concurrency, isolates a slow provider, and prevents thread-pool exhaustion from turning a payment slowdown into a full outage.
3. What would you monitor after the redesign, and what thresholds would trigger action?
Read the worked answer
Monitor four layers. Application: service action p95 duration and call count per screen, aggregate execution time, rows returned per aggregate, and error rate. Database: CPU, disk latency, wait stats, and top queries by total time. Asynchronous: BPT queue depth, running and failed instances, per-batch duration, timer overruns, and duplicate-processing attempts. Cache and front end: hit ratio, thread-pool usage, and session count.
Useful thresholds: p95 screen render above 1.5 seconds; any service action above 500 ms; any aggregate returning more than 5,000 rows; BPT queue depth growing for five consecutive minutes; a timer still running when its next scheduled run begins; database CPU above 80% for ten minutes. Add a synthetic flash-sale load test before each major campaign, and alert on trends, not only on absolute peaks, so the team sees the next bottleneck before customers do.