Order fulfillment: partial failures and idempotent integrations
A retailer processes 20,000 orders per day in Pega. Each order must call inventory, payment, tax, warehouse, and shipping. Inventory and payment can fail independently; warehouse is batch, every 15 minutes, and shipping is synchronous with a 10-second timeout. The current design calls all five systems in a single data transform during case creation. When payment succeeds but inventory fails, the case is stuck in New with no retry. Duplicate orders occur after timeouts: 0.8% of orders are submitted twice. The operations team manually reconciles about 300 orders per week. Peak is 3x on Black Friday. The business requires an order status visible within 2 minutes and no duplicate shipments. There are 12 order types with different fulfillment rules.
1. Redesign order creation and fulfillment to handle five external systems, partial failures, and duplicate submissions.
Read the worked answer
Do not call all five systems in a single data transform during case creation. Persist the order case first with an idempotency key from the client request, then run a Fulfillment subprocess with separate integration steps for inventory, payment, tax, warehouse, and shipping. Inventory and payment can run in parallel but must have compensation: if payment authorizes and inventory fails, void or release the authorization and set the order to Inventory Exception. Use queue processors for each external service, with retry, backoff, dead-letter, and a correlation ID stored on the case. Tax can be synchronous if it is fast, but it should still be timeout-bounded. Warehouse is batch, so create an allocation request and wait for a callback or scheduled poll; shipping is synchronous with a 10-second timeout, but treat timeout as unknown and reconcile by idempotency key. The tradeoff is more case states and integration records, but it prevents the current stuck-in-New and duplicate-order failures.
2. How do you achieve visible order status within two minutes and no duplicate shipments during Black Friday 3x peaks?
Read the worked answer
Status visibility comes from event-driven updates: each integration writes a fulfillment event record and updates a summary status on the case, and the case view reads the summary, not five live systems. Target under two minutes by using asynchronous queue processors with short polling intervals and by prioritizing status events over batch warehouse work. For no duplicate shipments, the shipping request must carry a deterministic shipment key based on order ID plus fulfillment attempt. Before calling shipping, check the shipment record; after a timeout, query by that key rather than blindly retrying. Use an outbox pattern: write the shipment request to the Pega database in the same transaction as the case update, then a queue processor sends it. At 3x peak, partition queues by order type and apply back-pressure; do not increase concurrency against a timed-out shipping service. The tradeoff is eventual status, but two minutes is achievable with event updates, whereas live federation would fail under peak.
3. How would you make the design extensible for 12 order types without creating 12 case types?
Read the worked answer
Use one Order case type with an OrderType property and a decision table that selects fulfillment steps, service levels, and validation rules. Circumstance sections and processes by order type only where the flow genuinely differs; otherwise use configuration parameters. For example, digital goods may skip warehouse, while bulky goods may require a delivery slot. Keep the integration layer generic: a FulfillmentStep data class with step name, service, timeout, retry policy, and compensation action. The case process iterates or routes to the configured steps. This avoids 12 case types and 12 sets of duplicated integrations. The risk is over-generalization; I would allow a specialized case type only if an order type has a materially different lifecycle, not just different fields. I would also add a regression suite that runs all 12 configurations through the same fulfillment harness.