Payment Retries That Must Not Charge Twice

Split the microflow at the external call. The UI microflow should create the Order and a PaymentAttempt with a server-generated idempotency key, commit both, and enqueue an asynchronous task; the page then shows Pending and polls or refreshes. The task queue worker calls the provider with the idempotency key in the request header or body. Because the key is persisted and unique, a retry after a timeout sends the same key and the provider can return the original result instead of charging again. On a timeout, do not mark the order Failed. Mark the attempt Unknown and call the provider's status endpoint with the same key or reference; only a confirmed decline becomes Failed. Use exponential backoff for retryable errors and a maximum attempt count. Keep the database transaction short: commit the attempt before the call, and commit the result in a new transaction. Do not hold an open transaction across a 3.5-second p99 call, because that holds locks and consumes connection pool capacity. Enforce a unique constraint on PaymentAttempt.IdempotencyKey and handle the duplicate-create exception as a signal that another worker already owns the attempt. The UI receives an immediate acknowledgment rather than waiting for the provider.

Separate the three side effects into explicit states. Inventory should be reserved, not decremented, when the order is created: create an InventoryReservation with an expiry time and a unique order reference. On confirmed payment, convert the reservation to a committed decrement in an idempotent microflow keyed by Order. On a confirmed decline, release the reservation. On Unknown, leave the reservation until reconciliation resolves the payment, then release or commit. This prevents selling stock for an unpaid order and prevents double decrement when a retry succeeds. The confirmation email should not be sent inside the payment transaction. Commit the payment result, then emit an after-commit event or enqueue an email task keyed by Order and event type. Add a unique constraint on an EmailLog record so a runtime restart or duplicate task cannot send twice. Catch errors per step, not with one broad handler: a provider 400 is permanent and should fail the order; a 500, timeout or connection reset is retryable; an inventory conflict is a business exception that needs a user-visible message. Store the provider response code and correlation ID on PaymentAttempt for support. Run a reconciliation job every five minutes for attempts stuck in Unknown longer than two minutes, and alert on the count rather than relying on customer complaints.

Add PaymentAttempt with Order, idempotency key, attempt number, provider reference, status, amount, currency, request timestamp, response code, response body hash and correlation ID. Keep one row per provider call rather than overwriting a single PaymentStatus field, because the history is what proves whether a duplicate happened. Add a unique constraint on provider reference when it is returned, and a unique constraint on idempotency key. Add PaymentReconciliation with the last checked timestamp and the resolved status. For the existing incidents, join PaymentAttempt to Order and group by amount, customer and provider reference within a short window; the duplicates should show two successful attempts with different provider references or the same reference recorded twice. Build a monitoring page and alerts for attempts in Unknown over two minutes, attempts with more than three retries, and any order with more than one successful attempt. Log the idempotency key in the provider request and in Mendix logs so support can search both sides. Also monitor task queue backlog and failure count, because a silent queue backlog looks like stuck orders. These measures turn the investigation from anecdote into a queryable audit trail and give the team a regression check after the redesign.