Duplicated Orders After a JMS Failure
An Order flow does: HTTP listener, validate, insert an order header into a database, publish to a JMS queue, respond 201. When the JMS broker is unavailable the flow fails after the database insert has committed, so the client retries and a second order is created. The current error handler is a single on-error-propagate that sets status 500 and returns error.description together with the Java stack trace. A database deadlock occurs a few times an hour and is transient. Consumers also complain that validation errors and infrastructure errors look identical in the response body.
1. Restructure the error handling: which errors map to which responses, and where does retry belong?
Read the worked answer
Match on error type, don't collapse everything into one handler. Use a shared error-handling sub-flow with variables for httpStatus, errorCode, and correlationId.
Suggested mapping:
APP:VALIDATION/APIKIT:BAD_REQUEST/APIKIT:NOT_FOUND→ 400/404, body{code, message, correlationId}.DB:DEADLOCK/DB:CONNECTIVITY/HTTP:TIMEOUT(transient) → retried where they occur, then 503 withRetry-After.JMS:*→ 503, or 202 if the order is durably persisted and the publish is deferred.MULE:UNKNOWN/ anything unmapped → 500 with a generic message.
Never return error.description or a stack trace. Log the full error and stack trace server-side with the correlation id, return only the id.
Retry placement: wrap only the database insert in a try scope with until-successful for DB:DEADLOCK, with a short delay. Do not put the retry around the whole flow - that is how the duplicate orders and inflated latency appear.
2. How do you stop the duplicate orders? Compare the transaction option with the idempotency option.
Read the worked answer
Transaction option: a try scope with transactionAction BEGIN_OR_JOIN can make the database work atomic, and Mule can enlist JMS in the same transaction with XA-capable resources. Two cautions: XA across a database and a broker is operationally expensive and fragile, and without XA the JMS publish is not covered - so this usually does not deliver the guarantee people assume it does.
Pragmatic pattern - transactional outbox:
- In one database transaction, insert the order and an outbox row.
- Commit, then publish to JMS (or a scheduled job does).
- Mark the outbox row sent; retry publishes from the outbox on failure.
Idempotency layer: enforce a unique constraint on the client-supplied order key so a retry cannot create a second row, and add an Object Store guard keyed on that same value with a TTL to short-circuit repeats quickly.
Client contract: if the order is persisted but the publish is pending, respond 202 with the order id and let the consumer poll - a 500 invites exactly the blind retry that duplicates the record.
3. A duplicate arrives while the broker is still down. Walk through what `on-error-continue` versus `on-error-propagate` does to the response, and what the API should return.
Read the worked answer
on-error-propagate rethrows after the handler runs: the error propagates to the HTTP listener, which produces the response. The flow stops - nothing after the failing component executes. Use it when the caller must know the operation failed.
on-error-continue swallows the error and the flow proceeds with the next component. Use it only when the remaining work is still valid - for example, a JMS publish failure handled by writing to the outbox and continuing to respond.
For this case:
- The database row is committed, so a 500 would be a lie about state and would trigger a destructive retry.
- With
on-error-continuearound the publish, plus the outbox, the flow returns 202 Accepted with the order id and a status link. - If a duplicate key is detected, return 409 Conflict with the existing order id, which tells the consumer the request was already applied.
- In all cases the body carries the stable error code and correlation id, never the stack trace.
Caveat: if the error occurs after the response has begun to be written, the HTTP status is already committed - which is another reason to do the risky work before the response and to keep the handler at the flow level where it can still set the status.