Redesigning an 850-to-SAP Business Process for SAP Outages and Duplicate AS2 Resends
A supplier receives 3,000 EDI 850 purchase orders per day, peaking at 20 per minute, over AS2 from a single retailer. The current business process is fully synchronous: the AS2 adapter receives the message, deenvelopes and translates the 850, performs an MQ put to SAP, translates an 855, and returns the AS2 MDN. SAP is taken down for a 10-minute maintenance window twice a week. During those windows the MQ put fails, the business process faults, and no MDN is returned before the partner’s 30-second timeout. The retailer resends the same 850, and the supplier has created duplicate sales orders in SAP. The team cannot change the retailer’s AS2 timeout or retry policy. They can change the business process, add persistence, and use asynchronous processing.
1. Where should the MDN be generated relative to SAP processing, and why? What must be persisted before acknowledging?
Read the worked answer
The MDN should acknowledge receipt and syntactic/basic validation of the AS2 message, not successful creation of a sales order in SAP. The retailer's 30-second timeout is shorter than a SAP maintenance window, so coupling the MDN to SAP availability guarantees timeout and resend. Once we acknowledge, we own durable delivery to SAP; before we acknowledge, we must be sure we can recover the payload even if the process or node fails.
Before returning the MDN I would persist, in a durable store:
- the raw AS2 payload and the MIME headers needed to reconstruct the message;
- sender/receiver IDs and the AS2 message ID;
- the EDI envelope keys used for duplicate detection, typically ISA13, GS06, and ST02 plus the sender/receiver qualifiers;
- a processing status such as RECEIVED or VALIDATED, and a work-item key.
Only after that persistence commits should the MDN be returned. If validation fails, return an error MDN or a 997 as agreed, not a positive MDN. The trade-off is that a positive MDN creates a delivery obligation: the asynchronous worker must retry until SAP accepts the order or an operator resolves the exception. That is preferable to duplicate orders, but it requires monitoring of the work queue and a dead-letter path.
2. How would you make the SAP MQ put idempotent and handle retries during the 10-minute outage?
Read the worked answer
Idempotency has to exist at two levels: AS2/EDI redelivery and business order creation.
At the EDI level, store a unique key from ISA13/GS06/ST02 and reject a resend with the same key before translation. The retailer's resend should hit this key, so the duplicate never reaches SAP. At the SAP level, derive a deterministic MQ message identifier or correlation ID from the purchase order number plus the EDI control key, and have the SAP interface check that key before creating an order. The MQ put itself is not inherently idempotent, so the consumer must enforce it.
For retries, the inbound business process should hand off to a separate worker process and persist a work item with status PENDING. On MQ failure, the worker writes the error and schedules a retry with backoff, for example 1, 3, 10, and 30 minutes, rather than a tight loop. During a planned 10-minute SAP outage the backlog simply accumulates in the work table; during an unplanned outage the same mechanism applies. Cap attempts and route exhausted items to an exception queue with an alert. Do not mark the AS2 message failed after the MDN has been sent; mark the work item failed and keep the payload for replay. This separates transport acknowledgment from business completion and prevents the retailer's resend from creating a second SAP order.
3. What business process structure would you use to keep peak throughput at 20 per minute while SAP is down, and how would you monitor it?
Read the worked answer
I would split the current monolithic process into a short inbound process and a separate outbound worker process.
The inbound process does only what must happen before the MDN: receive, validate AS2 and EDI envelopes, check the duplicate key, persist the payload and work item, and return the MDN. It should complete in seconds and must not call SAP. The worker process is triggered from the persisted work queue, performs translation if not already done, executes the MQ put with retry, and records completion. Workers can scale independently and can be paused during planned SAP windows without affecting AS2 receipt.
At 20 per minute, the queue absorbs a 10-minute outage as roughly 200 work items, which is small. The risk is not volume but head-of-line blocking: if workers retry the same failing SAP connection synchronously, they consume all worker threads. The retry design should release the worker and reschedule, so other work can proceed once SAP returns.
Monitoring should include: AS2 receipt rate and MDN response time; duplicate-key rejects; work-item queue depth and age; MQ put success rate; retry counts; and the age of the oldest PENDING item. Alert on oldest-item age, not just count, because a stuck item is more dangerous than a large but draining queue. Also monitor BP engine thread saturation and database pool usage so a SAP slowdown does not silently consume all capacity.