Shipment Validation and Carrier Booking

Split by transaction requirement, not by convenience.

• Validation → synchronous, pre-operation, on Create and Update. Read the truck capacity from InputParameters/Target and throw InvalidPluginExecutionException with a user-readable message. Pre-operation runs before the row is written and inside the transaction, so an overweight shipment is never persisted. Filtering attributes on the Update step prevent it firing on unrelated edits.
• Booking → never synchronous. A 3-second external call inside a synchronous step holds the user's form and the database transaction open, and a sandbox step that exceeds the two-minute limit is killed mid-flight - which is exactly the "no booking, no audit" symptom reported. Book asynchronously: a post-operation asynchronous plugin, or better, a post-operation step that posts to an Azure Service Bus queue which an Azure Function drains and calls the carrier from.
• Audit → asynchronous, after the outcome is known. Writing the audit row synchronously before the carrier responds records an event that has not happened.

Verify: throttle the carrier stub to 3 s with a 20% failure rate in Test and confirm the form stays responsive and every failure produces an audit row.

Queue with idempotency, then drain at a controlled rate.

• 200/minute is roughly 3.3/second sustained - trivial in aggregate, but the asynchronous service is throttled and its backlog is invisible to users, so a synchronous burst converted to async jobs simply moves the delay. Measure the async job backlog during the shift change before assuming it drains in time.
• Service Bus queue + Azure Function gives explicit control: sessions or partitioned queues for ordering per truck, MaxConcurrentCalls sized to the carrier's tolerance, and a dead-letter queue that a human can replay. This is the recommended shape because the carrier exposes no queue and can fail.
• Idempotency: generate a BookingId (GUID) on the Shipment record before dispatch and send it as the carrier's idempotency key; on timeout, retry with the same key rather than assuming failure. Use Service Bus duplicate detection so a retried enqueue does not produce a second call.
• Backoff: exponential retry on 429/5xx, capped attempts, then dead-letter plus a Shipment status of "Booking failed" so the operation is visible rather than silent.

Verify: replay the same message five times in Test and confirm the carrier stub records one booking; kill the Function mid-flight and confirm the message returns to the queue.

Constrain the trigger, do not rely on the depth guard.

• The platform enforces a maximum cascading depth and will eventually stop the chain, but it stops it by throwing an error on a real user's save - treat depth as a safety net, not a design.
• Filtering attributes on the Update step: register the step to fire only when the columns that matter (Weight, TruckId, Status) change. An audit-only update that touches AuditWrittenOn then never fires.
• Check context.Depth at the top of the handler and return early above a threshold you define (commonly 2), and set Status transitions explicitly so a re-entrant update is a no-op.
• Write the audit into its own table from the asynchronous step, so the audit insert cannot roll back the booking and cannot feed back into the Shipment pipeline.
• Also watch: the async step executes as the record owner, not as the calling user - if the owner lacks privileges on the audit table, you get intermittent failures that only appear for certain queues; and ITracingService output is essential because async failures surface only in system jobs.

Verify: create a shipment in Test with a plugin trace, confirm one booking call and one audit row, then edit only a non-filtered column and confirm zero calls.