Error Handling and Retry Logic for API Rate Limits
A scenario syncs contacts from a CRM to a marketing automation platform (MAP) every 15 minutes. The MAP API has a rate limit of 100 requests per minute. The scenario processes about 200 contacts per run, making one API call per contact to update or create. During peak times, the MAP returns HTTP 429 (Too Many Requests) errors. Currently, the scenario fails and sends an email alert, and the next scheduled run starts from the beginning, causing duplicate processing and missed contacts. The team wants the scenario to handle rate limits gracefully and ensure all contacts are synced without duplicates.
1. What error handling strategy would you implement to handle the 429 errors?
Read the worked answer
Implement a retry mechanism with exponential backoff. In Fusion, you can attach an error handler to the MAP module: use 'Resume' with a retry. However, Fusion's built-in retry (in module settings) can be configured: set 'Number of retries' to 3, 'Interval between retries' to 60 seconds (or use exponential). But since the rate limit is per minute, a fixed delay might suffice. Better: use an error handler route that catches the 429 and uses a 'Sleep' module to wait (e.g., 60 seconds) then retries the operation. Alternatively, use the 'Break' directive with a retry. Also, consider batching: instead of one call per contact, use bulk API endpoints if available. To avoid duplicates, ensure the MAP module uses an external ID or email as a unique key, so upsert is idempotent. Additionally, implement a 'Data store' to track processed contacts within a run, so if the scenario restarts, it skips already processed ones. But since it's scheduled every 15 minutes, a full restart might reprocess; using a data store with a timestamp or status can help. Another approach: use a 'Queue' to process contacts sequentially with rate limiting, but that's complex. The key is to handle 429 with retry and ensure idempotent operations.
2. How would you prevent duplicate contacts from being created if the scenario fails midway and restarts?
Read the worked answer
To prevent duplicates, the MAP module should perform an upsert based on a unique identifier (e.g., email). Most MAP connectors have an 'Update or Create' operation. Ensure that the module is configured to search for an existing contact by email first, then update if found, else create. This makes the operation idempotent. Additionally, you can use a data store to record which contacts have been successfully synced in the current run, but since the run restarts, you need a persistent record. A better approach: use a 'last sync' timestamp and only process contacts modified since then, but that doesn't solve duplicates if a contact was partially processed. The upsert approach is the most reliable. Also, consider enabling 'Sequential processing' and handling errors per contact so that one failure doesn't abort the whole scenario. Use an error handler on the MAP module that logs the error and continues, then at the end, report failures. This way, each contact is attempted once per run, and the upsert prevents duplicates.