Order Status Custom Component Token Expiry and Timeout
A logistics company uses a custom component to return order status. It is Node.js, deployed as an OCI Function behind API Gateway, and invoked about 8,000 times per day. The component calls an order API with an OAuth 2.0 client-credentials token. The token is requested once at module load, stored in a global variable, and reused; the code assumes a 60-minute lifetime. At peak there are 40 concurrent conversations; p95 latency is 4.6 seconds against a 3-second target. Three percent of calls return 401 after the service has been warm for an hour, and the dialog says 'Order service unavailable' because the component throws and the flow's error transition goes to an agent. The order API supports idempotency keys, but the component does not send one. The custom component has a 10-second timeout. Builders want to fix this by increasing the timeout to 30 seconds.
1. Diagnose the 401 errors and the latency. Why is increasing the timeout to 30 seconds unlikely to help?
Read the worked answer
The token is fetched once per warm function instance and stored in a module global. Serverless instances are reused, but the token has an absolute expiry. If the code checks only that a token exists, not that it is still valid, the instance keeps sending an expired token after 60 minutes. The order API returns 401. Cold starts fetch a new token, so failures cluster on long-lived warm instances, which matches the 'after an hour' symptom. There is no refresh-on-401 path.
Latency is a separate problem. p95 of 4.6 seconds can come from cold starts, the downstream order API, or token acquisition. A 10-second component timeout is not the immediate cause of 4.6-second responses. Raising it to 30 seconds would make users wait longer, consume concurrency, and still return 401. Before changing anything, instrument the component with per-invocation metrics: token age, cold-start flag, downstream latency, retry count, and HTTP status. Also check whether API Gateway or the channel has its own timeout shorter than 30 seconds.
2. Design the token refresh, retry, and idempotency behavior for the custom component. Include how secrets should be handled.
Read the worked answer
Use a token cache with an expiry timestamp and a safety skew, for example refresh five minutes before expiry. On a 401, refresh once and retry the order call exactly once. Use a shared in-flight promise or mutex so 40 concurrent conversations do not trigger 40 token requests. Keep the token in memory only and never log it.
Send an idempotency key derived from the conversation or session identifier plus the order number, so a retry after a timeout cannot create a duplicate action. Apply a downstream request timeout shorter than the component timeout, such as 2.5 seconds, and fail fast with a structured result: status, errorCode, retryable, and correlationId. Do not raise the custom component timeout to 30 seconds; keep it within the platform limit and design for a fast, controlled failure. Store the client secret in OCI Vault or environment configuration, not in source. Log correlation IDs and status codes, but redact order details and customer PII.
3. How should the dialog flow consume the component's result so that retryable failures do not all become agent transfers?
Read the worked answer
Define explicit output variables such as orderStatus, errorCode, and retryable. In the flow, branch on errorCode instead of using a single catch-all error transition. For a timeout or 5xx marked retryable, show a brief message and invoke the component once more; if it fails again, offer agent transfer or callback. For 401 after a refresh, fail closed and alert operations rather than retrying indefinitely. For 404, tell the user the order was not found and re-prompt for the order number. For 429, back off and offer a retry later. Test with mocked 401, 500, timeout, and 404 responses to confirm each branch and to verify that the user is not asked to repeat information already collected.