Routing Telemetry to Multiple Business Systems with Different SLAs
A utility company ingests smart-meter readings through IoT Hub. Three downstream systems need the data: (1) a billing pipeline that must receive every reading exactly once and tolerates 15-minute latency; (2) a real-time outage dashboard that needs sub-5-second latency and can tolerate occasional loss; (3) a cold archive that stores raw messages for seven years. The team currently uses a single consumer reading from the built-in endpoint and fanning out in application code. During peak, the billing pipeline occasionally reprocesses messages after consumer restarts, causing duplicate charges. The archive team wants to avoid maintaining its own compute.
1. What message-routing architecture best fits these three consumers, and why?
Read the worked answer
The three consumers have incompatible delivery contracts, so a single application-level fan-out conflates them. A better architecture uses IoT Hub message routing to separate concerns:
- Billing (exactly-once, 15-min latency): route to a durable endpoint such as a Service Bus queue or Storage queue, then process with idempotent handlers. True exactly-once end-to-end is not achievable over at-least-once transports; you get effectively-once via idempotency keys (for example device ID + reading timestamp or a message ID) and deduplication in the consumer or in the downstream store.
- Outage dashboard (sub-5s, loss-tolerant): route to Event Hubs or the built-in endpoint with a dedicated consumer group and a fast consumer; accept at-least-once and possible loss.
- Archive (7 years, no compute): route directly to a storage account (blob/ADLS) using a routing rule with the appropriate encoding, so no custom compute is required.
Judgement: match each consumer's SLA to an endpoint and delivery guarantee, and solve exactly-once with idempotency rather than claiming a transport guarantees it.
2. Why does the billing pipeline reprocess messages after restarts, and how should duplicates be handled?
Read the worked answer
Reprocessing after restart is expected: the built-in endpoint and most queue transports are at-least-once. If the consumer checkpoints or completes messages after doing work, a crash between work and checkpoint causes redelivery.
Handling options:
- Idempotency key derived from stable message content (device ID + reading timestamp + sequence). The billing handler checks whether the key was already processed and skips duplicates. This requires a durable dedupe store with a retention window at least as long as the redelivery window.
- Transactional outbox / conditional write so the charge and the dedupe record commit atomically.
- Checkpoint ordering: complete the message only after the charge is durably recorded, reducing (but not eliminating) duplicates.
Judgement: the correct answer is not "switch to exactly-once transport" - that does not exist here. It is to make the billing operation idempotent and to align checkpointing with durable side effects. Duplicate charges are a business-logic defect, not merely a transport setting.
3. What should the team verify about routing rules and the fallback route before relying on this design?
Read the worked answer
Routing rules have evaluation semantics that commonly cause silent data loss or duplication if misunderstood:
- First-match wins. A message is sent to the first route whose condition matches; it is not fanned out to all matching routes. If billing and archive both need the same message, either use a single route to an endpoint that fans out, or design conditions so each message matches exactly the intended route set - which may require multiple routes with complementary conditions or routing to a shared endpoint.
- Fallback route. Messages matching no route go to the fallback (default: built-in endpoint). If the fallback is disabled and no route matches, messages are dropped. The team must confirm the fallback is enabled or that conditions are exhaustive.
- Conditions and message properties. Rules match on message body, properties, or twin tags; verify the property names and that devices actually set them.
- Endpoint health and dead-lettering. Confirm downstream endpoints exist, are healthy, and that failures surface as metrics rather than silent loss.
Judgement: "route to three systems" is not a single fan-out unless explicitly designed; first-match semantics and fallback behavior must be validated with test messages before go-live.