The Dashboard That Won't Refresh in Time
An executive dashboard is built on a Designer Cloud workflow that reads three cloud sources, joins them, and publishes a summary. Users complain the dashboard shows yesterday's numbers until late morning. The workflow is scheduled hourly from 06:00, but each run takes 50–70 minutes and the first run often overlaps the second. Governance requires that published outputs be traceable to a specific source snapshot and that only approved users can change the published dataset. Two analysts have been editing the workflow directly to 'speed it up.'
1. What are the distinct problems here, and how would you separate them?
Read the worked answer
Three separable problems are tangled together:
- Latency/overlap: a 50–70 minute run on an hourly schedule means runs overlap, contend for resources, and the dashboard reads whichever run last succeeded - hence 'yesterday's numbers until late morning.'
- Traceability: outputs must map to a specific source snapshot; if runs overlap and sources move, the published number's provenance is ambiguous.
- Change control: two analysts editing the live workflow bypasses governance and makes both latency and provenance unpredictable.
Separate them because the fixes differ: latency is an engineering problem, provenance is a metadata/versioning problem, and change control is a permissions/process problem. Treating them as one 'make it faster' task is why it has stayed broken.
2. Propose an approach that addresses latency and traceability together, respecting the governance constraints.
Read the worked answer
- Decouple cadence from freshness: if true freshness is not needed, run less often than the runtime (e.g., every 2 hours) so runs never overlap; if freshness is needed, optimize the run rather than stacking schedules. Overlapping schedules are the direct cause of the stale-morning symptom.
- Snapshot pinning: have each run read a fixed, dated source snapshot and stamp the output with that snapshot id and run id. This makes each published number traceable to a specific input, satisfying the provenance requirement regardless of schedule.
- Publish atomically: write to a staging output and promote to the published dataset only on successful completion, so the dashboard never reads a half-finished run.
- Lock down editing: restrict edit rights on the production workflow to an approved group; analysts work on a copy/branch and promote through review. This restores change control and stops ad-hoc 'speed-ups' from silently altering published logic.
Confirm the fix by checking that consecutive publishes carry distinct, correct snapshot ids and that no two runs overlap.