The Branch That Ran on Stale Data
A workflow has two parallel branches fed from one input: Branch A aggregates daily sales and writes a summary table; Branch B applies a cleansing rule that depends on a reference table refreshed weekly. The reference table is loaded by a separate scheduled workflow that runs at 02:00; the main workflow runs at 01:30. On Mondays the summary numbers are consistently wrong and are corrected manually by mid-morning. The team wants to add a wait/delay step before Branch B so it 'runs after the reference is ready.'
1. Diagnose the Monday-only failure and assess the proposed delay.
Read the worked answer
The main workflow starts at 01:30, before the weekly reference refresh at 02:00, so on the day the reference changes (Monday) the cleansing branch uses last week's reference. A fixed delay is fragile: it guesses at the refresh duration and fails again whenever the refresh is slow or late. It also couples correctness to a magic number nobody will remember to update.
Better diagnosis framing: this is a dependency ordering problem, not a timing problem. The correct fix is to make the main workflow depend on the reference being complete, not merely on time having passed.
Confirm by checking the reference table's last-modified timestamp versus the main run's start on a failing Monday; the reference will be from the prior week.
2. Design a dependency mechanism that removes the Monday failure without a hard-coded delay.
Read the worked answer
Make readiness observable and gate on it:
- Sentinel/manifest: the weekly refresh writes a completion marker (or a run-date column) only when finished. The main workflow checks for the marker for the current cycle and fails fast if absent.
- Timestamp assertion: alternatively, assert that the reference's max load date equals the expected cycle date; if not, stop before Branch B runs.
- Scheduling: move the main workflow after the refresh window and make the refresh trigger the main workflow on success, so ordering is enforced by the scheduler rather than by clock arithmetic.
- Fail loud, don't wait forever: a bounded retry with a clear error is preferable to an open-ended wait that hides a broken upstream.
The tested principle: encode dependencies as explicit readiness checks and fail-fast gates, not as sleeps or staggered clocks.