Denodo Scheduler: Jobs, Data Movement, and Incremental Loads
What this domain tests
The Scheduler is a separate server that automates work: exports, cache refreshes, data movement between systems, replication and maintenance. Expect job anatomy, triggers, dependencies and incremental-load mechanics.
Job structure
A job defines what runs and where the data goes. Common types are VDP query/export, data movement (extract from a source and load into a cache or another database), replication between VDP servers, and maintenance such as cache refresh. Data movement jobs pair a source element with a destination and options for insert, truncate-and-insert, or update. Jobs are stored in the Scheduler's own metadata and run independently of the VDP client.
Triggers and dependencies
Triggers are time-based schedules, on-demand execution, or dependency on another job finishing. Dependencies express pipelines: load the dimension before the fact table. Execution history records each run's status and row counts, which is the first place to look when a report is stale.
Incremental loads
The incremental option reads only rows beyond a stored high-water mark. You nominate a field - usually a timestamp or a monotonically increasing sequence - and the Scheduler persists the last processed value. The next run filters on that value. This is efficient and simple, but it only captures inserts unless the field is updated on modification.
Worked example
Nightly loading of new orders from Oracle into a cached view. Reasoning: choose incremental with order_id as the high-water mark. If orders can be updated after insert, order_id never changes, so updated rows are missed - either use a last_modified timestamp with a small overlap window, or run a periodic full refresh. Check whether deletes occur at source; a high-water mark can never see them, so a separate reconciliation job is required.
Traps that cost marks
- Using a source-side clock for the mark and a scheduler-side clock for the filter, creating timezone gaps.
- Assuming incremental loads capture updates and deletes.
- Building dependencies that never fire because the upstream job fails silently.
- Forgetting that a cache refresh job is what keeps a full-cache view current.