Data Integration and Development (DataWorks)
DataWorks is where most operational questions live. Know its layers: Data Integration for data synchronization (batch offline sync and real-time sync), DataStudio for authoring and scheduling nodes, Operation Center for monitoring and rerunning instances, and Data Quality and Data Map for governance and lineage.
Scheduling is the highest-yield topic. Nodes run on a cron-like schedule with a bizdate (business date) parameter; a node scheduled daily at 00:00 with bizdate T-1 processes yesterday's data. Downstream nodes reference upstream outputs, and the scheduler builds a DAG from those dependencies. Traps cluster here: candidates forget that a node only runs after its upstream succeeds, misread bizdate as the actual run date, and assume a failed node auto-retries forever (retry counts are configured). Another trap is confusing supplemental data (backfill) with a normal rerun - backfill generates instances for historical bizdates.
Data Integration readers and writers are configured per data source, with channel and concurrency settings controlling throughput; splitting keys prevent a single reader thread from becoming a bottleneck. Real-time sync uses a different mechanism than batch sync and should not be described as scheduled.
Worked example: a daily node must aggregate orders for the previous day. Schedule it at 01:00, set the SQL to filter on ${bizdate} (yesterday's partition), and let the scheduler pass bizdate automatically. If a three-day outage occurs, use supplemental data for those three bizdates rather than rerunning today's instance three times.
Rule of thumb: bizdate is the data's date, not the clock; dependencies define order; backfill is for history.