Airflow Fundamentals study materials
Airflow Fundamentals study resources

Airflow Fundamentals: DAG Dependencies

Airflow Fundamentals · Work through your exam, one topic at a time.

All guide sections

DAG Dependencies

Two kinds of dependency

Candidates must separate intra-DAG dependencies from inter-DAG dependencies. Intra-DAG dependencies are edges between tasks inside one DAG: task_a >> task_b means B waits for A. These edges drive state propagation - if A fails, B becomes upstream_failed and is not run. Operators like TriggerDagRunOperator and sensors are how DAGs relate to each other.

Cross-DAG patterns

To make DAG B follow DAG A, common approaches are: TriggerDagRunOperator (A explicitly triggers B), ExternalTaskSensor (B waits for a task in A to reach a state for the same logical date), or Datasets/Assets (A declares a dataset as an outlet; B schedules on that dataset and runs when it is updated). The sensor approach is fragile if schedules drift, because the sensor waits for a matching execution date by default; dataset scheduling avoids clock alignment problems.

Traps to watch

A frequent trap is assuming ExternalTaskSensor waits for "the latest run" - it matches execution dates unless configured otherwise, so mismatched schedules cause long waits or timeouts. Another is thinking dependencies transfer data: edges only order execution; data must move through XComs, storage, or an external system. Also remember trigger_rule: the default all_success blocks downstream tasks when any upstream is skipped; all_done, one_success, and none_failed change that behavior and appear often in questions.

Rule of thumb

Prefer dataset-driven or explicit trigger dependencies over time-aligned sensors when the upstream schedule can shift.