DAG Runs
Anatomy of a run
A DAG run is one execution of a DAG for a specific logical_date (also called execution date). Each run contains task instances, one per task, each with its own state: queued, running, success, failed, upstream_failed, skipped, or up_to_date. Run types include scheduled, manual, backfill, and dataset. Run state is derived from task states - a run is successful when all its tasks succeed (subject to trigger rules).
Scheduling and backfill
Scheduled runs are created by the scheduler as intervals elapse. A manual trigger creates a run immediately with a chosen logical date. A backfill creates runs for historical intervals, which is how you reprocess past data; with catchup=True, the scheduler itself performs backfill-like creation of missed runs. Clearing a task instance re-queues it within its existing run, creating a new attempt while preserving the run.
Traps to watch
Don't equate logical_date with wall-clock execution time: a run for 2024-01-01 on a daily schedule typically executes just after 2024-01-02 begins. Don't assume re-running a DAG overwrites history - each run is recorded separately, and multiple runs can share a logical date if triggered manually. Also, depends_on_past makes a task wait for the previous run's instance, which can stall a whole chain if one historical run failed.
Rule of thumb
Treat logical_date as the data interval label, not the start time. When debugging "why didn't it run," check run creation (schedule, catchup, pause state) before checking task execution.