Airflow DAG Authoring: Error Handling, Retries, and SLAs

Failure semantics

Tasks end in success, failed, upstream_failed, skipped, or up_for_retry. retries and retry_delay control automatic re-attempts; retry_exponential_backoff and max_retry_delay shape the schedule. execution_timeout kills a task that runs too long; sla (or the newer sla_miss_callback on the DAG) records a missed deadline but does not stop or retry the task. on_failure_callback, on_success_callback, and on_retry_callback fire per task; on_failure_callback on the DAG fires once per failed run. EmailOperator and email_on_failure are legacy alerting paths.

Traps candidates hit

• SLA misses are notifications, not enforcement. A question implying an SLA stops a task is a trap.
• retries counts re-attempts, not total attempts. retries=2 means up to three tries.
• trigger_rule='all_done' lets downstream cleanup run after failures; all_success skips it.
• execution_timeout is per attempt, and a timed-out task consumes a retry if retries remains.
• Callbacks receive context, so signature must accept it.
• depends_on_past failures cascade into subsequent intervals.

Rule of thumb

Retries handle transient failures; alerts and callbacks handle persistent ones. Design the graph so cleanup and notification tasks use all_done and never block the happy path.

Worked example

PythonOperator(task_id='load', python_callable=load, retries=3, retry_delay=timedelta(minutes=5), retry_exponential_backoff=True, execution_timeout=timedelta(hours=1), on_failure_callback=alert). Up to four attempts, each capped at one hour, with an alert after the final failure.