Airflow Fundamentals study materials
Airflow Fundamentals study resources

Airflow Fundamentals: Basic Operators (e.g. PythonOperator)

Airflow Fundamentals · Work through your exam, one topic at a time.

All guide sections

Basic Operators (e.g. PythonOperator)

Operators and tasks

An operator is a template for a task. Instantiating it inside a DAG creates a task node. The PythonOperator calls a Python callable (python_callable) with optional op_args and op_kwargs; the @task decorator is the modern equivalent and returns a value that becomes an XCom automatically. Other basics include BashOperator (runs a shell command), EmptyOperator (a no-op placeholder for structure), and transfer operators for moving data between systems.

Templating and idempotency

Operators declare template_fields, which let Jinja expressions like {{ ds }} or {{ ti.xcom_pull(...) }} be rendered at runtime. A task should be idempotent: rerunning it for the same logical date should produce the same result, because retries and clears happen. Use the logical date (ds, data_interval_start) rather than datetime.now() for partition logic.

Traps to watch

Don't call the function at DAG parse time - PythonOperator(python_callable=my_func()) executes immediately and passes the return value; the correct form passes the function object without parentheses. Don't expect return values to flow automatically between plain operators: only the TaskFlow API or explicit xcom_push/xcom_pull moves data. Also remember operator arguments like retries and retry_delay belong on the task, not inside the callable.

Worked example

def load(ds):
    write_partition(ds)

task = PythonOperator(task_id="load", python_callable=load,
                      op_kwargs={"ds": "{{ ds }}"}, dag=dag)

The callable is passed, not invoked, and ds is templated at runtime.