Airflow Fundamentals study plan

Label each core component (scheduler, webserver, metadata database, executor, worker, triggerer, DAG files processor) on a hand-drawn architecture diagram and state one responsibility per component

Trace the lifecycle of a single task instance from DAG parsing through queued, running, and terminal states, noting which component acts at each step

Compare LocalExecutor, CeleryExecutor, and KubernetesExecutor by writing three sentences on when each is appropriate

Categorize five real-world use cases (ETL, ML pipeline, report generation, infrastructure maintenance, backfill) and identify which Airflow feature each relies on

Write three DAGs from scratch using the TaskFlow API and the classic operator style, each with at least four tasks and explicit start and end dates

Configure schedule intervals using a cron expression, a timedelta, and a preset, and predict the resulting run dates before checking them

Set catchup, max_active_runs, retries, retry_delay, and dagrun_timeout on a DAG and explain the effect of each in writing

Trigger a deliberate failure and diagnose it using task logs, then fix it and confirm a green run

Build one DAG using the bitshift operators to express a fan-out then fan-in pattern and verify the rendered graph matches intent

Implement cross-DAG dependency using an ExternalTaskSensor or a dataset-triggered schedule and explain the tradeoff

Trigger backfills over a date range and compare the resulting run states in the grid view

Clear and rerun a subset of tasks and describe how downstream task states change

Write a PythonOperator task that pulls a value from XCom and one that pushes a value, then inspect the stored value in the UI

Create a Connection and a Variable through the UI and the CLI, and consume both inside a DAG without hardcoding credentials

Implement a sensor with a timeout and a poke interval, and demonstrate what happens when it times out

Explain in writing why large payloads should not be passed through XCom and propose an alternative

List the ten most useful CLI commands (dags list, tasks test, dags trigger, tasks states-for-dag-run, connections add, variables set, and similar) and run each once

Use the CLI to test a single task in isolation without running the whole DAG

Navigate the UI to locate a failed task run, read its logs, and clear it from the grid view

Identify where the UI exposes DAG parsing errors and audit one intentionally broken DAG

Use the UI to compare a successful run and a failed run of the same DAG and document the differences visible in each view

Filter DAG runs by state and date range and reconcile the count against the CLI output

Explain how the scheduler decides which runs to create under catchup versus no catchup, using evidence from your own DAGs

Review all eight domains and list any objective you cannot yet explain aloud

Design and build a capstone DAG that ingests data, transforms it, and writes an output, with retries and alerting configured

Parameterize the capstone with DAG params or a Variable so it can run for two different date ranges

Add a branching or conditional path and verify both branches execute under different inputs

Stress the DAG with a backfill of at least five runs and confirm all complete successfully

Revisit every domain and answer at least five self-written questions per domain without notes

Re-explain the architecture diagram from week one from memory and correct any errors

Practice timed CLI drills covering DAG listing, triggering, testing, and inspecting task states

Review the week six gap list and confirm each item is now resolved