Unexpected Dag schedules for catchup=false

Question

I'm learning basics of Airflow (apache-airflow==1.10.1, MacBook OSX) and unable to understand the actual schedules created for dag runs.

Created a simple DAG with single PythonOperator:

some start_date in the past
schedule_interval: schedule every 10 minutes
catchup is False because I DO NOT want to to any backfill

The system results in following dag runs (execution_date, /start_date)

(2019-01-14 01:57:10.404054, 2019-01-14 02:17:10.410499)
(2019-01-14 02:07:10.404054, 2019-01-14 02:17:12.226403)
(2019-01-14 02:17:10.404054,2019-01-14 02:27:11.797695)

Considering, I activated the schedule around 2019-01-14T02:17:10(that’s today UTC 14th Jan19 @2:17:10am), I had expected system to only created schedule in line # 2. & 3, but not the one in line # 1.?

Here's code:

default_args = {
    'owner': 'ga_mp', 'depends_on_past': False,
    'start_date': datetime(2019, 1, 10, 4, 20, 00),
}
dag = DAG(dag_id = 'my_dag_v1',
          default_args=default_args,
          schedule_interval=timedelta(minutes=10),
          catchup=False
       )

Thanks a lot!

kaxil kaxil · Accepted Answer · 2020-05-11T23:12:47

I have explained this in detail in https://stackoverflow.com/a/61740904/5691525

There was a bug that created extra DagRun which is fixed in Airflow Master and would be available in Airflow 1.10.11

Unexpected Dag schedules for catchup=false

2 Answers