How do I poll and schedule a DAG in Apache Airflow?

0
votes

How to schedule a job in Airflow to poll and run every 30 minutes between 1- 3 ? I am trying it with the following cron syntax for schedule_interval value

  • Cron syntax 1: 0 1-3 ***
  • Cron syntax 2: 0 0/5 1,2,3 * * ?

Will cron syntax 2 work in airflow or any additional code is required to poll the DAG from 1-3 for every 30 minutes?

1
cronairflow-schedulerairflow

1 Answers

0
votes

Are you referring to hours 1-3 or days 1-3 of every month? And do you want to include 03:30?

Assuming hours and including 03:30, this cron will do: 0,30 1-3 * * *

Assuming days of month and including 03:30, this cron will do: 0,30 * 1-3 * *

There are a few cron-expression validators on the internet to check your expression, e.g. https://crontab.guru (there's a button "next" to see the next 5 times).

Or, if you prefer code, you could try croniter:

from croniter import croniter
from datetime import datetime

base = datetime(2021, 1, 1)
iter = croniter("0,30 1-3 * * *", base)

for _ in range(10):
    print(iter.get_next(datetime))

# 2021-01-01 01:00:00
# 2021-01-01 01:30:00
# 2021-01-01 02:00:00
# 2021-01-01 02:30:00
# 2021-01-01 03:00:00
# 2021-01-01 03:30:00
# 2021-01-02 01:00:00
# 2021-01-02 01:30:00
# 2021-01-02 02:00:00
# 2021-01-02 02:30:00