Airflow: Run DAG every minute

Question

Hi I have my DAG parameters structured as so

default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2017, 9, 26), 'schedule_interval': "* * * * *", 'email': ['[email protected]'], 'email_on_failure': False, 'email_on_retry': False, }

But I'm not seeing my Airflow scheduler scheduling these tasks a minute apart or anywhere at all. I have also consulted this question but it does not seem to be working as well. Am I missing something when constructing my DAG?

Is the DAG enabled? When you go to the main page of the UI, there should be an on/off toggle by each DAG. — Daniel Huang
Is your scheduler running? Did you go through the documentation at airflow.incubator.apache.org/… ? — Matthijs Brouns
Yes the scheduler is running and the DAG is enabled. I'm getting weird interval, not the every minute update as I expected — Minh Mai

Mike Mike · Accepted Answer · 2017-09-28T13:16:56

I'm not 100% sure about this (and just started digging into the airflow code to try and verify) but you might need to pass the schedule interval into the DAG like so:

dag = DAG(DAG_NAME, schedule_interval='* * * * *', default_args=default_args)

I had tried to do it like you did and had no luck either. This seemed to work for me though I have 0 12 * * * as my schedule interval.

Airflow: Run DAG every minute

1 Answers