Apache Airflow not scheduling tasks

5
votes

I have installed apache-airflow (version v1.9.0) along with python 2.7. To test whether its installed properly I tried to trigger a tutorial DAG from the interactive view in browser. The interface shows that the DAG is running, but the scheduler doesn't show any activity.

Below are the steps I tried

  1. Install airflow
pip install apache-airflow
  1. Install crypto for setting encryption using fernet key
pip install apache-airflow[crypto]
  1. Generate a fernet_key and add in airflow.cfg file: 
from cryptography.fernet import Fernet
fernet_key= Fernet.generate_key()
print(fernet_key)
  1. Initialise a airflow sqlite db 
airflow initdb
  1. Start the airflow webserver
airflow webserver -p 8080
  1. Start the airflow scheduler in a different window
airflow scheduler
  1. Trigger the tutotial DAG on the Airflow page at localhost:8080

After following these steps, I am not able to see any movement in my scheduler window, which just keeps me showing 

INFO - Heartbeating the process manager
INFO - Heartbeating the executor

I have tried it running in the local environment as well as in a virtual environment. I have also tried running through SequentialExecutor and LocalExecutor

the task is not triggered even when I try to trigger it through the terminal by airflow trigger_dag tutorial

I am working on mac OS High Sierra Version 10.13.3

3
pythonpython-2.7airflowmacos-high-sierraairflow-scheduler
can you run following commands? airflow list_dags and airflow list_tasks <dag_id> - Mubin
also can you test a task with the follwing. airflow test <dag_id> <task_id> 2018-06-06 - Mubin
I am facing exactly the same issue. - Ankit Basarkar

3 Answers

1
votes

The switch next to each task is on "OFF" by default. that was it for me

0
votes

By default, all the dags will be paused at the start. You have to unpause the dags and trigger them.

0
votes

After triggering your dag you have to turn on your DAG as by default it is off. You can turn this on by using AIRFLOW UI.