I am trying to have the Airflow scheduler & webserver running in the background on the ubuntu EC2 instance. Whenever I start the upstart job, the airflow-webserver job goes straight to stop/waiting. Here is the .conf file that I have saved in /etc/init. (I used anaconda to install airflow)
description "Airflow webserver daemon"
start on started networking
stop on (deconfiguring-networking or runlevel [016])
respawn
respawn limit 5 30
setuid unbuntu
setgid unbuntu
env AIRFLOW_CONFIG=/home/ubuntu/airflow/airflow.cfg
env AIRFLOW_HOME=/home/ubuntu/airflow/
export AIRFLOW_CONFIG
export AIRFLOW_HOME
exec /home/ubuntu/anaconda3/bin/airflow webserver
When I check the logs, this is what I am seeing:
starting airflow-webserver...
/home/airflow/airflow/
/home/airflow/airflow/airflow.cfg
[2018-07-23 19:44:29,341] {__init__.py:45} INFO - Using executor SequentialExecutor
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2018-07-23 19:44:29,883] {models.py:189} INFO - Filling up the DagBag from /home/ubuntu/airflow/dags
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
=================================================================
But then nothing after that.
