Apache Airflow 2.0 Celery Executor Installation

0
votes

I understand that for Airflow CeleryExecutor, celery and rabbitmq (or redis) are required. Are the requirements fulfilled by simply running pip install apache-airflow[celery,rabbitmq] or do we need to run sudo yum install rabbitmq-server and pip install celery?

My question is for Airflow 2.0+

1
rabbitmqceleryairflow

1 Answers

0
votes

  1. If you install apache-airflow[celery] on a fresh virtual environment you will see that it already installs celery for you. Thus, pip install celery is redundant.

  2. You do need rabbitMQ server (the actual process). Alternatively, you can use rabbitMQ in docker.

  3. Not sure about apache-airflow[rabbitmq], because I know that Celery use amqp protocol to communicate with rabbitMQ. amqp==5.0.6 package is being installed with celery's installation (which happened by pip install apache-airflow[celery]).