3
votes

I am trying to install airflow (distributed mode) in WSL, I got the setup of Airflow webserver, Airflow Scheduler, Airflow Worker, Celery (3.1) and RabbitMQ. While running the Airflow Scheduler it is throwing out this error (below) even though the backend is set up.

ERROR

Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/airflow/executors/celery_executor.py", line 92, in sync state = task.state File "/usr/local/lib/python3.6/dist-packages/celery/result.py", line 398, in state return self._get_task_meta()['status'] File "/usr/local/lib/python3.6/dist-packages/celery/result.py", line 341, in _get_task_meta return self._maybe_set_cache(self.backend.get_task_meta(self.id)) File "/usr/local/lib/python3.6/dist-packages/celery/backends/base.py", line 288, in get_task_meta meta = self._get_task_meta_for(task_id) AttributeError: 'DisabledBackend' object has no attribute '_get_task_meta_for'

https://issues.apache.org/jira/browse/AIRFLOW-1840

This is the exact error I am getting but couldn't find a solution.

Result Backend-

result_backend = db+postgresql://postgres:****@localhost:5432/postgres

broker_url = amqp://rabbitmq_user_name:rabbitmq_password@localhost/rabbitmq_virtual_host_name

Help please, gone through almost all the documents but couldn't find a solution

1

1 Answers

0
votes

I was facing the same issue on celery version - 3.1.26.post2 (with rabitmq,postgresql and airflow),the reason for this issue is the dictionary used in celery base.py file at(lib/python3.5/site-packages/celery/app/base.py) does not capture celery backend at key CELERY_RESULT_BACKEND instead it captures at key result_backend.

So the solution here is go to _get_config function available in base.py file at(lib/python3.5/site-packages/celery/app/base.py),at the end of the function before returning dictionary s add the below code.

s['CELERY_RESULT_BACKEND'] = s['result_backend'] #code to be added

return s

This solved the problem.