Gunicorn multiple flask workers and bound tasks celery

Question

Following this guide(Complex Example: Showing Status Updates and Results section) i have 2 flask endpoints for starting a bound task (POST request) and retreiving tasks result by its id (GET request to /status/<task_id>).

Running flask app as flask run in one shell and celery worker -A app.celery -l info in another one, it is possible to run task and then get its result by GET request to the /status/ endpoint.

After adding gunicorn and setting number of workers to 3, POST requests run normally, but getting status of a specific running task is a problem, as it can't get task(task.info is None). There is a chance of getting task result by this status endpoint, but if i correctrly understand the problem, it depends on which flask instance gunicorn redirects a request to.

I dont set any specific celery setting, only broker and result_backend(using RabbitMQ).

How to correctly configure the gunicorn+flask+celery for this sort of tasks?

Answer 1

Fixed by using redis as result backend(or any other than RPC, i believe).

According to the documentation

The RPC result backend (rpc://) is special as it doesn’t actually store the states, but rather sends them as messages. This is an important difference as it means that a result can only be retrieved once, and only by the client that initiated the task. Two different processes can’t wait for the same result.