Celery Tasks with eta get removed from RabbitMQ

Question

I'm using Django 1.6, RabbitMQ 3.5.6, celery 3.1.19.

There is a periodic task which runs every 30 seconds and creates 200 tasks with given eta parameter. After I run the celery worker, slowly the queue gets created in RabbitMQ and I see around 1200 scheduled tasks waiting to be fired. Then, I restart the celery worker and all of the waiting 1200 scheduled tasks get removed from RabbitMQ.

How I create tasks: my_task.apply_async((arg1, arg2), eta=my_object.time_in_future)

I run the worker like this: python manage.py celery worker -Q my_tasks_1 -A my_app -l

CELERY_ACKS_LATE is set to True in Django settings. I couldn't find any possible reason.

Should I run the worker with a different configuration/flag/parameter? Any idea?

1) Are you sure you restart only the worker not the whole rabbit server? 2) Maybe you set some task ttl time and it simply gets removed after this time? — matino
1) I am sure that only workers are restarted. 2) There is no ttl set to task in any configuration. It is what default is. But if I keep the workers open, the tasks remains in the queue. — Emin Bugra Saral

the_drow the_drow · Accepted Answer · 2016-02-01T09:10:38

As far as I know Celery does not rely on RabbitMQ's scheduled queues. It implements ETA/Countdown internally.
It seems that you have enough workers that are able to fetch enough messages and schedule them internally.
Mind that you don't need 200 workers. You have the prefetch multiplier set to the default value so you need less.

Celery Tasks with eta get removed from RabbitMQ

1 Answers