Running two celery workers in a server for two django application

2
votes

I have a server in which two django application are running appone, apptwo for them, two celery workers are started with commands:

celery worker -A appone -B --loglevel=INFO


celery worker -A apptwo -B --loglevel=INFO

Both points to same BROKER_URL = 'redis://localhost:6379'

redis is setup with db 0 and 1

I can see the task configured in these two apps in both app's log, which is leading to warnings and errors.

Can we configure in django settings such that the celery works exclusively without interfering with each other's tasks?

1
pythondjangorediscelery

1 Answers

7
votes

You can route tasks to different queues. Start Celery with two different -Q myqueueX and then use different CELERY_DEFAULT_QUEUE in your two Django projects.

Depending on your Celery configuration, your Django setting should look something like:

CELERY_DEFAULT_QUEUE = 'myqueue1'

You can also have more fine grained control with:

@celery.task(queue="myqueue3")
def some_task(...):
  pass

More options here:

How to keep multiple independent celery queues?