Celery workers concurrency

Question

I have a python script that is to be a celery task. My application needs a number of concurrent workers(from the same script) because each worker needs to process the messages sequentially. I cannot use the parameter concurrency = n because then the messages are not processed sequentially which is kind of a requirement.

So what I have done is set up 'n' different queues in the message broker and then start 'n' different workers and each of those workers processes each queue separately, and I have put concurrency = 1 in each worker so each queue is processed sequentially. The workers are from the same script so to start those workers I have to copy the script n times and run all of them parallel in different threads.

This does not seem to me to be the most efficient way to achieve this as the code is copied a number of times. Is there any other way to achieve this?

Let me know if any more details are required.

The part 'I cannot use the parameter concurrency = n because then the messages are not processed sequentially which is kind of a requirement' is very confusing. Sequential and concurrent processing are totally opposite approaches. In case you need concurrent processing but task should be taken out of queue in FIFO manner — it's totally doable (usually through configuration of your queue). — Igor Pomaranskiy
Sorry if I am not totally clear, let me try to clarify. So in each queue, the messages need to be processed one by one because the operation that needs to be done on the current message depends on the previous message. So, if I had started a worker with concurrency = n, then that worker will have n processes, each of which will take up a message and it would be impossible to discern the exact previous message for each of those processes. — Tarique