how to see all celery tasks pushed in rabbitmq queue

Question

I've got django project with celery 2.5.5 and rabbitmq backend on debian 6. I've got over 6000 tasks of different types in one queue. There were some bug in code and I need to list all tasks in that queue and pull out some of them. All I need is findout all task ids in rabbitmq queue. I cant findout way how to connect rabbitmq queue and list it's content, best without starting up management plugin.

Great would be something pythonish like:

    import somelib
    conn = somelib.server(credentials, vhost)
    queue = conn.get_queue(queue_name)
    messages = queue.get_messages()

But any other tool to list such queue helps. Found out some tool installed using npm, but debian 6 does not know npm and building it from source is not quite pleasant way.

Or something to backup rabbitmq queues in human readable form is also appreciated.

Thanks for ideas

Pavel

Answer 1

You can use celery flower library to do that.

It will provide you with multiple features like displaying task progress and history, showing the task details and graphs and statistics in a pretty dashboard-style interface.

Below are some screenshots for reference.

Task Dashboard:

Worker tasks:

Task info:

Answer 2

If you are up for a premade interface you will like flower. It shows you all tasks in a nice web view.

If you are however trying to process each task programmatically flower isn't the right thing, since it doesn't support it. Then you would have to use a rabbitmq/AMQP library for python which has been discussed about e.g. here: Good Python library for AMQP

With this it should definitely be possible to do your imagined code in some or another way, but you'll have to read into that, since I've been fine with celery and flower for now.