Python Django Celery, terminating running task not working

Question

I create an web app, where users can run multiple scripts (celery tasks).

I went through this artcle and found out that i have to use --pool (prefork, eventlet or gevent) not solo because more scripts can run in one time.

What i need is to be able to terminate the running task if user needs to do so therefore i need to use only prefork . I found that eventlet and gevent do not have implemented revoke method.

Now the issue. When i run --pool with eventlet or gevent , the task is received and executed. But i cannot terminate it as said. When i run --pool with prefork , it should be able to terminate the task, but when i run the task it is only received but not even executed.

And the last think i do no realy understand. I tried to start with --pole=solo just to try revoking tasks (but i cannot use this later, because i need to run more scripts in one time). And revoke method here does nothing.

app.control.revoke(task_id, terminate=True)

So my question is what should i use, to be able to terminate tasks and to be able to run more tasks in one time.

Update for Krukas:

When I run celery by celery -A TNS worker --loglevel=INFO --without-gossip --without-mingle --without-heartbeat -Ofair --pool=gevent --concurrency=2

and then in the app run the script celery shows

[2020-01-04 18:13:33,213: INFO/MainProcess] Received task: api.tasks.task_1[d4e37b3b-67a7-49b9-879f-5157bcc28585]
[2020-01-04 18:13:43,418: INFO/MainProcess] Task api.tasks.task_1[d4e37b3b-67a7-49b9-879f-5157bcc28585] succeeded in 10.202999999999975s: None

so the task was executed, when I run Celery by celery -A TNS worker -l info --without-gossip --without-mingle --without-heartbeat -Ofair --pool=prefork --concurrency=2

the log shows

[2020-01-04 18:21:39,255: INFO/MainProcess] Received task: api.tasks.task_1[4071904e-e014-4075-828e-58a0a87c781f]
[2020-01-04 18:21:40,696: INFO/SpawnPoolWorker-4] child process 9708 calling self.run()
[2020-01-04 18:21:40,696: INFO/SpawnPoolWorker-3] child process 8428 calling self.run()

And the script is executed.

Krukas Krukas · Accepted Answer · 2020-01-03T14:41:05

Take a look at this documentation for revoke https://docs.celeryproject.org/en/latest/userguide/workers.html#worker-persistent-revokes

There are two important things:

revoke only works with RabbitMQ or Redis broker.
If you don't have statedb enabled and your workers are restarted it will lose the revoke information.

Python Django Celery, terminating running task not working

1 Answers