Why does this python program sometimes fail to exit?

Question

I wrote a test program, which has two processes. The father process gets data from a Queue, and the child puts data into it. There is a signal handler which tells the program to exit. However, it does not exit sometimes when I send the signal SIGTERM to the pid(child process) I printed, and it seems to be having a deadlock.

import os
import sys 
import multiprocessing
import time
import signal

bStop = False
def worker(que):
    signal.signal(signal.SIGTERM,sighandler)
    print 'worker:',os.getpid()
    for i in range(100000000):
        que.put(i)

    print 'STOP'

def sighandler(num,frame):
    print 'catch signal'
    q.put('STOP')
    sys.exit(0)
q = multiprocessing.Queue(100)
p = multiprocessing.Process(target=worker,args=(q,))
p.start()

for item in iter(q.get,'STOP'):
    print 'get',item
    pass
print 'main stop'
p.join()

Wow... with 100,000,000 you weren't kidding about that range, were you? — cwallenpoole
Well, it's been running in the background for 20 minutes and hasn't gotten to 50,000,000 yet — cwallenpoole
@cwallenpoole Did you find the problem I mentioned above ? you should send SIGTERM to the child process rather than waiting itself to exit. — renenglish
I think you are blocking in queue.put method and signals in Python do not interrupt operations that is blocked in the native code. But I don't understand why is the parent process not clearing the queue. — MK.

John John · Accepted Answer · 2011-06-21T05:43:14

Unless you are running python 3 you should be using xrange instead of range for a loop that large. Python tends to choke once it exceeds a certain list size and so you really really need to move to generators by that point.

That very well could be the issue your seeing right now.

Why does this python program sometimes fail to exit?

1 Answers