Proper signal order to safely stop process

2
votes

I have a long-running Python process that I want to be able to terminate in the event it gets hung-up and stops reporting progress. But I want to signal it in a way that allows it to safely cleanup, in case it hasn't completely hung-up and there's still something running that can respond to signals gracefully. What's the best order of signals to send before outright killing it?

I'm currently doing something like:

def safe_kill(pid):
    for sig in [SIGTERM, SIGABRT, SIGINT, SIGKILL]:
         os.kill(pid, sig)
         time.sleep(1)
         if not pid_exists(pid):
             return

Is there a better order? I know SIGKILL bypasses the process entirely, but is there any significant difference between SIGTERM/SIGABRT/SIGINT or do they all have the same effect as far as Python is concerned?

2
pythonsignals
You're going to need the process to register a signal handler if you want it to clean up after any of those signals. I don't think Python turns any of those into exceptions; it just stops. - user2357112 supports Monica
I think the answer here depends on the python process that you're trying to gracefully kill -- Does it register any signal handlers? If so, are they documented? If it doesn't register any handlers, then I guess it doesn't really matter what you send first ;-) - mgilson
@user2357112 -- SIGINT gets turned into KeyboardInterrupt IIRC ... - mgilson
Right, forgot Ctrl-C was SIGINT. I thought it was SIGHUP or something. - user2357112 supports Monica
If your Python process gets hung-up, shouldn't you fix the root cause for why it gets hung-up? :) I see people using daemons to restart such processes, when they should actually be putting in the effort to make it not get stuck at all. Who restarts the restarter? - Asclepius

2 Answers

2
votes

I believe the proper way for stopping a process is SIGTERM followed by SIGKILL after a small timeout.

I don't think that SIGINT and SIGABRT are necessary if that process handles signals in a standard way. SIGINT is usually handled the same way as SIGTERM and SIGABRT is usually used by process itself on abort() (wikipedia).

Anything more complex than a small script usually implements custom SIGTERM handling to shutdown gracefully (cleaning up all the resources, etc).

For example, take a look at Upstart. It is an init daemon - it starts and stops most of processes in Ubuntu and some other distributions. The default Upstart behavior for stopping a process is to send SIGTERM, wait 5 seconds and send SIGKILL (source - upstart cookbook).

You probably should do some testing to determine the best timeout for your process.

1
votes

You need to register a signal handler, as you would do in C.

import signal
import sys

def clean_termination(signal):
    # perform your cleanup
    sys.exit(1)

# register the signal handler for the signals specified in the question
signal.signal(signal.SIGTERM, clean_termination)
signal.signal(signal.SIGABRT, clean_termination)

Note that Python maps the SIGINT signal to a KeyboardInterrupt exception, that you can catch with a regular except statement.