python - Popen waiting for child process even when the immediate child has terminated

Question

I'm working with Python 2.7 on Windows 8/XP.

I have a program A that runs another program B using the following code:

p = Popen(["B"], stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
return

B runs a batch script C. C is a long running script and I want B to exit even though C has not finished. I have done it using the following code (in B):

p = Popen(["C"])
return

When I run B, it works as expected. When I run A however, I expected it to exit when B exits. But A waits until C exits even though B has already exitted. Any ideas on what's happening and what possible solutions could be?

Unfortunately, the obvious solution of changing A to look like B is not an option.

Here is a functional sample code to illustrate this issue: https://www.dropbox.com/s/cbplwjpmydogvu2/popen.zip?dl=1

Any input is much appreciated.

If I get this right, you have program Am which runs program B which runs program C. Program A also runs program C. Is this correct? — Some programmer dude
No, program A doesn't run program C directly. It would be great if you took a look at attached example. Thanks. — khattam
Does supplying close_fds=True to the second Popen() call help? I'm guessing that C inherits the stdout/stderr pipes from A and thus A waits until C closes them. — Simon

jfs jfs · Accepted Answer · 2012-11-06T18:17:02

You could provide start_new_session analog for the C subprocess:

#!/usr/bin/env python
import os
import sys
import platform
from subprocess import Popen, PIPE

# set system/version dependent "start_new_session" analogs
kwargs = {}
if platform.system() == 'Windows':
    # from msdn [1]
    CREATE_NEW_PROCESS_GROUP = 0x00000200  # note: could get it from subprocess
    DETACHED_PROCESS = 0x00000008          # 0x8 | 0x200 == 0x208
    kwargs.update(creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)  
elif sys.version_info < (3, 2):  # assume posix
    kwargs.update(preexec_fn=os.setsid)
else:  # Python 3.2+ and Unix
    kwargs.update(start_new_session=True)

p = Popen(["C"], stdin=PIPE, stdout=PIPE, stderr=PIPE, **kwargs)
assert not p.poll()

[1]: Process Creation Flags for CreateProcess()

python - Popen waiting for child process even when the immediate child has terminated

2 Answers