multiple threads require multiple pipes

Question

I recently asked how to get the python shell to get a subprocess output. and got a very useful respond - ie.e. feeding the POpen a stdout=subprocess.PIPE

then use the p.stdout.readline() and feed the result to print()

        p = subprocess.Popen(args, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE, cwd=StartPath, shell=False)
        it = iter(p.stdout.readline, b'')
        sRet = "## decode Error ##"
        for line in it:
            try:
                sRet = line.rstrip().decode('utf-8')
            except:
                pass

            print(sRet[:PYTHON_WINDOW_WIDTH])

        viRet = p.wait()

however, I am worried about what is happening when i am calling this POpen from multiple threads simultaniously.

the same subprocess.PIPE is being fead to each POpen. to the extend that each readline is getting results from both subprocesses.

besides getting somewhat quirky results, each loop (for line in it:)

seems to wait until both threads subprocesses have finished before moving on

this is not what I want.

So is there a way I can have a PIPE of my one (one per thread) so that I can depend on each thread being totally independant?

Thanks in advance

dmjalund dmjalund · Accepted Answer · 2011-12-06T04:41:34

It looks like I found my answer - using os.pipe()

the result looks something like this:

        r,w = os.pipe()

        my_stdout=os.fdopen(r)
        os.close(w)

        p = subprocess.Popen(args, stdout=my_stdout, stdin=subprocess.PIPE, stderr=my_stdout, cwd=StartPath, shell=False)
        it = iter(p.stdout.readline, b'')
        sRet = "## decode Error ##"
        for line in it:
            try:
                sRet = line.rstrip().decode('utf-8')
            except:
                pass

            with MyGlobals.PrintLock:
                print(sRet[:PYTHON_WINDOW_WIDTH])

        viRet = p.wait()

multiple threads require multiple pipes

1 Answers