python - Execute multiple .py files at the same time

Question

I have three .py files that I want to run at the same time in a Python script file. I initially called subprocess.call() three times (once for each .py file) but remembered that it blocks until the command is finished. I tried subprocess.Popen(['screen', 'python_file']) since I believe it doesn't block but when I was checking for the processes with screen -ls there was only one process running. How do I get all three programs running at the same time with a Python script? Should I be using the multiprocessing or multithreading library?

Edit: The other processes aren't supposed to finish since they're running in an infinite loop. Here's exactly what I had in my Python script file. I'm using screen because each .py file has stdout logged onto the terminal and I want to be able to see what is being logged for each one.

subprocess.Popen(['screen', './submitter.py'])
subprocess.Popen(['screen', './worker.py'])
subprocess.Popen(['screen', './tester.py'])

Popen already does what you want. If you can see only one process then it simply means that the other processes already finished their work(or maybe they exited due to an error). — Bakuriu
Any reason why you are using 'screen'? Its not needed just to run the files. — tdelaney
@tdelaney An example of what is being displayed when I run screen -ls? — Petesta
What you've done works for me. Perhaps the .py files failed. You could create them all in a list, sleep a few seconds and then print out return codes for p in procs:print p.returncode. It'll be None if the proc is still running. — tdelaney

Unknown Unknown · Accepted Answer · 2013-08-13T18:23:56

If you want to use multiprocessing, you can try this:

import multiprocessing

def worker(file):
    # your subprocess code


if __name__ == '__main__':
    files = ["path/to/file1.py","path/to/file2.py","path/to/file3.py"]
    for i in files:
        p = multiprocessing.Process(target=worker, args=(i,))
        p.start()

python - Execute multiple .py files at the same time

1 Answers