python - How do I push a subprocess.call() output to terminal and file?

Question

I have subprocess.call(["ddrescue", in_file_path, out_file_path], stdout=drclog). I'd like this to display the ddrescue in the terminal as it's running and write the output to the file drclog. I've tried using subprocess.call(["ddrescue", in_file_path, out_file_path], stdout=drclog, shell=True), but that gives me an input error into ddrescue.

ddrescue shows the process of the copy in this format; GNU ddrescue 1.18.1 Press Ctrl-C to interrupt rescued: 0 B, errsize: 0 B, current rate: 0 B/s ipos: 0 B, errors: 0, average rate: 0 B/s opos: 0 B, run time: 1 s, successful read: 1 s ago Finished I'd like to be able to see this and have the final status written to a file (stdout). — p014k
If you want to write something to terminal and file at the same time you can use tee command. I'm not sure if it is possible to write part of the output to terminal and part of it to a file if the data is written into one output stream. — kraskevich
I'm familiar with tee, but how would I go about doing this in python? in_file_path and out_file_path are necessary inputs into ddrescue. — p014k
args should be a string when using shell=True, else the arguments get passed as parameters to the shell itself. For example, call(['echo $0 and $1', 'eggs', 'spam'], shell=True) prints "eggs and spam". — Eryk Sun

jfs jfs · Accepted Answer · 2014-09-22T07:11:55

If ddrescue doesn't change its output if its stdout/stderr are redirected to a pipe then you could use tee utility, to display output on the terminal and to save it to a file:

$ ddrescue input_path output_path ddrescue_logfile |& tee logfile

If it does then you could try to provide a pseudo-tty using script utility:

$ script -c 'ddrescue input_path output_path ddrescue_logfile' -q logfile

If it writes directly to a terminal then you could use screen to capture the output:

$ screen -L -- ddrescue input_path output_path ddrescue_logfile

The output is saved in screenlog.0 file by default.

To emulate the tee-based command in Python without calling tee utility:

#!/usr/bin/env python3
import shlex
import sys
from subprocess import Popen, PIPE, STDOUT

command = 'ddrescue input_path output_path ddrescue_logfile'
with Popen(shlex.split(command), stdout=PIPE, stderr=STDOUT, bufsize=1) as p:
    with open('logfile', 'wb') as logfile:
        for line in p.stdout:
            logfile.write(line)
            sys.stdout.buffer.write(line)
            sys.stdout.buffer.flush()

To call the tee-based command in Python using shell=True:

#!/usr/bin/env python
from pipes import quote
from subprocess import call

files = input_path, output_path, ddrescue_logfile
rc = call('ddrescue {} |  tee -a drclog'.format(' '.join(map(quote, files))),
          shell=True)

To emulate the script-based command:

#!/usr/bin/env python3
import os
import shlex
import pty

logfile = open('logfile', 'wb')
def read(fd):
    data = os.read(fd, 1024) # doesn't block, it may return less
    logfile.write(data) # it can block but usually not for long
    return data
command = 'ddrescue input_path output_path ddrescue_logfile'
status = pty.spawn(shlex.split(command), read)
logfile.close()

To call screen command in Python:

#!/usr/bin/env python3
import os
import shlex
from subprocess import check_call

screen_cmd = 'screen -L -- ddrescue input_path output_path ddrescue_logfile'
check_call(shlex.split(screen_cmd))
os.replace('screenlog.0', 'logfile')

python - How do I push a subprocess.call() output to terminal and file?

2 Answers