python - Printing to screen and writing to a file at the same time

Question

I found some code online that generally works, but I want to use it multiple times in the same program (write different things to different files, while still printing to the screen the whole time).

That is to say, when it closes, I think sys.stdout closes, so printing at all, and using this class again fails. I tried reimporting sys, and other dumb stuff, but I can't get it to work.

Here's the site, and the code groups.google.com/group/comp.lang.python/browse_thread/thread/d25a9f5608e473af/

import sys

class MyWriter:

    def __init__(self, stdout, filename):
        self.stdout = stdout
        self.logfile = file(filename, 'a')

    def write(self, text):
        self.stdout.write(text)
        self.logfile.write(text)

    def close(self):
        self.stdout.close()
        self.logfile.close()

writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer

print 'test' 

Answer 1

You are trying to reproduce poorly something that is done very well by the Python Standard Library; please check the logging module.

With this module you can do exactly what you want, but in a much simpler, standard, and extensible manner. You can proceed as follows (this example is a copy/paste from the logging cookbook):

Let’s say you want to log to console and file with different message formats and in differing circumstances. Say you want to log messages with levels of DEBUG and higher to file, and those messages at level INFO and higher to the console. Let’s also assume that the file should contain timestamps, but the console messages should not. Here’s how you can achieve this:

import logging

# set up logging to file - see previous section for more details
logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
                    datefmt='%m-%d %H:%M',
                    filename='/temp/myapp.log',
                    filemode='w')
# define a Handler which writes INFO messages or higher to the sys.stderr
console = logging.StreamHandler()
console.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger().addHandler(console)

# Now, we can log to the root logger, or any other logger. First the root...
logging.info('Jackdaws love my big sphinx of quartz.')

# Now, define a couple of other loggers which might represent areas in your
# application:

logger1 = logging.getLogger('myapp.area1')
logger2 = logging.getLogger('myapp.area2')

logger1.debug('Quick zephyrs blow, vexing daft Jim.')
logger1.info('How quickly daft jumping zebras vex.')
logger2.warning('Jail zesty vixen who grabbed pay from quack.')
logger2.error('The five boxing wizards jump quickly.')

When you run this, on the console you will see

root        : INFO     Jackdaws love my big sphinx of quartz.
myapp.area1 : INFO     How quickly daft jumping zebras vex.
myapp.area2 : WARNING  Jail zesty vixen who grabbed pay from quack.
myapp.area2 : ERROR    The five boxing wizards jump quickly.

and in the file you will see something like

10-22 22:19 root         INFO     Jackdaws love my big sphinx of quartz.
10-22 22:19 myapp.area1  DEBUG    Quick zephyrs blow, vexing daft Jim.
10-22 22:19 myapp.area1  INFO     How quickly daft jumping zebras vex.
10-22 22:19 myapp.area2  WARNING  Jail zesty vixen who grabbed pay from quack.
10-22 22:19 myapp.area2  ERROR    The five boxing wizards jump quickly.

As you can see, the DEBUG message only shows up in the file. The other messages are sent to both destinations.

This example uses console and file handlers, but you can use any number and combination of handlers you choose.

Answer 2

Easy-peasy with Python 3.3 and above

Starting with Python 3.3, doing so has become significantly easier since logging.basicConfig now accepts the handlers = argument.

import logging

level    = logging.INFO
format   = '  %(message)s'
handlers = [logging.FileHandler('filename.log'), logging.StreamHandler()]

logging.basicConfig(level = level, format = format, handlers = handlers)
logging.info('Hey, this is working!')

Note however, that certain Python modules may also be posting logging messages to the INFO level.

This is where it comes handy to create a custom logging level, called for example OK, 5 levels above the default INFO level and 5 levels below the default WARNING level.

Answer 3

Remove the line that's doing what you explicitly say you don't want done: the first line of close(), which closes stdout.

Answer 4

That is to say, when it closes, I think sys.stdout closes, so printing at all, and using this class again fails. I tried reimporting sys, and other dumb stuff, but I can't get it to work.

To answer your question, you should not be closing stdout. The python interpreter opens stdout, stdin and stderror at startup. In order for print to work, the interpreter requires stdout to be open. Reimporting sys does not do anything once a module has been loaded. You would need to reload the module. In this particular case, I am not sure a reload would fix the problem since sys.stdout allows stdout to be used as a file object.

Additionally, I think you have a bug in your code which may be causing print to break. In line 2 you are assigning a MyWriter object to sys.stdout. This may by closing stdout when the garbage collector deletes the unused stdout file object.

writer = MyWriter(sys.stdout, 'log.txt')
sys.stdout = writer

Answer 5

I know this is an old question, and the best answer is just to use logging for its intended purpose, but I just wanted to point out that if you're concerned only with affecting calls specifically to print (and not other interaction with sys.stdout), and you just want to paste a few lines into some old one-off script, there's nothing stopping you from simply reassigning the name to a different function which writes to two different files, since print is a function in Python 3+. You could even, god forbid, use a lambda with an or chain for the quickest, dirtiest solution out there:

old_print = print
log_file = open("logfile.log", "a")
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
print("Hello console and log file")
# ... more calls to print() ...
log_file.close()

Or for true fire-and-forget:

import atexit
old_print = print
log_file = open("logfile.log", "a")
atexit.register(log_file.close)
print = lambda *args, **kw: old_print(*args, **kw) or old_print(*args, file=log_file, **kw)
# ... do calls to print(), and you don't even have to close the file afterwards ...

It works fine assuming the program exits properly, but please no one use this in production code, just use logging :)

Edit: If you value some form of structure and want to write to the log file in real-time, consider something like:

from typing import Callable

def print_logger(
    old_print: Callable, 
    file_name: str,
) -> Callable:
    """Returns a function which calls `old_print` twice, specifying a `file=` on the second call.
    
    Arguments:
        old_print: The `print` function to call twice.
        file_name: The name to give the log file.
    """
    def log_print(*args, **kwargs):
        old_print(*args, **kwargs)
        with open(file_name, "a") as log_file:
            old_print(*args, file=log_file, **kwargs)
    return log_print

And then invoke as follows:

print = print_logger(print, "logs/my_log.log")