python logging module - terminal, stdout, stderr

Question

I want to log my output from a python (2.7) script using the logging module to three streams:

to terminal / standard output (root logger)
to a log file containing all messages (full logger, similar to stdout)
to a log file containing only warning and error messages (error logger, similar to stderr)

Loggers 2. and 3. need to be RotatingFileHandlers (maxBytes: 1024, backupCount: 3), overwriting old historical log files after some time.

With the below setup, it successfully writes the logs to terminal in the desired format and also creates the stdout.log and stderr.log files, but both files appear empty.

Any hints on what I am missing to correctly populate the log files?

This is the test script

test_logger.py

import logging.config

logging.config.fileConfig('logging.conf') 

logging.debug("DEBUG MESSAGE")
logging.info("INFO MESSAGE")
logging.warning("WARNING MESSAGE")
logging.error("ERROR MESSAGE")

a = []
try :
    b = a[0]
except :
    logging.exception("EXCEPTION MESSAGE")
# end try

This is the configuration for the logger

logging.conf

[loggers]
keys=root,fullLogger,errorLogger

[handlers]
keys=rootHandler,fullHandler,errorHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=rootHandler
propagate=1

[logger_fullLogger]
level=DEBUG
handlers=fullHandler
qualname=fullLogger
propagate=1

[logger_errorLogger]
level=WARNING
handlers=errorHandler
qualname=errorLogger
propagate=1

[handler_rootHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[handler_fullHandler]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('stdout.log','w',1024,3)

[handler_errorHandler]
class=logging.handlers.RotatingFileHandler
level=WARNING
formatter=simpleFormatter
args=('stderr.log','w',1024,3)

[formatter_simpleFormatter]
format=[%(asctime)s.%(msecs)03d] %(levelname)s :: %(message)s
datefmt=%Y-%m-%d %H:%M:%S

I expect the following outputs:

terminal

[2017-12-13 15:18:59.265] DEBUG :: DEBUG MESSAGE
[2017-12-13 15:18:59.265] INFO :: INFO MESSAGE
[2017-12-13 15:18:59.265] WARNING :: WARNING MESSAGE
[2017-12-13 15:18:59.265] ERROR :: ERROR MESSAGE
[2017-12-13 15:18:59.265] ERROR :: EXCEPTION MESSAGE
Traceback (most recent call last):
  File "C:\path\test_logger2.py", line 12, in <module>
    b = a[0]
IndexError: list index out of range

stdout.log

[2017-12-13 15:18:59.265] DEBUG :: DEBUG MESSAGE
[2017-12-13 15:18:59.265] INFO :: INFO MESSAGE
[2017-12-13 15:18:59.265] WARNING :: WARNING MESSAGE
[2017-12-13 15:18:59.265] ERROR :: ERROR MESSAGE
[2017-12-13 15:18:59.265] ERROR :: EXCEPTION MESSAGE
Traceback (most recent call last):
  File "C:\path\test_logger2.py", line 12, in <module>
    b = a[0]
IndexError: list index out of range

stderr.log

[2017-12-13 15:18:59.265] WARNING :: WARNING MESSAGE
[2017-12-13 15:18:59.265] ERROR :: ERROR MESSAGE
[2017-12-13 15:18:59.265] ERROR :: EXCEPTION MESSAGE
Traceback (most recent call last):
  File "C:\path\test_logger2.py", line 12, in <module>
    b = a[0]
IndexError: list index out of range

HeXor HeXor · Accepted Answer · 2017-12-13T14:52:11

One has to combine the three handlers under one root logger:

[loggers]
keys=root

[handlers]
keys=rootHandler,fullHandler,errorHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=rootHandler,fullHandler,errorHandler

##### TERMINAL HANDLER #####
# Setup for output to terminal
# level=DEBUG : Logging ALL messages

[handler_rootHandler]
class=logging.StreamHandler
level=DEBUG
formatter=simpleFormatter
args=tuple()

##### STDOUT HANDLER #####
# Setup for output to stdout.log file
# level=DEBUG : Logging ALL messages
# arg=2097152 : RotatingFileHandler with 2097152 Bytes (2 MB) max size
# arg=3 : RotatingFileHandler with 3 backup files

[handler_fullHandler]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('stdout.log','w',2097152,3)

##### STDERR HANDLER #####
# Setup for output to stderr.log file
# level=WARNING : Logging WARNING,ERROR and EXCEPTION messages
# arg=2097152 : RotatingFileHandler with 2097152 Bytes (2 MB) max size
# arg=3 : RotatingFileHandler with 3 backup files

[handler_errorHandler]
class=logging.handlers.RotatingFileHandler
level=WARNING
formatter=simpleFormatter
args=('stderr.log','w',2097152,3)

##### APPEARANCE #####
# Setup for formatting of log message
# e.g. [2017-12-13 16:00:50.983] DEBUG :: ext_module : DEBUG MESSAGE
# e.g. [2017-12-13 16:00:50.983] INFO :: ext_module : INFO MESSAGE
# e.g. [2017-12-13 16:00:50.983] WARNING :: ext_module : WARNING MESSAGE
# e.g. [2017-12-13 16:00:50.983] ERROR :: ext_module : ERROR MESSAGE
# e.g. [2017-12-13 16:00:50.983] ERROR :: ext_module : EXCEPTION MESSAGE

[formatter_simpleFormatter]
format=[%(asctime)s.%(msecs)03d] %(levelname)s :: %(module)s : %(message)s
datefmt=%Y-%m-%d %H:%M:%S

python logging module - terminal, stdout, stderr

1 Answers