How to use different formatters with the same logging handler in python

Question

Is it possible to log to a single destination (i.e. using one FileHandler) with multiple loggers (i.e. logging.getLogger("base.foo") and logging.getLogger("base.bar")), and use different formatters for each of the loggers.

To my understanding it's only possible to assign one formatter to each handle. Maybe it's possible to associate the formatter with a logger rather than the handler?

Answer 1

It's easy to dispatch to different formatters based on record.name. Below is prove-of-concept sample code:

import logging


class DispatchingFormatter:

    def __init__(self, formatters, default_formatter):
        self._formatters = formatters
        self._default_formatter = default_formatter

    def format(self, record):
        formatter = self._formatters.get(record.name, self._default_formatter)
        return formatter.format(record)


handler = logging.StreamHandler()
handler.setFormatter(DispatchingFormatter({
        'base.foo': logging.Formatter('FOO: %(message)s'),
        'base.bar': logging.Formatter('BAR: %(message)s'),
    },
    logging.Formatter('%(message)s'),
))
logging.getLogger().addHandler(handler)

logging.getLogger('base.foo').error('Log from foo')
logging.getLogger('base.bar').error('Log from bar')
logging.getLogger('base.baz').error('Log from baz')

Another way is to open file manually and create two stream handlers from it with different formatters.

Answer 2

Little fix to excellent Denis's solution.

Logging name system based on hierarchical structure:

The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo.

For example, when you setLevel() for some logger, this level will be also applied to child loggers. That's why you might want your formatter will be used for logger and it's child loggers too. For example, 'one.two' formatter should also be applied to 'one.two.three' logger (if no formatter for 'one.two.three' set). Here's version of DispatchingFormatter that do job (Python 3 code):

class DispatchingFormatter:
    """Dispatch formatter for logger and it's sub logger."""
    def __init__(self, formatters, default_formatter):
        self._formatters = formatters
        self._default_formatter = default_formatter

    def format(self, record):
        # Search from record's logger up to it's parents:
        logger = logging.getLogger(record.name)
        while logger:
            # Check if suitable formatter for current logger exists:
            if logger.name in self._formatters:
                formatter = self._formatters[logger.name]
                break
            else:
                logger = logger.parent
        else:
            # If no formatter found, just use default:
            formatter = self._default_formatter
        return formatter.format(record)

Example:

handler = logging.StreamHandler()
handler.setFormatter(DispatchingFormatter({
        'one': logging.Formatter('%(message)s -> one'),
        'one.two': logging.Formatter('%(message)s -> one.two'),
    },
    logging.Formatter('%(message)s -> <default>'),
))
logging.getLogger().addHandler(handler)

print('Logger used -> formatter used:')
logging.getLogger('one').error('one')
logging.getLogger('one.two').error('one.two')
logging.getLogger('one.two.three').error('one.two.three')  # parent formatter 'one.two' will be used here
logging.getLogger('other').error('other')

# OUTPUT:
# Logger used -> formatter used:
# one -> one
# one.two -> one.two
# one.two.three -> one.two
# other -> <default>

Answer 3

This work for you ? Different log level output formats and different log destinations, file vs stdout (and different levels for each destination):

    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)

    format_for_stdout = logging.Formatter('%(message)s')
    format_for_logfile = logging.Formatter('%(asctime)s: %(name)s: %(levelname)s: %(message)s')

    handler_logfile = logging.FileHandler('my_awesome_logs.log')
    handler_logfile.setLevel(logging.DEBUG)
    handler_logfile.setFormatter(format_for_logfile)

    handler_stdout = logging.StreamHandler()
    handler_stdout.setLevel(logging.INFO)
    handler_stdout.setFormatter(format_for_stdout)

    logger.addHandler(handler_logfile)
    logger.addHandler(handler_stdout)

    logging.addLevelName(logging.INFO, logging.getLevelName(logging.INFO))
    logging.addLevelName(logging.ERROR, logging.getLevelName(logging.ERROR))