Logging in multiple files in multiple folders independently - log4j JAVA

Question

I am working on a multithreading application in which i want to do logging at different locations using log4j.

Scenario:

Folder1 ---> log1.log
                    log2.log
                    log3.log

Folder2 ---> log1.log
                    log2.log
                    log3.log

I want my loggers to independently log into files.

Till now, i have created 3 RollingFileAppenders which corresponds to 3 loggers for Folder1 but the problem is "I will have around 100's of folders and i don't want to write Appenders and loggers for them in my log4j.xml file".

<appender name="logfile1" class="org.apache.log4j.RollingFileAppender">
    <param name="File" value="Folder1/log1.log" />
    <param name="Append" value="true" />
    <param name="MaxFileSize" value="10MB" />
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%m%n" />
    </layout>
</appender>

<appender name="logfile2" class="org.apache.log4j.RollingFileAppender">
    <param name="File" value="Folder1/log2.log" />
    <param name="Append" value="true" />
    <param name="MaxFileSize" value="10MB" />
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%m%n" />
    </layout>
</appender>

<appender name="logfile3" class="org.apache.log4j.RollingFileAppender">
    <param name="File" value="Folder1/log3.log" />
    <param name="Append" value="true" />
    <param name="MaxFileSize" value="10MB" />
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%m%n" />
    </layout>
</appender>

<logger name="Logger1" additivity="false">
    <level value="INFO" />
    <appender-ref ref="logfile1" />
</logger>

<logger name="Logger2" additivity="false">
    <level value="INFO" />
    <appender-ref ref="logfile2" />
</logger>

<logger name="Logger3" additivity="false">
    <level value="INFO" />
    <appender-ref ref="logfile3" />
</logger>

The folders will be created at runtime

MY APPROACH:
I want to instantiate my loggers according to my folder names and keep them in a hashmap like

HashMap<FolderName, HashMap<LoggerType, LoggerObject>>

FolderName: String 
LoggerType: it will be an Enum of log1, log2 and log3 
LoggerObject: It will a log4j Logger object

So, i will provide one method to get the LoggerObject like

public Logger getLoggerObject(String folderName, LoggerType loggerType){
     \\logic
     return loggerObject;
}

Logging.getLoggerObject("Folder1", LoggerType.log1).info("I am writing to log1.log file in Folder1");

Questions:
1. I am not able to configure my LoggerObjects at runtime. Help me in doing that!
2. How to configure my loggers for multiple folder locations and multiple logging files?

D.B. D.B. · Accepted Answer · 2018-07-28T15:15:14

Since you said you're open to switching to log4j2 I would suggest you do that and then use the RoutingAppender similar to the example in the log4j2 FAQ regarding dynamically writing to separate log files.

Here is some sample code to illustrate how you could change the folder and log file on the fly using the ThreadContext:

First we create 2 classes that implement Runnable:

package runners;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

public class Runner1 implements Runnable{
    private static final Logger log = LogManager.getLogger();

    public void run() {
        //Set up the context before getting logger
        ThreadContext.put("logFolder", "Folder1");
        ThreadContext.put("logFileName", "log1");

        //Generate some logs
        log.info("here's the first thread");

        //Wait a while so that threads interleave
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        //Generate more logs
        log.debug("some debug in first thread");
        log.info("finishing first thread");

    }
}

and

package runners;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

public class Runner2 implements Runnable{
    private static final Logger log = LogManager.getLogger();

    public void run() {
        //Set up the context before getting logger
        ThreadContext.put("logFolder", "Folder2");
        ThreadContext.put("logFileName", "log2");

        //Generate some logs
        log.info("here's the second thread");
        log.debug("some debug in second thread");

    }
}

Now a basic controller class to create and start 2 threads:

package runners;

public class Controller {

    public static void main(String[] args) {
        Thread t1 = new Thread(new Runner1());
        Thread t2 = new Thread(new Runner2());
        t1.start();
        t2.start();
    }

}

Lastly we need to configure log4j2:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <Routing name="MyRoutingAppender">
            <Routes pattern="$${ctx:logFolder}-$${ctx:logFileName}">
                <Route>
                    <File fileName="logs/${ctx:logFolder}/${ctx:logFileName}.log"
                        name="appender-${ctx:logFolder}-${ctx:logFileName}">
                        <PatternLayout>
                            <Pattern>[%date{ISO8601}][%-5level][%t] %m%n</Pattern>
                        </PatternLayout>
                    </File>
                </Route>
            </Routes>
        </Routing>
        <Console name="STDOUT" target="SYSTEM_OUT">
            <PatternLayout pattern="[%date{ISO8601}][%-5level][%t] %m%n" />
        </Console>

    </Appenders>
    <Loggers>
        <Logger name="runners" level="TRACE" additivity="false">
            <AppenderRef ref="STDOUT" />
            <AppenderRef ref="MyRoutingAppender" />
        </Logger>
        <Root level="WARN">
            <AppenderRef ref="STDOUT" />
        </Root>
    </Loggers>
</Configuration>

After you run the Controller you will see that two folders are generated with one file in each:

The logs from the first thread are in folder1 > log1 and logs from the second thread are in folder2 > log2

Note that I used a basic file appender but you can swap in a different appender within the routing appender to fit your needs.

Hope this helps to get you started.

Logging in multiple files in multiple folders independently - log4j JAVA

1 Answers