How to control log-message level for user Jobs/Tasks in Hadoop 1.2.1

1
votes

I need to print trace and debug messages from my hadoop mapreduce job, while keeping hadoop system messages at the default level (INFO by default).

I tried the following:

In the Job methods added logger message: 

package org.example.mapreducejob

public void reduce(Text extId, Iterable<MyData>> myDataItems, Context context)
        throws IOException, InterruptedException {
    log.debug("reduce(): i want this message to be printed in the userlogs");
    log.info("reduce(): Info message");
    ...
}

Then in $HADOOP_HOME/conf/log4.properties I added the line: log4j.logger.com.terapeak.etl=DEBUG

and restarted the node

However only "Info message' appears in user logs. Where should I change logger configuration to enable DEBUG messages for user jobs?

2
hadoop

2 Answers

2
votes

I found the answer. In the $HADOOP_HOME/conf there are two config files for log4.properties: 

log4j.properties
task-log4j.properties

To change log level for user jobs you need to make changes in task-log4j.properties

0
votes

Please try 

log4j.logger.org.apache.hadoop = DEBUG