word.set() method throws null pointer exception in map reduce hadoop

Question

I am new to map reduce programming and started my lessons with simple word count example. However, I am trying a different approach with it. I have two input files on my hdfs input folder. And I am trying to generate output like

anyword1 --> filename1     2
anyword2 --> filename2     3

I wrote a mapper class to concatenate the word and filename together at Key but when I set the key value in a Text, it throws null pointer exception. Can someone help and advise where I am doing wrong.

My mapper class

public static class TokenizerMapper 
       extends Mapper<Object, Text, Text,IntWritable>{

    private final static IntWritable one = new IntWritable(1);
    private Text word = null;
    private String fileText = null;

    public void map(Object key, Text value, Context context
                    ) throws IOException, InterruptedException {
      StringTokenizer itr = new StringTokenizer(value.toString());
      String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();
      String modifiedWord ="";
      fileName = "-->"+fileName;
      System.out.println("filename before word-->"+fileName);
      while (itr.hasMoreTokens()) {
        modifiedWord = itr.nextToken().toString();//+fileName;
        modifiedWord = modifiedWord + fileName;
        System.out.println("modified word-->"+modifiedWord);
        word.set(modifiedWord);
        context.write(word, one);
        System.out.println("Mapper context-->"+word);
      }
    }
  }

------ Exeption ----

[root@LinuxCentos7 hadoop]# hadoop jar /usr/local/mapreduceexample/WordCountEx3.jar /user/Siddharth/Input /user/Siddharth/output
17/06/09 23:32:29 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/06/09 23:32:32 INFO input.FileInputFormat: Total input paths to process : 2
17/06/09 23:32:32 INFO mapreduce.JobSubmitter: number of splits:2
17/06/09 23:32:32 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1497025644387_0011
17/06/09 23:32:33 INFO impl.YarnClientImpl: Submitted application application_1497025644387_0011
17/06/09 23:32:33 INFO mapreduce.Job: The url to track the job: http://LinuxCentos7:8088/proxy/application_1497025644387_0011/
17/06/09 23:32:33 INFO mapreduce.Job: Running job: job_1497025644387_0011
17/06/09 23:32:52 INFO mapreduce.Job: Job job_1497025644387_0011 running in uber mode : false
17/06/09 23:32:52 INFO mapreduce.Job:  map 0% reduce 0%
17/06/09 23:33:16 INFO mapreduce.Job:  map 100% reduce 0%
17/06/09 23:33:16 INFO mapreduce.Job: Task Id : attempt_1497025644387_0011_m_000000_0, Status : FAILED
Error: java.lang.NullPointerException
    at com.hadoop.WordCountEx3$TokenizerMapper.map(WordCountEx3.java:56)
    at com.hadoop.WordCountEx3$TokenizerMapper.map(WordCountEx3.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Jeffrey Chung Jeffrey Chung · Accepted Answer · 2017-06-09T18:42:39

Initialize the word variable with a Text instance:

private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
private String fileText = null;

public void map(Object key, Text value, Context context)
        throws IOException, InterruptedException {
    ...
}

word.set() method throws null pointer exception in map reduce hadoop

1 Answers