Different context types in hadoop for mapper and combiner

Question

Hello I'm trying to implement java hadoop application. I want to make mapper of <Object, Text, NaicsAreaPair, LongWritable> (so the output of mapper will be NaicsAreaPair as key and LongWritable as value). Then I need Combiner to be like <NaicsAreaPair,LongWritable, Text,AreaWagePair> so the input is correct with mapper output, but combiner output is different from mapper output.

I have in main class this configuration:

public static void main(String[] args) throws Exception {
 Configuration conf = new Configuration();
 Job job = Job.getInstance(conf, "NY statistics");
 job.setJarByClass(NYStatisticsOwnWritableComparable.class);
 job.setMapperClass(TokenizerMapper.class);
 job.setCombinerClass(Combiner.class);
 job.setReducerClass(IntSumReducer.class);
 job.setOutputKeyClass(NaicsAreaPair.class);
 job.setOutputValueClass(LongWritable.class);
 //job.setPartitionerClass(Rozdelovac.class);
 FileInputFormat.addInputPath(job, new Path(args[0]));
 FileOutputFormat.setOutputPath(job, new Path(args[1]));
 //job.setNumReduceTasks(3);
 System.exit(job.waitForCompletion(true) ? 0 : 1);
}

Here I have to say which output key and output value will be used. Is there any possibility to set it like ok for mapper use this output key and value but for combiner use different?

Thanks a lot for your answer

Pradeep Gollakota Pradeep Gollakota · Accepted Answer · 2015-05-09T19:49:27

1

votes

It is not. The Combiner output MUST be the same as Mapper output.

Different context types in hadoop for mapper and combiner

2 Answers