0
votes

I'm sorry if the title of the question is not really significant, but I haven't found a better way to describe this problem.

I'm writing some Kotlin extension methods for Hadoop, and yesterday I encounted a strange error that I don't understand. I wrote a couple of extension methods like this:

inline fun <reified T : InputFormat<*, *>, reified K : Mapper<*, *, *, *>> Job.addMultipleInputPath(path: Path) {
  MultipleInputs.addInputPath(this, path, T::class.java, K::class.java)
}

inline fun <reified T : OutputFormat<*, *>, reified Key : Any, reified Value : Any> Job.addMultipleNamedOutput(namedOutput: String) {
  MultipleOutputs.addNamedOutput(this, namedOutput, T::class.java, Key::class.java, Value::class.java)
}

inline fun <reified T : Mapper<*, *, KeyClass, ValueClass>, reified KeyClass : Any, reified ValueClass : Any> Job.setMapper() {
  this.setMapperClass<T>()
  this.mapOutput<KeyClass, ValueClass>()
}

An if I try to call them in my Driver class, (1) and (3) work but (2) gives me a compile error:

with(Job.getInstance()) {
  // works
  addMultipleInputPath<TextInputFormat, SensorsMapperType2>(secInput)

  // 2 type arguments expected for class TextOutputFormat<K: Any!, V: Any!> : FileOutputFormat<K!, V!> 
  // defined in org.apache.hadoop.mapreduce.lib.output
  addMultipleNamedOutput<TextOutputFormat, Text, SensorData>(HIGH_TEMP)

  // works
  setMapper<NGramMapper, Text, IntWritable>()
}

If I correct the call using <TextOutputFormat<Any, Any>> everything works fine, but I don't understand why this happens. Is there something that I'm missing on how reified and generics work in Kotlin?

1

1 Answers

2
votes

It's all about raw types, which kotlin doesn't allow.

This format works:

addMultipleNamedOutput<TextOutputFormat<Text, SensorData>, Text, SensorData>(HIGH_TEMP) //(1)

These two don't:

addMultipleNamedOutput<TextOutputFormat, Text, SensorData>(HIGH_TEMP) //(2)

So things get clear now. TextOutputFormat is illegal because Kotlin doesn't allow raw types. You have to specify the type arguments, which is missing in (2). Adding them result in format (1).

This looks a bit verbose, but there is currently no workaround (at least IMHO).