0
votes

I have a simple spark UDF that calls a Scala function. The Scala function currently uses 'Long' type as shown below

def someUDF(someList: List[String]) = udf { (inMap: Map[String, Long]) =>
    someFunc[Long](inMap, someList)
  }

Since spark doesn't support a Any type, is there a way to pass a generic type that could work for Long, Int or Double types without creating duplicate UDFs?

Thanks!

1

1 Answers

0
votes

I think you can define double function. Long and Int can be casted to double directly.