How to create UDF in Hive

0
votes

Hi I am new to hive i am trying to create UDF in hive called ConvertDateFormetUDF. While creating temporary function i am getting the following error please some help me.Can someone steps to be followedd? Please:

Hive> Add jar /home/cloudera/date.jar 
Added /home/cloudera/date.jar to class path
Added resource: /home/cloudera/date.jar
hive> CREATE TEMPORARY FUNCTION fun as 'com.db.acedq.opal.hive.ConvertDateFormatUDF';
java.lang.NoClassDefFoundError: org/apache/pig/EvalFunc
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)  
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.hive.ql.exec.FunctionTask.getUdfClass(FunctionTask.java:105)
at org.apache.hadoop.hive.ql.exec.FunctionTask.createFunction(FunctionTask.java:75)
at org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:63)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1374)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1160)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:973)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:893)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: org.apache.pig.EvalFunc
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 32 more
FAILED: Execution Error, return code -101 from    org.apache.hadoop.hive.ql.exec.FunctionTask
1
hadoophive
can you share your UDF code ? - Mukesh S
@MukeshS here it is my code package com.db.acedq.opal.hive; import java.io.IOException; import java.text.SimpleDateFormat; import org.apache.pig.EvalFunc; import org.apache.pig.data.Tuple; import org.mortbay.jetty.HttpParser.Input; public class ConvertDateFormatUDF extends EvalFunc<String>{ String empty = ""; public String exec(Tuple input) throws IOException { if ((input == null) || input.size() == 0 || (input.get(0)== null) || input.get(1) == null){ return null;} String field = (String)input.get(0).toString(); String ToFormat = (String) input.get(1).toString(); - Basu
String FromFormat = null; if (input.size()>2 && input.get(2)!= null){ FromFormat = (String) input.get(2).toString();} String val = ConvertDateFormatModule.convert(field,ToFormat,FromFormat); return val;} } - Basu
package com.db.acedq.opal.hive; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; public class ConvertDateFormatModule { public static String convert(String field, String toFormat, String fromFormat) { if (field == null || toFormat == null || field.toString().length()==0 || toFormat.toString().length() == 0) { return ""; } - Basu
else { SimpleDateFormat dateformat = new SimpleDateFormat( fromFormat == null ? "dd/mm/yyy":fromFormat); SimpleDateFormat TodateFormat = new SimpleDateFormat(toFormat); dateformat.setLenient(false); try { Date x = dateformat.parse(field); return TodateFormat.format(x); } catch (ParseException e) { String empty=""; return empty=""; } } } } - Basu

1 Answers

1
votes

Seems Like the java code of UDF you are creating is referring PIG libraries. While creating Hive UDF you should include hive library. I have mentioned whole Procedure in following link http://lets-do-something-big.blogspot.in/2014/02/hive-udf-example.html . Please let me know if you still face problem