Pig script cannot register UDF

1
votes

I have a simple Pig script that uses a Python UDF I have created. The script completes fine if I remove the UDF portion. But when I try to register my UDF I get the following error:

ERROR 2997: Encountered IOException. File pig_test/py_udf_substr.py does not exist

This is my UDF:

@outputSchema("chararray")
def get_fistsn(data,n):
    return data[:n]

This is my Pig script:

REGISTER 'pig_test/py_udf_substr.py' USING jython as pyudf;

A = load 'pig_test/sf.txt' using PigStorage(',')
as (Unique_flight_ID,Year,Month,Day,DOW,
Scheduled_departure_time,Scheduled_arrival_time,
Airline,Flight_number,Tail_number,Plane_model,
Seat_configuration,Departure_delay,Origin_airport,
Destination_airport,Distance_travelled,Taxi_time_in,
Taxi_time_out,Cancelled,Cancellation_code,target);

B = FOREACH A GENERATE Unique_flight_ID, pyudf.get_fistsn($0,3);

DUMP B;

I'm using HUE to run Pig. Both the data and the UDF are in the same HDFS location (pig_test).

2
pythonhadoopapache-pighueudf

2 Answers

1
votes

Based on the error you get, the issue is not with the script. Its IOException- The framework is unable to read the UDF. You can try giving the complete path of the UDF and see if works. Using a new terminal try opening the file using cat command or so and see if the path is correct or not.

ERROR 2997: Encountered IOException. File pig_test/py_udf_substr.py does not exist
0
votes

In the Editor, could you click on Properties, then Resources and select py_udf_substr.py as a file?

Then in your script, just do:

REGISTER 'py_udf_substr.py' USING jython as pyudf;