FileNotFoundException when running Python reducer

2
votes

I've implemented a U-SQL reducer in Python and it fails with a FileNotFoundException.

Unhandled exception from user code: "Could not find file 'D:\data\yarnnm\local\usercache\eba8b163-16c8-40b9-9723-43d521d4ab10\appcache\application_1480798662039_54518\container_e85_1480798662039_54518_01_000007\wd\42df9ef75185410d87f0cbf598f4bd55.output.0'."

Any ideas why?

1
pythonazure-data-lakeu-sql
The error is fairly straightforward, it can't locate the file it's being asked for. Have you checked to see if the file exists? - joebeeson

1 Answers

0
votes

We have encountered this as well. It seems to be a generic error that occurs when the user code throws an exception. My guess is that there is no output mechanism for errors in the user python code in ADLA and therefore this no output file found is thrown. Here is something we implemented to get the error back, or print statements that would normally be swallowed:

def usqlml_main(df):
try:
    stdout = sys.stdout
    sio = StringIO()
    sys.stdout = sio
    // script to run
    return pd.DataFrame({'h': [sio.getvalue()]})
except Exception as e:
    return pd.DataFrame({'h': [str(e)]})
";

Remember to change your produce to:

PRODUCE h string