Azure data lake analytics empty output file

Question

I need help, can't see the issue of this problem: I am provisioning an Azure Data Lake Store with a Stream Analytics jobs. File are tab separated and the job is running without errors.

I deployed an Azure Data Lake Analytics service to aggregate data like this:

@input = 
EXTRACT [applicationname] string,
        [clientip] string,
        [continent] string,
        [country] string,
        [province] string,
        [city] string,
        [latitude] string,
        [longitude] string
FROM "adl://mydatalakesotre.azuredatalakestore.net/instrumentationoutput/mystore/2017-10-22/{*}"
USING Extractors.Text(delimiter: '\t', skipFirstNRows: 1, silent: true);

OUTPUT @input 
TO "output/PowerBI_output.tsv"
USING Outputters.Tsv(outputHeader: true);

I can't find the way to make it working... I Have other 5 MB of input data, but the output got only the headers, as specify in the query... What am I missing.

Thanks for the help.

Nelson, it would help to provide some more information. I assume the files are not empty and your path is correct - but can you check the job graph that is generated and see what the input file nodes in the graph show? Are any/multiple files getting picked up and processed? Do they show data flowing out of the extract stage? Also, you didn't mention any errors, but maybe you want to take out the "silent:true' so you can see any errors that happened when rows are being read by the Extractor. — OmidA
Can you provide some sample data please? Also, the point of the silent parameter is to ignore errors. Maybe if you switch that off you will get some error information? — wBob

Michael Rys Michael Rys · Accepted Answer · 2017-10-31T09:53:28

As Bob mentions in the comment, you are getting an empty result because you most likely have a misalignment in your schema definition and the actual files that you extract from.

I suggest that you open the file in the ADL Tools of Visual Studio and use the CREATE EXTRACT statement wizard to create you the EXTRACT statement. If you still get error messages (after removing silent:true), please update your question with the detected error message and we will give you an updated answer.

Azure data lake analytics empty output file

1 Answers