is there a way to delete a file after some operations done in u-sql script?

Question

I want to delete some files in the Azure Data Lake Store after some operations(Extraction Method) done using the U-SQL script. Is there any way to delete files using functions or any other way in U-SQL?

I know that U-SQL can be used to only read the files but I want to delete some files. We can do the same using .NET SDK but I want to delete right after the U-SQL completes.

I dont think this supported yet, there is a suggestion item for ADLS which requests exactly what you asked for raise almost few months back. feedback.azure.com/forums/327234-data-lake/suggestions/… — frictionlesspulley

candidson candidson · Accepted Answer · 2017-10-09T16:21:43

Deleting files directly from U-SQL is not supported, at least not that I know of.

What about using Azure Data Factory. You could use Data Factory to orchestrate the U-SQL Jobs (https://docs.microsoft.com/en-us/azure/data-factory/v1/data-factory-usql-activity) and in the end (after the last Data Lake Analytics activity), you could use a Custom C# Activity (https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-dotnet-custom-activity) to delete the respective files.

is there a way to delete a file after some operations done in u-sql script?

1 Answers