Connection to Azure data lake Gen 2 from azure function

Question

I'm trying to connect to Azure Data Lake Storage Gen2 from an Azure Function to import some XML files and convert them to JSON. But my code is not working:

var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;  
var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
var result = adlsFileSystemClient.FileSystem.Open(adlsAccountName, "/Test/xml.xml");

This returns an error: The remote name could not be resolved + "azuredatalakestore.net" while actually DNS suffix should have been different.

Answer 1

As of now, no SDK is supported for ADLS Gen2, but you can use ADLS Gen2 rest api instead, do some create / read / delete operation.

For example, you can write code like below with sas token authentication(or you can also use the shared key authentication):

            string sasToken = "?sv=2018-03-28&ss=b&srt=sco&sp=rwdl&st=2019-04-15T08%3A07%3A49Z&se=2019-04-16T08%3A07%3A49Z&sig=xxxx";
            string url = "https://xxxx.dfs.core.windows.net/myfilesys1/app.JPG" + sasToken;
            var req = (HttpWebRequest)WebRequest.CreateDefault(new Uri(url));

            //you can specify the Method as per your operation as per the api doc
            req.Method = "HEAD"; 
            var res = (HttpWebResponse)req.GetResponse();

            //your other code

Answer 2

Taken from Known issues with Azure Data Lake Storage Gen2

SDK support for Data Lake Storage Gen2 accounts
There aren’t SDKs available that will work with Data Lake Storage Gen2 accounts.

Answer 3

SDKs are available now: https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet

My example:

StorageSharedKeyCredential credential = new StorageSharedKeyCredential(_configuration["AccountName"], _configuration["AccountKey"]);

serviceClient = new DataLakeServiceClient(new Uri(_configuration["DFSURL"]), credential);

usage_container = serviceClient.GetFileSystemClient(_configuration["BlobContainer"]);


DataLakeDirectoryClient directoryClient = usage_container.GetDirectoryClient(country);
directoryClient.CreateFileAsync(xmlFileName, headers);