Install Azure U-SQL Extensions to run R/Python scripts locally?

1
votes

We can extend U-SQL scripts with R/Python code in Azure Data Lake Analytics, but how can we do it locally?

2
azure-data-lakeu-sql

2 Answers

1
votes

  1. Install U-SQL Advanced Analytics extensions in your Data Lake Analytics Account
    1.1 Launch your Azure Portal
    1.2 Navigate to your Data Lake Analytics Account
    1.3 Click Sample Scripts
    1.4 Click More and select Install U-SQL Extensions
    1.5 Wait until the extensions have finished installing (2GB)
    1.6 Have you waited? Then go to your Data Lake Analytics Account
    1.7 Navigate to your default Data Lake Store Account
    1.8 Click on Data Explorer and verify that a folder /usqlext exists

  2. Get your USQL Local Run path
    2.1 Launch your Visual Studio
    2.2 Select Tools > Options > Azure Data Lake > General
    2.3 Under U-SQL Local Run, find and copy the value for DataRoot
    2.4 The value will look like this: C:\Users\username\AppData\Local\USQLDataRoot

  3. Copy U-SQL Advanced Analytics extensions from Azure to your localhost
    3.1 Use a powershell script or ... go to the next line
    3.2 Launch Microsoft Azure Storage Explorer (great tool, install it)
    3.3 Locate your default Data Lake Store, the one of your Data Lake Analytics Account
    3.4 Open data explorer and Download the folder /usqlext to your USQL Local Run's path
    3.5 The full path should look like this: C:\Users\username\AppData\Local\USQLDataRoot\usqlext

  4. Final step, register all Azure U-SQL Extensions under U-SQL Local Run
    4.1 Launch your Visual Studio
    4.2 Start a new U-SQL project
    4.3 Open the file C:\Users\username\AppData\Local\USQLDataRoot\usqlext\RegisterAll.usql
    4.4 Copy the text into your own U-SQL script
    4.5 Run it in Local Execution mode
    4.6 Once the script finishes...

You will be able to use all the U-SQL Advanced Analytics features (Python, R, Cognitive) on your own machine and explore all the samples in \usqlext\samples!

Have a nive day!

0
votes

This answer does not apply directly to the OP, but this article is the closest match to the issue that I had. Knowing others may find this article while searching for the solution, I am recording my answer to this thread.

Problem: In the Azure Portal (not locally), if you choose "Install Azure U-SQL Extensions", the job eventually fails with a non-descript error.

What is happening behind the scenes is that all of the files are copying into storage, but the assemblies fail to register. We have to create a job manually to register the assemblies.

Answer:

  1. Open Data Explorer
  2. Navigate to /usqlext
  3. Download the file "RegisterAll.usql". Open in Notepad, and copy the text
  4. Create a new Job. Paste in the text.
  5. Execute the job.

Assemblies will be registered (verify by checking Catalog->master->Assemblies). You can now run the Cognition and Python samples.