Authorizing Data Lake linked services through Visual Studio Data Factory Project

4
votes

I have an Azure Data Factory Visual Studio Project in which I am using Azure Data Lake linked services. When I create them, I have to authorize them initially. But the given authorization expires after a certain time period, which is in days.

I cannot find an option to re-authorize the linked services. I tried re-authorizing another linked service from the portal and using that authorization for the linked service created in VS, but that wouldn't work.

I had to delete and re-create the linked services for fresh authorization.

Is this feature missing or is there another way to get a fresh authorization for Data Lake linked services in VS?

2
azure-data-factoryazure-data-lake
I know you can "reauthorize" within the portal. I have not seen a way to do that from VS though. In the portal just open the ADL linked service to edit and it should show the authorize button again at the top and you can click it and re enter your credentials. This is how I have been doing it. – JustLogic
Yes, that we can do. But that doesn't work for a linked service created through VS. I am assuming the the authorization is linked with the name of the linked service as well. – Tayyab Anwar

2 Answers

0
votes

Currently, there is no explicit re-authorize in Visual Studio. You will have to create a new linked service in VS to get the authorization code and you can use that in the existing linked service. Using the Azure portal, you can click the 'Authorize' button again to get new authorization codes and then click 'Deploy'

0
votes

You can also use a service principle to what you need, not sure if this was supported back in 2016.

The only caveat is that you need Azure Active Directory (Azure AD).

 "typeProperties": {
    //"authorization": "urn:ietf:wg:oauth:2.0:XXX",
    //"sessionId": "XXX",
    "dataLakeStoreUri": "https://XXX.azuredatalakestore.net/webhdfs/v1",
    "accountName": "ACCOUNT_NAME_XXX",
    "subscriptionId": "SUBSCRIPTION_GUID",
    "resourceGroupName": "RESOURCE_GROUP_NAME",
    "servicePrincipalId": "APPLICATION_CLIENT_ID",
    "servicePrincipalKey": "APPLICATION_SECRET_KEY",
    "tenant": "XXX.onmicrosoft.com"
  }

In Azure AD, you can create an application and then add access to azure data lake for the application. use this application's client id / client secret in the APPLICATION_CLIENT_ID, APPLICATION_SECRET_KEY instead of authorization and sessionId which expires.

You can then control the expiry of client ID and client secret from the Azure AD center of azure portal.

Hope this helps.

update : this is a good starting point for registering applications in Azure AD https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-integrating-applications