Error 403 User not authorized when trying to access Azure Databricks API through Active Directory

Question

I have been following the document: https://docs.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

to create a service principal and use it to access Databricks. I already have a databricks workspace configured and have used it to create a cluster. Then I've followed the process mentioned in the document, created a service principal and obtained the two tokens: AD Access token and management access token. However, I am unable to use the API.

The final call cURL command after configuration:

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
-H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \
https://<databricks-instance>/api/2.0/clusters/list

With the tokens and the other info substitutued provides the following result:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 User not authorized.</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /api/2.0/clusters/list. Reason:
<pre>    User not authorized.</pre></p>
</body>
</html>

I have tried a couple of things to resolve this, namely changing the original access token request to use the interactive flow using the authorization_code grant type and so on, but that just gives me an Invalid access token error.

Is there something wrong with the above configuration? Am I missing some permissions?

Update: Came to the Access control (IAM) page and the app does not seem to be included there.

Are you either a Contributor or Owner role on the Databricks workspace resource in Azure ? — Axel R.
The service principal must be added to the workspace either as part of the admin user login or using the Add service principal endpoint. — Carl Zhao
@Axel I am a contributor but despite that I was not able to provide access. Have requested access from my admin separately. — Mkl Rjv
If my answer is helpful for you, you can accept it as answer( click on the check mark beside the answer to toggle it from greyed out to filled in.). See meta.stackexchange.com/questions/5234/… This can be beneficial to other community members. Thank you. — Carl Zhao

Carl Zhao Carl Zhao · Accepted Answer · 2020-09-03T04:09:14

You must grant a role to the service principal.

This is the result of my test that I did not grant a role to the service principal. The error is the same as yours:

Next, grant roles to the service principal according to the following process:

Azure Portal>Azure Databricks>Azure Databricks Service>Access control (IAM)>Add a role assignment>Select the role you want to grant and find your service principal>save

Finally, use the service principal to get the token.(Don’t forget to grant permissions to service principals and grant administrator consent)

Get an Azure Active Directory access token: Get the Azure Management Resource endpoint token: Use the management endpoint access token to access the Databricks REST API:

Error 403 User not authorized when trying to access Azure Databricks API through Active Directory

1 Answers