unauthorized_client: Client is unauthorized to retrieve access tokens using this method

Question

I'm a GSuite Administrator with nearly 300 users and I'm migrating away from GSuite. I need to download all our user files they've created/uploaded.

I'm starting small with writing a Python script that will show me user files in a list and seems like I'm stuck with the overwhelming authorization issues.

1. I've created a project in Google Console, and created a Service Account with private key (json based) and GSuite Domain-wide delegation checkbox ticked
2. In my GSuite Admin panel I've added the newly created client ID and permission scope in Manage API access to these scopes: https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/admin.datatransfer,https://www.googleapis.com/auth/admin.directory.user, https://www.googleapis.com/auth/admin.directory.group

So good according to the documentation (https://developers.google.com/admin-sdk/directory/v1/guides/delegation)

3. I'm creating a resource object from the ServiceAccountCredentials and building an object based off API name/version "drive" and "v3" respectively and trying to get files list according to Google's quickstart (https://developers.google.com/drive/api/v3/quickstart/python):

    from googleapiclient.discovery import build
    from oauth2client.service_account import ServiceAccountCredentials
   
    SERVICE_ACCOUNT_EMAIL = "[email protected]"
    SERVICE_ACCOUNT_PKCS12 = "./service_key.json"
   
    def create_directory_service(user_email):
      credentials = ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT_PKCS12, scopes=['https://www.googleapis.com/auth/drive'])
      credentials = credentials.create_delegated(user_email)
   
      return build('drive', 'v3', credentials=credentials)
   
   
    resource = create_directory_service('[email protected]')
   
    results = resource.files().list(
      pageSize=10, fields="nextPageToken, files(id, name)"
    ).execute()
   
    items = results.get('files', [])
   
    print(items)
   

It looks totally correct, but I get this error message:

Connected to pydev debugger (build 181.5087.37)

Traceback (most recent call last): File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1664, in main() File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1658, in main globals = debugger.run(setup['file'], None, None, is_module) File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1068, in run pydev_imports.execfile(file, globals, locals) # execute the script File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/Users/probe/Projects/drive_getter/src/dconnect.py", line 16, in pageSize=10, fields="nextPageToken, files(id, name)" File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper return wrapped(*args, **kwargs) File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/googleapiclient/http.py", line 835, in execute method=str(self.method), body=self.body, headers=self.headers) File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/googleapiclient/http.py", line 162, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/oauth2client/transport.py", line 159, in new_request credentials._refresh(orig_request_method) File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/oauth2client/client.py", line 749, in _refresh self._do_refresh_request(http) File "/Users/probe/Projects/drive_getter/drive_getter/lib/python3.6/site-packages/oauth2client/client.py", line 819, in _do_refresh_request raise HttpAccessTokenRefreshError(error_msg, status=resp.status) oauth2client.client.HttpAccessTokenRefreshError: unauthorized_client: Client is unauthorized to retrieve access tokens using this method.

Any idea what was done wrong in the process? Again - My goal is to list and later download all user files from all GSuite users, so I was thinking of looping my user emails and applying the same logic to all of them until I get all the files downloaded.

Thanks for any cooperation!

Answer 1

Client is unauthorized to retrieve access tokens using this method.

There are sevral diffrent types of clients in Google Develoepr console.

• Browser - Used for web applications.
• Native - used for installed applications
• Service account - used for server to server communication
• Mobile

The code used to authenticate with these different types is understandably different.

The code you are using appears to be the correct code for a python with a service account. Although from_json_keyfile_name accepts a p12 file and not a .json file.

There for the client must be in corect. Go back to Google developer console create a service account client and download the correct .json file.

Service account python

from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = '/path/to/service.json'

credentials = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES)

I recommend using the .json file over the p12 file i suspect google will discontinue using these at some point in the future.

Answer 2

The solution was simple.

First of all, to what I could find, google.oauth2 library supports domain wide delegation functions only if you use P12 key and construct an object based on these credentials.

Second, please pay attention to the filename of p12 key you are downloading, I was renaming the key filename to something shorter and didn't pay attention that Google includes public key fingerprint in the filename itself.

So the solution is, all the steps described in my question post, create service account with p12 key, tick domain wide delegation, and the working code would look like this:

from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials

SERVICE_ACCOUNT_EMAIL = "[email protected]"
PK12_FILE = "./private-key-original-filename.p12"
PK12_PASSWORD = "notasecret"
SCOPES = ['https://www.googleapis.com/auth/drive']

def create_directory_service(user_email):
    credentials = ServiceAccountCredentials.from_p12_keyfile(
        SERVICE_ACCOUNT_EMAIL,
        PK12_FILE,
        PK12_PASSWORD,
        scopes=SCOPES
    )

    credentials = credentials.create_delegated(user_email)

    return build('drive', 'v3', credentials=credentials)


bryan = create_directory_service('[email protected]')

results = bryan.files().list(
    pageSize=10, fields="nextPageToken, files(id, name, mimeType, parents)"
).execute()

items = results.get('files', [])

print(items)

Bonus: If you'd ever want to download files from GDrive and preserve folder structure (Google doesn't tell you whether a file hash is a folder or not), use mimeType extracted and parent ID to build a stack of parents (folders), create them automatically while looping your list and then place the files within their respective parent ID.