Download an entire public folder from Google-Drive using Python or wget/curl without authentication

Question

I would like to download an entire public folder from Google-drive from script (python, wget, terminal, etc.).
The procedure shall be accomplished without authentication, as it's a public folder which is accessible for anyone who has the link.

Link example: https://drive.google.com/drive/folders/1Gt-W8jMrADizXYGF5QZwJh_Gc8QpKflX

In case that it's not possible to directly download the entire folder, then it would be sufficient to just being able to list its content (files), without authentication, and then I'll be able to download each file separately. How to obtain such a listing feature?

Note:
I found many similar discussions, but all were assuming either file-download or authentication, and I couldn't find a match for that specific ask, e.g.:

In your situation, for example, is this CLI tool useful? github.com/tanaikech/goodls But, when the files in the public folder, at least, it is required to use an API key because the file list is required to be retrieved from the public folder. If this was not useful, I apologize. — Tanaike
Hi Tanaike, it surely seems relevant and I wasn’t aware of that CLI. I will give it a try, thank you! — Shahar Gino
Thank you for replying. If that was not useful for your situation, I apologize. And, if you cannot use the API key, for example, how about creating the file list from the public folder by manual operation or Google Apps Script? By this, you can download the files without the API key. — Tanaike
That's an excellent idea, thanks Tanaike. I also found this Google Apps Script, with your inspiration, thanks --> gist.github.com/mesgarpour/07317e81e9ee2b3f1699 — Shahar Gino
Thank you for replying. If your issue was resolved, can you post it as an answer? By this, it will be useful for other users who have the same issue. — Tanaike

Shahar Gino Shahar Gino · Accepted Answer · 2021-07-06T05:42:13

I've ended up applying the following code (tested, works well):

import urllib.request
from getfilelistpy import getfilelist
from os import path, makedirs, remove, rename

def download_googledrive_folder(remote_folder, local_dir, gdrive_api_key, debug_en):

    success = True

    if debug_en:
        print('[DEBUG] Downloading: %s --> %s' % (remote_folder, local_dir))
    else:
        try:
            resource = {
                "api_key": gdrive_api_key,
                "id": remote_folder.split('/')[-1].split('?')[0],
                "fields": "files(name,id)",
            }
            res = getfilelist.GetFileList(resource)
            print('Found #%d files' % res['totalNumberOfFiles'])
            destination = local_dir
            if not path.exists(destination):
                makedirs(destination)
            for file_dict in res['fileList'][0]['files']:
                print('Downloading %s' % file_dict['name'])
                if gdrive_api_key:
                    source = "https://www.googleapis.com/drive/v3/files/%s?alt=media&key=%s" % (file_dict['id'], gdrive_api_key)
                else:
                    source = "https://drive.google.com/uc?id=%s&export=download" % file_dict['id']  # only works for small files (<100MB)
                destination_file = path.join(destination, file_dict['name'])
                urllib.request.urlretrieve(source, destination_file)

        except Exception as err:
            print(err)
            success = False

    return success

I could not find a way to achieve my original goal, i.e. downloading public folder from Google-Drive without any credentials/keys, but the above code is a good compromise for me, as it only requires a key and not full credentials.

Note that there are 2 options here --> with or without providing the Google API Key (gdrive_api_key) for the source URL.
From my experience, the option without the API key works well for small files (< ~100MB), while the option with the API key appear to be more robust and works for any size.

Download an entire public folder from Google-Drive using Python or wget/curl without authentication

1 Answers