5
votes

Hi I am stuck here with this situation, I have tried a number of different approaches but I cannot get what I am after.

Situation: I have a large number of files in folders hosted on google drive. For instance a single client folder for reports has between 2000-10000 files(this will only get bigger), I want to be able to list these files with there title and file id and parent folder id so I can create direct links to the files. From my understanding I am able to do this with google drive: drive-api-client-php I have setup a project and have a Client ID and Client secret, I have downloaded subversion composer and google-api and drive-api-client-php and use XAMPP

What I have done so far: I have been through googles documentation and watch tutorial video they offer. I have tried the google sample they have available on there web page and it is the beginnings of what I need. This will list the files in the folder with the ID and Title or what ever fields you choose - two issues here is that you cannot specify the folder you want the results of and the max number of results is limited to 1000. I'm in need of 10,000.

I have also looked at some google apps script here, which seems to be used like vba in excel to pull file ids and titles etc, however this is also limited to max number of results and then time out issues. This is the exact type of result I want just with out the limitations.

My Question is: What are my options on listing 10,000 file ID's with Titles in a specified folder of google drive and what is the process from start to finish as I feel like I might be missing a fundamental point early on which is effecting my result possibilities. I am flexible on this and I can work with the data after it is extracted just getting the data is key for me first. look forward to your reply's, many thanks. I hope this is clear enough and makes sense.

2

2 Answers

2
votes

Try this code in Apps Script: You need to edit this code, and replace your folder ID. Open a folder in Google Drive, and in the browser address bar, you see:

https://drive.google.com/drive/folders/Your_Folder_ID_Here

Copy out the folder ID, and replace it in the code below. The Advanced Google Services must be explicitly enabled in two places, inside Apps Script and on your Developer Console.

//This requires the Drive API To be turned on in the Advanced Google Services
function listFilesInFolder() {
  var query = 'trashed = false and ' +
      "'Your Folder ID Here' in parents";

  var filesInFolder, pageToken;

  do {
    filesInFolder = Drive.Files.list({
      q: query,
      maxResults: 100,
      pageToken: pageToken
    });
    if (filesInFolder.items && filesInFolder.items.length > 0) {
      for (var i = 0; i < filesInFolder.items.length; i++) {
        var thisFile = filesInFolder.items[i];
        //Logger.log('%s (ID: %s)', thisFile.title, thisFile.id);
        //To Do - Output content to file
        . . . . Code to output content . . . . 
      }
    } else {
      Logger.log('No files found.');
    }
    pageToken = filesInFolder.nextPageToken;
  } while (pageToken);
}

If you're looking for code written in PHP, you'll need to use something different.

2
votes

This is what I came up with, thanks to the help of the above code and some other code I found, put the two together and it does what I need. It may be useful for some one else.

Thanks

function listFilesInFolder() {
var MAX_FILES = 2000; //use a safe value, don't be greedy
var id = 'FOLDER_ID_HERE'; 
var scriptProperties = PropertiesService.getScriptProperties();
var lastExecution = scriptProperties.getProperty('LAST_EXECUTION');
var sheet = SpreadsheetApp.getActiveSheet();
var data;
if( lastExecution === null )
lastExecution = '';

var continuationToken = scriptProperties.getProperty('IMPORT_ALL_FILES_CONTINUATION_TOKEN');
  var iterator = continuationToken == null ?
  DriveApp.getFolderById(id).getFiles() : DriveApp.continueFileIterator(continuationToken);


 try { 
   for( var i = 0; i < MAX_FILES && iterator.hasNext(); ++i ) {
   var file = iterator.next();
   var dateCreated = formatDate(file.getDateCreated());
    if(dateCreated > lastExecution)
     processFile(file);
    data = [
       i,
       file.getName(),
       file.getId()
      ];        

    sheet.appendRow(data);
}
} catch(err) {
 Logger.log(err);
}

if( iterator.hasNext() ) {
 scriptProperties.setProperty('IMPORT_ALL_FILES_CONTINUATION_TOKEN', iterator.getContinuationToken());
 } else { // Finished processing files so delete continuation token
 scriptProperties.deleteProperty('IMPORT_ALL_FILES_CONTINUATION_TOKEN');
 scriptProperties.setProperty('LAST_EXECUTION', formatDate(new Date()));
 }
 }

function formatDate(date) { return Utilities.formatDate(date, "GMT", "yyyy-MM-dd HH:mm:ss"); }

function processFile(file) {
var id = file.getId();
var name = file.getName();
//your processing...
Logger.log(name);

}