Get email senders in batch from Gmail API for over 10k emails

Question

I'm making an Chrome Extension which interacts with Gmail API. Basically it needs to extract email senders from emails that match some keyword. Problem is that sometimes there are over 10K emails matching this keyword. I utilize gapi.client.gmail.users.messages.list to fetch all emails that match entered keyword, but this only returns email and thread IDs, so I need to call gapi.client.gmail.users.messages.get for each email ID retrieved from messages.list. So there are over 10K requests to Gmail API, and I run into ERR_INSUFFICIENT_RESOURCES error in Chrome. To prevent this error, I set some timeout between calls to messages.get but then it would take ages to complete...

Is there some recommended way to get this amount of emails from Gmail API?

Iván Nokonoko Iván Nokonoko · Accepted Answer · 2018-03-16T08:15:54

According to documentation, one way to improve performance is by batching requests. Currently it has a limit of 100 requests per batch, but it is still 100 times fewer requests.

EDIT: Also you can use fields parameter in query to request the fields you want from the messages, since both messages.list and messages.get can return a whole users.messages Resource.

For example:

var xhr = new XMLHttpRequest;
xhr.onload = () => {
    var resp = JSON.parse(xhr.response);
    var wholeFirstMessage = atob(resp.messages[0].raw);
    console.log(wholeFirstMessage);
};
xhr.open("GET", "https://www.googleapis.com/gmail/v1/users/userId/messages?fields=messages(id,threadId,raw)");
xhr.send();

NOTE: The code sample above ignores pageToken and maxResults from the XHR parameters for simplicity' sake. Those parameters would be needed for a long list of messages.

Get email senders in batch from Gmail API for over 10k emails

2 Answers