Drive API files.list query with 'not' parameter returns empty pages

2
votes

I'm using the Drive API to list files from a collection which do not contain a certain string in their title.

My query looks something like this: files().list(q="'xxxxx' in parents and not title contains 'toto'")

In my drive collection, I have 100 files, all contain the string "toto" in their title except for let's say 10 files.

I'm using pagination to retrieve the results 20 by 20, so I'm expecting to get only one page with the 10 files corresponding to my request. Surprisingly, the API returns 5 pages, with the first 4 having no results but with a nextToken page, and the files which are compliant with my request only come with the fifth page.

I'm still trying some use-cases here but it seems that it has something to do with the "not" operator. Like if the request was made without it, therefore returning 5 pages, but the results not corresponding to the request being removed from the response. It's very disturbing for me as I'm looking for the best performance here, and obviously having to make 5 requests to Drive instead of one single is not good for me. I'm also noticing that the results don't always come in the last page. I made the test with another collection, the results show up in the second page, but I still get 3 empty pages after that.

Am I missing something here ? Is this kind of behaviour "normal" ? I mean imagine if I had 1000 documents in my collection, having to make 50 requests to find only a few is not what I expect.

2
google-drive-api

2 Answers

1
votes

I have similar problem in files.list API. I tried to receive all three folders under root folder. I received result only on 342nd page. After several hours of researching I found some regularity in this strange behavior.

As I understood, the Drive API works in this way:

  1. Detects something like index that best match your query
  2. Selects first 20 records using index from step 1
  3. Applies your filter: removes records that do not match your query
  4. Rest is returned to you (maybe empty) with next page token.

The nextPageToken is looks like just OFFSET for the first record on next page in decided index, maybe it contains some information about query or index.

After base64 decode this token I found appropriate record number for next result in 121st position in decoded token. Previously I built index of tokens using maxResults=1.

This is crazy, but I have no other explanation for observable behavior.

It is very useful for server because server do a very small work for search. From other side this algorithm must produce a lot of requests for pagenate whole list. But limitation for requests per second solve this problem.

Only You can do is pagenage and skip empty results. Do not forget about limitation of number of requests.

Do not try to find errors on your side. This is how Google Drive API works.

0
votes

contains operator is working as a prefix matcher at the moment.title contains 'toto' will match "totolong" and "toto", but not "blahtoto".