Google App Engine blobstore error: BlobNotFoundError

Question

I use the Google App Engine blobstore to hold a blob of user data -- anywhere from a few hundred bytes up to a few hundred KB in size. The blob_info is saved as a property on a datastore entity.

Occasionally in production, reads from the blobstore will fail with a BlobNotFoundError('',). The exception doesn't provide any detail and I can't figure out why the failures are occurring.

According to Google's documentation:

"If blob does not refer to an actual Blobstore value, fetch_data raises a BlobNotFoundError." https://developers.google.com/appengine/docs/python/blobstore/functions#fetch_data

"The fetch_data() function could not find a Blobstore value that corresponds with the given BlobInfo or BlobKey value." https://developers.google.com/appengine/docs/python/blobstore/exceptions#BlobNotFoundError

What is most puzzling about this, is that the failures are intermittent.

Below is my code for reading-from / writing-to the blobstore. A read is only attempted if the blob_info (read from the datastore) is not None.

Any suggestions?

def read(blob_info):
    blob_reader = blobstore.BlobReader(blob_info.key(), buffer_size=358400)
    try:
        data = blob_reader.read()
    finally:
        blob_reader.close()

    return data


def write(data, mime_type):
    file_name = files.blobstore.create(mime_type=mime_type)

    with files.open(file_name, 'a') as f:
        f.write(data)

    files.finalize(file_name)

    blob_key = files.blobstore.get_blob_key(file_name)

    # This is a hack to handle an apparent GAE delay synchronizing the blobstore
    for i in range(1,3):
        if blob_key:
            break
        else:
            time.sleep(0.05)
            blob_key = files.blobstore.get_blob_key(file_name)

    new_blob_info = blobstore.BlobInfo.get(str(blob_key))

    return new_blob_info

Patrick Simpson Patrick Simpson · Accepted Answer · 2013-04-10T05:31:10

I think your hack for the GAE delay is running three times without success, giving blob_key the value None. In the line

new_blob_info = blobstore.BlobInfo.get(str(blob_key))

str(blob_key) will convert to 'None', i.e. a string containing the word None. So you are returning a blob_info with a key, it just contains the word 'None'.

You could quickly fix it by checking for an explicit None after the loop.

The proper fix would be to somehow avoid the race condition, but I have no idea why that occurs; your code looks normal to me. I've done roughly the same on Java and there it seems to work fine. Keep in mind that blobstore file support is experimental and not really very well documented. Could you rework your code to use the raw blobstore API (i.e. with upload URLs)?

Google App Engine blobstore error: BlobNotFoundError

1 Answers