Google Cloud Function throwing a 404 error

Question

I created a Cloud Function which is set to be triggered by a Finalize/Create event on a Google Cloud Storage bucket.

I dumped 10k images into the bucket, and the majority of the images ran the Cloud Function successfully, while some threw the following exception and I'm just confused on what is causing this error.

The error is returning a 404, but when I check the bucket, the image is there. Does anyone have experience with fixing the following exception?

For some reason the Cloud Function is checking a directory that does not exist.

It should check the following directory:

https://storage.googleapis.com/download/storage/v1/b/hidden-name/images%2F3683740a95fa3600d4d9a220c9e0b472.jpg

However, it is checking the following directory instead, which does not exist (/o folder does not exist):

https://storage.googleapis.com/download/storage/v1/b/hidden-name/o/images%2F3683740a95fa3600d4d9a220c9e0b472.jpg

This is the exception I'm getting:

Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 721, in download_to_file transport, file_obj, download_url, headers, start, end, raw_download File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 645, in _do_download download.consume(transport) File "/env/local/lib/python3.7/site-packages/google/resumable_media/requests/download.py", line 153, in consume self._process_response(result) File "/env/local/lib/python3.7/site-packages/google/resumable_media/_download.py", line 171, in _process_response response, _ACCEPTABLE_STATUS_CODES, self._get_status_code File "/env/local/lib/python3.7/site-packages/google/resumable_media/_helpers.py", line 96, in require_status_code *status_codes google.resumable_media.common.InvalidResponse: ('Request failed with status code', 404, 'Expected one of', , ) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function _function_handler.invoke_user_function(event_object) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function event_context.Context(**request_or_event.context)) File "/user_code/main.py", line 44, in for_each_new_image create_thumb_move_to_thumbs_bucket(bucket_name,blob_name) File "/user_code/main.py", line 56, in create_thumb_move_to_thumbs_bucket blob_in_bucket.download_to_filename(download_tmp_filepath) File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 761, in download_to_filename raw_download=raw_download, File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 724, in download_to_file _raise_from_invalid_response(exc) File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 2161, in _raise_from_invalid_response raise exceptions.from_http_status(response.status_code, message, response=response) google.api_core.exceptions.NotFound: 404 GET https://storage.googleapis.com/download/storage/v1/b/hidden-name/o/images%2F3683740a95fa3600d4d9a220c9e0b472.jpg?alt=media: ('Request failed with status code', 404, 'Expected one of', , )

Cloud function entry method:

def for_each_new_image(event, context):   
    blob_name = event['name'].lower()
    bucket_name = event['bucket']   
    create_thumb_move_to_thumbs_bucket(bucket_name,blob_name)

Method which throws the error:

def create_thumb_move_to_thumbs_bucket(bucket_name,blob_name):     
    bucket = _storage_client.get_bucket(bucket_name)
    blob_in_bucket = bucket.blob(blob_name)

    blob_name = blob_name.replace("ugc_images/","")    
    thumbnail_filename = blob_name.replace(".","-sm.")

    thumbnail_tmp_filepath = '/tmp/{}'.format(thumbnail_filename)       
    download_tmp_filepath = '/tmp/{}'.format(blob_name)    

    blob_in_bucket.download_to_filename(download_tmp_filepath)

Line which throws the error:

blob_in_bucket.download_to_filename(download_tmp_filepath)

Have you uploaded the same file twice ? The Object Finalize will trigger even though it is overwritten. If I click on second URL, it says { "error": { "code": 400, "message": "Only media downloads requests can be sent to /download/*. Re-send request to the same path, but without the /download prefix.", "errors": [ { "message": "Only media downloads requests can be sent to /download/*. Re-send request to the same path, but without the /download prefix.", "domain": "global", "reason": "notDownload" } ] } } — Vikram Shinde
@VikramShinde Nope, all unique images. Also i made the URLs dummy, meaning they aren't live — Machine Learning
@Chris32 Instead of adding all of the lines of code, i added the method/line where the exception is thrown: "blob_in_bucket.download_to_filename(download_tmp_filepath)" — Machine Learning
Could you add enough code to make it reproducible. This will also give us more context to be able to help — Soni Sol

Aseem Aseem · Accepted Answer · 2020-07-13T04:01:05

I was having the same error. I was trying to a access a file which did not exist in the bucket.

Google Cloud Function throwing a 404 error

1 Answers