Error “No URLs matched” When copying Google cloud bucket data to my local computer?

Question

I am trying to download a folder which is inside my Google Cloud Bucket, I read from google docs gsutil/commands/cp and executed below the line.

gsutil cp -r appengine.googleapis.com gs://my-bucket

But i am getting the error

CommandException: No URLs matched: appengine.googleapis.com

Edit

By running below command

gsutil cp -r gs://logsnotimelimit .

I am getting Error

IOError: [Errno 22] invalid mode ('ab') or filename: u'.\logsnotimelimit\appengine.googleapis.com\nginx.request\2018\03\14\14:00:00_14:59:59_S0.json_.gstmp'

Answer 1

What is the appengine.googleapis.com parameter in your command? Is that a local directory on your filesystem you are trying to copy to the cloud bucket?

The gsutil cp -r appengine.googleapis.com gs://my-bucket command you provided will copy a local directory named appengine.googleapis.com recursively to your cloud bucket named my-bucket. If that's not what you are doing - you need to construct your command differently.

I.e. to download a directory named folder from your cloud bucket named my-bucket into the current location try running gsutil cp -r gs://my-bucket/folder .

-- Update: Since it appears that you're using a Windows machine (the "\" directory separators instead of "/" in the error message) and since the filenames contain the ":" character - the cp command will end up failing when creating those files with the error message you're seeing.

Answer 2

Just wanted to help people out if they run into this problem on Windows. As administrator:

• Open C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\gsutil\gslib\utils
• Delete copy_helper.pyc
• Change the permissions for copy_helper.py to allow writing
• Open copy_helper.py
• Go to the function _GetDownloadFile
• On line 2312 (at time of writing), change the following line

download_file_name = _GetDownloadTempFileName(dst_url)

to (for example, objective is to remove the colons):

download_file_name = _GetDownloadTempFileName(dst_url).replace(':', '-')

• Go to the function _ValidateAndCompleteDownload
• On line 3184 (at time of writing), change the following line

final_file_name = dst_url.object_name

to (for example, objective is to remove the colons):

final_file_name = dst_url.object_name.replace(':', '-')

• Save the file, and rerun the gsutil command
• FYI, I was using the command gsutil -m cp -r gs://my-bucket/* . to download all my logs, which by default contain : which does not bode well for Windows files!

Hope this helps someone, I know it's a somewhat hacky solution, but seeing as you never need (should have) colons in Windows filenames, it's fine to do and forget. Just remember that if you update the Google SDK you'll have to redo this.

Answer 3

I got same issue and resolved it as below.

1. Open a cloud shell, and copy objects by using gsutil command.

gsutil -m cp -r gs://[some bucket]/[object] .

2. On the shell, zip those objects by using zip command.

zip [some file name].zip -r [some name of your specific folder]

3. On the shell, copy the zip file into GCS by using gsutil command.

gsutil cp [some file name].zip gs://[some bucket] .

4. On a Windows Command Prompt, copy the zip file in GCS by using gsutil command.

gsutil cp gs://[some bucket]/[some file name].zip .

I wish this information helps someone.