How to upload files in a bucket on Google Cloud Storage using python snippets.py?

Question

I am trying to learn how to upload files in a bucket on Google Cloud Storage. I found this code: https://github.com/GoogleCloudPlatform/python-docs-samples/tree/master/storage/cloud-client This can do a bunch of operations like create-bucket, delete-bucket, list, upload and download …

But, I am unable to run the code.

I tried:

python snippets.py [-h] scene.appspot.com list

Error:

error: argument command: invalid choice: 'scene-maker.appspot.com' (choose from 'create-bucket', 'delete-bucket', 'get-bucket-labels', 'add-bucket-label', 'remove-bucket-label', 'list', 'list-with-prefix', 'upload', 'download', 'delete', 'metadata', 'make-public', 'signed-url', 'rename', 'copy')

I tried:

python snippets.py [-h] list scene.appspot.com
python snippets.py [-h] list gs://scene.appspot.com
python snippets.py [-h] list "gs://scene.appspot.com"
python snippets.py [-h] list bucket_name

Error:

error: unrecognized arguments: scene.appspot.com
error: unrecognized arguments: gs://scene.appspot.com

I tried:

python snippets.py list [-h] scene.appspot.com

Error:

error: argument command: invalid choice: '[-h]' (choose from 'create-bucket', 'delete-bucket', 'get-bucket-labels', 'add-bucket-label', 'remove-bucket-label', 'list', 'list-with-prefix', 'upload', 'download', 'delete', 'metadata', 'make-public', 'signed-url', 'rename', 'copy')

I tried:

python snippets.py [-h] list

Error:

Traceback (most recent call last):
  File "snippets.py", line 322, in <module>
    list_blobs(args.bucket_name)
  File "snippets.py", line 88, in list_blobs
    bucket = storage_client.get_bucket(bucket_name)
  File "C:\Anaconda2\lib\site-packages\google\cloud\storage\client.py", line 172, in get_bucket
    bucket = Bucket(self, name=bucket_name)
  File "C:\Anaconda2\lib\site-packages\google\cloud\storage\bucket.py", line 113, in __init__
    name = _validate_name(name)
  File "C:\Anaconda2\lib\site-packages\google\cloud\storage\_helpers.py", line 39, in _validate_name
    'Bucket names must start and end with a number or letter.')
ValueError: Bucket names must start and end with a number or letter.

When I run: gsutil ls I get:

gs://scene.appspot.com/
gs://staging.scene.appspot.com/

How to use python snippets.py command? Ultimately, I want users to be able to upload files form a web browser to Cloud Storage.

Dan Cornilescu Dan Cornilescu · Accepted Answer · 2017-11-16T20:44:48

You're having trouble invoking the correct command syntax.

error: argument command: invalid choice: '[-h]'

You shouldn't literally have [-h] in the command. The square brackets are just a standard notation to indicate optional arguments. So you either use -h in the command (which will indicate that you request the command's help/usage screen instead of actually attempting execution of the command) or leave it out completely.

From the page you referenced:

To run this sample:

$ python snippets.py

usage: snippets.py [-h]
                   bucket_name
                   {create-bucket,delete-bucket,get-bucket-labels,add-bucket-label,remove-bucket-label,list,list-with-prefix,upload,download,delete,metadata,make-public,signed-url,rename,copy}
                   ...

So the command expects the first argument (after snippets.py) to be the bucket name, followed by a specific operation to be performed. Depending on the operation other arguments may follow.

The [-h] you used is interpreted as an argument and throws the parser off:

in the 1st attempt [-h] is interpreted as the bucket name and scene.appspot.com as the operation. But that operation is not a valid one - hence the error.
in the 2nd group of attempts [-h] is interpreted as the bucket name, 'list' as the operation (a valid one this time), but then you have another argument (your attempts to specify the bucket name) which is unexpected for the list operation - hence the errors.
in the 3rd attempt list is interpreted as the bucket name and [-h] as the operation - again an invalid one - hence the error.
in the 4th attempt [-h] is interpreted as the bucket name, 'list' as the operation (valid). This time the number of arguments and the operation appear correct and the code proceeds further to validate the bucket name, which fails as [-h] is not a valid one - hence the error.

So I'd try (I'm unsure about the actual expected format for the bucket name):

python snippets.py scene.appspot.com list
python snippets.py gs://scene.appspot.com list

To upload file to the bucket (use the correct bucket name format determine from the previous attempt, I'll assume it's scene.appspot.com from now on), it's probably something starting with:

python snippets.py scene.appspot.com upload

To see the expected syntax for this operation add the -h option:

python snippets.py -h scene.appspot.com upload

Note: this whole method is command-driven, uploading files from a web browser to Cloud Storage is a different story altogether. I'd ask that as a separate question, with lots of details, as there can be many answers to it.

How to upload files in a bucket on Google Cloud Storage using python snippets.py?

1 Answers