how to restrict google cloud storage upload

0
votes

I have a mobile application that uses Google Cloud Storage. The application allows each registered user to upload a specific number of files.

My question is, is there a way to do some kind of checks before the storage upload? Or do I need to implement a separate reservation API of sorts that OKs an upload step? Any alternative suggestions are welcome too, of course.

1
google-cloud-platformgoogle-cloud-storage
What do you mean by "some kind of checks"? – Himanshu
You have two methods 1) Verify and then upload the data thru one of the Google Compute services 2) Verify and then use presigned URLs for directly uploading data to Cloud Storage from the client. The first method costs additional money for egress bandwidth. The second method is preferred. – John Hanley
Are you storing the files with any specific achitecture? for example a folder per user or something like that? – Soni Sol
@JohnHanley #2 verify step is our own custom verification correct? And by presigned URLs you mean the default google storage url ( i am using the java storage apis) – jayaram S
Yes, your software verifies the access rights for the user. Then your code generates a Google Cloud Storage Signed URL cloud.google.com/storage/docs/access-control/signed-urls – John Hanley

1 Answers

1
votes

warning: Not an authoritative answer. Happy to accept removal or update requests.

I am not aware of any GCS or Firebase Cloud Storage mechanisms that will inherently limit the number of files (objects) that a given user can create. If it were me, this is how I would approach the puzzle.

I would create a database (eg. Firestore / Datastore) that has a key for each user and a value which is the number of files they have uploaded. When a user wants to upload a new file, it would first make a REST call to a Cloud Function that I would write. This Cloud Function would implicitly know the identity of the calling user. It would look up the record in the database and determine if we are allowed to upload a new file. If no, then return an error and end of story. If yes, then increment the value in the database. Next I would create a GCS "signed URL" that can be used to permit an upload. It would be that signed URL that the Cloud Function would return. The app that now wishes to upload can use that signed URL to perform the actual upload.

I would also add metadata to each file uploaded to identify the logical uploader (user) of the file. That can be then used for reconciliation if needed. We could examine all the files in the bucket and re-build the database of how many files each user had uploaded.

A possible alternative to this story is for the Cloud Function to not return a signed-url but instead receive the data to be uploaded in the same request. If the check on number of files passes, then the Cloud Function could be a proxy to a GCS write to create the file directly. This alternative needs to be carefully examined as a function of the sizes of the files to be uploaded. If the files are large this may be a very poor solution. We want to be in and out of Cloud Functions as quickly as possible and holding a Cloud Function "around" to service data pass through isn't great. We may want to look at Cloud Run in that case as it supports concurrency in the instance without increasing the cost per call.