I am trying to download a file from the cloud storage bucket using the curl request as explained here - https://cloud.google.com/storage/docs/downloading-objects#download-object-json
In the above docs, In step1, I see the access token is getting generated from the Oauth 2.0 playground. However, I want to programmatically generate the token and send the CURL request. \
Is there any way to get the access token through any script? Probably from another CURL request using the service account?
Using OAuth 2.0 for Server to Server Applications
Bash OAuth 2.0 JWT script for Server to google Server Applications
#1. Create a service account with storage permissions to download the objects and download the p12 key file
#2. Convert p12 key to pem
PRIVATE_KEY=privateKey.pem
openssl pkcs12 -in privatekey.p12 -nodes -nocerts --passin pass:notasecret > $PRIVATE_KEY
#3. Create an object and uploaded to storage
cat file
#This is a testing file
gsutil cp file gs://your-bucket/
#4.Create a JSON Web Token (JWT, pronounced, "jot") which includes a header, a claim set, and a signature.
ALG=RS256
TYP=JWT
JSON_HEADER=$( jq -n --arg alg "$ALG" --arg typ "$TYP" '{alg: $alg, typ: $typ}' )
JSON_HEADER_ENCODED=`echo -n $JSON_HEADER | openssl base64 -e`
[email protected]
SCOPE=https://www.googleapis.com/auth/cloud-platform
AUD=https://oauth2.googleapis.com/token
IAT=$(date +%s)
EXP=$(($IAT + 3600))
JSON_CLAIM=$( jq -n --arg iss "$ISS" --arg scope "$SCOPE" --arg aud "$AUD" --arg exp "$EXP" --arg iat "$IAT" '{iss: $iss, scope: $scope, aud: $aud, exp: $exp, iat: $iat}');
JSON_CLAIM_ENCODED=`echo -n $JSON_CLAIM | openssl base64 -e`
HEAD_AND_CLAIM_TR=`echo -n "$JSON_HEADER_ENCODED.$JSON_CLAIM_ENCODED" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
echo $HEAD_AND_CLAIM_TR
SIGNATURE_ENCODED=`echo -n "$HEAD_AND_CLAIM_TR" | openssl dgst -sha256 -sign $PRIVATE_KEY | openssl base64 -e`
SIGNATURE_TR=`echo -n "$SIGNATURE_ENCODED" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
echo $SIGNATURE_TR
JWT_ASSERTION="$HEAD_AND_CLAIM_TR.$SIGNATURE_TR"
echo $JWT_ASSERTION
#5. Request an access token from the Google OAuth 2.0 Authorization Server.
RESPONSE=`curl -H "Content-type: application/x-www-form-urlencoded" -X POST "https://oauth2.googleapis.com/token" -d \
"grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=$JWT_ASSERTION" `
#6. Handle the JSON response that the Authorization Server returns.
BEARER=`echo $RESPONSE | jq '.access_token'`
#7. Download the object from GCS
curl -X GET \
-H "Authorization: Bearer $BEARER" \
-o "test_file" \
"https://storage.googleapis.com/storage/v1/b/your-bucket/o/file?alt=media"
#8. Test it
cat test_file
#This is a testing file
With CURL, you can use this command gcloud auth print-access-token. To achieve this, you need to be authenticated with your user credentials gcloud auth login
If you only have a service account key file (because your script don't run locally or on Google Cloud environment) you can load the credentials like this: gcloud auth activate-service-account --key-file=<YOUR FILE>
Then, your CURL command looks like this
curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-o "SAVE_TO_LOCATION" \
"https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/OBJECT_NAME?alt=media"
the $() execute a linux command and print the output
If you want to create this access token programmatically, the Google Auth library can help you to achieve this. Let us know your favorite language if you want code samples.
You can get this access token, if you are logged to Cloud SDK (gcloud) on the same machine you are running cURL command.
The process looks like this:
gcloud auth activate-service-account [ACCOUNT-NAME] --key-file=/path/to/service-key.json --project=[PROJECT_ID]$(gcloud auth print-access-token)Here is Google instruction to this process.
I replicated this on my testing project and such command worked perfectly:
curl -X GET \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-o "test.jpg" \
"https://storage.googleapis.com/storage/v1/b/[bucket-name]/o/original.jpg?alt=media"
