“Invalid value at 'ciphertext' (TYPE_BYTES)”

Question

SITUATION:

I am trying to download and decrypt some data from my google cloud bucket.

For encryption and decryption, I use:

https://cloud.google.com/kms/docs/quickstart#decrypt_data

Sadly, I get an error : "Invalid value at 'ciphertext' (TYPE_BYTES)".

I know the cyphertext is correct, I believe this may be an issue with the type of data expected by the Google KMS API, i.e.: when retrieving the encrypted data, my code somehow changed it's type before sending the POST request to the Google KMS API.

What did I do wrong and how do I fix it ?

CODE:

gcs.bucket(bucketName)
.file('mysecret.txt.encrypted.txt')
.download({ destination: 'mysecret.txt.encrypted.txt' })
.then(() => {
    fs.readFile('mysecret.txt.encrypted.txt', (err, data) => {
        if (err) throw err;
        console.log("DATA: "+data);
        var formData = {
           ciphertext: data, 
        };
        request.post({
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ...'
            },
            url: 'https://cloudkms.googleapis.com/v1/projects/kms-raimarketplace/locations/global/keyRings/.../cryptoKeys/...:decrypt',
            form: formData
        },
        function (err, httpResponse, body) {
            if (err) {
                console.log("ERROR: "+err);
            }
            else {
                console.log("BODY: "+body);
            }
            console.log(err, body);
        });
    });
}).catch(e => {
    console.error('getEnv.js: There was an error: ${JSON.stringify(e, undefined, 2)}');
});

OUTPUT:

BODY: {
  "error": {
    "code": 400,
    "message": "Invalid value at 'ciphertext' (TYPE_BYTES), ",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "field": "ciphertext",
            "description": "Invalid value at 'ciphertext' (TYPE_BYTES), "
          }
        ]
      }
    ]
  }
}

I'm not familiar with the request library you are using. It appears that the "form" option for request.post formats the body as application/x-www-form-urlencoded instead of JSON. Instead, try using the "json" option. request.post({json: formData}) — Russ Amos
How did you get the encrypted data? The base64 encoding can be tricky: I'm wondering here if your ciphertext is either not base64-encoded for transport or if it's double-encoded. — Tim Dierks
@RussAmos BODY: [object Object] null { error: { code: 400, message: 'Invalid JSON payload received. Unknown name "type" at \'ciphertext\': Cannot find field.\nInvalid JSON payload received. Unknown name "data" at \'ciphertext\': Cannot find field.', status: 'INVALID_ARGUMENT', details: [ [Object] ] } } — TheProgrammer
If you're Googling this like I did about 4 hours ago...and, like me, you habitually add newlines to your text files (including mysecret.txt.encrypted.txt...), you probably ran into the same issue I did, which is that data includes a self-induced trailing newline -- which is indeed not valid Base64. Assuming the rest of the contents of mysecret.txt.encrypted.txt is already base64 encoded, try passing ciphertext: data.toString().trim(). — Timothy Johns

Tim Dierks Tim Dierks · Accepted Answer · 2018-01-31T18:55:45

I think you are possibly not passing base64-encoded ciphertext; this is the error returned when such decoding fails (although this message can occur in other situations as well).

I created a world-usable key which anyone can use to decrypt, you should be able to use this to test.

Here's the code I used to create the ciphertext. (You will not be able to run this code because I have not opened up encrypt rights on the key. This code is straight out of the Cloud KMS Quickstart with my key inserted and the base64 encoding inline)

curl -s -X POST "https://cloudkms.googleapis.com/v1/projects/cloud-kms-demonstration/locations/global/keyRings/insecure-example-key-ring/cryptoKeys/insecure-example-key:encrypt" \
-d "{\"plaintext\":\"$(echo hello world | base64)\"}" \
  -H "Authorization:Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type:application/json"

When I run this, I get the following output (I ran it in GCP Cloud Shell logged in as my gmail account, which I'd granted encrypt rights on the key):

{
  "name": "projects/cloud-kms-demonstration/locations/global/keyRings/insecure-example-key-ring/cryptoKeys/insecure-example-key/cryptoKeyVersions/1",
  "ciphertext": "CiQAOay+bYR0HXjYHihHZmjPnu0ZEdOm4/HW4S6ZBifwWfFnL1QSNQBA6mRpHiq1MPQ7MerLyJ+gFJdeQooBFU0K0YmGlxRy5Ke/2eV16yR0viHPII6flFpzxnmD"
}

So then I ran the following command; you will see that I copied and pasted the ciphertext from the output of the encrypt operation. Anyone who is authenticated will be able to run this command to decrypt the ciphertext with my key, so feel free to run it now:

curl -s -X POST "https://cloudkms.googleapis.com/v1/projects/cloud-kms-demonstration/locations/global/keyRings/insecure-example-key-ring/cryptoKeys/insecure-example-key:decrypt" \
  -d "{\"ciphertext\":\"CiQAOay+bYR0HXjYHihHZmjPnu0ZEdOm4/HW4S6ZBifwWfFnL1QSNQBA6mRpHiq1MPQ7MerLyJ+gFJdeQooBFU0K0YmGlxRy5Ke/2eV16yR0viHPII6flFpzxnmD\"}" \
  -H "Authorization:Bearer $(gcloud auth application-default print-access-token)" \
  -H "Content-Type:application/json"

And I got the result:

{
  "plaintext": "aGVsbG8gd29ybGQK"
}

Which is the base64 encoding of my plaintext:

$ echo "aGVsbG8gd29ybGQK" | base64 -d
hello world

If I was using a library, it's possible that it would handle the base64 encoding as a part of transport requirements. So hopefully you can use this known-good ciphertext and a key you can decrypt it with to debug your own code. Good luck!

Note: As the names indicate, do not use the key I've provided for anything secure (this is why I didn't open encrypt rights on the key).

“Invalid value at 'ciphertext' (TYPE_BYTES)”

1 Answers