How to disable message encryption for gcp pubsub emulator?

Question

I'm using the official GCP PubSub emulator to test integration locally.

I'd like to send messages via classic curl/postman tools but it is getting complicated because this emulator requires encryption of incoming messages.

For instance, if we send it like this:

curl --location --request POST 'http://localhost:8091/v1/projects/my-project/topics/transactions:publish' \
--header 'Content-Type: application/json' \
--data-raw '{"messages":[{"data":"{\"foo\":\"baz\"}","attributes":{}}]}'

Then, I'm getting 400:

{
    "error": {
        "code": 400,
        "message": "Payload isn't valid for request.",
        "status": "INVALID_ARGUMENT"
    }
}

due to invalid incoming messages. It requires encryption and if I sniff the encrypted body it works.

But it is overwhelming to encrypt messages running it locally.

In order to disable encryption in GCP I can follow this guide but it is not applicable to local emulators run - there is no GCP environment or I don't know how to do it.

Are there any options to disable emulator decryption? If not where to report it, there is no GitHub project for this.

guillaume blaquiere guillaume blaquiere · Accepted Answer · 2020-11-11T13:31:28

Ok, it took time for me to understand, but I think you mixed 2 things: encryption and encoding.

The data value in PubSub isn't provided encrypted but encoded in base64. There isn't encryption required here. Base64 encoding is a raw encoding to prevent data loss, encoding type, special characters, binary data and boring compatibility things.

Note: On your local computer with pubsub emulator, the data aren't encrypted at rest and in transit. On Google Cloud, with PubSub service, the data are encrypted in transit and at rest

With curl you can use this command (with linux OS)

curl --location --request POST 'http://localhost:8091/v1/projects/my-project/topics/transactions:publish' \
--header 'Content-Type: application/json' \
--data-raw "{\"messages\":[{\"data\":\"$(echo "{\"foo\":\"baz\"}" | base64 -)\",\"attributes\":{}}]}"

Yes, backslash are boring...

I don't know how to do with postman

How to disable message encryption for gcp pubsub emulator?

1 Answers