Why is Ruby base64 encoded string different from all other base64 encoded strings?

Question

For end-to-end encrypted communication between a client and a server, I am implementing an encryption/decryption algorithm.

However, they (encryption/decryption and base64 encoding/decoding) work fine only when it's in Ruby.

But the actual problem I see is with the Base64 encoding of Ruby.

For example, let's say I have this (32 bytes) AES key:

"\"1\xAF\xC7\xC0\xA6\xC9\xBA\xD6\x9F\xBA\xD2\xC9\xBE\x0F\x8E*\x88\x87(\x9B\xCBp\x15!/\x13\x8F\xCE\xFB\x15\x9B"

that I am using to encrypt data in AES algorithm.

I want to send this key to a client in Base64 encoded format. For that, I am doing (two ways, each produces different encoded output):

Key in double quotes

Base64.urlsafe_encode64("\"1\xAF\xC7\xC0\xA6\xC9\xBA\xD6\x9F\xBA\xD2\xC9\xBE\x0F\x8E*\x88\x87(\x9B\xCBp\x15!/\x13\x8F\xCE\xFB\x15\x9B")
# => "IjGvx8CmybrWn7rSyb4PjiqIhyiby3AVIS8Tj877FZs="

Key in single quotes

Base64.urlsafe_encode64('\"1\xAF\xC7\xC0\xA6\xC9\xBA\xD6\x9F\xBA\xD2\xC9\xBE\x0F\x8E*\x88\x87(\x9B\xCBp\x15!/\x13\x8F\xCE\xFB\x15\x9B')
# => "XCIxXHhBRlx4QzdceEMwXHhBNlx4QzlceEJBXHhENlx4OUZceEJBXHhEMlx4QzlceEJFXHgwRlx4OEUqXHg4OFx4ODcoXHg5Qlx4Q0JwXHgxNSEvXHgxM1x4OEZceENFXHhGQlx4MTVceDlC"

Output 1 is different from all other libraries: Java, Swift, and an online site, another site, which all produce the same output.

Output 2 is the same with other libraries with respect to the output encoding. But I have issues converting AES key and AES encrypted data to be used in single quotes, which is not possible, as I have encrypted data that already have those single quotes and other illegal characters, for which Ruby's Base64 encoding does not work correctly.

Any help would be appreciated.

Do you really mean to include the double quote in the beginning also? That might be the reason it’s different. Have you decoded both versions and checked what comes back and that it’s identical? — Sami Kuhmonen
"For end to end encrypted communication between client and server, I have been implementing the encryption/decryption algorithm." Don't. Strong, secure, and free encryption software already exists and it's easy to use. Writing your own encryption code is not a good idea and the result will not be secure. Instead, enable SSL for your application server and use all the time you saved to add useful features to your app. — Jordan Running
... Observe what happens if you puts the strings with the different quotes. The type of quote matters in Ruby. — Dave Newton
And if you're implementing your own encryption algorithm, I echo the others in saying for your own benefit, don't. It's not clear if you're using a separate library or if you're actually attempting your own crypto. — Dave Newton
@jordan running yes, I am not writing my own encryption algo, instead I am using Ruby's default OpenSSL library and its AES encryption technics. — Bhavesh Kshatriya

lacostenycoder lacostenycoder · Accepted Answer · 2019-03-28T14:22:54

The problem is that when you use single quote you will get a different result:

a = "\"1\xAF\xC7\xC0\xA6\xC9\xBA\xD6\x9F\xBA\xD2\xC9\xBE\x0F\x8E*\x88\x87(\x9B\xCBp\x15!/\x13\x8F\xCE\xFB\x15\x9B"
#=> "\"1\xAF\xC7\xC0\xA6ɺ֟\xBA\xD2ɾ\u000F\x8E*\x88\x87(\x9B\xCBp\u0015!/\u0013\x8F\xCE\xFB\u0015\x9B"

b = '\"1\xAF\xC7\xC0\xA6\xC9\xBA\xD6\x9F\xBA\xD2\xC9\xBE\x0F\x8E*\x88\x87(\x9B\xCBp\x15!/\x13\x8F\xCE\xFB\x15\x9B'
#=> "\\\"1\\xAF\\xC7\\xC0\\xA6\\xC9\\xBA\\xD6\\x9F\\xBA\\xD2\\xC9\\xBE\\x0F\\x8E*\\x88\\x87(\\x9B\\xCBp\\x15!/\\x13\\x8F\\xCE\\xFB\\x15\\x9B"

a == b
=> false
a.bytes.count
=> 32
b.bytes.count
=> 108
a.length
=> 29
b.length
=> 108

So you see single quotes will double escape the escapes.

However, as Jordan Running mentions in comments you should not roll your own encryption, it is not secure please read this answer to understand why.

It is a much better idea to use ruby's openssl gem. For instructions on how to use it, please refer to documentation here.

Why is Ruby base64 encoded string different from all other base64 encoded strings?

2 Answers