Encryption is hard: AES encryption to Hex

Question

So, I've got an app at work that encrypts a string using ColdFusion. ColdFusion's bulit-in encryption helpers make it pretty simple:

encrypt('string_to_encrypt','key','AES','HEX')

What I'm trying to do is use Ruby to create the same encrypted string as this ColdFusion script is creating. Unfortunately encryption is the most confusing computer science subject known to man.

I found a couple helper methods that use the openssl library and give you a really simple encryption/decryption method. Here's the resulting string:

"\370\354D\020\357A\227\377\261G\333\314\204\361\277\250"

Which looks unicode-ish to me. I've tried several libraries to convert this to hex but they all say it contains invalid characters. Trying to unpack it results in this:

string = "\370\354D\020\357A\227\377\261G\333\314\204\361\277\250"
string.unpack('U')
ArgumentError: malformed UTF-8 character
  from (irb):19:in `unpack'
  from (irb):19

At the end of the day it's supposed to look like this (the output of the ColdFusion encrypt method):

F8E91A689565ED24541D2A0109F201EF

Of course that's assuming that all the padding, initialization vectors, salts, cypher types and a million other possible differences all line up.

Here's the simple script I'm using to encrypt/decrypt:

def aes(m,k,t)
  (aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc').send(m)).key = Digest::SHA256.digest(k)
  aes.update(t) << aes.final
end

def encrypt(key, text)
  aes(:encrypt, key, text)
end

def decrypt(key, text)
  aes(:decrypt, key, text)
end

Any help? Maybe just a simple option I can pass to OpenSSL::Cipher::Cipher that will tell it to hex-encode the final string?

Your string isn't Unicode. It's just a bunch of bytes. You just need to convert each byte in its hexadecimal numeric form. The first one, 370 in octal, is F8 in hexadecimal. — Joey
I can't help you with encryption part, but to display a byte sequence (in Ruby usually presented as an ordinary String) as a string of hexadecimal numbers, you can use s.each_byte.map{|b| b.to_s(16)}.join. I don't think Unicode is related here anyhow. — Mladen Jablanović

Oleh Khomey Oleh Khomey · Accepted Answer · 2011-10-23T16:32:02

I faced similar issue today. Mine was a bit trickier, b/c I didn't have access to CF code - only text to encrypt, key and SHA256 of encrypted result.

Official documentation for Encrypt function says:

key: String. Key or seed used to encrypt the string.

For the CFMX_COMPAT algorithm, any combination of any number of characters; used as a seed used to generate a 32-bit encryption key.

For all other algorithms, a key in the format used by the algorithm. For these algorithms, use the GenerateSecretKey function to generate the key.

In my case I was provided with a 32 chars long MD5 of some string which was a key I had to use.

Since we can not directly use own key for AES ancryption - I had to convert it to the same format GenerateSecretKey has.

After some digging it appeared that GenerateSecretKey creates random 16 bytes long binary string and encodes it to Base64. But – this is important – decodes it back internally during encryption process.

So this is working solution:

CF code:

plain_key = "REPLACE_ME_WITH_32_HEX_UPPERCASED_CHARS"; // this can be MD5 of some secret string
encoded_key = ToBase64(BinaryDecode(plain_key, "Hex"));
base64EncodedResult = Encrypt("PLAIN TEXT", key, "AES", "Base64");

Ruby code:

plain_key = "REPLACE_ME_WITH_32_HEX_UPPERCASED_CHARS"
encoded_key = plain_key.unpack('a2' * 16).map(&:hex).pack('c' * 16) # same as in CF except encoding to Base64
aes_encrypted = Aes.encrypt('PLAIN TEXT', encoded_key)
base64_encoded_result = ActiveSupport::Base64.encode64s(aes_encrypted)

Aes module code:

# Two changes comparing to author's code:
# 1) AES-128-ECB instead of AES-256-CBC
# 2) No key conversion to SHA256

module Aes
  def self.aes(m,k,t)
    (aes = OpenSSL::Cipher::Cipher.new('aes-128-ecb').send(m)).key = k
    aes.update(t) << aes.final
  end

  def self.encrypt(text, key)
    aes(:encrypt, key, text)
  end

  def self.decrypt(text, key)
    aes(:decrypt, key, text)
  end
end

Encryption is hard: AES encryption to Hex

2 Answers