AES-128-CBC encryption with OpenSSL/C++ and PHP/Mcrypt: the 1st block is decrypted only

Question

I write the program that has to exchange encrypted data with PHP-based web service. I use C++ with OpenSSL to encrypt data with AES-128 in CBC mode. I send base64-encoded data (IV and ciphertext) to HTTP server and PHP has to decrypt the data with Mcrypt. However, only the 1st block is decrypted successfully, other blocks become garbage. I don't understand at all: how we can decrypt the 1st block only in CBC mode? If IV, key or algorithm settings like key size/block size/rounds count are wrong, we can not receive the 1st block decrypted correctly; if decryption parameters are OK, how is it possible that other blocks are not decrypted? When I decrypt the same ciphertext that PHP fails to decrypt with OpenSSL/C++, decryption succeeds. The same is true for PHP: I can encrypt and decrypt data in CBC mode. But for any reason OpenSSL EVP_aes_128_cbc and mcrypt 'rijndael-128' are not compatible. My decryption code follows:

$chipher = mcrypt_module_open('rijndael-128', '', 'cbc', '');  
mcrypt_generic_init($chipher, $key, $iv);
$decrypted_data = mdecrypt_generic($chipher, $encrypted_data);

Is it bug of mcrypt or there are any way to encrypt data with OpenSSL AES-128-CBC and decrypt it with PHP mcrypt?

Is there a reason why you can't use a standard protocol like TLS which is designed for a use case like this? — ntoskrnl
I am thinking about some crack protection; it is usually trivial on Windows to add trusted root CA and intercept HTTPS traffic with tool like Fiddler; that is why I decided to use custom encryption instead of SSL, so atacker will have to disassemble & modify my code to crack the system instead of just standard tools usage like Fiddler. — Vitalii
But once the master secret key is "cracked", then all users of your whole system are compromised. You need to decide whether you are looking for security or obfuscation. — ntoskrnl
When you give program to users who potentially may try to crack it, obfuscation is the only option; however, I think that nonstandard packing + custom encryption is more complex to reverse engeneer than standard HTTPS protocol usage; disassembling is generally more complex task than additional CA installation to bypass HTTPS encryption in my opinion. Of couse, any software still crackable, but I want to make this job more complex. — Vitalii
hm ... the only reasons i could think of in this scenario are these: block 0(iv) and block 1 are transmitted ok, but then some other stuff was inserted into the datastream, like a separation char between the blocks... can you confirm that the output of your encryption has the right size, like size of iv + x * blocksize? --- can you confirm that you don't mix up modes of operation? like CBC on the one side and PCBC on the other — DarkSquirrel42

Vitalii Vitalii · Accepted Answer · 2013-11-22T15:57:21

Well, the error was on C++ side. The data I encrypted came by parts, so I had to call EVP_CipherUpdate multiple times; because on repeated call EVP_CipherUpdate copies to output buffer too much data and only last block only, I had to call EVP_CipherInit_ex repeatedly to clean internal buffers in OpenSSL context. And when I called EVP_CipherInit_ex, it reset IV, so after some point decryption became impossible. The solution is to save last version of IV (context.iv points to last IV, and context.oiv points to original one) in buffer after each EVP_CipherUpdate call and pass it to EVP_CipherInit_ex to allow OpenSSL to encrypt data by parts. When I did it, mcrypt was able to decrypt entire data and even remove padding. So, mcrypt is excellent, and problems I saw are my program bug and not mcrypt.

AES-128-CBC encryption with OpenSSL/C++ and PHP/Mcrypt: the 1st block is decrypted only

1 Answers