OpenSSL decrypt JSBN encrypted

Question

I'm having troubles decrypting in PHP some data that I have encrypted in JS using RSA:

Some data (32bytes, short enough for RSA) is encrypted client side using JSEncrypt, and the server's public key.

The server decrypts it using:

openssl_private_decrypt(base64_decode($result_obj['data']), $decrypted, $pkcs_private_key);

But this returns 44bytes, so obivously this is wrong! I've checked byte by byte, and it seems like it's almost a atter of deleting bytes at some positions in the decrypted string, but not that simple either.

I've checked the padding openssl supports, and it uses PKCS#1v1.5type1, whereas JSEncrypt seems to use PKCS#1v1.5type2. Could it be the problem? I've seen people that have problems with openssl using PKCS#1v1.5type2, so Isuspect it is able to use this padding, but I cannot figure out how... Any help appreciated!

EDIT: To tell more about encryption system:

The data to encrypt client side is a 32-byte array. In this example, I will be using the array [182, 13, 97, 94, 164, 102, 129, 70, 192, 52, 94, 65, 243, 190, 57, 48, 153, 161, 46, 32, 122, 64, 53, 237, 62, 130, 60, 1, 22, 184, 28, 231].

It gets encrypted using:

arr2str(arr: number[]): string {
    var result = '';
    for (var i = 0; i < arr.length; i++) {
        result += String.fromCharCode(arr[i]);
    }
    return result;
}

encryptRSA(data: number[], key: string): string {
    var enc = new window.JSEncrypt();
    enc.setPublicKey(key);
    return enc.encrypt(this.arr2str(data));
}

The key is the PKCS#1 public key resulting of the server's private key. We will be using key

-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCyJ824ZiwMiCMrrHrDq1IKLwL8\nQWg+ZzwMprrG85k0nxEB8ZJn+s2lXhS4pOE0Nu6I9XiXjtyDbnT8kQvaWLve593v\nDzC16wP9IKrAdmeV9CExMzKAHbFSvNTTn3TWjaKy9OnH+7Uv/VVn63AQZXaqvY/W\nbPVdTKn4Nx7vl+laOwIDAQAB\n-----END PUBLIC KEY-----

-----BEGIN RSA PRIVATE KEY----- MIICXAIBAAKBgQCyJ824ZiwMiCMrrHrDq1IKLwL8QWg+ZzwMprrG85k0nxEB8ZJn +s2lXhS4pOE0Nu6I9XiXjtyDbnT8kQvaWLve593vDzC16wP9IKrAdmeV9CExMzKA HbFSvNTTn3TWjaKy9OnH+7Uv/VVn63AQZXaqvY/WbPVdTKn4Nx7vl+laOwIDAQAB AoGASP3b4HgkBgJk/ojNR4vSsg9u1rFpp1+ej8Rj9A1sMM4XJse151ovlVhFfx02 k7EJ7B0+ikHjAQppbe1zgMMoPUuDOQc9VF2A2Tsf71kMagbQpNrLNiTIu6DNwzHI ivubmYBs73s2MyZmK7G8D/QRDs0qQNXdUfAKMBIUh9wQj0kCQQD5Urh18NWmW7w8 4MDFmjdalSbE9Dg38mfrlUne0KSCvwyX2zkoh/uc1eB+hqONwDkuw8VLTBgxDm+L 7jwOlmiVAkEAtu0uNEewMHi1oCIvRoS1n/UDqEHzuwFuxg+cNwAKJoN1ljqKIfqa jFLGawkyHIK2fLhP8OSQeyDi3kSoIMJzjwJAT0737FRqsdt2emsIBxNyTjcpuPby tyE921uGvwDhg9GgAOI0QWdYK2CBY94SQrIFvpF5veT7wQcVho6GviEsLQJABGj7 cC86RDDk0BOC6ERSzKRvjiLo6V1Demrt7TWHCR6qOxD2O5N7Hl7wgawbFSzhkWgw JTKdeRp13b3x/7gwaQJBAOpgGkEJKcwRFdamFYZwMGbueqkpqG/AmfNXblrOv70N CkB9YP3skoZ69+vFr1TJXfz23lHpwQdPkRXhjlc/gls= -----END RSA PRIVATE KEY-----

The server receives this encrypted data. It appears that JSEncrypt b64_encodes it, so we have to decrypt using

openssl_private_decrypt(base64_decode($result_obj['data']), $decrypted, $pkcs_private_key);

However, I receive the byte array:

[194, 182, 13, 97, 94, 194, 164, 102, 194, 129, 70, 195, 128, 52, 94, 65, 195, 179, 194, 190, 57, 48, 194, 153, 194, 161, 46, 32, 122, 64, 53, 195, 173, 62, 194, 130, 60, 1, 22, 194, 184, 28, 195, 167].

Maybe this has something to do with arr2str function of JS. However, I cannot see how not to use it, as JSEncrypt expects to encrypt a string. I thought this function did not modify the bytes...

For the sake of completeness of the example, JSEncrypt returns the encoded data:

E728nXaCUUSTzuGLB5QIkodddyUMUMR0rEM5Ad7qL3SEtGJVukMjsQt7NAaRyXz1P3n2qK/iBGcuUBy2bPg5pTwk1twVZc2BzXueZYcKxxOby8AkNTgF9YMPlh1FMjD5c0UAiwcb7DnykvbsulG4h+FlxEy+28eMTfRvjZmpq+4=

This is what openssl_private_decrypt tries to decrypt.

base64_decode($decrypted) yields a string of length 6, whose bytes are not even the first ones I am looking for, sadly. But maybe you meant something else? — Ottunger
Well, I've checked with strlen, as well as unpack("C*", $decrypted) that returns an array of length 6, and allows me to check byte per byte — Ottunger
The bytes I expect: [182, 13, 97, 94, 164, 102, 129, 70, 192, 52, 94, 65, 243, 190, 57, 48, 153, 161, 46, 32, 122, 64, 53, 237, 62, 130, 60, 1, 22, 184, 28, 231], the one I receive: [194, 182, 13, 97, 94, 194, 164, 102, 194, 129, 70, 195, 128, 52, 94, 65, 195, 179, 194, 190, 57, 48, 194, 153, 194, 161, 46, 32, 122, 64, 53, 195, 173, 62, 194, 130, 60, 1, 22, 194, 184, 28, 195, 167] — Ottunger
And count returns but the number 1, I suppose it looks for the number of dimensions of an array. — Ottunger

Ottunger Ottunger · Accepted Answer · 2016-09-07T06:02:12

For the posterity, I found the problem. Indeed it lies in the padding not compatible betwwen both libraries.

Therefore I set the decryption mode to "no padding" in PHP:

openssl_private_decrypt(base64_decode($result_obj['data']), $decrypted, $pkcs_private_key, OPENSSL_NO_PADDING);

And I have turned the unpadding done in JS in PHP:

function pkcs1unpad2($b, $bits = 4096) {
    $i = 0;
    $n = ($bits + 7) >> 3;
    $l = strlen($b);
    while($i < $l && ord($b[$i]) == 0)
        ++$i;
    if(ord($b[$i]) != 2)
        return null;
    ++$i;
    while(ord($b[$i]) != 0)
        if(++$i >= $l)
            return null;
    $ret = "";
    while(++$i < $l) {
        $c = ord($b[$i]) & 255;
        if($c < 128) {
            $ret .= chr($c);
        } elseif(($c > 191) && ($c < 224)) {
            $ret .= chr((($c & 31) << 6) | (ord($b[$i+1]) & 63));
            ++$i;
        } else {
            $ret .= chr((($c & 15) << 12) | ((ord($b[$i+1]) & 63) << 6) | (ord($b[$i+2]) & 63));
            $i += 2;
        }
    }
    return $ret;
}

pcks1unpad2($decrypted)

is what I expect!

OpenSSL decrypt JSBN encrypted

1 Answers