Encrypting AES key with RSA public key

Question

I'm writing a small application for transferring files, more or less as a way to learn more of the programmatic encryption underpinnings. The idea is to generate an RSA keypair, exchange public keys, and send the AES iv and key over for further decryption. I want to encrypt the AES key with the receivers RSA public key, like so:

// encode the SecretKeySpec
private byte[] EncryptSecretKey ()
{
    Cipher cipher = null;
    byte[] key = null;

    try
    {
        cipher = Cipher.getInstance("RSA/ECB/NOPADDING");
        // contact.getPublicKey returns a public key of type Key
        cipher.init(Cipher.ENCRYPT_MODE, contact.getPublicKey() );
        // skey is the SecretKey used to encrypt the AES data
        key = cipher.doFinal(skey.getEncoded());
    }
    catch(Exception e )
    {
        System.out.println ( "exception encoding key: " + e.getMessage() );
        e.printStackTrace();
    }
    return key;
}

I then write out the key value to the receiver, and decrypt it like so:

private SecretKey decryptAESKey(byte[] data )
{
    SecretKey key = null;
    PrivateKey privKey = null;
    Cipher cipher = null;

    System.out.println ( "Data as hex: " + utility.asHex(data) );
    System.out.println ( "data length: " + data.length );
    try
    {
        // assume this loads our private key
        privKey = (PrivateKey)utility.loadLocalKey("private.key", false);

        cipher = Cipher.getInstance("RSA/ECB/NOPADDING");
        cipher.init(Cipher.DECRYPT_MODE, privKey );
        key = new SecretKeySpec(cipher.doFinal(data), "AES");

        System.out.println ( "Key decrypted, length is " + key.getEncoded().length );
        System.out.println ( "data: " + utility.asHex(key.getEncoded()));
    }
    catch(Exception e)
    {
        System.out.println ( "exception decrypting the aes key: " + e.getMessage() );
        e.printStackTrace();
        return null;
    }

    return key;
}

In console, on the other side, I get this as output:

read_bytes for key: 16
data length: 16
Data as hex: <hex string>
Key decrypted, length is 256
java.security.InvalidKeyException: Invalid AES key length: 256 bytes

Furthermore, if I create a byte array of size 16 and put the cipher.doFinal(data) output into it, the array is seemingly resized to 256 bytes (.length says so, at least). Why would this be, and further, what am I doing incorrectly?

edit
I solved this, and thought I'd post the issue in case someone runs into this. The problem, it turns out, was the RSA/ECB/NOPADDING. For some odd reason it was screwing up my creation of the SecretKey when I transferred it over to the client. It might have something to do with how I'm generating keypairs (i'm using getInstance("RSA") for that), but I'm not entirely sure.

Hi hat, please post your complete answer as an answer instead of editing it in your question (you tell us what the problem was, not the answer). Then you can accept your own answer after a while. Or accept mine, which explains why you should use "RSA/ECB/PKCS1Padding" instead of "RSA/ECB/NoPadding"... — Maarten Bodewes
Note that you should use integrity protection (e.g. a signature) when you create an online protocol that performs encryption. Even RSA encryptiom may be succeptable to e.g. padding oracle attacks. Not having integrity protection is a common mistake, although this issue is limited if you use PKCS#1 v1.5 (implied by "RSA/ECB/PKCS1Padding"). — Maarten Bodewes
It's my understanding that the core javax.crypto library does not support any other form of block cipher, thus I'm sort of stuck with it. — hat
At minimum it's possible to construct CTR from ECB. You just need to be careful to create a good nonce. But I'd be surprised if the library didn't at least support CBC mode. — CodesInChaos

hat hat · Accepted Answer · 2012-03-18T20:32:39

As owlstead mentioned, you cannot just use "raw" RSA without padding for encryption/decryption. For one it is very insecure, and for another, the Java libraries do not even support it. Below is the working code for the encryption/decryption of the AES key using RSA keypairs.

private byte[] EncryptSecretKey ()
{
    Cipher cipher = null;
    byte[] key = null;

    try
    {
        // initialize the cipher with the user's public key
        cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
        cipher.init(Cipher.ENCRYPT_MODE, contact.getPublicKey() );
        key = cipher.doFinal(skey.getEncoded());
    }
    catch(Exception e )
    {
        System.out.println ( "exception encoding key: " + e.getMessage() );
        e.printStackTrace();
    }
    return key;
}

Decryption of the AES key looks like this:

private SecretKey decryptAESKey(byte[] data )
{
    SecretKey key = null;
    PrivateKey privKey = null;
    Cipher cipher = null;

    try
    {
        // this is OUR private key
        privKey = (PrivateKey)utility.loadLocalKey(
                                ConfigFrame.privateKeyLocation, false);

        // initialize the cipher...
        cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
        cipher.init(Cipher.DECRYPT_MODE, privKey );

        // generate the aes key!
        key = new SecretKeySpec ( cipher.doFinal(data), "AES" );
    }
    catch(Exception e)
    {
        System.out.println ( "exception decrypting the aes key: " 
                                               + e.getMessage() );
        return null;
    }

    return key;
}

Encrypting AES key with RSA public key

3 Answers