java.security.InvalidKeyException: Wrong key size during decryption

Question

During decryption, I am getting a mix of either "Wrong key size" or "Given final block not properly padded", dependent on which OS I am running.

On Win7, using IBMJCE or SUNJCE (both Java8), decryption fails 25% of the time:

javax.crypto.BadPaddingException: Given final block not properly padded at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:811) at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:676) at com.sun.crypto.provider.DESedeCipher.engineDoFinal(DESedeCipher.java:294) at javax.crypto.Cipher.doFinal(Cipher.java:2087)

On mac, using SUNJCE, decryption fails 100% of the time:

java.security.InvalidKeyException: Wrong key size at com.sun.crypto.provider.DESedeCrypt.init(DESedeCrypt.java:69) at com.sun.crypto.provider.CipherBlockChaining.init(CipherBlockChaining.java:91) at com.sun.crypto.provider.CipherCore.init(CipherCore.java:469) at com.sun.crypto.provider.DESedeCipher.engineInit(DESedeCipher.java:197) at javax.crypto.Cipher.implInit(Cipher.java:791) at javax.crypto.Cipher.chooseProvider(Cipher.java:849) at javax.crypto.Cipher.init(Cipher.java:1348)

Using DESEde, i believe the key size needs to be 24, I can see that on windows, after decryption, it is always 24 bytes, while on mac, it is never 24 bytes.

Here's the starting point. The exception is always throw during decryptWithSymmetricKey. Note, i short-cycled much of the code (specific for DESede), couldn't narrow it down any further (very new to security space).

public static void runtest() throws Exception {
    String symmPad = "DESede/CBC/PKCS5Padding";
    String asymmPad = "RSA/ECB/OAEPWithSHA-256AndMGF1Padding";
    String pubKeyFp = "somekey";
    String stringToEncrypt = "abcdefg";

    KeyGenerator kgen = KeyGenerator.getInstance(DESEDE);
    kgen.init(112);
    SecretKey secKey = kgen.generateKey();

    String encryptedKey = encryptSymmetricKey(secKey, asymmPad);
    String encryptedData = encryptDataWithSymmetricKey(stringToEncrypt, secKey, symmPad);

    String decryptedKey = decryptWithPrivateKey(encryptedKey, pubKeyFp, asymmPad);
    String decryptedData = decryptWithSymmetricKey(encryptedData, decryptedKey, symmPad);
}

Here we encrypt the symmetric key, key length is 24 in both environments

private static String encryptSymmetricKey(SecretKey secKey, String asymmPadding) throws Exception {
    KeyPair keyPair = getKeyPair("self4");
    Cipher cipher = Cipher.getInstance(asymmPadding);

    OAEPParameterSpec ospec = new OAEPParameterSpec(SHA256, MGF1, MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
    cipher.init(Cipher.ENCRYPT_MODE, keyPair.getPublic(), ospec);

    String secKeyEncoded = new String(secKey.getEncoded());
    byte[] encrypted = cipher.doFinal(secKeyEncoded.getBytes());

    char[] encoded = Hex.encodeHex(encrypted);
    return new String(encoded);
}

Here we encrypt our string with the symmetric key

private static String encryptDataWithSymmetricKey(String data, SecretKey secretKey, String symmPadding) throws Exception {
    Cipher cipher = Cipher.getInstance(symmPadding);
    IvParameterSpec iv = new IvParameterSpec(new byte[8]);
    cipher.init(Cipher.ENCRYPT_MODE, secretKey, iv);

    byte[] encrypted = cipher.doFinal(data.getBytes());
    char[] encoded = Hex.encodeHex(encrypted);
    return new String(encoded);
}

Decrypting and decoding the symmetric key is when i first see variable length key on mac.

public String decryptWithPrivateKey(String encryptedData, String pubKeyFp, String asymmPadding) throws Exception {
    loadKeystores();
    String alias = fingerPrintAliasMap.get(pubKeyFp);

    KeyPair keyPair = getKeyPair(alias);
    Cipher cipher = Cipher.getInstance(asymmPadding);

    OAEPParameterSpec oParamSpec = new OAEPParameterSpec(SHA256, MGF1, MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
    cipher.init(Cipher.DECRYPT_MODE, keyPair.getPrivate(), oParamSpec);

    byte[] decoded = Hex.decodeHex(encryptedData.toCharArray());
    byte[] decrypted = cipher.doFinal(decoded);
    System.out.println("decoded and decrypted key length: " + decrypted.length); // 24 on windows, random on mac

    return new String(Hex.encodeHex(decrypted));
}

The failures happen here - on windows, it fails during cipher.doFinal 25% of the time, on mac, it fails at the cipher.init 100% of the time.

public String decryptWithSymmetricKey(String encryptedHexData, String symmKey, String symmPadding) throws Exception {

    byte[] key = Hex.decodeHex(symmKey.toCharArray());
    SecretKey skeySpec = new SecretKeySpec(key, DESEDE);
    IvParameterSpec iv = new IvParameterSpec(new byte[8]);

    Cipher cipher = Cipher.getInstance(symmPadding);
    cipher.init(Cipher.DECRYPT_MODE, skeySpec, iv); // mac: Wrong key size

    byte[] decoded = Hex.decodeHex(encryptedHexData.toCharArray());
    byte[] deciphered = cipher.doFinal(decoded); // windows: Given final block not properly padded

    return new String(deciphered);
}

I assume if I solve this on mac, it should solve it on windows as well.

Artjom B. Artjom B. · Accepted Answer · 2015-09-18T17:15:07

The problem are the following two lines in the encryptDataWithSymmetricKey method:

String secKeyEncoded = new String(secKey.getEncoded());
byte[] encrypted = cipher.doFinal(secKeyEncoded.getBytes());

Since you generated your DES key randomly, it will most likely contain characters that are not printable. By calling new String(bytes), you're silently dropping those unprintable characters which breaks your key. Use instead:

byte[] encrypted = cipher.doFinal(secKey.getEncoded());

Other considerations:

Also, I'm not sure if your key size is correct if you generate 112 bits. I think Java expects the key size including the parity bits which would be 128 for that security level. You probably should use 192 bits which gives a much better security level (only 112 bit for DESede with a 192 bit key = 168 bit without counting parity bits).

You shouldn't use DESede. Even AES-128 provides better security.

You please don't use a static IV of zero bytes. Use a strong randomness source like SecureRandom to generate an IV for every encryption. It doesn't have to be secret, so you can simply prepend it to the ciphertext and not forget to slice it off before decryption.

You're not authenticating your symmetric ciphertexts. You either need to use an encrypt-then-MAC approach with a strong MAC like HMAC-SHA256 or use an authenticated mode of operation like GCM or EAX.

java.security.InvalidKeyException: Wrong key size during decryption

1 Answers

Other considerations: