Using ObjectOutputStream to send a byte array of a public key, encrypt something, and send the encrypted version back

Question

So I have a server side public key and private key, my aim is to send the client the public key, the client will encrypt a string with the key, then send the bytes through a stream, and the server will decrypt the byte array.

Exception:

javax.crypto.BadPaddingException: Decryption error

Code:

Sending the encoded key.

                    handler.getOos().writeObject(publicKey.getEncoded());
                    handler.getOos().flush();

Receiving the byte array (of the encoded key):

                        Object o = ois.readObject();
                        if (o instanceof byte[]) {
                            JChat.get().setServerPublicKey(KeyFactory.getInstance("RSA").generatePublic(new X509EncodedKeySpec((byte[]) o)));
                            JChat.get().go();
                        }

The go() method (here I use a DataOutputStream to send the byte array):

public void go() {
    String text = "hello darkness my old friend";
    byte[] encrypted = encrypt(text, serverPublicKey);
    try {
        handler.getDos().write(encrypted);
        handler.getDos().flush();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Reading the byte array, on the server side:

                        int count = dis.available();
                        byte[] in = new byte[count];
                        dis.readFully(in);
                        System.out.println(Server.decrypt(in, Server.get().getPrivateKey()));

The decryption method throws this exception:

javax.crypto.BadPaddingException: Decryption error
at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:380)
at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:291)
at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:363)
at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:389)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
at com.archiepking.Server.decrypt(Server.java:97)
at com.archiepking.net.ClientHandler$1.run(ClientHandler.java:44)
at java.lang.Thread.run(Thread.java:745)

Any suggestions as to what I am doing wrong? Please note:

Dos = DataOutputStream Dis = DataInputStream Oos = ObjectOutputStream Ois = ObjectInputStream

I am using two different sockets, one for sending objects and one for datatypes (as my chat application will need both).

What can I do to fix this error?

FURTHER INFORMATION: Generation of keys:

KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
        keyPairGenerator.initialize(1024);
        KeyPair keyPair = keyPairGenerator.genKeyPair();
        byte[] publicKeyBytes = keyPair.getPublic().getEncoded();
        FileOutputStream fosPublic = new FileOutputStream("public");
        fosPublic.write(publicKeyBytes);
        fosPublic.close();

        byte[] privateKeyBytes = keyPair.getPrivate().getEncoded();
        FileOutputStream fosPrivate = new FileOutputStream("private");
        fosPrivate.write(privateKeyBytes);
        fosPrivate.close();

        publicKey = keyPair.getPublic();
        privateKey = keyPair.getPrivate();

Answer 1

If you are using an ObjectOutputStream why bother converting the public key to a byte array using getEncoded? You can serialize the object directly. e.g. handler.getOos().writeObject(publicKey); Or if you have to use the encoded version, then remove the ObjectOutputStream and use ByteArrayOutputStream instead.

Answer 2

The problem is that you are using DataInputStream.available() to determine how many bytes to read. That method does not do what you apparently think that it does.

From the Javadoc of this method:

Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking by the next caller of a method for this input stream. The next caller might be the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.

It just returns the number of bytes that can be read without blocking, which can be far less than the actual number of bytes that you sent, especially if you are using network Sockets to send/receive that data.

The solution:

• before writing the bytes, write an int with the writeInt method that contains the number of bytes that you're writing
• before reading the bytes, call readInt to read the number of bytes that will follow, and construct a byte array of the right length from that number.