Why don't DataOutputStream.writeChars(str) and String(byte[]) use the same encoding?

Question

I'm writing some marshaling/unmarshaling routines for a class project and am a bit perplexed about Java's default behavior in this case. Here are my "naive" subroutines for writing and reading strings to and from byte streams:

protected static void write(DataOutputStream dout, String str)
        throws IOException{
    dout.writeInt(str.length());
    dout.writeChars(str);
}

protected static String readString(DataInputStream din)
        throws IOException{
    int strLength = 2*din.readInt(); // b/c there are two bytes per char
    byte[] stringHolder = new byte[strLength];
    din.read(stringHolder);
    return new String(stringHolder);
}

Unfortunately, this simply doesn't work; the characters are written in UTF-16 format by default, but String(byte[]) seems to assume that each byte will contain a character, and since ASCII characters all start with a 0 byte in UTF-16, the constructor appears to just give up and return an empty string. The solution is to change readString to specify that it must use UTF-16 encoding:

protected static String readString(DataInputStream din)
        throws IOException{
    int strLength = 2*din.readInt();
    byte[] stringHolder = new byte[strLength];
    din.read(stringHolder);
    return new String(stringHolder, "UTF-16");
}

My question is, why is this necessary? Since Java uses UTF-16 for strings by default, why wouldn't it assume that UTF-16 is being used when reading chars from bytes? Or, alternatively, why wouldn't it just encode the chars as bytes in the first place by default? In short, why don't the default behaviors of the writeChars() method and the String(byte[]) constructor parallel each other?

How about using DataOutputStream.writeUTF() and DataInputStream.readUTF()? — Boris the Spider
....wow, that's way more convenient. Thank you. I'm still curious about why my method didn't work as expected, but I think I'll switch to using those methods instead. — Kyle Strand

Boris the Spider Boris the Spider · Accepted Answer · 2013-02-18T00:11:29

The issue is you are writing using the underlying char[] which is essentialy a byte[] that represents a UTF-16 representation of a string, see the javadoc.
You are then reading using the String(byte[] bytes) constructor, which is designed for reading data encoded with the system default encoding, in your case presumably this is UTF-8.
You need to be consistent, in fact the DataOutputStream.writeUTF() and DataInputStream.readUTF() functions are designed especially for this.
If you want use the underlying byte[] for some reason you can get the UTF-8 representation of the String easily using String.getBytes("UTF-8"), again, see the javadoc.
To simplify matters you could just use an ObjectOutputStream and an ObjectInputStream and that would serialize the actual String to the stream rather than just its char[] representation.

Why don't DataOutputStream.writeChars(str) and String(byte[]) use the same encoding?

2 Answers