Chained GZipStream/DeflateStream and CryptoStream (AES) breaks when reading

Question

I want to compress and then encrypt my data, and for improved speed (by not having to write to byte arrays and back) decided to chain the streams used for compression and encryption together.

It works perfectly when I write (compress and encrypt) the data, but when I try to read the data (decompress and decrypt), the Read operation breaks - simply calling Read once reads exactly 0 bytes, because the first Read always returns 0. Looping as in the below code almost works, except that at a certain point, Read stops returning anything > 0 even though there's still data to be read.

Everything before those last few bytes are decompressed and decrypted perfectly.

The number of bytes left when that happens remains the same for the same plaintext; for example, it's always 9 bytes for a certain string, but always 1 byte for another.

The following is the relevant encryption and decryption code; any ideas as to what could be going wrong?

Encryption:

// Create the streams used for encryption.
using (MemoryStream msEncrypt = new MemoryStream())
{
    using (ICryptoTransform encryptor = aes.CreateEncryptor(aes.Key, aes.IV))
    using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
    using (DeflateStream zip = new DeflateStream(csEncrypt, CompressionMode.Compress, true))
    {
        zip.Write(stringBytes, 0, stringBytes.Length);
        csEncrypt.FlushFinalBlock();

Decryption:

// Create the streams used for decryption.
using (MemoryStream msDecrypt = new MemoryStream())
{
    // Writes the actual data (sans prepended headers) to the stream
    msDecrypt.Write(stringBytes, prependLength, stringBytes.Length - prependLength);
    // Reset position to prepare for read
    msDecrypt.Position = 0;
    // init buffer to read to
    byte[] buffer = new byte[originalSize];

    using (ICryptoTransform decryptor = aes.CreateDecryptor(aes.Key, aes.IV))
    using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
    using (DeflateStream zip = new DeflateStream(csDecrypt, CompressionMode.Decompress))
    {
        // Hangs with "offset" at a small, deterministic number away from originalSize (I've gotten 9 less and 1 less for different strings)
        // Loop fixed as per advice
        int offset = 0;
        while (offset < originalSize)
        {
            int read = zip.Read(buffer, offset, originalSize - offset);
            if (read > 0)
                offset += read;
            else if (read < 0)
                Console.WriteLine(read); // Catch it if it happens.
        }
        // Hangs with "left" at a small, deterministic number (I've gotten 9 and 1 for different strings)
        /*
        for (int left = buffer.Length; left > 0; )
            left -= zip.Read(buffer, 0, left);
        */

Solution (Modification to Encryption):

// Create the streams used for encryption.
using (MemoryStream msEncrypt = new MemoryStream())
{
    using (ICryptoTransform encryptor = aes.CreateEncryptor(aes.Key, aes.IV))
    using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
    {
        using (DeflateStream zip = new DeflateStream(csEncrypt, CompressionMode.Compress, true))
            zip.Write(stringBytes, 0, stringBytes.Length);
        //Flush after DeflateStream is disposed.
        csEncrypt.FlushFinalBlock();

You are overwriting from byte 0 each time. You will need to add an offset variable that starts at 0 and increases by the bytes read each time - otherwise you keep overwriting the start of the buffer. — Marc Gravell
Thanks for catching that; you're right there - I replaced the loop with the following: int offset = 0; while (offset < originalSize) { offset += zip.Read(buffer, offset, originalSize - offset); } Unfortunately, that didn't solve the problem; I still get that 1 byte left and it hangs. (In retrospect, the decypted data wouldn't have been perfect if the faulty loop were the culprit). — cervellous
yes, when I saw that it wouldn't fix the issue of hanging bytes I removed it as an answer and added it as a comment instead. I just thought it might save you a second round of head-scratching wondering why the buffer was borked :) — Marc Gravell
This might be zany... Can you try adding a bytesRead variable, that catches the value of Read, and then only run the logic etc if this is positive? I wonder if it is returning a negative at the end (iirc this would be a valid terminator as from memory it just states non-positive is end-of-data, although I haven't checked the documentation on this lately) — Marc Gravell
Tried that, and it never returns negative :< The current docs seem to indicate it only returns the number of bytes read or 0 for finishing - although that's kinda odd, since I always get 0 on the first call to Read (the subsequent calls return positives until the hang). — cervellous

svick svick · Accepted Answer · 2011-07-23T19:01:10

The problem lies in the following line:

csEncrypt.FlushFinalBlock();

If you remove that, the code will work.

The reason is that when you write to DeflateStream, not all data is written to the underlying stream. That happens only when you call Close() or Dispose() explicitly or implicitly by leaving the using block.

So in your code, this happens:

You Write() all of the data to the DeflateStream, which in turn writes most of the data to the underlying CryptoStream.
You call csEncrypt.FlushFinalBlock(), which closes the CryptoStream.
You leave the using block of the DeflateStream, which tries to write the rest of the data to the already closed CryptoStream.
You leave the using block of the CryptoStream, which would call FlushFinalBlock(), if it wasn't called already.

The correct sequence of events is:

You Write() all of the data to the DeflateStream, which in turn writes most of the data to to the underlying CryptoStream.
You leave the using block of the DeflateStream, which writes the rest of the data to the already closed CryptoStream.
You leave the using block of the CryptoStream, which calls FlushFinalBlock().

Although I would expect that writing to a closed stream would fail with an exception. I'm not sure why that doesn't happen.

Chained GZipStream/DeflateStream and CryptoStream (AES) breaks when reading

2 Answers