C# networkstream compression - Sharpziplib, DotNetZip, gzipstream all give errors on my stream

Question

I have a pair of C# client-server programs that communicate using a networkstream. Everything works fine as it is without compression. Now I'd like to get the bandwidth-usage down, so I want to use a compressing wrapperstream around my networkstream.

I have tried SharpZipLib, DotNetZip, C#'s own GZipStream - but I can get none of them to work.

SharpZipLib has problems flushing, and applying the fix specified here: http://community.sharpdevelop.net/forums/p/7855/22139.aspx results in an exception "Header checksum illegal".

Using DotNetZip's DeflateStream results in a ZLibException("Bad state (invalid stored block lengths)");

GZipStream gives me a System.IO.InvalidDataException stating "The magic number in GZip header is not correct. Make sure you are passing in a GZip stream.".

The way I've implemented it is that everytime an array of byte has to be sent by my framework, I create a new Compression stream wrapper around the existing networkstream, write the bytes to the compression stream, and then flush, close & dispose it. This to make sure that each WriteMessage(byte[] blah) uses it's own state-independent compressionstream that will be flushed immediately. I've taken care to not let any of the streams close the original network stream.

                using (System.IO.Stream outputStream = CreateOutputStreamWrapper(_networkStream))
                {
                    outputStream.Write(messageBytes, 0, messageBytes.Length);
                    outputStream.Flush();
                    outputStream.Close();
                    outputStream.Dispose();
                }

Basicly, my DecompressionStream is created as follows (optionals commented out)

protected System.IO.Stream CreateInputStreamWrapper(System.IO.Stream inInputStream)
{
        //return new DeflateStream(inInputStream, CompressionMode.Decompress, true);
        //return new BZip2InputStream(inInputStream, true);
        return new GZipStream(inInputStream, System.IO.Compression.CompressionMode.Decompress, true);
}

and started as

_inputStream.BeginRead(_buffer, 0, _buffer.Length, new AsyncCallback(ReceiveCallback), null);

then in the ReceiveCallback, the data is read, the stream is flushed, closed and disposed:

            //Get received bytes count
            var bytesRead = _inputStream.EndRead(ar);
            _inputStream.Flush();
            _inputStream.Close();
            _inputStream.Dispose();

and immediately create a new inputStream by calling CreateInputStreamWrapper again.

So what's going on ? Since all compression-stream implementations are failing with errors that come down to "there's an error in the datastream" I have a hunch it must be me and my code. On the other hand, if I remove the compression and just use the networkstream there's no problem, which makes me think the problem must lie with the compression-code.

Does this sound familiar to anyone ? And while we're at it, does anyone know of any (other) compression stream implementations that are suited to wrap around a networkstream ?

You need a compression library that supports partial flush. The libraries you mentioned are, afaik, all built for compressing files, not continuous streams of data. — dtb
I agree. But I've tried the way of the "full flush", basicly using these libs to compress a single message and then flush, close and dispose. This should basicly make each network-message a completely separate entity, being created, written to, flushed and closed. But as anyone can see by my post here, this doesn't work, so if you have any tips on compression that would support continuous streams of data - please let me know :) — Pygmy
That should work. But you need to reassemble each message before decompressing it; you can't just read random chunks from the stream and throw them at the decompressor. A NetworkStream does not preserve message boundaries. — dtb
Seriously, F me. I was so tied up with the compression I failed to realize that OFCOURSE the resulting bytes read by EndRead were not a completed message. stupid, stupid, stupid. Thank you very much. — Pygmy

Pygmy Pygmy · Accepted Answer · 2012-03-13T08:26:48

Just in case anyone else ever reads this, DotNetZip's ZLib streams have a FlushMode flag that enables you to set up flushing compatible for networking stuff ('Sync' and 'Full' modes).

C# networkstream compression - Sharpziplib, DotNetZip, gzipstream all give errors on my stream

1 Answers