GZipStream reads beyond the end of my data parts

Question

My program downloads via HttpWebRequest one file which consists of several stringed up gzip-compressed files/parts.

So when I decompress my responseStream, the GZipStream closes itself after each file part. That is not a big problem, since I can create a new one for each file, but the thing is: The GZipStream reads beyond each file, i.e. the beginning of the next file.

This is a problem, since I can't apply Seek() to my responseStream to return to the offset of the next file, so this next file is basically lost.

The most obvious solution to me was to copy the NetworkStream into a MemoryStream before decompression. But I don't want the whole file loaded into memory, not even the file parts, only a defined buffer size (f.e. 512kB).

==================EDITED====================

My new solution, thanks to Tarik

============================================

Download Process:

  using (DownloadStream dlStream = new DownloadStream(responseStream, file.compressedSize))
  using (GZipStream zip = new GZipStream(dlStream, CompressionMode.Decompress, true))
  {
       await zip.CopyToAsync(fs);
  }

DownloadStream class:

class DownloadStream : Stream
{
    Stream strm;
    int len;
    int pos;

    public DownloadStream(Stream netStream, int fileSize)
    {
        strm = netStream;
        len = fileSize;
        pos = 0;
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int rest = len - pos;
        int nRead;

        if (count > rest)
        {
            nRead = strm.Read(buffer, 0, rest);
        }
        else
        {
            nRead = strm.Read(buffer, 0, count);
        }
        pos += nRead;

        return nRead;
    }

    public override bool CanRead
    {
        get
        {
            return true;
        }
    }

//...add all other must-overrideables throwing NotImplementedException.

}

And that's already all I needed. As one can see, it also supports the asynchronous decompression task and uses almost no memory. Thank you very much vor this simple solution Tarik! :)

Can you determine in advance the size of each compressed file? — Tarik
Sure, when you take a look into the code there is file.compressedSize. Also I saved all the offsets/seek points of the file parts within the stream and the uncompressed size as well. — Showdown
6 years, can't believe this didn't get an upvote. Very helpful, thanks — Thomas Williams

Tarik Tarik · Accepted Answer · 2015-02-10T20:15:23

I would create a class that implements a stream. I would pass to this class constructor the responsestream and the size of data it should read before indicating an EOF. Internally this stream would read from the underlying responsestream and would stop reading upon hitting the specified limit. I would instanciate these stream classes one at a time with the desired size and pass them to the GZipStream.

GZipStream reads beyond the end of my data parts

1 Answers