boost::iostreams decompression returns an empty stream

Question

I have a seemingly basic issue that I've been tackling for the past couple of days and can't seem to find an answer for. I have read multiple answers for similar issues, but none seems to work for me.

I have been simply trying to use boost's compression / decompression functions, but I figure I must be using it incorrectly. My goal is quite simple - create a string, compress it and write to file, read the compressed file, decompress it and print to std::cout (or any other stream for that matter). I used the following code:

// write files
std::stringstream str;
str << "SampleString";

std::ofstream outFile_raw("rawFile");
std::ofstream outFile_comp("compressedFile);

boost::iostreams::filtering_istream filteredString;
filteredString.push(boost::iostreams::zlib_compressor());
filteredString.push(str);

boost::iostreams::copy(str, outFile_raw);
boost::iostreams::copy(filteredString, outFile_comp);

\\read files
std::ifstream inFile_raw("rawFile");
std::ifstream inFile_comp("compressedFile);

boost::iostreams::filtering_istream filteredInput;
filteredInput.push(boost::iostreams::zlib_decompressor());
filteredInput.push(inFile_comp);

boost::iostreams::copy(inFile_raw, std::cout);
boost::iostreams::copy(filteredInput, std::cout);

The writing phase works as it should - two files are created, one with an expected size (stringLength bytes), and a smaller one. Upon reading, for some reason only the raw file is read successfully, while the compressed one prints nothing.

I have tried several methods in order to debug this issue:

Getting the stream length with seekg and tellg, in order to verify that the stream is indeed empty
using stream << filteredStream.rdbuf(); instead of boost::iostreams::copy
Using filtering_istreambuf insteaf of filtering_istream (what is the difference? When should I use each one?)
Using other compression algorithms, such as gzip and bzip2.
Adding inFile_comp.close(); before and after pushing it to the filtering stream (One approach does not help, the other yields an iostream stream error exception.

I feel like there is a basic concept I am missing, can anyone please help me figure it out?

EDIT: After further investigating the issue, I have come to an interesting situation- During zlib and gzip compression, the entire process is completed successfully for a small file (~5KB, which is compressed to ~200 bytes), yet an "iostream stream error" exception is thrown for a larger file (~55MB, compressed to ~320KB).

For bzip2 compression, the exception is thrown in both cases. Could it have something to do with the decompressor buffer size? I am getting pretty hopeless at this stage..

Thank you all in advance!

sehe sehe · Accepted Answer · 2017-03-30T11:57:10

You're using str twice. After using it the first time it is at end-of-stream, so nothing will be read the second time.

Besides this, either explicitly flush your streams and buffers, or limit the scope of the objects so it happens automatically:

Live On Coliru

#include <boost/iostreams/filter/zlib.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/copy.hpp>
#include <fstream>
#include <iostream>
#include <sstream>

int main() {
    // write files
    {
        std::stringstream str("SampleString");
        std::ofstream outFile_raw("rawFile");
        boost::iostreams::copy(str, outFile_raw);
    }
    {
        std::stringstream str("SampleString");
        std::ofstream outFile_comp("compressedFile");

        boost::iostreams::filtering_istream filteredString;
        filteredString.push(boost::iostreams::zlib_compressor());
        filteredString.push(str);

        boost::iostreams::copy(filteredString, outFile_comp);

        outFile_comp.flush();
        outFile_comp.close();
    }

    //read files
    {
        std::ifstream inFile_raw("rawFile");
        boost::iostreams::copy(inFile_raw, std::cout << "\n inFile_raw: ");
    }
    {
        std::ifstream inFile_comp("compressedFile");

        boost::iostreams::filtering_istream filteredInput;
        filteredInput.push(boost::iostreams::zlib_decompressor());
        filteredInput.push(inFile_comp);

        boost::iostreams::copy(filteredInput, std::cout << "\n filteredInput: ");
    }
}

Prints:

 inFile_raw: SampleString
 filteredInput: SampleString

boost::iostreams decompression returns an empty stream

1 Answers