How to zlib compress a QByteArray?

Question

I would like to maintain interoperability between every other application on the planet (including web applications) when compressing text. Since qCompress and qUncompress seem to go against the grain, I'm trying to use zlib directly from my Qt application.

I will accept the simplest (most minimal) answer that shows me how to use the zlib library with a QByteArray directly OR modify the output of qCompress so that it can be used outside of a Qt application.

Here's my embarrassing attempt:

QByteArray tdata = QString("Oh noes!").toUtf8();
QByteArray cdata;
uLongf len = 12 + 1.002*tdata.length();
compress(&cdata, &len, &tdata, tdata.length());

And the error:

error: cannot convert 'QByteArray*' to 'Bytef*' for argument '1' to 'int compress(Bytef*, uLongf*, const Bytef*, uLong)'

Then I tried using QByteArray::constData()

compress(cdata.constData(), &len, &tdata, tdata.length());

But got the following error:

error: invalid conversion from 'const char*' to 'Bytef*'

I have no idea what a Bytef is so I start looking in the zlib sources to investigate. But all I can find for this is in QtSources/src/3rdparty/zlib/zconf.h

# define Bytef                 z_Bytef

So now I'm just lost.

Answer 1

Based on this note in qUncompress, I think it's pretty easy.

Note: If you want to use this function to uncompress external data that was compressed using zlib, you first need to prepend a four byte header to the byte array containing the data. The header must contain the expected length (in bytes) of the uncompressed data, expressed as an unsigned, big-endian, 32-bit integer.

So you can probably just compress it like this:

QByteArray tdata = QString("Oh noes!").toUtf8();
QByteArray compressedData = qCompress(tdata);
compressedData.remove(0, 4);

Answer 2

Here is some code I once wrote which gets as input a pointer to a byte array, the number of bytes to compress and the compression level and then uses zlib to compress the input. The result is returned in a string.

 enum compressionLevel
 {
    clFast,
    clSmall,
    clDefault
 };

 const size_t ChunkSize = 262144; //256k default size for chunks fed to zlib

 void compressZlib(const char *s, size_t nbytes, std::string &out, compressionLevel l /*= clDefault*/ )
 {
    int level = Z_DEFAULT_COMPRESSION;
    switch (l)
    {
    case clDefault:
        level = Z_DEFAULT_COMPRESSION; break;
    case clSmall:
        level = Z_BEST_COMPRESSION; break;
    case clFast:
        level = Z_BEST_SPEED; break;
    };

    z_stream strm;
    strm.zalloc = Z_NULL;
    strm.zfree = Z_NULL;
    strm.opaque = Z_NULL;
    int ret = deflateInit(&strm, level);
    if (ret != Z_OK)
    {
        throw std::runtime_error("Error while initializing zlib, error code "+ret);
    }
    size_t toCompress = nbytes;
    char *readp = (char*)s;
    size_t writeOffset = out.size();
    out.reserve((size_t)(nbytes * 0.7));
    while (toCompress > 0)
    {
        size_t toRead = std::min(toCompress,ChunkSize);
        int flush = toRead < toCompress ? Z_NO_FLUSH : Z_FINISH;
        strm.avail_in = toRead;
        strm.next_in = (Bytef*)readp;
        char *writep = new char[ChunkSize];
        do{
            strm.avail_out = ChunkSize;
            strm.next_out = (Bytef*)writep;
            deflate(&strm, flush);
            size_t written = ChunkSize - strm.avail_out;
            out.resize(out.size() + written);
            memcpy(&(out[writeOffset]), writep, written);
            writeOffset += written;
        } while (strm.avail_out == 0);
        delete[] writep;
        readp += toRead;
        toCompress -= toRead;
    }
    (void)deflateEnd(&strm);
 }

Maybe this helps you to solve your problem, I guess using the cdata.constData() you can directly call this function

Answer 3

Just to help you out with the last section of your question here:

I have no idea what a Bytef is so I start looking in the zlib sources to investigate.

For the definitions of Byte and Bytef, look at lines 332 and 333 of zconf.h, as well as line 342:

332    #if !defined(__MACTYPES__)
333    typedef unsigned char  Byte;  /* 8 bits */
...
338    #ifdef SMALL_MEDIUM
339       /* Borland C/C++ and some old MSC versions ignore FAR inside typedef */
340    #  define Bytef Byte FAR
341    #else
342       typedef Byte  FAR Bytef;    

The definition of FAR is for mixed-mode MSDOS programming, otherwise it is not defined as anything (see lines 328-330 of zconf.h).

Thus the zlib typedefs Bytef and Byte are basically the same as unsigned char on most platforms. Therefore you should be able to-do the following:

QByteArray tdata = QString("Oh noes!").toUtf8();
QByteArray cdata(compressBound(tdata.length()), '\0');
uLongf len = compressBound(tdata.length());
compress(reinterpret_cast<unsigned char*>(cdata.data()), &len, 
         reinterpret_cast<unsigned char*>(tdata.data()), tdata.length());