JavaScript implementation of Gzip [closed]

209
votes

I'm writing a Web application that needs to store JSON data in a small, fixed-size server-side cache via AJAX (think: Opensocial quotas). I do not have control over the server.

I need to reduce the size of the stored data to stay within a server-side quota, and was hoping to be able to gzip the stringified JSON in the browser before sending it up to the server.

However, I cannot find much in the way of JavaScript implementations of Gzip. Any suggestions for how I can compress the data on the client side before sending it up?

9
javascriptajaxcompressiongzip
You are sending it up to the server. That's why there are the notions of "upload" and "download". Maybe that's why you are getting answers that tell you "the server can do it". – Tomalak
A proper implementation of this is probably tricky, since javascript is single threaded. It would probably have to compress in batches, using setTimeout(), so that the UI doesn't lock up while compressing. – August Lilleaas
perhaps you could write your own compression algorithm – Captain kurO
@AugustLilleaas now you can use webworkers to do this :) – Captain Obvious

9 Answers

144
votes

Edit There appears to be a better LZW solution that handles Unicode strings correctly at http://pieroxy.net/blog/pages/lz-string/index.html (Thanks to pieroxy in the comments).

I don't know of any gzip implementations, but the jsolait library (the site seems to have gone away) has functions for LZW compression/decompression. The code is covered under the LGPL.

// LZW-compress a string
function lzw_encode(s) {
    var dict = {};
    var data = (s + "").split("");
    var out = [];
    var currChar;
    var phrase = data[0];
    var code = 256;
    for (var i=1; i<data.length; i++) {
        currChar=data[i];
        if (dict[phrase + currChar] != null) {
            phrase += currChar;
        }
        else {
            out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
            dict[phrase + currChar] = code;
            code++;
            phrase=currChar;
        }
    }
    out.push(phrase.length > 1 ? dict[phrase] : phrase.charCodeAt(0));
    for (var i=0; i<out.length; i++) {
        out[i] = String.fromCharCode(out[i]);
    }
    return out.join("");
}

// Decompress an LZW-encoded string
function lzw_decode(s) {
    var dict = {};
    var data = (s + "").split("");
    var currChar = data[0];
    var oldPhrase = currChar;
    var out = [currChar];
    var code = 256;
    var phrase;
    for (var i=1; i<data.length; i++) {
        var currCode = data[i].charCodeAt(0);
        if (currCode < 256) {
            phrase = data[i];
        }
        else {
           phrase = dict[currCode] ? dict[currCode] : (oldPhrase + currChar);
        }
        out.push(phrase);
        currChar = phrase.charAt(0);
        dict[code] = oldPhrase + currChar;
        code++;
        oldPhrase = phrase;
    }
    return out.join("");
}
57
votes

I had another problem, I did not want to encode data in gzip but to decode gzipped data. I am running javascript code outside of the browser so I need to decode it using pure javascript.

It took me some time but i found that in the JSXGraph library there is a way to read gzipped data.

Here is where I found the library: http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/ There is even a standalone utility that can do that, JSXCompressor, and the code is LGPL licencied.

Just include the jsxcompressor.js file in your project and then you will be able to read a base 64 encoded gzipped data:

<!doctype html>
</head>
<title>Test gzip decompression page</title>
<script src="jsxcompressor.js"></script>
</head>
<body>
<script>
    document.write(JXG.decompress('<?php 
        echo base64_encode(gzencode("Try not. Do, or do not. There is no try.")); 
    ?>'));
</script>
</html>

I understand it is not what you wanted but I still reply here because I suspect it will help some people.

41
votes

We just released pako https://github.com/nodeca/pako , port of zlib to javascript. I think that's now the fastest js implementation of deflate / inflate / gzip / ungzip. Also, it has democratic MIT licence. Pako supports all zlib options and it's results are binary equal.

Example:

var inflate = require('pako/lib/inflate').inflate; 
var text = inflate(zipped, {to: 'string'});
17
votes

I ported an implementation of LZMA from a GWT module into standalone JavaScript. It's called LZMA-JS.

14
votes

Here are some other compression algorithms implemented in Javascript:

8
votes

I did not test, but there's a javascript implementation of ZIP, called JSZip:

https://stuk.github.io/jszip/

0
votes

I guess a generic client-side JavaScript compression implementation would be a very expensive operation in terms of processing time as opposed to transfer time of a few more HTTP packets with uncompressed payload.

Have you done any testing that would give you an idea how much time there is to save? I mean, bandwidth savings can't be what you're after, or can it?

-4
votes

Most browsers can decompress gzip on the fly. That might be a better option than a javascript implementation.

-5
votes

You can use a 1 pixel per 1 pixel Java applet embedded in the page and use that for compression.

It's not JavaScript and the clients will need a Java runtime but it will do what you need.