HN Explorer

24 comments

[ 5.4 ms ] story [ 62.0 ms ] thread
udp
Pretty cool, but the thought of inflate/deflate running in Javascript makes me cringe. How fast is this?
fl3x
Edit: The numbers are only for the C implementation so they don't add much to the discussion. Sorry for posting too quickly!

I don't know about this library in particular, but there's some numbers available on http://liblzg.bitsnbites.eu/ for a similar library.

(The same author has also made a library for self extracting javascript code: http://crunchme.bitsnbites.eu/. Pretty cool, but probably not that useful outside of the demo scene.)

cleverjake
While I haven't used this in production, I can say that for zipping up stuff that was already generated client side it is usually quicker than sending the data to the server to be zipped, then getting it back. While this seems like a just-because-its-cool sort of hack, it actually has awesome implications. Zipped file creation from offline apps? Yes please.
bradleyland
I just tested with a 42 MB text log file under OS X 10.7.2.

Using zip 3.0:

    $ time zip logfile.zip logfile.log

    adding: logfile.log (deflated 89%)

    real	0m1.170s
    user	0m0.947s
    sys	0m0.030s
Resulting file size: 4,551,020 bytes

Using Chrome 16.0.912.77:

    4.7s - 1st drop
    5.5s - 2nd drop
    5.2s - 3rd drop
    5.1s - 4th drop
    Average 5.125s
    (timed using stopwatch)
Resulting file size: 4,547,885 bytes

Size: Advantage Zip.js by 3,135 bytes (roughly 3k; 6.8% better than zip) Speed: Advantage zip by 3.955s (77% better than Zip.js)

halter73
Zip.js only has a .069% size advantage over zip 3.0 given your numbers (4,551,020 bytes & 4,547,885 bytes respectively).

Your calculation of 6.8% better than zip just seemed too impressive.

bradleyland
Sorry, was off by a factor of 100.

Showing the math:

    zip        = 4,551,020
    zipjs      = 4,547,885
    diff       = zip - zipjs    => 3,135 bytes
    pctng_diff = diff/zip × 100 => 0.0688856564%
jorangreef
Anyone know of a Javascript implementation of Snappy? There's a binding for Node, but not a Javascript implementation which would be useful in the browser considering IndexedDB and Websockets support binary data.
gmac
No, but I've been using an LZMA library [1] to compress data sent over binary WebSockets.

LZMA is nice for streaming data to many clients (i.e. compress once, decompress many times), because although compression time/CPU is high (LZMA > BZIP2 > GZIP), compression ratio is very good (LZMA > BZIP2 > GZIP) and decompression not too bad (BZIP2 > LZMA > GZIP) [2].

[1] http://code.google.com/p/js-lzma/

[2] Disclaimer: I should check, but haven't yet, whether these inequalities hold for this JS LZMA implementation relative to the available GZIP and BZIP2 ones.

jorangreef
Thank you, I spotted BZIP2 and LZMA. Looking for compression/decompression cpu cost of X < GZIP < BZIP2 < LZMA where compression ratio is of secondary concern.
gildas
I am the author of the library. Just to be clear about my motivations, I wrote this library for chrome extensions. It's a simple way to export/import large user data.
tantalor
Can you tell us why you don't rely on built-in compression? I would guess both local storage and HTTP POST automatically compress files... is this not the case?
gildas
My need is to extract data (archived web pages) from a WebSQL database (or HTML5 filesytem). So the user can export it to a file. Otherwise, he couldn't export the extension data. All is done on client-side so I cannot use HTTP post.
pooriaazimi
Also check out: http://jszip.stuartk.co.uk/

I used it a week ago for a simple project (a kinda secure file system - with PHP and MongoDB for backend). I would encrypt a file on my server (AES) and then send the encrypted data to user's browser. Then I could decrypt the data with JavaScript and used JSZip to make user's browser download the decrypted files.

jonknee
Why go to all the hassle of encrypting/decrypting and not just use HTTPS?
pooriaazimi
Because it was an assignment for my 'Information & Communication Security' class and I had to do it without SSL/TLS! And it was a fun little project - I learnt how to use JQuery and worked with PHP, MongoDB and JavaScript. I usually code in Java and ObjC and it was a fun break...
gildas
Before writing zip.js, I tried this library. It was really great but id does not scale at all: it means the created zip file is not "streamed" so it can use a lot of RAM when creating large files.
etherealG
any idea what would be involved in getting this working in IE?
gildas
Since it heavily relies on new HTML5 APIs, I think the simplest solution is to wait for IE10 ;).
mrpollo
Might have been my poor connection or the file size limit, I just got this screen shot http://dl.dropbox.com/u/9694286/zip.js/ScreenShot2012-02-01a...

Here is the zip output http://dl.dropbox.com/u/9694286/zip.js/123Example.zip

And finally the guilty upload http://dl.dropbox.com/u/9694286/zip.js/search-engine-optimiz...

Hope it helps debug or something!

gildas
Thanks for the report. The zip file seems to be valid: I can open it without any issues with 7zip (on Win7). The "read zip demo" [1] is also able to open it and extract the compressed PDF. What zip software do you use?

[1] http://gildas-lormeau.github.com/zip.js/demos/demo2.html

mrpollo
Native from OSX 10.7
gildas
Thanks! Unfortunately, I don't own a Mac so it will take some time to fix this bug.