Limitations in Chrome "Unzippin"

[mikaelhm]

Hi,

I am working on an offline web-app where user need to load data from zip-files, and jszip does a really good job there.

Do you have any idea why Chrome is chrashing (not just hanging) much earlier than Firefox and Safari?

I did a very small test trying 4 different file sizes (10.2 MB, 12.7 MB, 30.6 MB, 51.1 MB).

Browser	Version	File size working	File size chrashing
Chrome	33.0.1750.152	10.2 MB	12.7 MB
Safari	7.0.2 (9537.74.9)	51.1 MB	--
Firefox	26.0	51.1 MB	--

I did not try anything higher than 51.1MB as this is already much bigger than what we need.

Machine Info
Model Identifier	MacBookPro11,3
Processor Name	Intel Core i7
Processor Speed	2.3 GHz
Memory	16 GB
System Version	OS X 10.9.2 (13C64)

[dduponchel]

I suspect a memory issue (but 10MB seems low). Could you share the files and the code snippet using JSZip ?
It could come from the size of the uncompressed content and/or the operations you do on the loaded file.

[mikaelhm]

Hi,

I did some further investigation and found some interesting results.

The problem is not unzipping the file, it is reading it with asText. However, the problem only exists in Chrome. Any Idea Why

Here's the small example i used:
http://github.com/mikaelhm/jszip-test/

The testFiles folder has the zip files I am using (random generated csv-files)

The page uses console.time to measure timing (see js console for results).

[dduponchel]

The method asText() will try to decode the content as an unicode string. In Firefox we have the TextEncoder/TextDecoder API (don't know if Safari has it) which allow a fast / memory efficient conversion. On Chrome, this API exists but doesn't seem to be active (it is active on Chrome Canary if I remember correctly).
In that case we fallback to a manual conversion which seems to cause your issue.

If you load files that will always contains ASCII text, you can use .asBinary() instead. This method will return a binary string : it is strictly equal to an ASCII text but you will get garbage text with non ASCII unicode characters. That's not a solution, of course.

I'll see if I can optimize this method on Chrome.

[mikaelhm]

@dduponchel that makes sense :)

Neither of my Chrome or Safari versions have the TextDecoder.
But Interestingly Safari does not crash when I try to unzip the biggest file, where chrome does.

So maybe there can be some optimizations.

[dduponchel]

I didn't find any way to optimize the current method. But I read your file as a string without crashing the page in chrome with a FileReader :

var ab = zipEntry.asArrayBuffer();
var blob = new Blob([ab]);
var reader = new FileReader();
reader.onload = function(e) {
    var txt = e.target.result;
};
reader.readAsText(blob);

I only tested Firefox and Chrome but it should work in other browsers too.

[mikaelhm]

@dduponchel
Thanks! that is an interesting approach!

I did a small test.
It shows that this approach give a drastic speed up in Chorme, some speedup in Safari, but some slowdown in Firefox.

Test file: 208000.csv.zip

Browser	direct asText time	using FileReader
Chrome	3779ms	419ms
Firefox	299ms	364ms
Safari	11828ms	8452ms

I know this is a very limited example, which really doesn't show much. I have also seen cases where the FileReader approach is faster in Firefox, so I think I will stick with that for now.

[dduponchel]

I'm glad I could help you :-)
JSZip currently has a synchronous API and will need some serious thoughts to provide a consistent asynchronous one, but that's definitely something to try.

[puzrin]

@dduponchel do you mean async or chunking? It's not mandatory to make chunking async. Async can be useful only for tricks:

• utf8 recode via filereader api
• cache via indexeddb
• unfreeze interface via webworker or setTimeout.

Chunking solves another problems:

• double buffering (when 100MB file requires 300-500MB tо proceed)
• better memory allocation and more effective gc.

IMHO, migrating to blobs (when possible) for internal data store should help.

[dduponchel]

@puzrin I was thinking of async (mainly because the FileReader API could help a lot). Chunking is also a path to explore !

[puzrin]

@dduponchel i've added strings support to pako (now in master only), and can confirm, that string operations in chrome are very fast. You can check 2 new benchmarks, when inflate/deflate get 1mb binary/utf8 strings on input and output. Speed difference is not significant.

Probably, it's some mistake in jszip, that breaks JIT inline caches. I guess, after fixing it you can also drop TextEncoder/TextDecoder as useless overcomplication.

[dduponchel]

That's interesting. I'm almost sure that JSZip triggers deoptimizations at the wrong places. Time to use hydra I guess :-)

[puzrin]

Found bug in my utf8 decoding part. Speed is significantly slower, compared to pure inflate. (~19mb/sec against 97mb/sec on my mac mini). But note, that even 19 mb/sec is not "very slow".

Temporary out of ideas. Decoding speed constantly jumps between 19 and 40 mb/sec, and i can't find WTF happens. Looks like floating deopt. Last decoder is here https://github.com/nodeca/pako/blob/2cd0309b62efbf86a162bb928c4b58fed457e2cf/lib/utils/strings.js#L107-L172 . Note, that it relies on small chunks (inflate chunked output), and effectively uses String.fromCharCode.apply on array of words. You also use this method, but not for utf8/utf16 recoding.