Geek Answers Handbook

Memory Consumption of Sparse Arrays in Node.js

I wrote a small program that creates arrays that work for quite some time (almost forever ;-)):

var results = []; var i = 1; while (true) { console.log(i++); results.push([]); }

When instead of an empty array I create a sparse array of length i , the program crashes pretty quickly:

 var results = []; var i = 1; while (true) { console.log(i); results.push(new Array(i++)); }

Actually, I get to i equal to 17424, then I get an error message telling me

 FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory Abort trap: 6

and Node.js brings me back to the console. Since the only difference is that the second creates “larger” empty arrays than the first, this means that an empty sparse array of length n takes n times the space of an empty array of length 1 .

Am I right about this (in particular, Node.js)?

One more question: if I ran

 var results = []; var i = 1; while (true) { console.log(i); var temp = []; temp[i++] = i; results.push(temp); }

then I get up to 1286175, then crash again:

 FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory Abort trap: 6

Why does this behave differently than the other two options?

PS: I use Node.js 0.12.0 to run this on OS X.

+5
source share
2 answers

When you declare an array with size

 Array(1024);

You make it allocate space for 1024 elements. It should take this space forward because this form of declaring an array is an optimization indicating

"I need you to reserve 1024 places so as not to resize the array when I click on it with more elements."

As you probably know, declaring an array using just [] still allows you to click on it an unlimited number of elements, however the array does not resize (most likely memcpy() ) backstage to allow this behavior.

EDIT:

The reason you get much higher iterations in the second example is because you are now using a sparse array. With a rare array making

 var arr = [] arr[1000000] = 1;

Doesn't mean that your array now uses 1,000,000 records in memory. Contrast this with a dense array

 var arr = Array(1000000);

which explicitly tells runtime to reserve an array that can store 1,000,000 records in memory.

Related StackOverflow question: fooobar.com/questions/6253 / ...

+6
source

V8, the JS engine in Node, uses 4 bytes for each element in a seemingly empty array. The best way to find out for sure is to create empty arrays in Chrome and use the profiler to find out how much extra size it used. For more details on how you can do this, see https://developer.chrome.com/devtools/docs/heap-profiling ...

+4
source

More articles:


All Articles