Memory leak in Express.js with EventSource

Question

I think I ran into a memory leak using the Express application when connecting the number x of EventSource to it. After connecting clients and sending their x messages and disconnecting them, my Express application only releases a small amount of allocated heap / RSS.

To confirm this, I saved Heapdump at server startup and one after connecting 7,000 clients to it and sending x messages to each client. I waited a while to give the GC the opportunity to clear before taking a picture of the heap.

To compare these heap snapshots, I loaded them into the Chrome Developer Profile view and selected the "Compare" mode.

My questions:

1) How to interpret these numbers? (For reference, see a screenshot of the attached heap snapshot.)

2) For example, it seems that Socket objects almost do not contain any objects at all, is this true?

3) Can you give me more tips to investigate the problem?

Answer 1

You can be free from memory leaks and, as a bonus, avoid the garbage collector. All you have to do is poll objects.

You can do something like

var clientsPool = new Array(1000); var clientsConnected = []; 

When a new client connects, you execute

 var newClient = clientsPool.pop(); //set your props here clientsConnected.push(newClient); 

This is an amazing way to avoid garbage collection and prevent memory leaks. Of course, there is a bit more work there, and you will need to handle this carefully, but it is completely worth the performance.

There's a terrific talk about it, here you go https://www.youtube.com/watch?v=RWmzxyMf2cE

Answer 2

As for my comment ...

Javascript cannot clear the memory section if something points to it about 2 years ago, someone found an exploit, and it quickly closed, and it works like this:

 var someData = ["THIS IS SOME DATA SAY IT WAS THE SIZE OF A SMALL APPLICATION"]; var somePointer = someData[0]; delete someData; 

Then they entered the application into somePointer, because it was a reference to a memory location when there was no data. hey presto you have entered the memory.

So, if there is a link, as mentioned above somePointer = someData[0]; , you cannot free memory until you delete someData , so you need to delete all links to everything that you want to clear in your case ALL_CLIENTS.push(this); on line 64, makes your system memory accessible through ALL_CLIENTS, so you can do

Line 157

 _.each(ALL_CLIENTS, function(client, i) { var u; // holds a undefined value (null, empty, nothing) client.close(); //delete ALL_CLIENTS[i]; ALL_CLIENTS[i] = u; ALL_CLIENTS.unused++; }); 

In another note, this is not a memory leak, a memory leak means that you had this server, you close it if the memory is not freed after you left it, then you have a memory leak, if it clears the memory behind it, not a leak, it's just bad memory management.

Thanks to @Magus for pointing out that deleting is not the best thing you could use, but I would never recommend you implement a restrictive structure, but you could try

Line 27: ALL_CLIENTS.unused = 0;

Line 64:

 var u; if(ALL_CLIENTS.unused > 0){ for(var i = 0; i < ALL_CLIENTS.length; i++){ if(ALL_CLIENTS[i] == u){ ALL_CLIENTS[i] = this; ALL_CLIENTS.unused--; i = ALL_CLIENTS.length; } } }else{ ALL_CLIENTS.push(this); }