Why does Node.js have incremental memory usage?

Question

I have a gameserver.js file that is larger than 100 KB. And I continued to check my task manager after each update in my browser and continued to see that the memory usage of node.exe continued to grow for each update. I use the ws module here: https://github.com/websockets/ws and realized that you know that there is most likely a memory leak in my somewhere ...

So, in order to double check and isolate the problem, I created the test.js file and placed it in the default ws code block:

 var WebSocketServer = require('ws').Server , wss = new WebSocketServer({ port: 9300 }); wss.on('connection', function connection(ws) { ws.on('message', function incoming(message) { console.log('received: %s', message); }); }); 

And ran it:

Now I check the use of node.exe memory:

The incremental part that confuses me:

If I update my browser, which connects to this 9300 websocket server, then looks back at my task manager .. it shows:

Now it is at 14,500 K

And it continues to grow on every update, so theoretically, if I just refresh, it will go through the roof. Is this intended? Is there a memory leak in the ws module somewhere maybe? The whole reason I ask is because I thought that after a few minutes or when the user closes the browser, he will return, but this is not so.

And the main reason I wanted to do this test was because I decided that I had a memory leak in my personal code and just wanted to check if it was me or vice versa. Now I'm at a standstill.

Answer 1

Observing increased memory with Node.js is completely normal behavior. Node.js constantly analyzes your current code, generates optimized code, returns to non-optimized code (if necessary), etc. This requires quite a bit of memory even for the simplest applications (Node.js itself is mostly written in JavaScript, which follows the same optimizations / deoptimizations as your own code).

In addition, a process may be given more memory when necessary, but many operating systems only remove allocated memory from a process when they decide that it is needed elsewhere (i.e., another process). Thus, the application can consume 1 GB of RAM at peak, then loading garbage, usage drops to 500 MB, but the process can still support 1 GB.

Memory Leak Detection

To properly analyze memory usage and memory leaks, you should use Node.js process.memoryUsage() .

You must configure an interval that resets the use of this memory to a file, that is, every second, and then apply "stress" to your application for several seconds (that is, for web servers, it produces several thousand requests). Then take a look at the results and see if the memory is increasing, or if it follows a robust increase / decrease scheme.

Memory Leak Detection

The best tool for this is most likely node-heapdump . You use it with a Chrome debugger.

• Start your application and apply the initial stress (this is to create optimized code and "warm up" your application).
• While the application is not working, create a heapdump file
• Perform one additional operation (i.e., another request) that you suspect is likely to cause a memory leak - this is probably the hardest part, especially for large applications
• Generate another heapdump file
• Download both heapdumps to the Chrome debugger and compare them - if there is a memory leak, you will see that there are some objects that were allocated during this single request, but were not subsequently released
• Examine the object to determine where the leak occurs.

I had the opportunity to investigate the reported memory leak in the Sails.js structure - you can see a detailed description of the analysis (including beautiful graphs, etc.) on this question .

There is also a detailed article about working with heapdumps from StrongLoop - I suggest taking a look at it.

Answer 2

The garbage collector is not called constantly because it blocks your process. Therefore, the V8 launches the GC when it considers it necessary.

To find out if you have a memory leak, I suggest starting the GC manually after each request to check if your memory is growing. Usually, if you do not have a memory leak, your memory should not increase. Because the GC will clear all unused objects. If your memory continues to grow after a GC call, you have a memory leak.

To start the GC manually, you can do it, but attention! Do not use it in production; this is just a way to clear the memory and see if you have a memory leak.

Launch Node.js as follows:

 node --expose-gc --always-compact test.js 

He will take out the garbage collector and make him be aggressive. Call this method to start the GC:

 global.gc(); 

Call this method after each hit on your server and see if the GC clears the memory or not.

You can also do two heaps of your process before and after the request to see the difference.

Do not use it in the manufacturing process or in the design. This is just a way to see if you have a memory leak or not.