All geek questions in one place

NodeJS: What is the correct way to handle TCP socket streams? Which separator should I use?

From what I understood here , "V8 has a generation garbage collector that moves randomly. Node Can't get a pointer to raw string data to write to the socket." therefore, I should not store the data coming from the TCP stream in a string, especially if this string becomes larger than Math.pow(2,16) bytes. (hope I'm right so far ..)

What is the best way to handle all the data coming from a TCP socket? So far I have tried to use _:_:_ as a separator, because I think it is somehow unique and will not interfere with other things.

The sample data that would come would be something_:_:_maybe a large text_:_:_ maybe tons of lines_:_:_more and more data

Here is what I tried to do:

 net = require('net'); var server = net.createServer(function (socket) { socket.on('connect',function() { console.log('someone connected'); buf = new Buffer(Math.pow(2,16)); //new buffer with size 2^16 socket.on('data',function(data) { if (data.toString().search('_:_:_') === -1) { // If there no separator in the data that just arrived... buf.write(data.toString()); // ... write it on the buffer. it part of another message that will come. } else { // if there is a separator in the data that arrived parts = data.toString().split('_:_:_'); // the first part is the end of a previous message, the last part is the start of a message to be completed in the future. Parts between separators are independent messages if (parts.length == 2) { msg = buf.toString('utf-8',0,4) + parts[0]; console.log('MSG: '+ msg); buf = (new Buffer(Math.pow(2,16))).write(parts[1]); } else { msg = buf.toString() + parts[0]; for (var i = 1; i <= parts.length -1; i++) { if (i !== parts.length-1) { msg = parts[i]; console.log('MSG: '+msg); } else { buf.write(parts[i]); } } } } }); }); }); server.listen(9999);

Whenever I try console.log('MSG' + msg) , it prints the entire buffer, so it is useless to see if something is working.

How can I process this data correctly? Does the lazy module work even if this data is not line-oriented? Is there any other module for processing non-line oriented threads?

+5
source share
2 answers

Actually it was said that additional work is going on because Node has to take this buffer and then insert it into v8 / pass it to a string. However, doing toString () in the buffer is not better. So far, as far as I know, there is no good solution for this, especially if your ultimate goal is to get the string and trick it. Its one of the things that Ryan mentioned @nodeconf as an area to do the work.

As for the delimiter, you can choose whatever you want. Many binary protocols allow you to include a fixed header, so you can put things in a normal structure that includes length many times. Thus, you cut the well-known header and get information about the rest of the data without the need for repetition throughout the buffer. Using such a scheme, you can use a tool such as:

Third-party buffers can be accessed through array syntax, and they can also be sliced ​​using .slice ().

Finally check here: https://github.com/joyent/node/wiki/modules - find a module that parses the simple tcp protocol and seems to do it well and read some code.

+4
source

You should use the new stream2 api. http://nodejs.org/api/stream.html

Here are some very useful examples: https://github.com/substack/stream-handbook

https://github.com/lvgithub/stick

+2
source

Source: https://habr.com/ru/post/927636/

More articles: